Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve DAG Text Formatting #975

Merged
merged 9 commits into from
Jan 26, 2024
Merged

Conversation

plypaul
Copy link
Contributor

@plypaul plypaul commented Jan 13, 2024

Description

This PR:

  • Adds Dag.text_structure() / DagNode.text_structure() to make it easier to generate the text representation of a DAG.
  • Removes previous methods that were used for text formatting a DAG.
  • Updates the logic for how the max. column width was handled when formatting a DAG to reduce the column width of snapshot files.

@cla-bot cla-bot bot added the cla:yes label Jan 13, 2024
@plypaul plypaul added the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jan 13, 2024
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 13, 2024 04:14 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 13, 2024 04:14 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 13, 2024 04:14 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 13, 2024 04:14 — with GitHub Actions Inactive
@plypaul plypaul marked this pull request as ready for review January 13, 2024 04:21
@github-actions github-actions bot removed the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jan 13, 2024
Copy link
Contributor

@courtneyholcomb courtneyholcomb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

<!-- from_source = SqlSelectStatementNode(node_id=ss_10) -->
<!-- where = None -->
<!-- distinct = False -->
<SqlSelectStatementNode>
<!-- description = Pass Only Elements: ['visits',] -->
<!-- node_id = ss_10 -->
<!-- description = "Pass Only Elements: ['visits',]" -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we default to double quotes for these descriptions for better consistency? Not a strong opinion here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean how it uses " in some cases but ' in others? It seems that's the behavior of repr() / pprint.pformat() if it's a string or a string containing a quote.

@plypaul plypaul force-pushed the plypaul--75--mf-format-snapshots branch 2 times, most recently from ff71ec2 to 9b9bb09 Compare January 19, 2024 20:42
Base automatically changed from plypaul--75--mf-format-snapshots to main January 19, 2024 20:50
@plypaul plypaul force-pushed the plypaul--83--improve-dag-formatting branch from bd04368 to fc4fe7b Compare January 19, 2024 20:54
@@ -111,6 +112,10 @@ def accept_dag_node_visitor(self, visitor: DagNodeVisitor[VisitorOutputT]) -> Vi
"""Visit this node."""
return visitor.visit_node(self)

def text_structure(self, formatter: MetricFlowDagTextFormatter = MetricFlowDagTextFormatter()) -> str:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change warms my cold grinchy heart....

@plypaul plypaul force-pushed the plypaul--83--improve-dag-formatting branch from fc4fe7b to 89d5fe4 Compare January 23, 2024 01:28
@plypaul plypaul added the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jan 23, 2024
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 23, 2024 22:11 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 23, 2024 22:11 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 23, 2024 22:11 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 23, 2024 22:11 — with GitHub Actions Inactive
@github-actions github-actions bot removed the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jan 23, 2024
This adds a .text_structure() to the DAG classes for easier conversion to a text
representation.
These are no longer necessary as the DAG classes now have a .text_structure().
This updates the logic for how max with is tracked / handled when recursively
formatting a DAG  to properly handle the indented / recursive case.
@plypaul plypaul force-pushed the plypaul--83--improve-dag-formatting branch from 89d5fe4 to 3d3d82a Compare January 26, 2024 11:45
@plypaul plypaul added the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jan 26, 2024
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 26, 2024 11:45 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 26, 2024 11:45 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 26, 2024 11:45 — with GitHub Actions Inactive
@plypaul plypaul temporarily deployed to DW_INTEGRATION_TESTS January 26, 2024 11:45 — with GitHub Actions Inactive
@github-actions github-actions bot removed the Run Tests With Other SQL Engines Runs the test suite against the SQL engines in our target environment label Jan 26, 2024
@plypaul plypaul merged commit eea6fbe into main Jan 26, 2024
23 checks passed
@plypaul plypaul deleted the plypaul--83--improve-dag-formatting branch January 26, 2024 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants