Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a Validation Rule for Duplicate Metrics in a Query #939

Merged
merged 1 commit into from
Dec 21, 2023

Conversation

plypaul
Copy link
Contributor

@plypaul plypaul commented Dec 15, 2023

Resolves #943

Description

This PR add a validation to check that queries do not contain the same metric more than once, as that would result in a duplicate column in the generated SQL.

@cla-bot cla-bot bot added the cla:yes label Dec 15, 2023
Copy link

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

@plypaul plypaul changed the title Add a Query Validation Rule for Duplicate Metrics in a Query. Add a Validation Rule for Duplicate Metrics in a Query Dec 15, 2023
@plypaul plypaul force-pushed the plypaul--58.14--resolve-ambiguous-queries branch 3 times, most recently from 98374e5 to 23edb4e Compare December 16, 2023 03:47
Base automatically changed from plypaul--58.14--resolve-ambiguous-queries to main December 16, 2023 04:00
@plypaul plypaul force-pushed the plypaul--58.15--add-duplicate-metric-validation-rule branch 3 times, most recently from 93d40d3 to 5497354 Compare December 18, 2023 20:23
@plypaul plypaul force-pushed the plypaul--58.15--add-duplicate-metric-validation-rule branch from 5497354 to e9f1657 Compare December 18, 2023 20:27
@plypaul plypaul marked this pull request as ready for review December 19, 2023 05:32
Copy link
Contributor

@tlento tlento left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an awful lot of code to do what amounts to assert len(set(metrics)) == len(metrics). We are certainly working very hard to accommodate this use of the visitor pattern, so I suppose there's that.

I'd very much prefer to dispense with yet another set of recursive DAG walks in favor of just validating the resolver output for correctness, seems like that'd be cleaner, simpler to build and update, and easier to reason about. I cannot come up with a use case where we'd need to do any kind of stateful dependency validation at this stage in the query processor.

That said, all of this..... exists, so I guess we may as well proceed and see if the hypothesis holds that this will really end up being the equivalent of map(resolver.resolved_metrics, lambda x: validator.validate(x))

@plypaul
Copy link
Contributor Author

plypaul commented Dec 21, 2023

This is an awful lot of code to do what amounts to assert len(set(metrics)) == len(metrics)

As previously mentioned, the value of the query validation framework is not there since most of the intended rules for it were moved out. I decided to keep it around for a little bit in case new use cases pop up and if not, I was going to take it out. If you want to go ahead and do that early, go ahead.

@plypaul plypaul merged commit 6cb59eb into main Dec 21, 2023
12 checks passed
@plypaul plypaul deleted the plypaul--58.15--add-duplicate-metric-validation-rule branch December 21, 2023 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SL-1455] [Feature] Add a Query Validation Rule for Repeated Metrics in a Query
2 participants