Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dc/8546 semantic models in graph selection #8589

Merged
merged 10 commits into from
Sep 26, 2023

Conversation

dave-connors-3
Copy link
Contributor

@dave-connors-3 dave-connors-3 commented Sep 7, 2023

resolves #8546

  • adds SemanticModelSelectionMethod to allow for dbt ls -s "semantic_model:*"
  • adds ability to dbt ls --resource-type semantic_model

as of 9/7

SemanticModelSelectionMethod works

❯ dbt ls -s +semantic_model:orders                      
15:10:25  Running with dbt=1.7.0-b2
15:10:25  Registered adapter: snowflake=1.7.0-b1
15:10:25  Found 5 models, 1 analysis, 1 source, 0 exposures, 0 metrics, 514 macros, 0 groups, 1 semantic model
davebt.numbers # model
semantic_model:davebt.orders # semantic model
  • dbt ls does not include semantic models
  • dbt ls --resource-type semantic_model yields no nodes

Problem

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@cla-bot cla-bot bot added the cla:yes label Sep 7, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Sep 7, 2023

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

@codecov
Copy link

codecov bot commented Sep 7, 2023

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (aa86fdf) 86.61% compared to head (89d6aad) 86.59%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8589      +/-   ##
==========================================
- Coverage   86.61%   86.59%   -0.02%     
==========================================
  Files         176      176              
  Lines       25678    25731      +53     
==========================================
+ Hits        22240    22283      +43     
- Misses       3438     3448      +10     
Flag Coverage Δ
integration 83.35% <68.85%> (-0.01%) ⬇️
unit 65.14% <55.73%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
core/dbt/cli/params.py 100.00% <ø> (ø)
core/dbt/compilation.py 95.84% <100.00%> (-0.35%) ⬇️
core/dbt/graph/cli.py 94.35% <100.00%> (ø)
core/dbt/task/list.py 93.38% <100.00%> (+4.69%) ⬆️
core/dbt/graph/selector_methods.py 91.03% <74.07%> (-0.70%) ⬇️
core/dbt/contracts/graph/nodes.py 95.09% <54.16%> (-1.06%) ⬇️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jtcohen6
Copy link
Contributor

jtcohen6 commented Sep 8, 2023

TODO: finish ability to dbt ls --resource-type semantic_model

Found it:

DEFAULT_INCLUDES: List[str] = ["fqn:*", "source:*", "exposure:*", "metric:*"]

Should be:

DEFAULT_INCLUDES: List[str] = ["fqn:*", "source:*", "exposure:*", "metric:*", "semantic_model:*"]

Then semantic models will be included when --select is not passed.

It does feel like there's a real opportunity for refactoring here. It feels odd that sources/exposures/semantic_models/metrics are "pointer" node types, as opposed to the "logical" node types (models/seeds/snapshots/tests/analyses), and only those are included by the fqn:* selection.

But I think that's all out of scope for something we want to backport to v1.6!

@jtcohen6
Copy link
Contributor

Thanks for the solid start @dave-connors-3 !!

@MichelleArk is going to take this one over the finish line, so we can get it merged & backported for the next v1.6.x patch

@MichelleArk MichelleArk added the user docs [docs.getdbt.com] Needs better documentation label Sep 25, 2023
@MichelleArk MichelleArk marked this pull request as ready for review September 26, 2023 09:59
@MichelleArk MichelleArk requested a review from a team as a code owner September 26, 2023 09:59
@MichelleArk MichelleArk requested review from martynydbt and gshank and removed request for martynydbt September 26, 2023 09:59
@MichelleArk MichelleArk merged commit bb4214b into main Sep 26, 2023
50 checks passed
@MichelleArk MichelleArk deleted the dc/8546-semantic-models-in-graph-selection branch September 26, 2023 22:23
@FishtownBuildBot
Copy link
Collaborator

Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#4133

@FishtownBuildBot
Copy link
Collaborator

Opened a new issue in dbt-labs/docs.getdbt.com: dbt-labs/docs.getdbt.com#4134

gshank added a commit that referenced this pull request Sep 27, 2023
* Dc/8546 semantic models in graph selection (#8589)

(cherry picked from commit bb4214b)

* Update test_list.py to remove config "access" and remove group
comparison from SemanticModel since there is no group

---------

Co-authored-by: dave-connors-3 <[email protected]>
Co-authored-by: Gerda Shank <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.6.latest cla:yes user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-3073] [Bug] semantic model doesn't work in the "plus" node selection
5 participants