Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Regression] 1.9.pre lost the warning at the project level (i.e. start of the run) that notified users that a model depends on a now deprecated model #10833

Closed
2 tasks done
jeremyyeo opened this issue Oct 8, 2024 · 3 comments · Fixed by #10852
Labels
bug Something isn't working regression

Comments

@jeremyyeo
Copy link
Contributor

jeremyyeo commented Oct 8, 2024

Is this a regression in a recent version of dbt-core?

  • I believe this is a regression in dbt-core functionality
  • I have searched the existing issues, and I could not find an existing issue for this regression

Current Behavior

Pre-1.9 (1.8, 1.7), dbt emitted a "project level" warning that a model was using a now deprecated model. This seems to have been removed in 1.9 (versionless in dbt Cloud).

Expected/Previous Behavior

Bring that warning back :)

Steps To Reproduce

Project setup:

# dbt_project.yml
name: my_dbt_project
profile: all
config-version: 2
version: "1.0.0"

models:
  my_dbt_project:
    +materialized: table

# models/schema.yml
models:
  - name: foo
    deprecation_date: 1999-01-01 00:00:00.00+00:00
-- models/foo.sql
select 1 id

-- models/bar.sql
select * from {{ ref('foo') }}

Try running with 1.8:

$ dbt run --no-partial-parse

07:00:33  Running with dbt=1.8.7
07:00:33  Registered adapter: postgres=1.8.2
07:00:33  [WARNING]: Model foo has passed its deprecation date of 1999-01-01T00:00:00+00:00. This model should be disabled or removed.
07:00:33  [WARNING]: While compiling 'bar': Found a reference to foo, which was deprecated on '1999-01-01T00:00:00+00:00'. 
07:00:33  Found 2 models, 417 macros
07:00:33  
07:00:33  Concurrency: 4 threads (target='pg')
07:00:33  
07:00:33  1 of 2 START sql table model public.foo ........................................ [RUN]
07:00:33  1 of 2 OK created sql table model public.foo ................................... [SELECT 1 in 0.05s]
07:00:33  2 of 2 START sql table model public.bar ........................................ [RUN]
07:00:33  2 of 2 OK created sql table model public.bar ................................... [SELECT 1 in 0.02s]
07:00:33  
07:00:33  Finished running 2 table models in 0 hours 0 minutes and 0.22 seconds (0.22s).
07:00:33  
07:00:33  Completed successfully
07:00:33  
07:00:33  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2

Warning tells us that model bar is using a deprecated foo:

07:00:33  [WARNING]: While compiling 'bar': Found a reference to foo, which was deprecated on '1999-01-01T00:00:00+00:00'. 

Swap over to 1.9 pre and run again:

$ dbt run --no-partial-parse

07:01:31  Running with dbt=1.9.0-b2
07:01:31  Registered adapter: postgres=1.9.0-b1
07:01:31  [WARNING]: Model foo has passed its deprecation date of 1999-01-01T00:00:00+00:00. This model should be disabled or removed.
07:01:31  Found 2 models, 424 macros
07:01:31  
07:01:31  Concurrency: 4 threads (target='pg')
07:01:31  
07:01:32  1 of 2 START sql table model public.foo ........................................ [RUN]
07:01:32  1 of 2 OK created sql table model public.foo ................................... [SELECT 1 in 0.05s]
07:01:32  2 of 2 START sql table model public.bar ........................................ [RUN]
07:01:32  2 of 2 OK created sql table model public.bar ................................... [SELECT 1 in 0.02s]
07:01:32  
07:01:32  Finished running 2 table models in 0 hours 0 minutes and 0.22 seconds (0.22s).
07:01:32  
07:01:32  Completed successfully
07:01:32  
07:01:32  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2

Which is admittedly less helpful than before.

Relevant log output

No response

Environment

- OS: macOS
- Python: 3.11.9
- dbt (working version): 1.8.7
- dbt (regression version): 1.9.0-b2

Which database adapter are you using with dbt?

postgres

Additional Context

Since dbt Cloud versionless is on pre - then dbt Cloud also lost this warning message. Though you can go back to 1.7 to get this warning message back:

image
@jeremyyeo
Copy link
Contributor Author

This is a more reasonable scenario where this warning is helpful - 2 versions of foo, one is deprecated:

# models/schema.yml
models:
  - name: foo
    latest_version: 2
    versions:
      - v: 1
        deprecation_date: 1999-01-01 00:00:00.00+00:00
      - v: 2
-- models/foo_v1.sql
select 1 id

-- models/foo_v2.sql
select 2 id

-- models/bar.sql
select * from {{ ref('foo', v=1) }}
$ dbt run --no-partial-parse
07:23:55  Running with dbt=1.8.7
07:23:55  Registered adapter: postgres=1.8.2
07:23:55  [WARNING]: While compiling 'bar': Found a reference to foo.v1, which was deprecated on '1999-01-01T00:00:00+00:00'. A new version of 'foo' is available. Migrate now: {{ ref('my_dbt_project', 'foo', v='2') }}.
07:23:55  [WARNING]: Model foo.v1 has passed its deprecation date of 1999-01-01T00:00:00+00:00. This model should be disabled or removed.
07:23:55  Found 3 models, 417 macros
07:23:55  
07:23:55  Concurrency: 4 threads (target='pg')
07:23:55  
07:23:55  1 of 3 START sql table model public.foo_v1 ..................................... [RUN]
07:23:55  2 of 3 START sql table model public.foo_v2 ..................................... [RUN]
07:23:55  1 of 3 OK created sql table model public.foo_v1 ................................ [SELECT 1 in 0.06s]
07:23:55  2 of 3 OK created sql table model public.foo_v2 ................................ [SELECT 1 in 0.06s]
07:23:55  3 of 3 START sql table model public.bar ........................................ [RUN]
07:23:55  3 of 3 OK created sql table model public.bar ................................... [SELECT 1 in 0.02s]
07:23:55  
07:23:55  Finished running 3 table models in 0 hours 0 minutes and 0.24 seconds (0.24s).
07:23:55  
07:23:55  Completed successfully
07:23:55  
07:23:55  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

$ dbt run --no-partial-parse
07:20:14  Running with dbt=1.9.0-b2
07:20:14  Registered adapter: postgres=1.9.0-b1
07:20:15  [WARNING]: Model foo.v1 has passed its deprecation date of 1999-01-01T00:00:00+00:00. This model should be disabled or removed.
07:20:15  Found 3 models, 424 macros
07:20:15  
07:20:15  Concurrency: 4 threads (target='pg')
07:20:15  
07:20:15  1 of 3 START sql table model public.foo_v1 ..................................... [RUN]
07:20:15  2 of 3 START sql table model public.foo_v2 ..................................... [RUN]
07:20:15  2 of 3 OK created sql table model public.foo_v2 ................................ [SELECT 1 in 0.07s]
07:20:15  1 of 3 OK created sql table model public.foo_v1 ................................ [SELECT 1 in 0.07s]
07:20:15  3 of 3 START sql table model public.bar ........................................ [RUN]
07:20:15  3 of 3 OK created sql table model public.bar ................................... [SELECT 1 in 0.02s]
07:20:15  
07:20:15  Finished running 3 table models in 0 hours 0 minutes and 0.25 seconds (0.25s).
07:20:15  
07:20:15  Completed successfully
07:20:15  
07:20:15  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

@AnotherGuitar
Copy link

AnotherGuitar commented Oct 14, 2024

I believe this warning is currently issued by DeprecatedReference, so I assume this means that dbt 1.9 (currently only available in dbt Cloud - versionless) removes the following code:

class DeprecatedReference(WarnLevel):
def code(self) -> str:
return "I067"
def message(self) -> str:
ref_model_version = ".v" + self.ref_model_version if self.ref_model_version else ""
msg = (
f"While compiling '{self.model_name}': Found a reference to {self.ref_model_name}{ref_model_version}, "
f"which was deprecated on '{self.ref_model_deprecation_date}'. "
)
if self.ref_model_version and self.ref_model_version != self.ref_model_latest_version:
coda = (
f"A new version of '{self.ref_model_name}' is available. Migrate now: "
f"{{{{ ref('{self.ref_model_package}', '{self.ref_model_name}', "
f"v='{self.ref_model_latest_version}') }}}}."
)
msg = msg + coda
return warning_tag(msg)
; would it be straight forward for the dbt Labs team to just add back and keep this code in 1.9 and beyond?

@gshank
Copy link
Contributor

gshank commented Oct 15, 2024

This happened in the code changes in #10221, where it only checks currently deprecated models instead of all models. We should be able to fix it by looping through references to the deprecated model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working regression
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants