[Bug] --exclude doesn't work when --defer flag is provided

[AndrewCanva]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

it runs the model in --exclude while it has --defer in the dbt command whether the model is included in --select or not.

Expected Behavior

dbt should exclude the model specified after --exclude whether it is included in --select when --defer is provided. In other words, --exclude overrides --select.

Steps To Reproduce

run a project with dbt run --select state:modified+1 --exclude m --state /tmp/tmp.daf --defer --target tgt

Relevant log output

same as the usual dbt logs but runs the excluded ones

Environment

- OS: ubuntu 20.04
- Python: 3.7.9
- dbt: 1.0.1

What database are you using dbt with?

redshift

Additional Context

No response

[ChenyuLInx]

Hey @AndrewCanva, thanks for opening this, and sorry for getting back to you late. I looked up the test case and seems like we don't have test coverage of mixing --defer and --exclude together.
To get more information related to this, I have two questions.

• Does the --exclude work as expected in the case --defer is not provided?
• are the model/s in --exclude exists in the --state /tmp/tmp.daf?

We probably should exclude those model/s regarding the answer to the questions, but more information is always helpful. Thanks!

[AndrewCanva]

Hey @AndrewCanva, thanks for opening this, and sorry for getting back to you late. I looked up the test case and seems like we don't have test coverage of mixing --defer and --exclude together. To get more information related to this, I have two questions.

• Does the --exclude work as expected in the case --defer is not provided?
• are the model/s in --exclude exists in the --state /tmp/tmp.daf?

We probably should exclude those model/s regarding the answer to the questions, but more information is always helpful. Thanks!

• --exclude works when `--defer is not provided
• the model in --exclude is in the --state /tmp/tmp.daf

[jtcohen6]

Hey @AndrewCanva, I haven't been able to reproduce this one!

1. Create two models, m and mplusone (where the the latter is select * from {{ ref('m') }})
2. dbt run --target prod
3. mv target/manifest.json state/
4. Change m in some way (e.g. add a SQL comment in models/m.sql)
5. dbt run --select state:modified+1 --exclude m --state state/ --defer --target dev

$ dbt run --select state:modified+1 --exclude m --state state/ --defer --target dev
09:29:54  Running with dbt=1.0.1
09:29:54  Unable to do partial parsing because config vars, config profile, or config target have changed
09:29:54  Found 2 models, 0 tests, 0 snapshots, 0 analyses, 167 macros, 0 operations, 1 seed file, 0 sources, 0 exposures, 0 metrics
09:29:54
09:29:55  Concurrency: 5 threads (target='dev')
09:29:55
09:29:55  1 of 1 START view model dbt_jcohen.mplusone..................................... [RUN]
09:29:55  1 of 1 OK created view model dbt_jcohen.mplusone................................ [CREATE VIEW in 0.06s]
09:29:55
09:29:55  Finished running 1 view model in 0.41s.
09:29:55
09:29:55  Completed successfully
09:29:55
09:29:55  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

And, when I check compiled/run SQL in target/ or logs/, I see that mplusone selects from the "prod" version of m

[AndrewCanva]

@jtcohen6 ah, I know why it doesn't exclude the model when I run it. it is not related to defer, what I did in a script is

local args="--exclude m"
dbt run --select state:modified+1 "${args}" --state /tmp/tmp.daf --defer --target tgt

when I remove the double quotes, it works fine

local args="--exclude m"
dbt run --select state:modified+1 ${args} --state /tmp/tmp.daf --defer --target tgt

the m isn't in the --select, what's confusing is that dbt ignores the --exclude and takes "${args}" as m.

that's probably because of argparse, I will close this issue if you think it is not a bug.

[jtcohen6]

Ahh, that's a weird one all right. We're talking about refactoring the dbt CLI (#4179, #4475). That work might include moving away from argparse, given some of its shortcomings. I think we should close the issue, and I'm happy to have it linked from there for visibility.