[Feature] --empty flag should be exposed in flags variable

[jlucas91]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When running dbt 1.8 EMPTY is not present on the flags variable.

Expected Behavior

When using the --empty flag, I would expect EMPTY to be set to true. This is would bring it in line with other first class flags like full-refresh and fail-fast.

Steps To Reproduce

1. ./dbt run --empty
2. Within a model, include {{ print(flags) }}
3. FLAGS is not present

Relevant log output

No response

Environment

- OS:
- Python:
- dbt:

Which database adapter are you using with dbt?

snowflake

Additional Context

It does not appear to be included in the flags module https://github.com/dbt-labs/dbt-core/blob/HEAD/core/dbt/flags.py

[dbeatty10]

Thanks for reaching out @jlucas91 !

This sounds more like a feature request to me. Can you share more why this seems like a bug to you?

[jlucas91]

I think it could go either way and no worries on my end if you'd prefer to reclassify.

It felt like a potential unintentional miss in the --empty flag release to me, which is why I leaned towards calling it a bug. While the documentation does state the code is the source of truth for what flags are included in the variable, it seemed strange for a significant and first class flag to be omitted from the interface.

[jtcohen6]

For what it's worth, I do buy this as a thing we should do!

Among other things, this might enable us to provide users with slightly more flexibility & customization while working around known edge cases of --empty.

As an example:

• [Bug] --empty flag not working on Pseudo-columns dbt-bigquery#1243 (comment)

[jlucas91]

That's exactly how I bumped into the flags limitation @jtcohen6 ! I'm doing a lot of checks like this right now to guard the various limitation cases we've bumped into:

{% set empty_run = '--empty' in flags.INVOCATION_COMMAND %}
{% if empty_run %}
-- compatibility code
{% else %}
-- model code
{% endif %}

That's pretty gross and will get nicer with the flags change. To take it one step further we've opted to implement a is_empty_run macro to clean up the syntax even more. A similar macro might be nice to roll into core DBT as a bit of syntactic sugar for users (similar to is_incremntal()).

[dsillman2000]

That's exactly how I bumped into the flags limitation @jtcohen6 ! I'm doing a lot of checks like this right now to guard the various limitation cases we've bumped into:

{% set empty_run = '--empty' in flags.INVOCATION_COMMAND %}
{% if empty_run %}
-- compatibility code
{% else %}
-- model code
{% endif %}

That's pretty gross and will get nicer with the flags change. To take it one step further we've opted to implement a is_empty_run macro to clean up the syntax even more. A similar macro might be nice to roll into core DBT as a bit of syntactic sugar for users (similar to is_incremntal()).

Hi @jlucas91 - this feature has a crucial bug (flags.INVOCATION_COMMAND) which prevents using it reliably the way you have above.

If you run dbt using programmatic invocation, then the flags.INVOCATION_COMMAND routes sys.argv directly. I'm seeing this when I invoke dbt programmatically from a pytest function:

def test_not_empty():
    dbt_runner = dbtRunner()
    run_result = dbt_runner.invoke(
        ["compile", "--select", "compile_empty_test"],
        project_dir=DBT_PROJECT_DIR,
        profiles_dir=DBT_PROJECT_DIR,
    )
    results = run_result.result.results
    assert len(results) == 1
    assert results[0].node.compiled_code.strip().lower() == "select 2"

In the model I "info-print" the flags.INVOCATION_COMMAND which reveals it harvested sys.argv from the pytest invocation:

INVOCATION_COMMAND='dbt -sv test_empty.py'

This unrelated bug prevents us from reliably using flags.INVOCATION_COMMAND for this purpose (or virtually anything else, so long as we're programmatically invoking dbt, e.g. in CI). Any other work-arounds available until "EMPTY" is added to flags?

(P.S. - I didn't manage to find this bug mentioned anywhere - should I open a separate issue to have it addressed? Thanks!)

[dbeatty10]

(P.S. - I didn't manage to find this bug mentioned anywhere - should I open a separate issue to have it addressed? Thanks!)

@dsillman2000 Yes, that would be great if you open a separate issue 👍

[dbeatty10]

Resolved by #10315