Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Unit tests fail when input has a column name with a quoted identifier #205

Open
2 tasks done
mpatek opened this issue May 12, 2024 · 5 comments · May be fixed by #204
Open
2 tasks done

[Bug] Unit tests fail when input has a column name with a quoted identifier #205

mpatek opened this issue May 12, 2024 · 5 comments · May be fixed by #204
Labels
bug Something isn't working community This PR is from a community member Medium Severity bug with minor impact that does not have resolution timeframe requirement quoting Issues related to dbt's quoting behavior unit tests Issues related to built-in dbt unit testing functionality

Comments

@mpatek
Copy link

mpatek commented May 12, 2024

Is this a new bug?

  • I believe this is a new bug
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

In BigQuery:

Supposing an input table with a reserved-word-named column like:

-- stg
SELECT id, loaded_at, `from` FROM {{ source('some', source') }}

And a downstream table like:

-- some_model
SELECT id, loaded_at FROM {{ ref('stg') }}
qualify row_number() over (partition by id order by loaded_at desc) = 1

And a unit test like:

version: 2

unit_tests:
  - name: test__some_model
    model: some_model
    given:
      - input: ref('stg')
        rows:
          - {id: "a", loaded_at: "2024-01-01"}
          - {id: "a", loaded_at: "2024-01-02"}
          - {id: "a", loaded_at: "2024-01-02"}
    expect:
      rows:
        - {id: "a", loaded_at: "2024-01-02"}

If I try dbt test --select test_type:unit

I get: Syntax error: Unexpected keyword FROM

Expected Behavior

Expect tests to pass without errors.

Steps To Reproduce

  1. Create input model with keyword-named column (e.g. from)
  2. Create output model that selects from the input model (not necessarily including the reserved-word-named column).
  3. Add unit test that puts data into input model and runs expectations on output model
  4. Run unit test

Relevant log output

No response

Environment

- OS: macOS Monterey
- Python: 3.11.7
- dbt-adapter: 1.1.0

Additional Context

Original Issue opened in the dbt-core repo: dbt-labs/dbt-core#10112

@jtcohen6
Copy link
Contributor

Thanks for reopening here @mpatek!

Prototype of potential solution:

@jtcohen6 jtcohen6 added Medium Severity bug with minor impact that does not have resolution timeframe requirement and removed triage labels May 13, 2024
@dbeatty10
Copy link
Contributor

See below for a minimal reproducible example that covers the maximal number of edge cases.

Minimal reprex

models/my_model.sql

select 1 as {{ adapter.quote("from") }}

models/_unit.yml

unit_tests:

  - name: test___my___model
    model: my_model
    given: []
    expect:
      rows:
        - {from: 1}

Run the model and its tests:

dbt build -s my_model

@dbeatty10 dbeatty10 added quoting Issues related to dbt's quoting behavior unit tests Issues related to built-in dbt unit testing functionality labels May 13, 2024
@dbeatty10 dbeatty10 changed the title [Bug] Unit tests fail when input contain reserved-word-named columns [Bug] Unit tests fail when input has a column name with a quoted identifier May 14, 2024
@robertf-b
Copy link

Also seeing this issue, it looks like #204 is fixing it - is that going anywhere?

@megetron3
Copy link

@dbeatty10 - we are curious about the status of this ticket and if there is an available workaround. This issue is currently blocking the unittests feature since all of our models use quoted column names in Snowflake.

@kdeggelman
Copy link

Echoing the call for some more visibility into the status of this issue, as it's also preventing us from being able to unit test many of our models. In our case many of our tables have a column called group.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working community This PR is from a community member Medium Severity bug with minor impact that does not have resolution timeframe requirement quoting Issues related to dbt's quoting behavior unit tests Issues related to built-in dbt unit testing functionality
Projects
None yet
7 participants