[Bug] python model - dbt.ref

[ArtrCtrs]

Is this a regression in a recent version of dbt-core?

• I believe this is a regression in dbt-core functionality
• I have searched the existing issues, and I could not find an existing issue for this regression

Current Behavior

I encounter an error when I try to dbt compile a python model that uses dbt.ref .
macro 'dbt_macro__ref' takes no keyword argument 'v'

Expected/Previous Behavior

In version 1.3.1, dbt compiles correctly the following python model code :
initial_dataset = dbt.ref("my_base_model")

Steps To Reproduce

1. install dbt-core 1.7.9 (& dbt-snowflake 1.7.2)
2. Create dbt python model containing initial_dataset = dbt.ref("my_base_model")
3. dbt compile
4. Get compilation error

Compilation Error in model my_python_model (models/my_python_model.py)
    macro 'dbt_macro__ref' takes no keyword argument 'v'

    > in macro build_ref_function (macros/python_model/python.sql)
    > called by macro py_script_postfix (macros/python_model/python.sql)
    > called by model my_python_model

Relevant log output

No response

Environment

- OS: tested on mac os 14.2.1 & Debian 10
- Python: tested on 3.8.16 & 3.12.2
- dbt (working version): 1.3.1
- dbt (regression version): 1.7.9 (possibly same bahavior since 1.5)

Which database adapter are you using with dbt?

snowflake

Additional Context

While testing locally, when I patch the core/dbt/include/global_project/macros/python_model/python.sql
file (build_ref_function method) to remove the v argument that raises the error from
{%- set resolved = ref(*_ref_args, v=_ref.get('version')) -%}
to
{%- set resolved = ref(*_ref_args) -%}
, it seems it compiles correctly.

Perhaps I'm missing something in how to use this.

This file was modified in this MR : https://github.com/dbt-labs/dbt-core/pull/7287/files#diff-a4206d061c239c90908b351474debd4a86881235c033efbbc0c6ddc81e8f7238L13

[dbeatty10]

Thanks for reaching out @ArtrCtrs !

Are you overriding the ref macro, by any chance?

If so, you'll want to follow the example here to update your ref macro.

[dbeatty10]

I looked at this a little closer, and there's two possibilities for changes to be made:

1. an update to the documentation for overriding the ref macro
2. updating the v argument (similar to your patch)

1. Overriding the ref macro

It might be as simple as making a slight update to the docs about overriding the ref macro.

Specifically, it looks like it needs to be this:

-- extract user-provided positional and keyword arguments
{% set version = kwargs.get('version') or kwargs.get('v') %}

Instead of this:

-- extract user-provided positional and keyword arguments
{% set version = kwargs.get('version') %}

2. Updating the v argument

Alternatively, we might need or want to do the change you tried out by changing this:

  {%- set resolved = ref(*_ref_args, v=_ref.get('version')) -%}

to this:

 {%- set resolved = ref(*_ref_args) -%}

But I don't know if it would preserve the version correctly that way. So I'm guessing it would need to be like this instead:

 {%- set resolved = ref(*_ref_args, version=_ref.get('version')) -%}

Reprex

models/my_base_model.sql

select 1 as id

models/my_staging_model.sql

select * from {{ ref("my_base_model", version=2) }}

models/my_python_model.py

def model(dbt, _):
    return dbt.ref("my_base_model", version=2)

models/_models.yml

models:
  - name: my_base_model
    latest_version: 2
    versions:
      - v: 2

macros/overrides.sql

{% macro ref() %}

-- extract user-provided positional and keyword arguments
{% set version = kwargs.get('version') %}
{% set packagename = none %}
{%- if (varargs | length) == 1 -%}
    {% set modelname = varargs[0] %}
{%- else -%}
    {% set packagename = varargs[0] %}
    {% set modelname = varargs[1] %}
{% endif %}

-- call builtins.ref based on provided positional arguments
{% set rel = None %}
{% if packagename is not none %}
    {% set rel = builtins.ref(packagename, modelname, version=version) %}
{% else %}
    {% set rel = builtins.ref(modelname, version=version) %}
{% endif %}

-- verify that the version portion for the `my_base_model` model is working correctly
{% if modelname == "my_base_model" %}
    {% set expected_version = 2 %}
    {% if modelname == "my_base_model" and version != expected_version %}
        {{ exceptions.raise_compiler_error("Expected version=" ~ expected_version ~ ". Got: version=" ~ version) }}
    {% endif %}
{% endif %}

{% do return(rel) %}

{% endmacro %}

Then run this command, and it will go 💥:

dbt compile

But if the override of the ref macro is changed to this, it should work:

-- extract user-provided positional and keyword arguments
{% set version = kwargs.get('version') or kwargs.get('v') %}

[ArtrCtrs]

Hi @dbeatty10 !
Indeed we are already overriding the ref macro, which explains the issue.

Thanks for a lot for your help !

Our macro has added explicit arguments similar to this :

{% macro ref(
    parent_model,
    param_1=var("param_1", False),
    param2=env_var("param_2", False)
) %}

I'm not too sure if it's possible to handle kwargs in a macro that already has named parameter, but it does seem to work when I add an explicit v argument

{% macro ref(
    parent_model,
    param_1=var("param_1", False),
    param2=env_var("param_2", False),
    v=None
) %}

Thanks again !

[dbeatty10]

You're welcome @ArtrCtrs -- glad it is working for you now 🎉

We've also updated the code example, so I'm going to close this as resolved via dbt-labs/docs.getdbt.com#5051.

[gaoshihang]

@dbeatty10 Hi, is this bug will be repair in the future version?

[dbeatty10]

@gaoshihang we believe we addressed this by updating the documentation here.

If you are having problems with versioned python models and you have an override of the ref macro, could you try the instructions above and let us know if you still think it is a bug?

[gaoshihang]

Thanks @dbeatty10 for your reply! I'll try this method. so what I need to do is create an Macro to overwrite def() with the code in https://docs.getdbt.com/reference/dbt-jinja-functions/builtins?

[dbeatty10]

@gaoshihang we were expecting that this would only affect people that already were overwriting the ref macro and then using dbt model versions along with a dbt python model.

Are you saying you got this error without already having an custom ref macro in your project?

[gaoshihang]

@dbeatty10 I just check the code, we already have a ref macro in project, and I remove the macro, the error is gone. I think now it's fine. thanks.

So if later we need overwriting the ref macos, we need to add this code, is that right?

[dbeatty10]

So if later we need overwriting the ref macos, we need to add this code, is that right?

Yep, that's right 👍

[gaoshihang]

many thanks to you @dbeatty10