Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Improve error messages when calling a non-existent macro in the config block of a model #10503

Open
3 tasks done
haje01 opened this issue Jul 30, 2024 · 0 comments
Open
3 tasks done
Labels
enhancement New feature or request triage

Comments

@haje01
Copy link

haje01 commented Jul 30, 2024

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Hi!

I'm using dbt 1.8.3 with Python 3.11.8

Calling a macro that doesn't exist in your project code will normally raise a RuntimeError, and will make the user aware that the macro is undefined.

select {{ calling_undefined_macro() }}

Error message:

 'calling_undefined_macro' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

However, calling undefined macro from the config block of the model, as shown in the following example, results in a strange error that is almost impossible for the user to notice.

{{
    config(foo = calling_undefined_macro())
}}

select 1 as id

Error message:

can not serialize 'Undefined' object                                                                                      01:40:35  Traceback (most recent call last):                                                                                File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/cli/requires.py", line 138, in wrapper                                                                                                                            result, success = func(*args, **kwargs)                                                                                                     ^^^^^^^^^^^^^^^^^^^^^                                                                                 File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/cli/requires.py", line 101, in wrapper                                                                                                                            return func(*args, **kwargs)                                                                                                     ^^^^^^^^^^^^^^^^^^^^^                                                                                            File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/cli/requires.py", line 218, in wrapper                                                                                                                            return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/cli/requires.py", line 247, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/cli/requires.py", line 294, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/cli/requires.py", line 320, in wrapper
    ctx.obj["manifest"] = parse_manifest(
                          ^^^^^^^^^^^^^^^
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/parser/manifest.py", line 1898, in parse_manifest
    manifest = ManifestLoader.get_full_manifest(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/parser/manifest.py", line 330, in get_full_manifest
    manifest = loader.load()
               ^^^^^^^^^^^^^
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/parser/manifest.py", line 525, in load
    self.write_manifest_for_partial_parse()
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/parser/manifest.py", line 811, in write_manifest_for_partial_parse
    manifest_msgpack = self.manifest.to_msgpack(extended_mashumaro_encoder)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 3, in __mashumaro_to_msgpack__
  File "<string>", line 85, in __mashumaro_to_msgpack__
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/dbt/parser/manifest.py", line 150, in extended_mashumaro_encoder
    return msgpack.packb(data, default=extended_msgpack_encoder, use_bin_type=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/haje01/.pyenv/versions/3.11.8/envs/dbt/lib/python3.11/site-packages/msgpack/__init__.py", line 36, in packb
    return Packer(**kwargs).pack(o)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "msgpack/_packer.pyx", line 294, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 300, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 297, in msgpack._cmsgpack.Packer.pack
  File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  File "msgpack/_packer.pyx", line 231, in msgpack._cmsgpack.Packer._pack
  [Previous line repeated 1 more time]
  File "msgpack/_packer.pyx", line 291, in msgpack._cmsgpack.Packer._pack
TypeError: can not serialize 'Undefined' object

The worst part is that these config issues happen even if the model is not the target of the direct run.

This issue is probably also related to the reporting in the link below.

#10270

Describe alternatives you've considered

No response

Who will this benefit?

No response

Are you interested in contributing this feature?

No response

Anything else?

No response

@haje01 haje01 added enhancement New feature or request triage labels Jul 30, 2024
@haje01 haje01 changed the title [Feature] Improve error messages when calling a non-existent macro in the config section of a model [Feature] Improve error messages when calling a non-existent macro in the config block of a model Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request triage
Projects
None yet
Development

No branches or pull requests

1 participant