Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-3402] python func returning None where using in JINGA created corrupted sql #9113

Closed
2 tasks done
obar1 opened this issue Nov 17, 2023 · 7 comments
Closed
2 tasks done
Labels
bug Something isn't working wontfix Not a bug or out of scope for dbt-core

Comments

@obar1
Copy link

obar1 commented Nov 17, 2023

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

pyhton() func returning None create issue in compiled see below

Expected Behavior

no None

Steps To Reproduce

20231117_160208

I have to write that hack to avoid to have corrupted sql

Relevant log output

see img

Environment

- OS: win
- Python: 3.9.0
- dbt: 1.6

Which database adapter are you using with dbt?

bigquery

Additional Context

done above - sorry I can put the code here -but it's easy to reporduce

@obar1 obar1 added bug Something isn't working triage labels Nov 17, 2023
@github-actions github-actions bot changed the title python func returning None where using in JINGA created corrupted sql [CT-3402] python func returning None where using in JINGA created corrupted sql Nov 17, 2023
@dbeatty10
Copy link
Contributor

Thanks for reaching out @obar1 !

Can you share more details of the step-by-step code that gave you a problem?

For example, please share the following so we can try to reproduce the bug that you are reporting:

  1. The code in text format
  2. dbt commands that you ran
  3. Log output showing the error message
  4. Any other relevant content to help us reproduce the bug you are reporting

@obar1
Copy link
Author

obar1 commented Nov 17, 2023

simple jinga code as in the image - then dbt compile - easy - I did at work and I cannot send snippet oout so I did a super simple version you guys can type and run

@dbeatty10
Copy link
Contributor

From the screenshot you shared, I'm not able to see anything amiss.

Are you able and willing to create a small reproducible example as described here?

If not, we'll close this as "can't repro" (short for "can't reproduce").

@obar1
Copy link
Author

obar1 commented Nov 22, 2023

hi @dbeatty10

try this in a_model.sql

{% set a_set = [ 'one', '2' , '333'] %}
  /* HI */

{% set appends = [] %}
{% for s in a_set  %}
  /* you should see this only */
  {{ appends.append(s)}}
  {{ appends.append(s) | replace('None','')}}
{% endfor %}

hit dbt compile and see target/compiled

it should be only ...

  /* HI */

  /* you should see this only */
  /* you should see this only */
  /* you should see this only */

the bounch of None you see are the problem :)

@dbeatty10
Copy link
Contributor

Thanks for sharing that example @obar1 👍

I saw the None strings that you described when following your example.

This output is just a result of how Jinja works as a string templater rather than anything that is specific to dbt. The key insights are:

  • {{ ... }} is used to output a string into a template.
  • {% ... %} is used for controlling the flow and logic of the template and doesn't output strings.

There's multiple ways that you can avoid a None string, and you already discovered one of them.

Here's another that uses {% ... %} instead of {{ ... }}:

{% set a_set = [ 'one', '2' , '333'] %}
  /* HI */

{% set appends = [] %}
{% for s in a_set %}
  /* you should see this only */
  {%- do appends.append(s) -%}
{% endfor %}

Since this is expected behavior, I'm going to close this a not a bug.

@dbeatty10 dbeatty10 closed this as not planned Won't fix, can't repro, duplicate, stale Nov 27, 2023
@dbeatty10 dbeatty10 added wontfix Not a bug or out of scope for dbt-core and removed triage labels Nov 27, 2023
@obar1
Copy link
Author

obar1 commented Nov 28, 2023

cool stuff it makes sense and I will use it for sure, maybe to be added in some docs? the side effects was to break the compiled silently ... not sure this is really what we want to let the user experience 😀 @dbeatty10 even if it s a jinja behaviour

@dbeatty10
Copy link
Contributor

@obar1 we don't want to fully replicate the Jinja docs, but I tried to propose some light modifications to the dbt docs in dbt-labs/docs.getdbt.com#4546.

Feel free to comment on that PR or open your own issue and/or PR to update the dbt docs here. At the bottom of each page in dbt docs, there is a link that you can click to submit a proposed change:

image

dbeatty10 added a commit to dbt-labs/docs.getdbt.com that referenced this issue Dec 18, 2023
#4546)

[Preview](https://docs-getdbt-com-git-dbeatty-jinja-statements-dbt-labs.vercel.app/docs/build/jinja-macros#jinja)

## What are you changing in this pull request and why?

Instigated by
dbt-labs/dbt-core#9113 (comment)

We want to briefly explain the differences between expressions that
print out and statements that don't.

At the same time, we don't want to fully replicate all the [Jinja
documentation](https://jinja.palletsprojects.com/en/3.0.x/templates/).

Here's a quick table to explain the behavioral differences between the
types of delimiters ("curlies"):

| Name       | Delimiter   | Executes | Prints |
|------------|-------------|:--------:|:------:|
| Expression | `{{ ... }}` |     ✅    |    ✅   |
| Statement  | `{% ... %}` |     ✅    |    ❌   |
| Comment    | `{# ... #}` |     ❌    |    ❌   |

The Jinja docs for the different [types of
delimiters](https://jinja.palletsprojects.com/en/3.0.x/templates/#synopsis:~:text=The%20default%20Jinja%20delimiters%20are%20configured%20as%20follows)
has the advantage of being brief, but lacks clarity especially as it
relates to whether statements print out or not (they don't). Since the
[“do” aka
expression-statement](https://jinja.palletsprojects.com/en/3.1.x/templates/#expression-statement)
is an [optional
extension](https://jinja.palletsprojects.com/en/3.0.x/extensions/#expression-statement)
in Jinja, it isn't documented in list of [control
structures](https://jinja.palletsprojects.com/en/3.0.x/templates/#list-of-control-structures).

## 🎩 

<img width="600" alt="image"
src="https://github.com/dbt-labs/docs.getdbt.com/assets/44704949/23a302e0-d68e-42b4-baca-ff1fb36c4fb0">


## Checklist
- [x] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [x] I have checked that the preview renders correctly
- [x] I have checked that the new links work correctly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working wontfix Not a bug or out of scope for dbt-core
Projects
None yet
Development

No branches or pull requests

2 participants