Skip to content

Commit

Permalink
Fix nullable constraint on array fields
Browse files Browse the repository at this point in the history
Fixes #4
  • Loading branch information
H-Max committed Jan 10, 2024
1 parent e2528a5 commit afe5dbf
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 9 deletions.
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ __pycache__/
_local

# Output directory of the package itself
**/output
**/target/bq2dbt
/target

# Distribution / packaging
build/
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,14 @@ I know this looks like something the codegen dbt package could do, but it was fa

We used this script to convert an existing project with missing contracts and with `SELECT *` statements to something for robust and implicit, and with existing tables in BigQuery. This is the main purpose of this.

# Install the script localy
# Instal from source

Just use pip:

```
pip install .
```

# Install from Github

```
# Latest version
Expand Down
12 changes: 7 additions & 5 deletions bq2dbt/bq2dbt.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,11 +128,13 @@ def bq2dbt():
"description": field.description or ""
}
if field.is_nullable == 'NO':
field_info = {**field_info, **{
"constraints": [
{"type": "not_null"}
]
}}
# BigQuery says array cannot be null, but they can and they don't support not_null constraint
if data_type not in ['ARRAY']:
field_info = {**field_info, **{
"constraints": [
{"type": "not_null"}
]
}}

yaml_data["models"][0]["columns"].append(field_info)
if '.' not in field.field_path:
Expand Down

0 comments on commit afe5dbf

Please sign in to comment.