-
Notifications
You must be signed in to change notification settings - Fork 177
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add tests to sources, snapshots and seeds when using TestBehavior.AFT…
…ER_EACH (#599) Previously Cosmos would only create a task group when using `TestBehavior.AFTER_EACH` for nodes of the type `DbtResourceType.MODEL`. This change adds the same behavior to snapshots and seeds. For this to work as expected with sources, we would need to create a default operator to handle `DbtResourceType.SOURCE`, which is outside the scope of the current ticket. Once this operator exists, sources will also lead to creating a task group. All the test selectors were tested successfully with dbt 1.6. This screenshot illustrates the validation of this feature, with an adapted version of Jaffle Shop: <img width="1502" alt="Screenshot 2023-10-13 at 19 43 52" src="https://github.com/astronomer/astronomer-cosmos/assets/272048/893ac557-1846-4ed1-b5e2-913a7bd38485"> The modifications that were done to jaffle_shop were: Appended the following lines to `dev/dags/dbt/jaffle_shop/models/schema.yml`: ``` seeds: - name: raw_customers description: Raw data from customers columns: - name: id tests: - unique - not_null snapshots: - name: orders_snapshot description: Snapshot of orders columns: - name: orders_snapshot.order_id tests: - unique - not_null ``` And created the file `dev/dags/dbt/jaffle_shop/snapshots/orders_snapshot.sql` with: ``` {% snapshot orders_snapshot %} {{ config( target_database='postgres', target_schema='public', unique_key='order_id', strategy='timestamp', updated_at='order_date', ) }} select * from {{ ref('jaffle_shop', 'orders') }} {% endsnapshot %} ``` Closes: #474
- Loading branch information
Showing
3 changed files
with
89 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters