Allow asset_checks
to be executed before handle_output
#20901
ion-elgreco
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Our pattern is generally to run data validation checks before writing to storage, this prevents incorrect data to ever end up in your target table/directory. Dagster how ever executes the asset_checks after handling the output, which means incorrect data now already get's persisted.
Asset checks concept is great but where it gets executed in the timeline is crucial.
Beta Was this translation helpful? Give feedback.
All reactions