-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DM-41650 Add step definitions to pipelines #392
Conversation
bfee149
to
31dc729
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think StepIR.__hash__
and StepIR.__eq__
as defined so far are going to be problematic, and I think it'll all go away if you hold them via dict
instead of set
(at least in Python, maybe in YAML).
31dc729
to
5ff4f4f
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #392 +/- ##
==========================================
+ Coverage 82.43% 82.49% +0.05%
==========================================
Files 91 91
Lines 10414 10470 +56
Branches 1964 1981 +17
==========================================
+ Hits 8585 8637 +52
- Misses 1486 1488 +2
- Partials 343 345 +2 ☔ View full report in Codecov by Sentry. |
5ff4f4f
to
86ac280
Compare
Added a directive to Pipelines that allows specifying which subsets correspond to an end to end processing step and what dimensions that those steps are expected to operate on.
86ac280
to
ba8243d
Compare
This key is a list of the labels for all subsets that are considered steps in end to end processing. | ||
Alongside each label, a step must be declared with the set of dimensions the step is expected to run over. | ||
An example of what the step syntax looks like can be seen in the example below. | ||
These bits of information allow campaign management / batch production software better reason about how to handle the processing workflow found withing a pipeline. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These bits of information allow campaign management / batch production software better reason about how to handle the processing workflow found withing a pipeline. | |
These bits of information allow campaign management / batch production software to better reason about how to handle the processing workflow found within a pipeline. |
Checklist
doc/changes