Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-41650 Add step definitions to pipelines #392

Merged
merged 1 commit into from
Jan 23, 2024
Merged

Conversation

natelust
Copy link
Contributor

@natelust natelust commented Dec 5, 2023

Checklist

  • ran Jenkins
  • added a release note for user-visible changes to doc/changes

Copy link
Member

@TallJimbo TallJimbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think StepIR.__hash__ and StepIR.__eq__ as defined so far are going to be problematic, and I think it'll all go away if you hold them via dict instead of set (at least in Python, maybe in YAML).

doc/changes/DM-41650.feature.md Outdated Show resolved Hide resolved
doc/lsst.pipe.base/creating-a-pipeline.rst Outdated Show resolved Hide resolved
doc/lsst.pipe.base/creating-a-pipeline.rst Outdated Show resolved Hide resolved
doc/lsst.pipe.base/creating-a-pipeline.rst Outdated Show resolved Hide resolved
python/lsst/pipe/base/pipelineIR.py Outdated Show resolved Hide resolved
python/lsst/pipe/base/pipelineIR.py Outdated Show resolved Hide resolved
python/lsst/pipe/base/pipelineIR.py Outdated Show resolved Hide resolved
python/lsst/pipe/base/pipelineIR.py Outdated Show resolved Hide resolved
python/lsst/pipe/base/pipelineIR.py Outdated Show resolved Hide resolved
Copy link

codecov bot commented Jan 18, 2024

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (4e68ae3) 82.43% compared to head (ba8243d) 82.49%.

Files Patch % Lines
python/lsst/pipe/base/pipelineIR.py 85.36% 3 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #392      +/-   ##
==========================================
+ Coverage   82.43%   82.49%   +0.05%     
==========================================
  Files          91       91              
  Lines       10414    10470      +56     
  Branches     1964     1981      +17     
==========================================
+ Hits         8585     8637      +52     
- Misses       1486     1488       +2     
- Partials      343      345       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Added a directive to Pipelines that allows specifying which
subsets correspond to an end to end processing step and what
dimensions that those steps are expected to operate on.
This key is a list of the labels for all subsets that are considered steps in end to end processing.
Alongside each label, a step must be declared with the set of dimensions the step is expected to run over.
An example of what the step syntax looks like can be seen in the example below.
These bits of information allow campaign management / batch production software better reason about how to handle the processing workflow found withing a pipeline.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
These bits of information allow campaign management / batch production software better reason about how to handle the processing workflow found withing a pipeline.
These bits of information allow campaign management / batch production software to better reason about how to handle the processing workflow found within a pipeline.

@natelust natelust merged commit 5ae894a into main Jan 23, 2024
15 checks passed
@natelust natelust deleted the tickets/DM-41650 branch January 23, 2024 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants