Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Combine some short tasks. #76

Merged
merged 11 commits into from
Oct 5, 2023
Merged

Combine some short tasks. #76

merged 11 commits into from
Oct 5, 2023

Conversation

williamrowell
Copy link
Collaborator

  • bcftools roh and bcftools stats for DeepVariant vcf
  • writing the cohort yaml and pedigree, as well as the phrank lookup
  • small variant normalization, slivar filter/annotate, slivar compound hets, and slivar tsv
  • svpack task compresses and indexes output

@williamrowell
Combine some short tasks.
0470637 
- bcftools roh and bcftools stats for DeepVariant vcf
- writing the cohort yaml and pedigree, as well as the phrank lookup
- small variant normalization, slivar filter/annotate, slivar compound hets, and slivar tsv
- svpack task compresses and indexes output
@williamrowell
Copy link
Collaborator Author

Depends on PacificBiosciences/wdl-dockerfiles#31
Depends on PacificBiosciences/wdl-dockerfiles#29

@williamrowell williamrowell self-assigned this Oct 4, 2023
@williamrowell williamrowell force-pushed the feature/combine-short-tasks branch from 9327a59 to 1e96f38 Compare October 4, 2023 04:53
gconcepcion
gconcepcion approved these changes Oct 5, 2023
View reviewed changes
Copy link
Contributor

@gconcepcion gconcepcion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There sure seems to be alot of optimization from combining tasks and removing unnecessary loading of images. also nice to see need for an image just for parse-cohort eliminated. The tests pass, looks good to me. what's the approx runtime delta?

@williamrowell
Copy link
Collaborator Author

These don't really save a ton of time, but they prevent spinning up extra nodes for 1- to 5-minute jobs. Trading off added complexity within tasks for reduced overhead of copying inputs and outputs around.

@williamrowell williamrowell merged commit 86d268c into develop Oct 5, 2023
1 check failed
@williamrowell williamrowell deleted the feature/combine-short-tasks branch October 5, 2023 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gconcepcion gconcepcion gconcepcion approved these changes

Assignees

@williamrowell williamrowell

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@williamrowell @gconcepcion