Parallelize pbsv call #75

williamrowell · 2023-10-03T05:13:24Z

Split pbsv call task into 14 sets of ~150-300Mbp, never splitting chromosomes.

depends on changes to pbsv_call and the addition of concat_vcf in Use grep to filter svsigs to filter svsigs within region. wdl-common#29
depends on a new input, "pbsv_splits": "dataset/GRCh38/human_GRCh38_no_alt_analysis_set.pbsv_splits.json", which is a json file with an Array[Array[String]]. Each Array[String] is a set of chromosomes to be processed by the same task.

…hromosomes.

williamrowell · 2023-10-04T00:58:18Z

gconcepcion

Looks good to me, basically just the changes we've been talking about the past few days

williamrowell added 2 commits October 2, 2023 21:46

Split pbsv call task into 14 sets of ~150-300Mbp, never splitting c…

08b9a85

…hromosomes.

Update wdl-common

cb85551

williamrowell self-assigned this Oct 3, 2023

github-actions bot and others added 3 commits October 3, 2023 05:23

update wdl-ci config file after successful tests

18f908b

pbsv_call produces a single output per call.

e998322

update wdl-ci config file after successful tests

502b353

williamrowell requested a review from gconcepcion October 3, 2023 05:46

williamrowell and others added 2 commits October 3, 2023 15:54

Combine indexing with concatenation. Add vcf concat test.

a57ec1a

update wdl-ci config file after successful tests

730bbcb

williamrowell and others added 4 commits October 4, 2023 09:35

Update submodule to use pbsv call with _optional_ regions.

c372f38

Use write_lines() to write non-region svisg.fofn.

b81f466

update wdl-ci config file after successful tests

6b73fd3

Merge branch 'develop' into feature/split-pbsv-call

7156b6e

gconcepcion approved these changes Oct 5, 2023

View reviewed changes

williamrowell merged commit d8d539e into develop Oct 5, 2023
1 check passed

williamrowell deleted the feature/split-pbsv-call branch October 5, 2023 22:11

Provide feedback