From 9e63d4d669dec32f05aa4e6473bebc7dd4e22d8e Mon Sep 17 00:00:00 2001 From: William Rowell Date: Thu, 14 Dec 2023 15:22:50 -0800 Subject: [PATCH 01/25] switching to branch with altered history --- workflows/wdl-common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/workflows/wdl-common b/workflows/wdl-common index fef058b8..eae8a3f4 160000 --- a/workflows/wdl-common +++ b/workflows/wdl-common @@ -1 +1 @@ -Subproject commit fef058b879d04c15c3da2626b320afdd8ace6c2e +Subproject commit eae8a3f4ab51c86bb67b5d2eb8941b6678784b22 From f2181196bf373dd1ccc757870b01163c934ca3bc Mon Sep 17 00:00:00 2001 From: William Rowell Date: Thu, 14 Dec 2023 15:24:06 -0800 Subject: [PATCH 02/25] feat: Update pbmm2 to 1.13.1 New container also has updated pysam script with silenced errors. --- README.md | 2 +- workflows/sample_analysis/sample_analysis.wdl | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 457449bb..4e5b694f 100644 --- a/README.md +++ b/README.md @@ -264,7 +264,7 @@ The Docker image used by a particular step of the workflow can be identified by | mosdepth | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/mosdepth) | | paraphase | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/paraphase) | | pb-cpg-tools | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/7481837d3b0f539adf4f64209a65cf28eebf3dba/docker/pb-cpg-tools) | -| pbmm2 | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/pbmm2) | +| pbmm2 | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/6d3d818dce6f97454ac29f4ed96637e450a19f63/docker/pbmm2) | | pbsv | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/f9e33a757e6d8cb15696ac930a2efd0fd7a885d8/docker/pbsv) | | pyyaml | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/f72e862bca2f209b9909e6043ef0197975762f27/docker/pyyaml) | | samtools | | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/samtools) | diff --git a/workflows/sample_analysis/sample_analysis.wdl b/workflows/sample_analysis/sample_analysis.wdl index 1f4f1a8d..77e6196c 100644 --- a/workflows/sample_analysis/sample_analysis.wdl +++ b/workflows/sample_analysis/sample_analysis.wdl @@ -317,7 +317,7 @@ task pbmm2_align { } runtime { - docker: "~{runtime_attributes.container_registry}/pbmm2@sha256:1013aa0fd5fb42c607d78bfe3ec3d19e7781ad3aa337bf84d144c61ed7d51fa1" + docker: "~{runtime_attributes.container_registry}/pbmm2@sha256:ed9dcb4db98c81967fff15f50fca89c8495b1f270eee00e9bec92f46d14d7e2f" cpu: threads memory: mem_gb + " GB" disk: disk_size + " GB" From 200efc34b5991bad3df49bf5b5b29bb17f2bd3eb Mon Sep 17 00:00:00 2001 From: William Rowell Date: Thu, 14 Dec 2023 15:27:55 -0800 Subject: [PATCH 03/25] feat: Update TRGT to 0.7.0 --- README.md | 2 +- workflows/sample_analysis/sample_analysis.wdl | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 457449bb..04f73119 100644 --- a/README.md +++ b/README.md @@ -270,7 +270,7 @@ The Docker image used by a particular step of the workflow can be identified by | samtools |
  • [samtools 1.14](https://github.com/samtools/samtools/releases/tag/1.14)
| [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/samtools) | | slivar |
  • [slivar 0.2.2](https://github.com/brentp/slivar/releases/tag/v0.2.2)
  • [bcftools 1.14](https://github.com/samtools/bcftools/releases/tag/1.14)
  • [vcfpy 0.13.3](https://github.com/bihealth/vcfpy/releases/tag/v0.13.3)
  • [pysam 0.19.1](https://github.com/pysam-developers/pysam/releases/tag/v0.19.1)
| [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/slivar) | | svpack |
  • [svpack 36180ae6](https://github.com/PacificBiosciences/svpack/tree/a82598ebc4013bf32e70295b83b380ada6302c4a)
  • [htslib 1.18](https://github.com/samtools/htslib/releases/tag/1.18)
  • [pysam 0.21.0](https://github.com/pysam-developers/pysam/releases/tag/v0.21.0)
    • | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/8edbc516abc0ff43ac279b48018003923721b054/docker/svpack) | -| trgt |
    • [trgt 0.5.0](https://github.com/PacificBiosciences/trgt/releases/tag/v0.5.0)
    • [samtools 1.18](https://github.com/samtools/samtools/releases/tag/1.18)
    • [bcftools 1.18](https://github.com/samtools/bcftools/releases/tag/1.18)
    • [pysam 0.21.0](https://github.com/pysam-developers/pysam/releases/tag/v0.21.0)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/d2a45e0213ac3fa631a51a48757c442d3ed550b6/docker/trgt) | +| trgt |
    • [trgt 0.7.0](https://github.com/PacificBiosciences/trgt/releases/tag/v0.7.0)
    • [samtools 1.18](https://github.com/samtools/samtools/releases/tag/1.18)
    • [bcftools 1.18](https://github.com/samtools/bcftools/releases/tag/1.18)
    • [pysam 0.21.0](https://github.com/pysam-developers/pysam/releases/tag/v0.21.0)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/9aae7d50a4beb08a1869dd3ce1a0567ee4e5cbc2/docker/trgt) | --- diff --git a/workflows/sample_analysis/sample_analysis.wdl b/workflows/sample_analysis/sample_analysis.wdl index 1f4f1a8d..cea9b8a8 100644 --- a/workflows/sample_analysis/sample_analysis.wdl +++ b/workflows/sample_analysis/sample_analysis.wdl @@ -502,7 +502,7 @@ task trgt { } runtime { - docker: "~{runtime_attributes.container_registry}/trgt@sha256:8c9f236eb3422e79d7843ffd59e1cbd9b76774525f20d88cd68ca64eb63054eb" + docker: "~{runtime_attributes.container_registry}/trgt@sha256:07e9d2be1590fc1dd45e6e2a6b9f5869ee6d83f5a1ca47c8ea84e114b70573b6" cpu: threads memory: "4 GB" disk: disk_size + " GB" @@ -545,7 +545,7 @@ task coverage_dropouts { } runtime { - docker: "~{runtime_attributes.container_registry}/trgt@sha256:8c9f236eb3422e79d7843ffd59e1cbd9b76774525f20d88cd68ca64eb63054eb" + docker: "~{runtime_attributes.container_registry}/trgt@sha256:07e9d2be1590fc1dd45e6e2a6b9f5869ee6d83f5a1ca47c8ea84e114b70573b6" cpu: threads memory: "4 GB" disk: disk_size + " GB" From 45cec15dd265671ce382895476f531543339e49b Mon Sep 17 00:00:00 2001 From: William Rowell Date: Thu, 14 Dec 2023 15:30:23 -0800 Subject: [PATCH 04/25] feat: Updated Paraphase to 3.0.0 - paraphase_vcfs now points to a tarball of VCFs; we implemented this change because there is a large, variable number of output VCFs, and we need a flatter output structure --- README.md | 4 ++-- workflows/main.wdl | 2 +- workflows/sample_analysis/sample_analysis.wdl | 14 +++++++++----- 3 files changed, 12 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 457449bb..3e267681 100644 --- a/README.md +++ b/README.md @@ -214,7 +214,7 @@ These files will be output for each sample defined in the cohort. | Array[Array[File]] | cpg_pileup_bigwigs | 5mCpG site methylation probability pileups generated by pb-CpG-tools | | | Array[File] | paraphase_output | Output generated by [Paraphase](https://github.com/PacificBiosciences/paraphase) | | | Array[[IndexData](https://github.com/PacificBiosciences/wdl-common/blob/main/wdl/structs.wdl)] | paraphase_realigned_bam | Realigned BAM for selected medically relevant genes in segmental duplications (with index), generated by Paraphase | | -| Array[Array[File]] | paraphase_vcfs | Phased Variant calls for selected medically relevant genes in segmental duplications, generated by Paraphase | | +| Array[File] | paraphase_vcfs | Tarball of phased variant calls for selected medically relevant genes in segmental duplications, generated by Paraphase | | | Array[[IndexData](https://github.com/PacificBiosciences/wdl-common/blob/main/wdl/structs.wdl)] | hificnv_vcfs | VCF output containing copy number variant calls for the sample from [HiFiCNV](https://github.com/PacificBiosciences/HiFiCNV) | | | Array[File] | hificnv_copynum_bedgraphs | Copy number values calculated for each region | | | Array[File] | hificnv_depth_bws | Bigwig file containing the depth measurements from HiFiCNV | | @@ -262,7 +262,7 @@ The Docker image used by a particular step of the workflow can be identified by | hiphase |
    • [HiPhase 1.0.0](https://github.com/PacificBiosciences/HiPhase/releases/tag/v1.0.0)
    • [samtools 1.18](https://github.com/samtools/samtools/releases/tag/1.18)
    • [bcftools 1.18](https://github.com/samtools/bcftools/releases/tag/1.18)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/d26db6204409dfeff56e169cdba0cc14bc272f15/docker/hiphase) | | htslib |
    • [htslib 1.14](https://github.com/samtools/htslib/releases/tag/1.14)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/htslib) | | mosdepth |
    • [mosdepth 0.2.9](https://github.com/brentp/mosdepth/releases/tag/v0.2.9)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/mosdepth) | -| paraphase |
    • [minimap2 2.17](https://github.com/lh3/minimap2/releases/tag/v2.17)
    • [samtools 1.14](https://github.com/samtools/samtools/releases/tag/1.14)
    • [paraphase 2.2.3](https://github.com/PacificBiosciences/paraphase/releases/tag/v2.2.3)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/paraphase) | +| paraphase |
    • [minimap2 2.26](https://github.com/lh3/minimap2/releases/tag/v2.26)
    • [samtools 1.18](https://github.com/samtools/samtools/releases/tag/1.18)
    • [paraphase 3.0.0](https://github.com/PacificBiosciences/paraphase)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/4f510e5f434cc138577853f56558b90e059fd770/docker/paraphase) | | pb-cpg-tools |
    • [pb-CpG-tools v2.3.2](https://github.com/PacificBiosciences/pb-CpG-tools/releases/tag/v2.3.2)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/7481837d3b0f539adf4f64209a65cf28eebf3dba/docker/pb-cpg-tools) | | pbmm2 |
    • [pbmm2 1.10.0](https://github.com/PacificBiosciences/pbmm2/releases/tag/v1.10.0)
    • [datamash 1.1.0](https://ftp.gnu.org/gnu/datamash/)
    • [pysam 0.16.0.1](https://github.com/pysam-developers/pysam/releases/tag/v0.16.0.1)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/pbmm2) | | pbsv |
    • [pbsv 2.9.0](https://github.com/PacificBiosciences/pbsv/releases/tag/v2.9.0)
    • [htslib 1.14](https://github.com/samtools/htslib/releases/tag/1.14)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/f9e33a757e6d8cb15696ac930a2efd0fd7a885d8/docker/pbsv) | diff --git a/workflows/main.wdl b/workflows/main.wdl index 0ee719aa..d334f526 100644 --- a/workflows/main.wdl +++ b/workflows/main.wdl @@ -130,7 +130,7 @@ workflow humanwgs { # per sample paraphase outputs Array[File] paraphase_output_jsons = sample_analysis.paraphase_output_json Array[IndexData] paraphase_realigned_bams = sample_analysis.paraphase_realigned_bam - Array[Array[File]] paraphase_vcfs = sample_analysis.paraphase_vcfs + Array[File] paraphase_vcfs = sample_analysis.paraphase_vcfs # per sample hificnv outputs Array[IndexData] hificnv_vcfs = sample_analysis.hificnv_vcf diff --git a/workflows/sample_analysis/sample_analysis.wdl b/workflows/sample_analysis/sample_analysis.wdl index 1f4f1a8d..73ed5983 100644 --- a/workflows/sample_analysis/sample_analysis.wdl +++ b/workflows/sample_analysis/sample_analysis.wdl @@ -231,7 +231,7 @@ workflow sample_analysis { # per sample paraphase outputs File paraphase_output_json = paraphase.output_json IndexData paraphase_realigned_bam = {"data": paraphase.realigned_bam, "data_index": paraphase.realigned_bam_index} - Array[File] paraphase_vcfs = paraphase.paraphase_vcfs + File paraphase_vcfs = paraphase.paraphase_vcfs # per sample hificnv outputs IndexData hificnv_vcf = {"data": hificnv.cnv_vcf, "data_index": hificnv.cnv_vcf_index} @@ -624,8 +624,8 @@ task paraphase { RuntimeAttributes runtime_attributes } - Int threads = 4 - Int mem_gb = 4 + Int threads = 8 + Int mem_gb = 16 Int disk_size = ceil(size(bam, "GB") + 20) command <<< @@ -638,17 +638,21 @@ task paraphase { --bam ~{bam} \ --reference ~{reference} \ --out ~{out_directory} + + cd ~{out_directory} \ + && tar zcvf ~{out_directory}.tar.gz ~{sample_id}_vcfs/*.vcf \ + && mv ~{out_directory}.tar.gz ../ >>> output { File output_json = "~{out_directory}/~{sample_id}.json" File realigned_bam = "~{out_directory}/~{sample_id}_realigned_tagged.bam" File realigned_bam_index = "~{out_directory}/~{sample_id}_realigned_tagged.bam.bai" - Array[File] paraphase_vcfs = glob("~{out_directory}/~{sample_id}_vcfs/*.vcf") + File paraphase_vcfs = "~{out_directory}.tar.gz" } runtime { - docker: "~{runtime_attributes.container_registry}/paraphase@sha256:186dec5f6dabedf8c90fe381cd8f934d31fe74310175efee9ca4f603deac954d" + docker: "~{runtime_attributes.container_registry}/paraphase@sha256:b9852d1a43485b13c563aaddcb32bacc7f0c9088c2ca007051b9888e9fe5617d" cpu: threads memory: mem_gb + " GB" disk: disk_size + " GB" From 546990c11874312a0075c79bfabd063a12099a1b Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 14 Dec 2023 23:43:55 +0000 Subject: [PATCH 05/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 414f0248..454f7884 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -142,7 +142,7 @@ }, "trgt": { "key": "trgt", - "digest": "ylzep5nroxhzjff43gkc6fs25ydor7dd", + "digest": "uaowbumdens55465girwarokc46hwgco", "tests": [ { "inputs": { @@ -178,7 +178,7 @@ }, "coverage_dropouts": { "key": "coverage_dropouts", - "digest": "3el45hg36hlyx5cswr3dkvqfg644cvbn", + "digest": "3wfgwjflppb5tifrwycmjzy727p7wzyi", "tests": [ { "inputs": { From 0ce53bea5e127da3b0ed302d9b9399c3956af740 Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 14 Dec 2023 23:49:22 +0000 Subject: [PATCH 06/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 414f0248..8a8100aa 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -245,7 +245,7 @@ }, "paraphase": { "key": "paraphase", - "digest": "gzktyxvdrw73el5khnudlpu23x34lbxv", + "digest": "p36f7pt2tghztqhtdgreyxohbjjndg3i", "tests": [ { "inputs": { From 511021e5f1cc8938646e52dd17a3b8af11464fe9 Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 15 Dec 2023 00:17:48 +0000 Subject: [PATCH 07/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 744d869f..a673b508 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -25,7 +25,7 @@ "tasks": { "pbmm2_align": { "key": "pbmm2_align", - "digest": "3r4icze5zkps7m6xoruzvnfzk2fp4gqd", + "digest": "ntqgonovwrgxezaewjfw7oec4t34yvhp", "tests": [ { "inputs": { From 4ad7323161d7178c6453f4b53042628f538f0721 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Thu, 14 Dec 2023 16:56:43 -0800 Subject: [PATCH 08/25] feat: Update to DeepVariant 1.6.0 Most changes are internal to the DeepVariant subworkflow, except that custom models should now be passed as a tarball. Instructions are found in the README in `wdl-common/wdl/workflows/deepvariant/README.md` --- README.md | 6 +++--- backends/aws/inputs.aws.json | 2 +- backends/azure/inputs.azure.json | 2 +- backends/gcp/inputs.gcp.json | 2 +- backends/hpc/inputs.hpc.json | 2 +- wdl-ci.config.json | 12 ++++++------ workflows/input_template.json | 2 +- workflows/main.wdl | 6 +++--- workflows/sample_analysis/inputs.json | 2 +- workflows/sample_analysis/sample_analysis.wdl | 6 +++--- workflows/wdl-common | 2 +- 11 files changed, 22 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 9ddf0538..04172822 100644 --- a/README.md +++ b/README.md @@ -172,8 +172,8 @@ These files are hosted publicly in each of the cloud backends; see `backends/${b | Type | Name | Description | Notes | | :- | :- | :- | :- | -| String? | deepvariant_version | Version of deepvariant to use \["1.5.0"\] | | -| [DeepVariantModel](https://github.com/PacificBiosciences/wdl-common/blob/main/wdl/structs.wdl)? | deepvariant_model | Optional alternate DeepVariant model file to use | | +| String? | deepvariant_version | Version of deepvariant to use \["1.6.0"\] | | + | File? | custom_deepvariant_model_tar | Optional alternate DeepVariant [custom model](https://github.com/PacificBiosciences/wdl-common/blob/8d868818b62345fdb64ab104238d406d65334b3e/wdl/workflows/deepvariant/README.md) to use | | | Int? | pbsv_call_mem_gb | Optionally set RAM (GB) for pbsv_call during cohort analysis | | | Int? | glnexus_mem_gb | Optionally set RAM (GB) for GLnexus during cohort analysis | | | Boolean? | run_tertiary_analysis | Run the optional tertiary analysis steps \[`false`\] | \[`true`, `false`\] | @@ -256,7 +256,7 @@ The Docker image used by a particular step of the workflow can be identified by | Image | Major tool versions | Links | | :- | :- | :- | | bcftools |
    • [bcftools 1.14](https://github.com/samtools/bcftools/releases/tag/1.14)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/bcftools) | -| deepvariant | User-defined; default is version [1.5.0](https://github.com/google/deepvariant/releases/tag/v1.5.0) | [DeepVariant GitHub](https://github.com/google/deepvariant) | +| deepvariant | User-defined; default is version [1.6.0](https://github.com/google/deepvariant/releases/tag/v1.6.0) | [DeepVariant GitHub](https://github.com/google/deepvariant) | | glnexus |
    • [glnexus v1.4.3](https://github.com/dnanexus-rnd/GLnexus/releases/tag/v1.4.3)
    | [GLnexus GitHub](https://github.com/dnanexus-rnd/GLnexus) | | hificnv |
    • [HiFiCNV v0.1.7](https://github.com/PacificBiosciences/HiFiCNV/releases/tag/v0.1.7)
    • [bcftools 1.16](https://github.com/samtools/bcftools/releases/tag/1.16)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/0b0fbe939648087e9fdea4497ae08dc76538ebf0/docker/hificnv) | | hiphase |
    • [HiPhase 1.0.0](https://github.com/PacificBiosciences/HiPhase/releases/tag/v1.0.0)
    • [samtools 1.18](https://github.com/samtools/samtools/releases/tag/1.18)
    • [bcftools 1.18](https://github.com/samtools/bcftools/releases/tag/1.18)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/d26db6204409dfeff56e169cdba0cc14bc272f15/docker/hiphase) | diff --git a/backends/aws/inputs.aws.json b/backends/aws/inputs.aws.json index 87b4f162..d4f0bf4f 100644 --- a/backends/aws/inputs.aws.json +++ b/backends/aws/inputs.aws.json @@ -64,7 +64,7 @@ "clinvar_lookup": "s3://dnastack-resources/dataset/slivar/clinvar_gene_desc.20221214T183140.txt" }, "humanwgs.deepvariant_version": "String (optional)", - "humanwgs.deepvariant_model": "WomCompositeType {\n model -> WomCompositeType {\n data -> File\ndata_index -> File \n}\nmetadata -> File \n}? (optional)", + "humanwgs.custom_deepvariant_model_tar": "File (optional)", "humanwgs.pbsv_call_mem_gb": "Int (optional, default = if N<=3: 64 else 96)", "humanwgs.glnexus_mem_gb": "Int (optional, default = 30)", "humanwgs.run_tertiary_analysis": "Boolean (optional, default = false)", diff --git a/backends/azure/inputs.azure.json b/backends/azure/inputs.azure.json index 8d603d54..12bb8153 100644 --- a/backends/azure/inputs.azure.json +++ b/backends/azure/inputs.azure.json @@ -64,7 +64,7 @@ "clinvar_lookup": "/datasetpbrarediseases/dataset/slivar/clinvar_gene_desc.20221214T183140.txt" }, "humanwgs.deepvariant_version": "String (optional)", - "humanwgs.deepvariant_model": "WomCompositeType {\n model -> WomCompositeType {\n data -> File\ndata_index -> File \n}\nmetadata -> File \n}? (optional)", + "humanwgs.custom_deepvariant_model_tar": "File (optional)", "humanwgs.pbsv_call_mem_gb": "Int (optional, default = if N<=3: 64 else 96)", "humanwgs.glnexus_mem_gb": "Int (optional, default = 30)", "humanwgs.run_tertiary_analysis": "Boolean (optional, default = false)", diff --git a/backends/gcp/inputs.gcp.json b/backends/gcp/inputs.gcp.json index 483998e7..2a09f898 100644 --- a/backends/gcp/inputs.gcp.json +++ b/backends/gcp/inputs.gcp.json @@ -64,7 +64,7 @@ "clinvar_lookup": "gs://pacbio-wdl/dataset/slivar/clinvar_gene_desc.20221214T183140.txt" }, "humanwgs.deepvariant_version": "String (optional)", - "humanwgs.deepvariant_model": "WomCompositeType {\n model -> WomCompositeType {\n data -> File\ndata_index -> File \n}\nmetadata -> File \n}? (optional)", + "humanwgs.custom_deepvariant_model_tar": "File (optional)", "humanwgs.pbsv_call_mem_gb": "Int (optional, default = if N<=3: 64 else 96)", "humanwgs.glnexus_mem_gb": "Int (optional, default = 30)", "humanwgs.run_tertiary_analysis": "Boolean (optional, default = false)", diff --git a/backends/hpc/inputs.hpc.json b/backends/hpc/inputs.hpc.json index e5a66215..859c387a 100644 --- a/backends/hpc/inputs.hpc.json +++ b/backends/hpc/inputs.hpc.json @@ -64,7 +64,7 @@ "clinvar_lookup": "/dataset/slivar/clinvar_gene_desc.20221214T183140.txt" }, "humanwgs.deepvariant_version": "String (optional)", - "humanwgs.deepvariant_model": "WomCompositeType {\n model -> WomCompositeType {\n data -> File\ndata_index -> File \n}\nmetadata -> File \n}? (optional)", + "humanwgs.custom_deepvariant_model_tar": "File (optional)", "humanwgs.pbsv_call_mem_gb": "Int (optional, default = if N<=3: 64 else 96)", "humanwgs.glnexus_mem_gb": "Int (optional, default = 30)", "humanwgs.run_tertiary_analysis": "Boolean (optional, default = false)", diff --git a/wdl-ci.config.json b/wdl-ci.config.json index a673b508..aedb80f9 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -598,7 +598,7 @@ "task_start_index": 0, "tasks_per_shard": 8, "total_deepvariant_tasks": 64, - "deepvariant_version": "1.5.0", + "deepvariant_version": "1.6.0", "runtime_attributes": "${default_runtime_attributes}" }, "output_tests": { @@ -637,12 +637,12 @@ "${resources_file_path}/deepvariant/${sample_id}.56.example_tfrecords.tar.gz" ], "total_deepvariant_tasks": 64, - "deepvariant_version": "1.5.0", + "deepvariant_version": "1.6.0", "runtime_attributes": "${default_runtime_attributes}" }, "output_tests": { - "tfrecord": { - "value": "${resources_file_path}/deepvariant/${sample_id}.GRCh38.call_variants_output.tfrecord.gz", + "tfrecords_tar": { + "value": "${resources_file_path}/deepvariant/${sample_id}.GRCh38.call_variants_output.tar.gz", "test_tasks": [ "compare_file_basename", "check_gzip" @@ -659,7 +659,7 @@ { "inputs": { "sample_id": "${sample_id}", - "tfrecord": "${resources_file_path}/deepvariant/${sample_id}.GRCh38.call_variants_output.tfrecord.gz", + "tfrecords_tar": "${resources_file_path}/deepvariant/${sample_id}.GRCh38.call_variants_output.tar.gz", "nonvariant_site_tfrecord_tars": [ "${resources_file_path}/deepvariant/${sample_id}.0.nonvariant_site_tfrecords.tar.gz", "${resources_file_path}/deepvariant/${sample_id}.8.nonvariant_site_tfrecords.tar.gz", @@ -674,7 +674,7 @@ "reference_index": "${datasets_file_path}/GRCh38/human_GRCh38_no_alt_analysis_set.fasta.fai", "reference_name": "GRCh38", "total_deepvariant_tasks": 64, - "deepvariant_version": "1.5.0", + "deepvariant_version": "1.6.0", "runtime_attributes": "${default_runtime_attributes}" }, "output_tests": { diff --git a/workflows/input_template.json b/workflows/input_template.json index 39632a22..8f9c0ca4 100644 --- a/workflows/input_template.json +++ b/workflows/input_template.json @@ -42,7 +42,7 @@ "clinvar_lookup": "File" }, "humanwgs.deepvariant_version": "String? (optional)", - "humanwgs.deepvariant_model": "WomCompositeType {\n model -> WomCompositeType {\n data -> File\ndata_index -> File \n}\nmetadata -> File \n}? (optional)", + "humanwgs.custom_deepvariant_model_tar": "File (optional)", "humanwgs.pbsv_call_mem_gb": "Int? (optional)", "humanwgs.glnexus_mem_gb": "Int? (optional)", "humanwgs.run_tertiary_analysis": "Boolean? (optional, default = false)", diff --git a/workflows/main.wdl b/workflows/main.wdl index d334f526..76a4bec2 100644 --- a/workflows/main.wdl +++ b/workflows/main.wdl @@ -13,8 +13,8 @@ workflow humanwgs { ReferenceData reference SlivarData? slivar_data - String deepvariant_version = "1.5.0" - DeepVariantModel? deepvariant_model + String deepvariant_version = "1.6.0" + File? custom_deepvariant_model_tar Int? pbsv_call_mem_gb Int? glnexus_mem_gb @@ -48,7 +48,7 @@ workflow humanwgs { sample = sample, reference = reference, deepvariant_version = deepvariant_version, - deepvariant_model = deepvariant_model, + custom_deepvariant_model_tar = custom_deepvariant_model_tar, default_runtime_attributes = default_runtime_attributes } } diff --git a/workflows/sample_analysis/inputs.json b/workflows/sample_analysis/inputs.json index 6a7ff8ca..0fd685ee 100644 --- a/workflows/sample_analysis/inputs.json +++ b/workflows/sample_analysis/inputs.json @@ -30,7 +30,7 @@ "population_vcfs": "Array[WomCompositeType {\n data -> File\ndata_index -> File \n}]? (optional)" }, "sample_analysis.deepvariant_version": "String", - "sample_analysis.deepvariant_model": "WomCompositeType {\n model -> WomCompositeType {\n data -> File\ndata_index -> File \n}\nmetadata -> File \n}? (optional)", + "sample_analysis.custom_deepvariant_model_tar": "File (optional)", "sample_analysis.default_runtime_attributes": { "preemptible_tries": "Int", "max_retries": "Int", diff --git a/workflows/sample_analysis/sample_analysis.wdl b/workflows/sample_analysis/sample_analysis.wdl index 9e31a348..a411a7bb 100644 --- a/workflows/sample_analysis/sample_analysis.wdl +++ b/workflows/sample_analysis/sample_analysis.wdl @@ -17,7 +17,7 @@ workflow sample_analysis { ReferenceData reference String deepvariant_version - DeepVariantModel? deepvariant_model + File? custom_deepvariant_model_tar RuntimeAttributes default_runtime_attributes } @@ -56,7 +56,7 @@ workflow sample_analysis { reference_fasta = reference.fasta, reference_name = reference.name, deepvariant_version = deepvariant_version, - deepvariant_model = deepvariant_model, + custom_deepvariant_model_tar = custom_deepvariant_model_tar, default_runtime_attributes = default_runtime_attributes } @@ -244,7 +244,7 @@ workflow sample_analysis { sample: {help: "Sample information and associated data files"} reference: {help: "Reference genome data"} deepvariant_version: {help: "Version of deepvariant to use"} - deepvariant_model: {help: "Optional deepvariant model file to use"} + custom_deepvariant_model_tar: {help: "Optional deepvariant model to use"} default_runtime_attributes: {help: "Default RuntimeAttributes; spot if preemptible was set to true, otherwise on_demand"} } } diff --git a/workflows/wdl-common b/workflows/wdl-common index eae8a3f4..ab5b669e 160000 --- a/workflows/wdl-common +++ b/workflows/wdl-common @@ -1 +1 @@ -Subproject commit eae8a3f4ab51c86bb67b5d2eb8941b6678784b22 +Subproject commit ab5b669ea508c240d8e7999a6b6b882883d8434b From c744195d35687b081341ff69b4f837e22108f12d Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 15 Dec 2023 01:20:46 +0000 Subject: [PATCH 09/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index aedb80f9..51643649 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -620,7 +620,7 @@ }, "deepvariant_call_variants": { "key": "deepvariant_call_variants", - "digest": "a6ksi3haiz5pye7p64c67zeeauit7gqf", + "digest": "kytisx2zskpyhpxi6kgs6e42tpip7aex", "tests": [ { "inputs": { @@ -654,7 +654,7 @@ }, "deepvariant_postprocess_variants": { "key": "deepvariant_postprocess_variants", - "digest": "afmkoy2hy4lcyolms7n6lgzuzudr7uxx", + "digest": "jsx4n7nswvlktdwsp6u3anzlbtmki3tk", "tests": [ { "inputs": { From 5327b57111172bf830200f39d38f455125e35c83 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Fri, 15 Dec 2023 10:22:25 -0800 Subject: [PATCH 10/25] updated submodule --- workflows/wdl-common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/workflows/wdl-common b/workflows/wdl-common index ab5b669e..dd3f6386 160000 --- a/workflows/wdl-common +++ b/workflows/wdl-common @@ -1 +1 @@ -Subproject commit ab5b669ea508c240d8e7999a6b6b882883d8434b +Subproject commit dd3f63866b9fd1eb8396409ed4ab6f33c822d008 From f48182768197402857b789d5621d20db872a823c Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 15 Dec 2023 18:56:25 +0000 Subject: [PATCH 11/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 51643649..e3153c15 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -570,7 +570,7 @@ "tasks": { "deepvariant_make_examples": { "key": "deepvariant_make_examples", - "digest": "35kzpf37semcoxs7frzvhjrc4zvwoyan", + "digest": "3jfeho4suf23enopqj23cx6ygviftqny", "tests": [ { "inputs": { @@ -620,7 +620,7 @@ }, "deepvariant_call_variants": { "key": "deepvariant_call_variants", - "digest": "kytisx2zskpyhpxi6kgs6e42tpip7aex", + "digest": "ehblbh453m7dvjesw2573kuuxitgaduj", "tests": [ { "inputs": { @@ -654,7 +654,7 @@ }, "deepvariant_postprocess_variants": { "key": "deepvariant_postprocess_variants", - "digest": "jsx4n7nswvlktdwsp6u3anzlbtmki3tk", + "digest": "ey7zqpajaeesvsg372rehhjmkpqld2qx", "tests": [ { "inputs": { From b3223436f000fc1e953729047c831fdb0483b583 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Fri, 15 Dec 2023 12:11:47 -0800 Subject: [PATCH 12/25] Update module to make correct custom directory --- workflows/wdl-common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/workflows/wdl-common b/workflows/wdl-common index dd3f6386..ec28a7f6 160000 --- a/workflows/wdl-common +++ b/workflows/wdl-common @@ -1 +1 @@ -Subproject commit dd3f63866b9fd1eb8396409ed4ab6f33c822d008 +Subproject commit ec28a7f674fc382aa13d1b16cc1f4027b695464a From fc8b47faa957fd05cea307cdd13a49b0f1454158 Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 15 Dec 2023 20:22:19 +0000 Subject: [PATCH 13/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index e3153c15..eb6fd9c2 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -620,7 +620,7 @@ }, "deepvariant_call_variants": { "key": "deepvariant_call_variants", - "digest": "ehblbh453m7dvjesw2573kuuxitgaduj", + "digest": "rywffaewvhwakdysxejsrqbz7bqwzt2o", "tests": [ { "inputs": { From f7d8b4c990efbd6ab0b87d5512078383c5e137e5 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Fri, 15 Dec 2023 12:57:36 -0800 Subject: [PATCH 14/25] fix: Handle missing output for paraphase. --- workflows/main.wdl | 2 +- workflows/sample_analysis/sample_analysis.wdl | 12 +++++++----- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/workflows/main.wdl b/workflows/main.wdl index d334f526..6bf71e84 100644 --- a/workflows/main.wdl +++ b/workflows/main.wdl @@ -130,7 +130,7 @@ workflow humanwgs { # per sample paraphase outputs Array[File] paraphase_output_jsons = sample_analysis.paraphase_output_json Array[IndexData] paraphase_realigned_bams = sample_analysis.paraphase_realigned_bam - Array[File] paraphase_vcfs = sample_analysis.paraphase_vcfs + Array[File?] paraphase_vcfs = sample_analysis.paraphase_vcfs # per sample hificnv outputs Array[IndexData] hificnv_vcfs = sample_analysis.hificnv_vcf diff --git a/workflows/sample_analysis/sample_analysis.wdl b/workflows/sample_analysis/sample_analysis.wdl index 9e31a348..7c976638 100644 --- a/workflows/sample_analysis/sample_analysis.wdl +++ b/workflows/sample_analysis/sample_analysis.wdl @@ -231,7 +231,7 @@ workflow sample_analysis { # per sample paraphase outputs File paraphase_output_json = paraphase.output_json IndexData paraphase_realigned_bam = {"data": paraphase.realigned_bam, "data_index": paraphase.realigned_bam_index} - File paraphase_vcfs = paraphase.paraphase_vcfs + File? paraphase_vcfs = paraphase.paraphase_vcfs # per sample hificnv outputs IndexData hificnv_vcf = {"data": hificnv.cnv_vcf, "data_index": hificnv.cnv_vcf_index} @@ -639,16 +639,18 @@ task paraphase { --reference ~{reference} \ --out ~{out_directory} - cd ~{out_directory} \ - && tar zcvf ~{out_directory}.tar.gz ~{sample_id}_vcfs/*.vcf \ - && mv ~{out_directory}.tar.gz ../ + if ls ~{out_directory}/~{sample_id}_vcfs/*.vcf &> /dev/null; then + cd ~{out_directory} \ + && tar zcvf ~{out_directory}.tar.gz ~{sample_id}_vcfs/*.vcf \ + && mv ~{out_directory}.tar.gz ../ + fi >>> output { File output_json = "~{out_directory}/~{sample_id}.json" File realigned_bam = "~{out_directory}/~{sample_id}_realigned_tagged.bam" File realigned_bam_index = "~{out_directory}/~{sample_id}_realigned_tagged.bam.bai" - File paraphase_vcfs = "~{out_directory}.tar.gz" + File? paraphase_vcfs = "~{out_directory}.tar.gz" } runtime { From 49998875cb18b4eddc38fc879378df99a8ec88a6 Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 15 Dec 2023 21:21:52 +0000 Subject: [PATCH 15/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index a673b508..6ab42c31 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -245,7 +245,7 @@ }, "paraphase": { "key": "paraphase", - "digest": "p36f7pt2tghztqhtdgreyxohbjjndg3i", + "digest": "xme5pugrmhvnys5vas6jfagm65eac4hz", "tests": [ { "inputs": { From c58779e213c716f0ee1cf26a4b5a1eda5ddccf84 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Tue, 23 Jan 2024 16:15:13 -0800 Subject: [PATCH 16/25] added hpc backend --- wdl-ci.config.json | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 580bcdd2..2326b6d6 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -1114,7 +1114,7 @@ "engines": { "c3213beb-97bc-4adc-9bd8-cdb3ac83b398": { "key": "c3213beb-97bc-4adc-9bd8-cdb3ac83b398", - "enabled": true, + "enabled": false, "name": "PacBio CoA" } }, @@ -1142,6 +1142,11 @@ "input_file_path": "/coac74908838b5dd7/inputs/small_dataset/chr6.p23", "resources_file_path": "/coac74908838b5dd7/inputs/wdl-ci/humanwgs", "datasets_file_path": "/datasetpbrarediseases/dataset" + }, + "abc123": { + "input_file_path": "/pbi/collections/appslabht/cromwell_output/testdata/inputs/chr6.p23", + "resources_file_path": "/pbi/collections/appslabht/cromwell_output/testdata/wdl-ci/humanwgs", + "datasets_file_path": "/pbi/collections/appslabht/cromwell_output/testdata/datasetpbrarediseases/dataset" } } } From a9bfac1755c6d395dab76dfbc77cace53a4d23cf Mon Sep 17 00:00:00 2001 From: William Rowell Date: Tue, 23 Jan 2024 16:17:02 -0800 Subject: [PATCH 17/25] added hpc backend --- wdl-ci.config.json | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 2326b6d6..07ac15d7 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -1116,6 +1116,11 @@ "key": "c3213beb-97bc-4adc-9bd8-cdb3ac83b398", "enabled": false, "name": "PacBio CoA" + }, + "abc123": { + "key": "abc123", + "enabled": true, + "name": "pacbio-hpc" } }, "test_params": { From 9521bec4f6cda76bc448ae7829686cbc92c9a73a Mon Sep 17 00:00:00 2001 From: William Rowell Date: Tue, 13 Feb 2024 18:47:39 -0800 Subject: [PATCH 18/25] feat(trgt): Updated to TRGT 0.8.0 --- workflows/sample_analysis/sample_analysis.wdl | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/workflows/sample_analysis/sample_analysis.wdl b/workflows/sample_analysis/sample_analysis.wdl index 09dceef6..4f62c82f 100644 --- a/workflows/sample_analysis/sample_analysis.wdl +++ b/workflows/sample_analysis/sample_analysis.wdl @@ -502,7 +502,7 @@ task trgt { } runtime { - docker: "~{runtime_attributes.container_registry}/trgt@sha256:07e9d2be1590fc1dd45e6e2a6b9f5869ee6d83f5a1ca47c8ea84e114b70573b6" + docker: "~{runtime_attributes.container_registry}/trgt@sha256:88eaa6b6c7d440a48d7f0036e46a2ce4b37cf5be8bd84921eaa69e3c11b98556" cpu: threads memory: "4 GB" disk: disk_size + " GB" @@ -545,7 +545,7 @@ task coverage_dropouts { } runtime { - docker: "~{runtime_attributes.container_registry}/trgt@sha256:07e9d2be1590fc1dd45e6e2a6b9f5869ee6d83f5a1ca47c8ea84e114b70573b6" + docker: "~{runtime_attributes.container_registry}/trgt@sha256:88eaa6b6c7d440a48d7f0036e46a2ce4b37cf5be8bd84921eaa69e3c11b98556" cpu: threads memory: "4 GB" disk: disk_size + " GB" From 78d735429e569a553cf8bfffe81cd85908aeaee7 Mon Sep 17 00:00:00 2001 From: github-actions <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 16 Feb 2024 21:22:07 +0000 Subject: [PATCH 19/25] update wdl-ci config file after successful tests --- wdl-ci.config.json | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/wdl-ci.config.json b/wdl-ci.config.json index 07ac15d7..179ef52d 100644 --- a/wdl-ci.config.json +++ b/wdl-ci.config.json @@ -142,7 +142,7 @@ }, "trgt": { "key": "trgt", - "digest": "uaowbumdens55465girwarokc46hwgco", + "digest": "fyt3gqmt5tfykbls33kbr62nw3d4rvhj", "tests": [ { "inputs": { @@ -178,7 +178,7 @@ }, "coverage_dropouts": { "key": "coverage_dropouts", - "digest": "3wfgwjflppb5tifrwycmjzy727p7wzyi", + "digest": "iac5c3nzugilbarb4xsat37n2j5hjov4", "tests": [ { "inputs": { From 5a18241fb37fdc560d626a41678930c7ecc5844c Mon Sep 17 00:00:00 2001 From: William Rowell Date: Fri, 16 Feb 2024 13:31:42 -0800 Subject: [PATCH 20/25] docs(trgt): Update documentation with new trgt version. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index a03c771c..b2d3e201 100644 --- a/README.md +++ b/README.md @@ -270,7 +270,7 @@ The Docker image used by a particular step of the workflow can be identified by | samtools |
    • [samtools 1.14](https://github.com/samtools/samtools/releases/tag/1.14)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/samtools) | | slivar |
    • [slivar 0.2.2](https://github.com/brentp/slivar/releases/tag/v0.2.2)
    • [bcftools 1.14](https://github.com/samtools/bcftools/releases/tag/1.14)
    • [vcfpy 0.13.3](https://github.com/bihealth/vcfpy/releases/tag/v0.13.3)
    • [pysam 0.19.1](https://github.com/pysam-developers/pysam/releases/tag/v0.19.1)
    | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3560fcc5a84e044067cea9c9a7669cfc2659178e/docker/slivar) | | svpack |
    • [svpack 36180ae6](https://github.com/PacificBiosciences/svpack/tree/a82598ebc4013bf32e70295b83b380ada6302c4a)
    • [htslib 1.18](https://github.com/samtools/htslib/releases/tag/1.18)
    • [pysam 0.21.0](https://github.com/pysam-developers/pysam/releases/tag/v0.21.0)
      • | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/8edbc516abc0ff43ac279b48018003923721b054/docker/svpack) | -| trgt |
      • [trgt 0.7.0](https://github.com/PacificBiosciences/trgt/releases/tag/v0.7.0)
      • [samtools 1.18](https://github.com/samtools/samtools/releases/tag/1.18)
      • [bcftools 1.18](https://github.com/samtools/bcftools/releases/tag/1.18)
      • [pysam 0.21.0](https://github.com/pysam-developers/pysam/releases/tag/v0.21.0)
      | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/9aae7d50a4beb08a1869dd3ce1a0567ee4e5cbc2/docker/trgt) | +| trgt |
      • [trgt 0.8.0](https://github.com/PacificBiosciences/trgt/releases/tag/v0.7.0)
      • [samtools 1.18](https://github.com/samtools/samtools/releases/tag/1.18)
      • [bcftools 1.18](https://github.com/samtools/bcftools/releases/tag/1.18)
      • [pysam 0.21.0](https://github.com/pysam-developers/pysam/releases/tag/v0.21.0)
      | [Dockerfile](https://github.com/PacificBiosciences/wdl-dockerfiles/tree/3bbc033f8f942b10a304b4fa907957a789c73ef7/docker/trgt) | --- From 75ef1c43b0d96209b0f48034f272b41e0f5a0284 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Fri, 16 Feb 2024 17:14:35 -0800 Subject: [PATCH 21/25] docs: Corrected the Zenodo download URL. --- backends/hpc/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/backends/hpc/README.md b/backends/hpc/README.md index 48f018a5..132a8cf4 100644 --- a/backends/hpc/README.md +++ b/backends/hpc/README.md @@ -41,7 +41,7 @@ Reference data is hosted on Zenodo at [10.5281/zenodo.8415406](https://zenodo.or ```bash # download the reference data bundle -wget https://zenodo.org/record/8415406/files/wdl-humanwgs.v1.0.2.resources.tgz +wget https://zenodo.org/records/8415406/files/wdl-humanwgs.v1.0.2.resource.tgz # extract the reference data bundle and rename as dataset tar -xzf wdl-humanwgs.v1.0.2.resources.tgz && mv static_resources dataset From 8c953c19aadef51e7ab98d42c3c370969555c5de Mon Sep 17 00:00:00 2001 From: William Rowell Date: Tue, 20 Feb 2024 10:41:16 -0800 Subject: [PATCH 22/25] fix: Change submodule url to relative. --- .gitmodules | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitmodules b/.gitmodules index 8c759522..6bc7de6b 100644 --- a/.gitmodules +++ b/.gitmodules @@ -1,3 +1,3 @@ [submodule "workflows/wdl-common"] path = workflows/wdl-common - url = https://github.com/PacificBiosciences/wdl-common + url = ../wdl-common.git From 7be71512a73f724db111d15e2ac8ebf0ee5713c4 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Tue, 20 Feb 2024 12:52:42 -0800 Subject: [PATCH 23/25] Updated license year. --- LICENSE | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/LICENSE b/LICENSE index aaea0c14..9d176165 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,4 @@ -Copyright (c) 2023, Pacific Biosciences of California, Inc. +Copyright (c) 2024, Pacific Biosciences of California, Inc. All rights reserved. From eb3bbeb224dddb5d2fa3575fd93171e4f94e7197 Mon Sep 17 00:00:00 2001 From: William Rowell Date: Tue, 20 Feb 2024 12:57:22 -0800 Subject: [PATCH 24/25] docs: Described restrictions on sample_id and cohort_id. --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index b2d3e201..c076bb09 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,7 @@ A cohort can include one or more samples. Samples need not be related, but if yo | Type | Name | Description | Notes | | :- | :- | :- | :- | -| String | cohort_id | A unique name for the cohort; used to name outputs | | +| String | cohort_id | A unique name for the cohort; used to name outputs | Alphanumeric characters, underscore (`_`), and dash (`-`) are allowed. | | Array[[Sample](#sample)] | samples | The set of samples for the cohort. At least one sample must be defined. | | | Array[String] | phenotypes | [Human Phenotype Ontology (HPO) phenotypes](https://hpo.jax.org/app/) associated with the cohort. If no particular phenotypes are desired, the root HPO term, `"HP:0000001"`, can be used. | | @@ -125,12 +125,12 @@ Sample information for each sample in the workflow run. | Type | Name | Description | Notes | | :- | :- | :- | :- | -| String | sample_id | A unique name for the sample; used to name outputs | | +| String | sample_id | A unique name for the sample; used to name outputs | Alphanumeric characters, underscore (`_`), and dash (`-`) are allowed. | | Array[[IndexData](https://github.com/PacificBiosciences/wdl-common/blob/main/wdl/structs.wdl)] | movie_bams | The set of unaligned movie BAMs associated with this sample | | | String? | sex | Sample sex | ["MALE", "FEMALE", `null`]. If the sex field is missing or `null`, sex will be set to unknown. Used to set the expected sex chromosome karyotype for TRGT and HiFiCNV. | | Boolean | affected | Is this sample affected by the phenotype? | \[`true`, `false`\] | -| String? | father_id | Paternal `sample_id` | | -| String? | mother_id | Maternal `sample_id` | | +| String? | father_id | Paternal `sample_id` | Alphanumeric characters, underscore (`_`), and dash (`-`) are allowed. | +| String? | mother_id | Maternal `sample_id` | Alphanumeric characters, underscore (`_`), and dash (`-`) are allowed. | ## [ReferenceData](workflows/humanwgs_structs.wdl) From c463d01ed6d3dc30bc107f8cf08cd6cb6c03ca1e Mon Sep 17 00:00:00 2001 From: William Rowell Date: Tue, 20 Feb 2024 12:58:43 -0800 Subject: [PATCH 25/25] docs: Noted that only DeepVariant 1.6 is supported. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index c076bb09..3dc28bb3 100644 --- a/README.md +++ b/README.md @@ -172,7 +172,7 @@ These files are hosted publicly in each of the cloud backends; see `backends/${b | Type | Name | Description | Notes | | :- | :- | :- | :- | -| String? | deepvariant_version | Version of deepvariant to use \["1.6.0"\] | | +| String? | deepvariant_version | Version of deepvariant to use \["1.6.0"\] | This release is only compatible with DeepVariant [1.6](https://github.com/google/deepvariant/releases/tag/v1.6.0). | | File? | custom_deepvariant_model_tar | Optional alternate DeepVariant [custom model](https://github.com/PacificBiosciences/wdl-common/blob/8d868818b62345fdb64ab104238d406d65334b3e/wdl/workflows/deepvariant/README.md) to use | | | Int? | pbsv_call_mem_gb | Optionally set RAM (GB) for pbsv_call during cohort analysis | | | Int? | glnexus_mem_gb | Optionally set RAM (GB) for GLnexus during cohort analysis | |