| - **Pre-processing**: Phasing, QC, variant filtering, variant annotation of the reference panel 
| The preprocessing mode is responsible to the preparation of the multiple input file that will be used by the phasing process. The main processes are :
- **Haplotypes phasing** of the reference panel using [**Shapeit5**](https://odelaneau.github.io/shapeit5/).
- **Filter** the reference panel to select only the necessary variants.
- **Chunking the reference panel** in a subset of region for all the chromosomes.
- **Extract** the positions where to perform the imputation.| -| **Phasing** |
| The phasing mode is the core mode of this pipeline. It is constituted of 3 main steps:
- **Phasing**: Phasing of the target dataset on the reference panel using either:
- [**Glimpse1**](https://odelaneau.github.io/GLIMPSE/glimpse1/index.html)
It's come with the necessety to compute the genotype likelihoods of the target dataset.
This step is done using [BCFTOOLS_mpileup](https://samtools.github.io/bcftools/bcftools.html#mpileup)
- [**Glimpse2**](https://odelaneau.github.io/GLIMPSE/glimpse2/index.html) For this step the reference panel is transformed to binary chunks.
- [**Stitch**](https://github.com/rwdavies/stitch)
- [**Quilt**](https://github.com/rwdavies/QUILT)
- **Ligation**: all the different chunks are merged together.
- **Sampling** (optional) | -| **Simulate** |
| The simulation mode is used to create artificial low informative genetic information from high density data. This allow to compare the imputed result to a *truth* and therefore evaluate the quality of the imputation. For the moment it is possible to simulate:
- Low-pass data by **downsample** BAM or CRAM using [SAMTOOLS_view -s]() at different depth
- Genotype data by **SNP selecting** the position used by a designated SNP chip.
The simulation mode will also compute the **Genotype likelihoods** of the high density data. | -| **Concordance** |
| This mode compare two vcf together to compute a summary of the differences between them. To do so it use either:
- [**Glimpse1**](https://odelaneau.github.io/GLIMPSE/glimpse1/index.html) concordance process.
- [**Glimpse2**](https://odelaneau.github.io/GLIMPSE/glimpse2/index.html) concordance process
- Or convert the two vcf fill to `.zarr` using [**Scikit allele**](https://scikit-allel.readthedocs.io/en/stable/) and [**anndata**](https://anndata.readthedocs.io/en/latest/) before comparing the SNPs. | -| **Postprocessing**|
| This final process unable to loop the whole pipeline for increasing the performance of the imputation. To do so it filter out the best imputed position and rerun the analysis using this positions. |
+| Mode | Flow chart | Description |
+| ------------------ | ---------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Preprocessing** | | The preprocessing mode is responsible to the preparation of the multiple input file that will be used by the phasing process. The main processes are :
- **Haplotypes phasing** of the reference panel using [**Shapeit5**](https://odelaneau.github.io/shapeit5/).
- **Filter** the reference panel to select only the necessary variants.
- **Chunking the reference panel** in a subset of region for all the chromosomes.
- **Extract** the positions where to perform the imputation. | +| **Phasing** |
| The phasing mode is the core mode of this pipeline. It is constituted of 3 main steps:
- **Phasing**: Phasing of the target dataset on the reference panel using either:
- [**Glimpse1**](https://odelaneau.github.io/GLIMPSE/glimpse1/index.html)
It's come with the necessety to compute the genotype likelihoods of the target dataset.
This step is done using [BCFTOOLS_mpileup](https://samtools.github.io/bcftools/bcftools.html#mpileup)
- [**Glimpse2**](https://odelaneau.github.io/GLIMPSE/glimpse2/index.html) For this step the reference panel is transformed to binary chunks.
- [**Stitch**](https://github.com/rwdavies/stitch)
- [**Quilt**](https://github.com/rwdavies/QUILT)
- **Ligation**: all the different chunks are merged together.
- **Sampling** (optional) | +| **Simulate** |
| The simulation mode is used to create artificial low informative genetic information from high density data. This allow to compare the imputed result to a _truth_ and therefore evaluate the quality of the imputation. For the moment it is possible to simulate:
- Low-pass data by **downsample** BAM or CRAM using [SAMTOOLS_view -s]() at different depth
- Genotype data by **SNP selecting** the position used by a designated SNP chip.
The simulation mode will also compute the **Genotype likelihoods** of the high density data. | +| **Concordance** |
| This mode compare two vcf together to compute a summary of the differences between them. To do so it use either:
- [**Glimpse1**](https://odelaneau.github.io/GLIMPSE/glimpse1/index.html) concordance process.
- [**Glimpse2**](https://odelaneau.github.io/GLIMPSE/glimpse2/index.html) concordance process
- Or convert the two vcf fill to `.zarr` using [**Scikit allele**](https://scikit-allel.readthedocs.io/en/stable/) and [**anndata**](https://anndata.readthedocs.io/en/latest/) before comparing the SNPs. | +| **Postprocessing** |
| This final process unable to loop the whole pipeline for increasing the performance of the imputation. To do so it filter out the best imputed position and rerun the analysis using this positions. |
## Pipeline output
@@ -97,11 +86,12 @@ For more details about the output files and reports, please refer to the
## Credits
-nf-core/phaseimpute was originally written by LouisLeNezet.
+nf-core/phaseimpute was originally written by Louis Le Nézet.
We thank the following people for their extensive assistance in the development of this pipeline:
-
+- Anabella Trigilla
+- Saul Pierotti
## Contributions and Support
@@ -110,7 +100,6 @@ If you would like to contribute to this pipeline, please see the [contributing g
For further information or help, don't hesitate to get in touch on the [Slack `#phaseimpute` channel](https://nfcore.slack.com/channels/phaseimpute) (you can join with [this invite](https://nf-co.re/join/slack)).
For further information or help, don't hesitate to get in touch on the [Slack `#phaseimpute` channel](https://nfcore.slack.com/channels/phaseimpute) (you can join with [this invite](https://nf-co.re/join/slack)).
-## Citations
## Citations
diff --git a/assets/chr_rename_add.txt b/assets/chr_rename_add.txt
new file mode 100644
index 00000000..c48a2640
--- /dev/null
+++ b/assets/chr_rename_add.txt
@@ -0,0 +1,39 @@
+1 chr1
+2 chr2
+3 chr3
+4 chr4
+5 chr5
+6 chr6
+7 chr7
+8 chr8
+9 chr9
+10 chr10
+11 chr11
+12 chr12
+13 chr13
+14 chr14
+15 chr15
+16 chr16
+17 chr17
+18 chr18
+19 chr19
+20 chr20
+21 chr21
+22 chr22
+23 chr23
+24 chr24
+25 chr25
+26 chr26
+27 chr27
+28 chr28
+29 chr29
+30 chr30
+31 chr31
+32 chr32
+33 chr33
+34 chr34
+35 chr35
+36 chr36
+37 chr37
+38 chr38
+X chrX
diff --git a/assets/chr_rename.txt b/assets/chr_rename_del.txt
similarity index 100%
rename from assets/chr_rename.txt
rename to assets/chr_rename_del.txt
diff --git a/assets/panel.csv b/assets/panel.csv
new file mode 100644
index 00000000..c99b845c
--- /dev/null
+++ b/assets/panel.csv
@@ -0,0 +1,3 @@
+panel,vcf,index,sites,tsv,legend,phased
+1000GP,1000GP.phased.vcf,1000GP.phased.vcf.csi,1000GP.sites,1000GP.tsv,,TRUE
+1000GP_RePhase,1000GP.vcf,1000GP.vcf.csi,,,,FALSE
diff --git a/assets/samplesheet.csv b/assets/samplesheet.csv
index 441a688f..217ef7c3 100644
--- a/assets/samplesheet.csv
+++ b/assets/samplesheet.csv
@@ -1,3 +1,3 @@
-sample,BAM,BAI
+sample,bam,bai
1_BAM_1X,/path/to/.bam,/path/to/.bai
1_BAM_SNP,/path/to/.bam,/path/to/.bai
diff --git a/assets/schema_input.json b/assets/schema_input.json
index ce442280..aca033f6 100644
--- a/assets/schema_input.json
+++ b/assets/schema_input.json
@@ -13,17 +13,17 @@
"errorMessage": "Sample name must be provided and cannot contain spaces",
"meta": ["id"]
},
- "BAM": {
+ "bam": {
"type": "string",
"pattern": "^\\S+\\.bam$",
"errorMessage": "BAM file must be provided, cannot contain spaces and must have extension '.bam'"
},
- "BAI": {
+ "bai": {
"errorMessage": "BAI file must be provided, cannot contain spaces and must have extension '.bai'",
"type": "string",
"pattern": "^\\S+\\.bai$"
}
},
- "required": ["sample", "BAM", "BAI"]
+ "required": ["sample", "bam", "bai"]
}
}
diff --git a/assets/schema_input_panel.json b/assets/schema_input_panel.json
new file mode 100644
index 00000000..49593176
--- /dev/null
+++ b/assets/schema_input_panel.json
@@ -0,0 +1,48 @@
+{
+ "$schema": "http://json-schema.org/draft-07/schema",
+ "$id": "https://raw.githubusercontent.com/nf-core/phaseimpute/master/assets/schema_input.json",
+ "title": "nf-core/phaseimpute pipeline - params.input_region schema",
+ "description": "Schema for the file provided with params.input_region",
+ "type": "array",
+ "items": {
+ "type": "object",
+ "properties": {
+ "panel": {
+ "type": "string",
+ "pattern": "^\\S+$",
+ "errorMessage": "Panel name must be provided and cannot contain spaces",
+ "meta": ["panel"]
+ },
+ "vcf": {
+ "type": "string",
+ "pattern": "^\\S+\\.vcf$",
+ "errorMessage": "Panel vcf file must be provided, cannot contain spaces and must have extension '.vcf'"
+ },
+ "index": {
+ "type": "string",
+ "pattern": "^\\S+\\.vcf\\.(tbi|csi)$",
+ "errorMessage": "Panel vcf index file must be provided, cannot contain spaces and must have extension '.vcf.tbi' or '.vcf.csi'"
+ },
+ "sites": {
+ "type": "string",
+ "pattern": "^\\S+\\.sites$",
+ "errorMessage": "Panel sites file must be provided, cannot contain spaces and must have extension '.sites'"
+ },
+ "tsv": {
+ "type": "string",
+ "pattern": "^\\S+\\.tsv$",
+ "errorMessage": "Panel tsv file must be provided, cannot contain spaces and must have extension '.tsv'"
+ },
+ "legend": {
+ "type": "string",
+ "pattern": "^\\S+\\.legend$",
+ "errorMessage": "Panel legend file must be provided, cannot contain spaces and must have extension '.legend'"
+ },
+ "phased": {
+ "type": "boolean",
+ "errorMessage": "Is the vcf given phased? Must be a boolean"
+ }
+ },
+ "required": ["panel", "vcf", "index", "phased"]
+ }
+}
diff --git a/conf/modules.config b/conf/modules.config
index f30dbfa6..0d09feb2 100644
--- a/conf/modules.config
+++ b/conf/modules.config
@@ -50,7 +50,7 @@ process {
// Panel preparation workflow
withName: VIEW_VCF_REGION {
ext.args = [
- "-Oz",
+ "--output-type z",
"--no-version"
].join(' ')
ext.prefix = { "${meta.id}_${meta.region}" }
@@ -69,6 +69,7 @@ process {
"-m 2",
"-M 2",
"-v snps",
+ "--output-type z",
"--no-version"
].join(' ')
ext.prefix = { "${meta.id}_SPNS" }
@@ -87,6 +88,7 @@ process {
"-m 2",
"-M 2",
"-v snps",
+ "--output-type z",
"--no-version"
].join(' ')
ext.prefix = { "${meta.id}_SITES" }
diff --git a/conf/test.config b/conf/test.config
index fbb2e5c5..e81ac0b4 100644
--- a/conf/test.config
+++ b/conf/test.config
@@ -22,8 +22,14 @@ params {
// Input data
// TODO nf-core: Specify the paths to your test data on nf-core/test-datasets
// TODO nf-core: Give any required params for the test so that command line flags are not needed
- input = 'https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/samplesheet/samplesheet_test_illumina_amplicon.csv'
+ input = "${projectDir}/tests/csv/bam.csv"
// Genome references
- genome = 'R64-1-1'
+ fasta = "https://raw.githubusercontent.com/nf-core/test-datasets/phaseimpute/data/reference_genome/21_22/hs38DH.chr21_22.fa"
+ panel = "https://raw.githubusercontent.com/nf-core/test-datasets/phaseimpute/data/panel/21_22/1000GP.chr21_22.s.norel.bcf"
+ phased = true
+
+ // Impute parameters
+ step = "impute"
+ tools = "glimpse1"
}
diff --git a/conf/test_full.config b/conf/test_full.config
index cb2a0fae..2e95f899 100644
--- a/conf/test_full.config
+++ b/conf/test_full.config
@@ -22,7 +22,7 @@ params {
map = "/groups/dog/llenezet/test-datasets/data/genetic_maps.b38/chr21.b38.gmap.gz"
genome = "GRCh38"
fasta = "/groups/dog/llenezet/script/phaseimpute/data/genome.fa"
-
+
// Resources increase incompatible with Github Action
max_cpus = 12
max_memory = '50.GB'
diff --git a/conf/test_panelprep.config b/conf/test_panelprep.config
index 32a78c5a..2c5fcc0b 100644
--- a/conf/test_panelprep.config
+++ b/conf/test_panelprep.config
@@ -20,9 +20,8 @@ params {
max_time = '6.h'
// Input data
- input = "tests/csv/panel.csv"
- input_region_file = "tests/csv/regionsheet.csv"
- outdir = "results/test_panelprep"
+ input = "${projectDir}/tests/csv/panel.csv"
+ input_region = "${projectDir}/tests/csv/regionsheet.csv"
genome = "GRCh38"
map = "/groups/dog/llenezet/test-datasets/data/genetic_maps.b38/chr21.b38.gmap.gz"
diff --git a/docs/NfCore_library.xml b/docs/NfCore_library.xml
index 21989d75..c7d26934 100644
--- a/docs/NfCore_library.xml
+++ b/docs/NfCore_library.xml
@@ -1,163 +1,163 @@
-$group
\n" - summary_section += "- \n"
- for (param in group_params.keySet()) {
- summary_section += "
- $param
- ${group_params.get(param) ?: 'N/A'} \n" - } - summary_section += "
| Process Name | \\n\\", + " \\ Software | \\nVersion | \\n
|---|---|---|
| CUSTOM_DUMPSOFTWAREVERSIONS | \\npython | \\n\\", + " \\3.11.7 | \\n
| \\n \\", + " \\ | yaml | \\n5.4.1 | \\n
| TOOL1 | \\ntool1 | \\n\\", + " \\0.11.9 | \\n
| TOOL2 | \\n\\", + " \\tool2 | \\n1.9 | \\n
| Workflow | \\nNextflow | \\n\\" + ], + [ + "CUSTOM_DUMPSOFTWAREVERSIONS:", + " python: 3.11.7", + " yaml: 5.4.1", + "TOOL1:", + " tool1: 0.11.9", + "TOOL2:", + " tool2: '1.9'", + "Workflow:" + ] + ], + "timestamp": "2024-01-09T23:01:18.710682" + } +} \ No newline at end of file diff --git a/modules/nf-core/custom/dumpsoftwareversions/tests/tags.yml b/modules/nf-core/custom/dumpsoftwareversions/tests/tags.yml new file mode 100644 index 00000000..405aa24a --- /dev/null +++ b/modules/nf-core/custom/dumpsoftwareversions/tests/tags.yml @@ -0,0 +1,2 @@ +custom/dumpsoftwareversions: + - modules/nf-core/custom/dumpsoftwareversions/** diff --git a/modules/nf-core/fastqc/main.nf b/modules/nf-core/fastqc/main.nf deleted file mode 100644 index 9e19a74c..00000000 --- a/modules/nf-core/fastqc/main.nf +++ /dev/null @@ -1,55 +0,0 @@ -process FASTQC { - tag "$meta.id" - label 'process_medium' - - conda "${moduleDir}/environment.yml" - container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ? - 'https://depot.galaxyproject.org/singularity/fastqc:0.12.1--hdfd78af_0' : - 'biocontainers/fastqc:0.12.1--hdfd78af_0' }" - - input: - tuple val(meta), path(reads) - - output: - tuple val(meta), path("*.html"), emit: html - tuple val(meta), path("*.zip") , emit: zip - path "versions.yml" , emit: versions - - when: - task.ext.when == null || task.ext.when - - script: - def args = task.ext.args ?: '' - def prefix = task.ext.prefix ?: "${meta.id}" - // Make list of old name and new name pairs to use for renaming in the bash while loop - def old_new_pairs = reads instanceof Path || reads.size() == 1 ? [[ reads, "${prefix}.${reads.extension}" ]] : reads.withIndex().collect { entry, index -> [ entry, "${prefix}_${index + 1}.${entry.extension}" ] } - def rename_to = old_new_pairs*.join(' ').join(' ') - def renamed_files = old_new_pairs.collect{ old_name, new_name -> new_name }.join(' ') - """ - printf "%s %s\\n" $rename_to | while read old_name new_name; do - [ -f "\${new_name}" ] || ln -s \$old_name \$new_name - done - - fastqc \\ - $args \\ - --threads $task.cpus \\ - $renamed_files - - cat <<-END_VERSIONS > versions.yml - "${task.process}": - fastqc: \$( fastqc --version | sed '/FastQC v/!d; s/.*v//' ) - END_VERSIONS - """ - - stub: - def prefix = task.ext.prefix ?: "${meta.id}" - """ - touch ${prefix}.html - touch ${prefix}.zip - - cat <<-END_VERSIONS > versions.yml - "${task.process}": - fastqc: \$( fastqc --version | sed '/FastQC v/!d; s/.*v//' ) - END_VERSIONS - """ -} diff --git a/modules/nf-core/fastqc/meta.yml b/modules/nf-core/fastqc/meta.yml deleted file mode 100644 index ee5507e0..00000000 --- a/modules/nf-core/fastqc/meta.yml +++ /dev/null @@ -1,57 +0,0 @@ -name: fastqc -description: Run FastQC on sequenced reads -keywords: - - quality control - - qc - - adapters - - fastq -tools: - - fastqc: - description: | - FastQC gives general quality metrics about your reads. - It provides information about the quality score distribution - across your reads, the per base sequence content (%A/C/G/T). - You get information about adapter contamination and other - overrepresented sequences. - homepage: https://www.bioinformatics.babraham.ac.uk/projects/fastqc/ - documentation: https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/ - licence: ["GPL-2.0-only"] -input: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - reads: - type: file - description: | - List of input FastQ files of size 1 and 2 for single-end and paired-end data, - respectively. -output: - - meta: - type: map - description: | - Groovy Map containing sample information - e.g. [ id:'test', single_end:false ] - - html: - type: file - description: FastQC report - pattern: "*_{fastqc.html}" - - zip: - type: file - description: FastQC report archive - pattern: "*_{fastqc.zip}" - - versions: - type: file - description: File containing software versions - pattern: "versions.yml" -authors: - - "@drpatelh" - - "@grst" - - "@ewels" - - "@FelixKrueger" -maintainers: - - "@drpatelh" - - "@grst" - - "@ewels" - - "@FelixKrueger" diff --git a/modules/nf-core/fastqc/tests/main.nf.test b/modules/nf-core/fastqc/tests/main.nf.test deleted file mode 100644 index 70edae4d..00000000 --- a/modules/nf-core/fastqc/tests/main.nf.test +++ /dev/null @@ -1,212 +0,0 @@ -nextflow_process { - - name "Test Process FASTQC" - script "../main.nf" - process "FASTQC" - - tag "modules" - tag "modules_nfcore" - tag "fastqc" - - test("sarscov2 single-end [fastq]") { - - when { - process { - """ - input[0] = Channel.of([ - [ id: 'test', single_end:true ], - [ file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/fastq/test_1.fastq.gz', checkIfExists: true) ] - ]) - """ - } - } - - then { - assertAll ( - { assert process.success }, - - // NOTE The report contains the date inside it, which means that the md5sum is stable per day, but not longer than that. So you can't md5sum it. - // looks like this:|
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls | |
| File type | Conventional base calls |