error while running the DNA-mapping

[sunta3iouxos]

Hallo again,
This something new:

DNA-mapping -i /mnt/Z/Shared\ folder/Shared\ data/DNA/NGS_AP06_apouikli_A006200409/ -o /mnt/Z/Shared\ folder/Shared\ data/DNA/NGS_AP06_apouikli_A006200409/bam/ -c /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/DNA-mapping/defaults.yaml --local -j 8 --fastqc --trim --trimmer fastp --trimmerOptions "--trim_poly_g --trim_poly_x -Q -L --correction" --dedup --plotFormat "pdf" --mapq 3 --cutntag --verbose --insertSizeMax 5000     /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/mm10.yaml
/home/tgeorgom/mambaforge/envs/snakePipes/bin/DNA-mapping:61: SyntaxWarning: invalid escape sequence '\-'
  "the first - (e.g., '\--very-fast'). Default: '%(default)s'.",

TMPDIR=/home/tgeorgom/temp/ PYTHONNOUSERSITE=True /home/tgeorgom/mambaforge/envs/snakePipes/bin/snakemake --use-conda --conda-prefix /home/tgeorgom/mambaforge/envs --latency-wait 300 --snakefile /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/DNA-mapping/Snakefile --jobs 8 --directory /mnt/Z/Shared folder/Shared data/DNA/NGS_AP06_apouikli_A006200409/bam --configfile /mnt/Z/Shared folder/Shared data/DNA/NGS_AP06_apouikli_A006200409/bam/DNA-mapping.config.yaml --keep-going --printshellcmds

PID: 364423

WorkflowError:
Config file must be given as JSON or YAML with keys at top level.
Error: snakemake returned an error code of 1, so processing is incomplete!

my snakePipes version

(snakePipes) tgeorgom@CECAD:~$ snakePipes version
version 2.5.3

freshly created environment

$ snakePipes  config

--- Final Updated Config ---------------------------------------------------------------------
config file: /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/defaults.yaml
clusterConfig: shared/cluster.yaml
configMode: manual
emailSender: None
max_thread: 8
oldConfig: None
onlySSL: False
organismsDir: shared/organisms
smtpPassword: None
smtpPort: 0
smtpServer: None
smtpUsername: None
snakemakeOptions:  --use-conda --conda-prefix /home/tgeorgom/mambaforge/envs
tempDir: /home/tgeorgom/temp/
toolsVersion: True
--------------------------------------------------------------------------------

and the info:

 snakePipes info
Organism YAML files:
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/SchizoSPombe_ASM294v2.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/GRCz10.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/hg38.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/dm3.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/mm10.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/hs37d5.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/dm6.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/mm9.yaml
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/mm10_gencodeM13.yaml

The workflow-generic defaults.yaml file is:
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/defaults.yaml

The default cluster.yaml file. Its defaults are overridden by the per-workflow cluster.yaml files:
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/cluster.yaml

Workflow-specific cluster.yaml and defaults.yaml files are in:
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/preprocessing
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/HiC
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/noncoding-RNA-seq
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/mRNA-seq
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/ATAC-seq
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/WGBS
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/createIndices
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/scRNAseq
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/ChIP-seq
    /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/DNA-mapping

Temp dir under /home/tgeorgom/temp/ exists and will be used.

I am using the /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/shared/organisms/mm10.yaml

and I am calling the pipeline with:

DNA-mapping -i /mnt/Z/Shared\ folder/Shared\ data/DNA/NGS_AP06_apouikli_A006200409/ -o /mnt/Z/Shared\ folder/Shared\ data/DNA/NGS_AP06_apouikli_A006200409/bam/ -c /home/tgeorgom/mambaforge/envs/snakePipes/lib/python3.12/site-packages/snakePipes/workflows/DNA-mapping/defaults.yaml --local -j 8 --fastqc --trim --trimmer fastp --trimmerOptions "--trim_poly_g --trim_poly_x -Q -L --correction" --dedup --plotFormat "pdf" --mapq 3 --cutntag --verbose --insertSizeMax 5000 mm10

[sunta3iouxos]

I think I solved the first one:
the pipeline does not like special characters, like spaces, even if it is under quotes.
maybe this can be addressed?
The second are the values for the bowtie2 this part:
SyntaxWarning: invalid escape sequence '-'
"the first - (e.g., '--very-fast'). Default: '%(default)s'.",
so something is wrong in the code there

[katsikora]

Hi,

It's good practice to remove spaces from paths, as they lead to all kind of issues in shell, such as those you have encountered.
I don't think I'd see it on our side 🤷

Could you be more explicit about the second error? Did you pass any --alignerOpts to DNA-mapping?

Also, why are you explicitly passing the defaults.yaml config from some (other?) snakePipes installation to DNA-mapping?

Best wishes,

Katarzyna

[sunta3iouxos]

I passed defaults.yaml to just trying to figure out where the error comes from.
I then by checking a couple of other things found that the error is due to the name.
I symling the fastq.gz to another folder and things are working.
I know about the spaces, but this is not of my doing 😢

now regarding that part:

/scratch/Theo/envs/snakePipes/bin/DNA-mapping:61: SyntaxWarning: invalid escape sequence '\-'
 "the first - (e.g., '\--very-fast'). Default: '%(default)s'.",

although the pipeline works very well (till now), I am always getting this error. I thought that is comes from bowtie, since I faintly remember the --very-fast string, but I can not tell for sure