Assess which files should be published

[nschcolnicov]

Current version publishes almost every file outputed by the processes. We probably should avoid publishing some of them, but would need a more informed opinion of an end-user or SME on which files to keep.

• Should we keep bam bai from genomequant in publishdir?
• Should we keep fastqc_raw and fastqc_trimmed files in publishdir? Only used for building multiqc (zip files), html reports are for user.
• Should we outuput fastq and bam files?
• Should we return fasta with their indices?

FYI @atrigila @apeltzer @grst @lpantano

[apeltzer]

In general I would be rather keen to publish only relevant information, e.g. metrics and smaller files but no BAM/FASTQ or similar files if not explicity specified by the user. Default should be conservative, e.g. not pushing these files to users.

• no BAM/BAI
• keep fastqc (not much data)
• no fastq/Bams
• fasta with index I am indifferent

After all, we could make these things configurable, e.g. --save_intermediates or similar and upon that either publish or not.

[lpantano]

agree with this, as far as the user can turn on saving those other files.

[apeltzer]

Sounds like a plan then :)

[atrigila]

Closed via #429