Singularity images naming convention and writing permission to cache library

[LouisLeNezet]

Hi, I'm running nextflow on an HPC and numerours singularity image are already available in a shared directory.
We do not have write access to it but it would be nice to use all the image already present.
The issue I'm facing is that the images are written as follow as multiqc-1.21--pyhdfd78af_0.img and nextflow is looking for depot.galaxyproject.org-singularity-multiqc-1.21--pyhdfd78af_0.img.

I guess that the issue originate from this code.

Would it be possible to configure this in the config file ?
Also, as we do not have writing access to the singularity directory nextflow might crash if the image is not present.

Here I remove the write access to singularity2 folder.
java.io.FileNotFoundException: work/singularity2/.depot.galaxyproject.org-singularity-samtools-1.19.2--h50ea8bc_0.img.lock (Permission denied)

In this case it would be nice to have a recovery mechanism, like using the default work/singularity in case the image is not present in the library ?

Thanks !

[pditommaso]

I think this is already supported by using env variables NXF_SINGULARITY_CACHEDIR and NXF_SINGULARITY_LIBRARYDIR. See. the docs for details https://www.nextflow.io/docs/latest/config.html#environment-variables

[LouisLeNezet]

Thanks ! That effectively solve my second issue for the write access.
Unfortunately I cannot change the name of the image on our cluster (e.g. samtools:1.19.2--h50ea8bc_0, multiqc:1.21--pyhdfd78af_0).
As they aren't in the form depot.galaxyproject.org-singularity-<tool>-<version>--<patch>.img, nextflow doesn't recognise them in the LIBRARYDIR and therefore pull them from the url.

[pditommaso]

Not sure how nextflow could help you

[LouisLeNezet]

What would be nice, would be to allow customisation of the simpleName() function in SingularityCache.groovy.
This would allow to change the way Nextflow would expect the name of a given image to be.
For example we could have in the Singularity scope a variable such as keepOnlyImgName that would do something like

    String simpleName(String imageUrl) {
        def p = imageUrl.indexOf('://')
        def name = p != -1 ? imageUrl.substring(p+3) : imageUrl
        if (keepOnlyImgName) {
            name = name.split('/').last()
        } else {
            name = name.replace(':','-').replace('/','-')
            name += ".img"
       }
       return name

[abretaud]

Just to add a precision: the shared directory mentioned by @LouisLeNezet is the CVMFS mountpoint corresponding to https://depot.galaxyproject.org/singularity/

It can only be mounted at /cvmfs/singularity.galaxyproject.org/all/ and the content (and image naming) can't be changed on our side