Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SAMTOOLS_FAIDX can't locate singularity image #82

Open
PBMCHUGA opened this issue Feb 2, 2023 · 9 comments
Open

SAMTOOLS_FAIDX can't locate singularity image #82

PBMCHUGA opened this issue Feb 2, 2023 · 9 comments
Labels
bug Something isn't working

Comments

@PBMCHUGA
Copy link

PBMCHUGA commented Feb 2, 2023

Description of the bug

Hello,

The process SAMTOOLS_FAIDX is not able to locate the singularity image :

FATAL: could not open image /data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif : failed to retrieve path for /data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif : lstat /data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif : no such file or directory

However there is a singularity image containing samtools at this specific location available from any node on the cluster (Managed with slurm)

Image location is precised like this in the config files :

withName: 'SAMTOOLS_FAIDX' {
	container = '/data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif '
}

Command used and terminal output

No response

Relevant files

No response

System information

No response

@PBMCHUGA PBMCHUGA added the bug Something isn't working label Feb 2, 2023
@maxulysse
Copy link
Member

@PBMCHUGA and you have the issue only with this process?

@PBMCHUGA
Copy link
Author

PBMCHUGA commented Feb 2, 2023

I can't say for steps after NFCORE_RNAVAR:RNAVAR:PREPARE_GENOME:SAMTOOLS_FAIDX but at least NFCORE_RNAVAR:RNAVAR:PREPARE_GENOME:GTF2BED process is completing succesfully. You can find attached my nextflow.log file

nextflow.log

@maxulysse
Copy link
Member

can you share what's in your cluster profile or in your custom config file?

@PBMCHUGA
Copy link
Author

PBMCHUGA commented Feb 3, 2023

Hi,

I added

cluster {
    process {
    executor = 'slurm'
        queue = 'brouette'
        /*queue = 'servoz'*/
}

into nextflow.config.txt file.

Also it contains includeConfig 'conf/singularity.config' refering to this file

Already used this process with sarek, rnaseq and rnafusion nf-core pipelines. It worked pretty well.

@maxulysse
Copy link
Member

I can see that you have:

	withName: 'SAMTOOLS.*' {
		container = '/data/projects/ptdom/pipelines/rnavar/sifs/samtools-1.15.1--h1170115_0.sif '
	}

In your conf/singularity.config, so I think it should cover SAMTOOL_FAIDX, can you remove the lines regarding SAMTOOL_FAIDX in your conf/singularity.config and try again?

@PBMCHUGA
Copy link
Author

PBMCHUGA commented Feb 3, 2023

It seems it goes succesfully through the process, after singularity image pulling in the working directory specified with -work-dir

@maxulysse
Copy link
Member

ok, I honestly have no idea why this idea fixed your problem, but I think we can close this issue then, right?

@PBMCHUGA
Copy link
Author

PBMCHUGA commented Feb 6, 2023

Hi,
It is more a workaround than a fix as we can’t have pre-built singularity image for this process. So it will re-pull singularity image at each pipeline run, unless singularity.cacheDir is properly set up. I’ll dive into it to figure out if the problem is linked to our infrastructure.

@maxulysse
Copy link
Member

I see.
I think the proper way to deal with it, is indeed to use singularity.cacheDir or NXF_SINGULARITY_CACHE_DIR.
Which should allow you not to set up such custom files to your singularity files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants