Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make nf-tests go faster #389

Closed
nschcolnicov opened this issue Aug 26, 2024 · 5 comments
Closed

Make nf-tests go faster #389

nschcolnicov opened this issue Aug 26, 2024 · 5 comments
Assignees

Comments

@nschcolnicov
Copy link
Contributor

Not sure about the priority of this ticket, or if we have enough time to tackle it, but nf-tests run in an OK time when running in CI because they run in parallel, but when having to update snapshots we have to run the sequentially, making them take around 1h. The scope of this ticket is to assess and implement (if possible) the following ideas for making them go faster:

  • Update tests to use bowtie index for faster runs
  • Subset test_full_filter_contamination test profile for running in CI nf-tests
  • Subset all nf-test input files

FYI @grst @atrigila @apeltzer

@nschcolnicov nschcolnicov converted this from a draft issue Aug 26, 2024
@atrigila
Copy link
Contributor

The second point was designed in #387

@apeltzer
Copy link
Member

+1 for using the bowtie indices, that makes sense 👍

@apeltzer
Copy link
Member

@atrigila suggested to run on not all 8 samples in the samplesheet but rather reduce to 3-4 maybe.

@nschcolnicov
Copy link
Contributor Author

nschcolnicov commented Aug 27, 2024

We will implement the following changes:

  • Reduce samplesheet size for all test cases, whenever possible.
  • Remove cleanup parameter from all tests, not doing anything.
  • Nice to have: Have bowtie index as input also for hairpin and mature fasta files, should we move it to igenomes?

Combine the following profiles into one (we saw that they were running redundant modules and subworkflows)

  • test, test_index and test_contamination: Combine and add bowtie_index as input
  • test_technical_repeats, test_no_genome and test_skipfastp

Dont combine these tests

  • test_umi : Add bowtie_index as input
  • test_mirgenedb

@nschcolnicov nschcolnicov self-assigned this Aug 27, 2024
@nschcolnicov
Copy link
Contributor Author

#394

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

5 participants
@apeltzer @atrigila @nschcolnicov and others