Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: Changed the function load_card_db to load all files globally instead of locally. #66

Merged
merged 5 commits into from
Jun 20, 2024

Conversation

VinzentRisch
Copy link
Collaborator

@VinzentRisch VinzentRisch commented Apr 5, 2024

solves #65

  • Removed the --local flag in the load function and the annotation functions.

Set up an environment for apple silicone

CONDA_SUBDIR=osx-64 mamba create -yn q2-amr \
  -c conda-forge -c bioconda -c qiime2 -c defaults \
  -c https://packages.qiime2.org/qiime2/2024.2/shotgun/released/ \
  qiime2 q2cli q2templates q2-types rgi

conda activate q2-amr
conda config --env --set subdir osx-64

pip install --no-deps --force-reinstall \
  git+https://github.com/misialq/rgi.git@py38-fix \

pip install git+https://github.com/qiime2/qiime2.git  

Run it locally

  1. First, clone the repo and checkout the PR branch:
git clone https://github.com/bokulich-lab/q2-amr.git
cd q2-amr
git fetch origin pull/66/head:pr-66
git checkout pr-66
pip install -e .

solves #51

  • Three transformers have been added from the formats CARDMAGsKmerAnalysisDirectoryFormat, CARDReadsAlleleKmerAnalysisDirectoryFormat and CARDReadsGeneKmerAnalysisDirectoryFormat to qiime2.Metadata. This enables the user to use these formats with the function metadata tabulate to explore the analysis output in a browser.
  • The function tabulate_data has been expanded. in this process also a bug has been fixed for the tabulation of the MAGs tables.

Set up an environment for apple silicone

CONDA_SUBDIR=osx-64 mamba create -yn q2-amr \
  -c conda-forge -c bioconda -c qiime2 -c defaults \
  -c https://packages.qiime2.org/qiime2/2024.2/shotgun/released/ \
  qiime2 q2cli q2templates q2-types rgi

conda activate q2-amr
conda config --env --set subdir osx-64

pip install --no-deps --force-reinstall \
  git+https://github.com/misialq/rgi.git@py38-fix \

pip install git+https://github.com/qiime2/qiime2.git  

Run it locally

  1. First, clone the repo and checkout the PR branch:
git clone https://github.com/bokulich-lab/q2-amr.git
cd q2-amr
git fetch origin pull/71/head:pr-71
git checkout pr-71
pip install -e .
  1. Download test files
    download reads.qza, mags.qza and card_db.qza:

  2. Test it out!

Run all commands

qiime amr annotate-reads-card --i-reads reads.qza --i-card-db card_db.qza --p-threads 10 --output-dir reads_amr_annotation
qiime amr annotate-mags-card --i-mag mags.qza --i-card-db card_db.qza --p-threads 10 --output-dir mags_amr_annotation

@codecov-commenter
Copy link

codecov-commenter commented Apr 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.29%. Comparing base (75d4efa) to head (4207c12).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #66   +/-   ##
=======================================
  Coverage   94.29%   94.29%           
=======================================
  Files          20       20           
  Lines        1491     1491           
=======================================
  Hits         1406     1406           
  Misses         85       85           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@VinzentRisch VinzentRisch linked an issue Apr 11, 2024 that may be closed by this pull request
Copy link
Contributor

@misialq misialq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question, as I don't remember how this DB loading worked exactly: where do the DB files get stored when either of those options is toggled?

@VinzentRisch
Copy link
Collaborator Author

Hi @misialq regarding your question.
When the local flag is added to the loading functions, a directory is created called localDB in your working directory and all database files get copied into that localDB directory. Then when an annotation function is called with the local flag it searches for this localDB in the working directory and uses the database files in that directory for the analysis.

When the files get loaded globally then the path to the files gets stored somewhere by RGI and every time an annotation function is called the database files that were loaded globally are being used for the analysis

So in my case the database files are now only in the database artifacts with these changes in this PR and before the database files were first copied from the database artifacts to a directory called localDB in a temp directory.

I haven't tried this out with parallelisation but i think that will also work as expected.

@ChristosMatzoros
Copy link
Contributor

Hey @VinzentRisch,

Can you add here a link to the documentation regarding the --local parameter?

Thanks

Copy link
Contributor

@misialq misialq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @VinzentRisch, thanks for the explanation! This is a conditional ✅: could you please test that it works also with parallelization? If yes, feel free to go ahead and merge 🚀

Copy link
Contributor

@ChristosMatzoros ChristosMatzoros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @VinzentRisch,
Everything has been tested on my side and is working perfectly! Feel free to merge after you address the request from @misialq. 🚀

@misialq
Copy link
Contributor

misialq commented May 8, 2024

Hey @VinzentRisch, what's up with this PR?

@VinzentRisch
Copy link
Collaborator Author

This works in testing also with parsl parallelisation.
With reads annotation and also with kmer analysis.

@VinzentRisch
Copy link
Collaborator Author

Hey @misialq can you please merge this for me? Thanks!

@misialq misialq merged commit 5ace3a6 into bokulich-lab:main Jun 20, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ENH: Change how the CARD database is loaded with function load_card
4 participants