Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: add actions to partition and collate mags #137

Closed
wants to merge 23 commits into from

Conversation

Sann5
Copy link
Contributor

@Sann5 Sann5 commented Feb 20, 2024

What's new

  • The action eggnog-diamond-search can take as input FeatureData[MAG] artifacts.
  • This action can be sped up considerably running it on individual parts of the input simultaneously. See this part of the eggnog wiki.
  • Therefore we introduce an action partition-feature-data-mags which splits the input into smaller artifacts on which eggnog-diamond-search can be run individually.
  • We also introduce a collate-feature-data-mags action which does the opposite, namely takes many FeatureData[MAG] and outputs one.
  • We also introduce the same actions but for SampleData[MAGs]; partition-sample-data-mags and collate-sample-data-mags
  • Closes ENH: enable parsl parallelization of eggnog-diamond-search #126

Set up an environment

# For linux: 
# export MY_OS="linux"
# For mac:
export MY_OS="osx" 
wget "https://data.qiime2.org/distro/shotgun/qiime2-shotgun-2023.9-py38-"$MY_OS"-conda.yml"
conda env create -n q2-shotgun --file qiime2-shotgun-2023.9-py38-osx-conda.yml
rm "qiime2-shotgun-2023.9-py38-"$MY_OS"-conda.yml"

Now clone the repo and checkout the PR branch (skip this step if you already have a local copy of the code):

# Remove q2-moshpit so you can install your local version.
conda activate q2-shotgun
conda remove q2-moshpit

# clone from GitHub
git clone [email protected]:bokulich-lab/q2-moshpit.git
cd q2-moshpit
gh pr checkout 137
pip install -e .

Run it locally

FeatrueData[MAG]

  1. Let's get you some data to play with:
cd <somewhere>
wget https://polybox.ethz.ch/index.php/s/WnZanIQECtwPEVu/download -O mags.qza
  1. Test it out!
### Partition
qiime moshpit partition-feature-data-mags --i-mags collated_mags.qza --o-partitioned-mags partitioned_mags --verbose

### Collate
iime moshpit collate-feature-data-mags --i-mags partitioned_mags --o-collated-mags collated_mags.qza --verbose

SampleData[MAGs]

  1. Let's get you some data to play with:
cd <somewhere>
wget https://polybox.ethz.ch/index.php/s/slGyW1uQbyxmyb9/download -O collated_mags.qza
  1. Test it out!
### Partition
qiime moshpit partition-sample-data-mags --i-mags mags.qza --o-partitioned-mags partitioned_mags --verbose --p-num-partitions 3
### Collate
qiime moshpit collate-sample-data-mags --i-mags partitioned_mags --o-collated-mags collated_mags.qza --verbose

Running the tests

pytest -W ignore -vv --pyargs q2_moshpit

@Sann5 Sann5 added the enhancement New feature or request label Feb 20, 2024
@Sann5 Sann5 self-assigned this Feb 20, 2024
@Sann5 Sann5 requested a review from misialq February 20, 2024 14:25
@misialq
Copy link
Contributor

misialq commented Mar 1, 2024

Hey @Sann5, this PR is being replaced by #144 (which builds on top of this one and where @Oddant1 is also directly adding support for eggnog parallelization).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ENH: enable parsl parallelization of eggnog-diamond-search
2 participants