Skip to content

Commit

Permalink
deploy: f316c0e
Browse files Browse the repository at this point in the history
  • Loading branch information
@misialq
misialq committed Dec 16, 2024
1 parent b021738 commit 22fe8ba
Show file tree
Hide file tree
Showing 3 changed files with 123 additions and 17 deletions.
66 changes: 58 additions & 8 deletions _sources/chapters/05_interoperability/export.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ artifacts will contain data in the original format that the underlying tool uses
export data from QIIME 2 and connect it with other tools.

```{warning}
QIIME 2 does not yet support exporting data from the cache. This means that you will need to manually copy the data from the
QIIME 2 does not yet support exporting data from the cache (see below). This means that you will need to manually copy the data from the
cache directory to a location where you can access it with other tools. In our examples, the cache directory is located directly
in the working directory and that is where we will copy the data from. Keep in mind that you should never temper with the files
in the cache directory directly, as this may lead to broken artifacts and failed analyses.
Expand All @@ -31,23 +31,73 @@ in the cache directory directly, as this may lead to broken artifacts and failed
If you have used Kraken 2 to [classify your reads](kraken-reads), you can export the resulting reports from the corresponding
QIIME 2 artifact and visualize them with [Pavian](https://github.com/fbreitwieser/pavian) which will allow you to explore the
taxonomic composition of your samples in an interactive way. To export the Kraken 2 reports, you can use the following commands:

`````{tab-set}
````{tab-item} Direct export
Support for extracting data out of the cache is not yet available but is coming soon! You can track the progress of
the issue [here](https://github.com/qiime2/qiime2/issues/822).
````
````{tab-item} Workaround 1: without using QIIME
```bash
UUID=$(cat ./cache/keys/kraken_reports_reads | grep 'data' | awk '{print $2}')
mkdir exported_reports
cp -r ./cache/data/$UUID/data/* exported_reports/
mkdir ./exported_reports
cp -r ./cache/data/$UUID/data/* ./exported_reports/
```
This will find the UUID of the reports artifact, use it to locate the data within the cache directory, create a directory
for the exported data and copy the files from the cache into it. You can then use those files (within the `exported_reports`
directory) with Pavian. To give it a quick try, navigate to [Pavian's demo site](https://fbreitwieser.shinyapps.io/pavian/)
and upload the exported files.
for the exported data and copy the files from the cache into it.
````
````{tab-item} Workaround 2: with QIIME
```bash
qiime tools cache-fetch \
--cache ./cache \
--key kraken_reports_reads \
--output-path ./kraken_reports_reads.qza
qiime tools export \
--input-path ./kraken_reports_reads.qza \
--output-path ./exported_reports
```
This will first fetch the reports artifact from the cache and then export it to the `exported_reports` directory.
````
`````

Once you got the data into the `exported_reports` directory, you can then use those with Pavian.
To give it a quick try, navigate to [Pavian's demo site](https://fbreitwieser.shinyapps.io/pavian/) and upload the exported files.

## Microbial pangenomics with Anvi'o
Another suite of tools you may be familiar with is the [Anvi'o](http://anvio.org/) platform. One of the workflows that Anvi'o
provides is the microbial pangenomics analysis, which can be used to explore the gene clusters within your samples. You
could export the MAGs obtained from the [binning step](mag-recovery) and use them as input to the `anvi-pan-genome` workflow, as
described [here](https://merenlab.org/2016/11/08/pangenomics-v2/). To export the MAGs, you can use the following command:
`````{tab-set}
````{tab-item} Direct export
Support for extracting data out of the cache is not yet available but is coming soon! You can track the progress of
the issue [here](https://github.com/qiime2/qiime2/issues/822).
````
````{tab-item} Workaround 1: without using QIIME
```bash
UUID=$(cat ./cache/keys/mags | grep 'data' | awk '{print $2}')
mkdir exported_mags
cp -r ./cache/data/$UUID/data/* exported_mags/
mkdir ./exported_mags
cp -r ./cache/data/$UUID/data/* ./exported_mags/
```
This will find the UUID of the MAGs artifact, use it to locate the data within the cache directory, create a directory
for the exported data and copy the files from the cache into it.
````
````{tab-item} Workaround 2: with QIIME
```bash
qiime tools cache-fetch \
--cache ./cache \
--key mags \
--output-path ./mags.qza
qiime tools export \
--input-path ./mags.qza \
--output-path ./exported_mags
```
This will first fetch the MAGs artifact from the cache and then export it to the `exported_mags` directory.
````
`````
72 changes: 64 additions & 8 deletions chapters/05_interoperability/export.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -351,7 +351,7 @@ <h2> Contents </h2>
export data from QIIME 2 and connect it with other tools.</p>
<div class="admonition warning">
<p class="admonition-title">Warning</p>
<p>QIIME 2 does not yet support exporting data from the cache. This means that you will need to manually copy the data from the
<p>QIIME 2 does not yet support exporting data from the cache (see below). This means that you will need to manually copy the data from the
cache directory to a location where you can access it with other tools. In our examples, the cache directory is located directly
in the working directory and that is where we will copy the data from. Keep in mind that you should never temper with the files
in the cache directory directly, as this may lead to broken artifacts and failed analyses.</p>
Expand All @@ -361,27 +361,83 @@ <h2>Visualizing Kraken 2 reports with Pavian<a class="headerlink" href="#visuali
<p>If you have used Kraken 2 to <a class="reference internal" href="../03_taxonomic_classification/reads.html#kraken-reads"><span class="std std-ref">classify your reads</span></a>, you can export the resulting reports from the corresponding
QIIME 2 artifact and visualize them with <a class="reference external" href="https://github.com/fbreitwieser/pavian">Pavian</a> which will allow you to explore the
taxonomic composition of your samples in an interactive way. To export the Kraken 2 reports, you can use the following commands:</p>
<div class="sd-tab-set docutils">
<input checked="checked" id="sd-tab-item-0" name="sd-tab-set-0" type="radio">
<label class="sd-tab-label" for="sd-tab-item-0">
Direct export</label><div class="sd-tab-content docutils">
<p>Support for extracting data out of the cache is not yet available but is coming soon! You can track the progress of
the issue <a class="reference external" href="https://github.com/qiime2/qiime2/issues/822">here</a>.</p>
</div>
<input id="sd-tab-item-1" name="sd-tab-set-0" type="radio">
<label class="sd-tab-label" for="sd-tab-item-1">
Workaround 1: without using QIIME</label><div class="sd-tab-content docutils">
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">UUID</span><span class="o">=</span><span class="k">$(</span>cat<span class="w"> </span>./cache/keys/kraken_reports_reads<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span><span class="s1">&#39;data&#39;</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>awk<span class="w"> </span><span class="s1">&#39;{print $2}&#39;</span><span class="k">)</span>
mkdir<span class="w"> </span>exported_reports
cp<span class="w"> </span>-r<span class="w"> </span>./cache/data/<span class="nv">$UUID</span>/data/*<span class="w"> </span>exported_reports/
mkdir<span class="w"> </span>./exported_reports
cp<span class="w"> </span>-r<span class="w"> </span>./cache/data/<span class="nv">$UUID</span>/data/*<span class="w"> </span>./exported_reports/
</pre></div>
</div>
<p>This will find the UUID of the reports artifact, use it to locate the data within the cache directory, create a directory
for the exported data and copy the files from the cache into it. You can then use those files (within the <code class="docutils literal notranslate"><span class="pre">exported_reports</span></code>
directory) with Pavian. To give it a quick try, navigate to <a class="reference external" href="https://fbreitwieser.shinyapps.io/pavian/">Pavian’s demo site</a>
and upload the exported files.</p>
for the exported data and copy the files from the cache into it.</p>
</div>
<input id="sd-tab-item-2" name="sd-tab-set-0" type="radio">
<label class="sd-tab-label" for="sd-tab-item-2">
Workaround 2: with QIIME</label><div class="sd-tab-content docutils">
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>qiime<span class="w"> </span>tools<span class="w"> </span>cache-fetch<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--cache<span class="w"> </span>./cache<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--key<span class="w"> </span>kraken_reports_reads<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--output-path<span class="w"> </span>./kraken_reports_reads.qza

qiime<span class="w"> </span>tools<span class="w"> </span><span class="nb">export</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--input-path<span class="w"> </span>./kraken_reports_reads.qza<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--output-path<span class="w"> </span>./exported_reports
</pre></div>
</div>
<p>This will first fetch the reports artifact from the cache and then export it to the <code class="docutils literal notranslate"><span class="pre">exported_reports</span></code> directory.</p>
</div>
</div>
<p>Once you got the data into the <code class="docutils literal notranslate"><span class="pre">exported_reports</span></code> directory, you can then use those with Pavian.
To give it a quick try, navigate to <a class="reference external" href="https://fbreitwieser.shinyapps.io/pavian/">Pavian’s demo site</a> and upload the exported files.</p>
</section>
<section id="microbial-pangenomics-with-anvi-o">
<h2>Microbial pangenomics with Anvi’o<a class="headerlink" href="#microbial-pangenomics-with-anvi-o" title="Link to this heading">#</a></h2>
<p>Another suite of tools you may be familiar with is the <a class="reference external" href="http://anvio.org/">Anvi’o</a> platform. One of the workflows that Anvi’o
provides is the microbial pangenomics analysis, which can be used to explore the gene clusters within your samples. You
could export the MAGs obtained from the <a class="reference internal" href="../02_mag_reconstruction/reconstruction.html#mag-recovery"><span class="std std-ref">binning step</span></a> and use them as input to the <code class="docutils literal notranslate"><span class="pre">anvi-pan-genome</span></code> workflow, as
described <a class="reference external" href="https://merenlab.org/2016/11/08/pangenomics-v2/">here</a>. To export the MAGs, you can use the following command:</p>
<div class="sd-tab-set docutils">
<input checked="checked" id="sd-tab-item-3" name="sd-tab-set-1" type="radio">
<label class="sd-tab-label" for="sd-tab-item-3">
Direct export</label><div class="sd-tab-content docutils">
<p>Support for extracting data out of the cache is not yet available but is coming soon! You can track the progress of
the issue <a class="reference external" href="https://github.com/qiime2/qiime2/issues/822">here</a>.</p>
</div>
<input id="sd-tab-item-4" name="sd-tab-set-1" type="radio">
<label class="sd-tab-label" for="sd-tab-item-4">
Workaround 1: without using QIIME</label><div class="sd-tab-content docutils">
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">UUID</span><span class="o">=</span><span class="k">$(</span>cat<span class="w"> </span>./cache/keys/mags<span class="w"> </span><span class="p">|</span><span class="w"> </span>grep<span class="w"> </span><span class="s1">&#39;data&#39;</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>awk<span class="w"> </span><span class="s1">&#39;{print $2}&#39;</span><span class="k">)</span>
mkdir<span class="w"> </span>exported_mags
cp<span class="w"> </span>-r<span class="w"> </span>./cache/data/<span class="nv">$UUID</span>/data/*<span class="w"> </span>exported_mags/
mkdir<span class="w"> </span>./exported_mags
cp<span class="w"> </span>-r<span class="w"> </span>./cache/data/<span class="nv">$UUID</span>/data/*<span class="w"> </span>./exported_mags/
</pre></div>
</div>
<p>This will find the UUID of the MAGs artifact, use it to locate the data within the cache directory, create a directory
for the exported data and copy the files from the cache into it.</p>
</div>
<input id="sd-tab-item-5" name="sd-tab-set-1" type="radio">
<label class="sd-tab-label" for="sd-tab-item-5">
Workaround 2: with QIIME</label><div class="sd-tab-content docutils">
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>qiime<span class="w"> </span>tools<span class="w"> </span>cache-fetch<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--cache<span class="w"> </span>./cache<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--key<span class="w"> </span>mags<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--output-path<span class="w"> </span>./mags.qza

qiime<span class="w"> </span>tools<span class="w"> </span><span class="nb">export</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--input-path<span class="w"> </span>./mags.qza<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--output-path<span class="w"> </span>./exported_mags
</pre></div>
</div>
<p>This will first fetch the MAGs artifact from the cache and then export it to the <code class="docutils literal notranslate"><span class="pre">exported_mags</span></code> directory.</p>
</div>
</div>
</section>
</section>

Expand Down
Loading

0 comments on commit 22fe8ba

Please sign in to comment.