Merge pull request #37 from sbslee/0.20.0-dev

0.20.0 dev

CHANGELOG.rst

@@ -1,6 +1,27 @@
 Changelog
 *********
 
+0.20.0 (2021-08-07)
+-------------------
+
+* Add new method :meth:`pymaf.MafFrame.matrix_waterfall_matched`.
+* Add new method :meth:`pymaf.MafFrame.plot_waterfall_matched`.
+* Add new method :meth:`pymaf.MafFrame.plot_tmb_matched`.
+* Add new method :meth:`pymaf.MafFrame.plot_mutated_matched`.
+* Add new method :meth:`pymaf.MafFrame.plot_oncoplot_matched`.
+* Deprecate method :meth:`pymaf.MafFrame.legend_handles`.
+* Add new method :meth:`common.legend_handles`.
+* Deprecate classes :class:`pyvcf.AnnFrame` and :class:`pymaf.AnnFrame`. Add new class :class:`common.AnnFrame`.
+* Rename :meth:`common.file2list` method to :meth:`convert_file2list`.
+* Add new method :meth:`common.convert_num2cat`.
+* Add new command :command:`fuc-undetm`.
+* Add new method :meth:`common.plot_annot_matched`.
+* Add new argument ``sheet`` to :command:`fuc-demux` command.
+* Add new class :class:`common.Variant`.
+* Add new method :meth:`pyvcf.rescue_filtered_variants`.
+* Add new arguments ``a_size`` and ``b_size`` to :meth:`pymaf.MafFrame.plot_regplot` method.
+* Rename ``hue`` and ``hue_order`` arguments in plotting methods to ``group_col`` and ``group_order``, respectively.
+
 0.19.0 (2021-07-31)
 -------------------
 
@@ -66,7 +87,7 @@ Changelog
 * Factor out ``proportion`` mode of :meth:`pymaf.MafFrame.plot_snvcls` method to new method :meth:`pymaf.MafFrame.plot_snvclsp`.
 * Factor out ``samples`` mode of :meth:`pymaf.MafFrame.plot_snvcls` method to new method :meth:`pymaf.MafFrame.plot_snvclss`.
 * Factor out ``titv`` mode of :meth:`pymaf.MafFrame.plot_snvcls` method to new method :meth:`pymaf.MafFrame.plot_titv`.
-* Deprecate :meth:`pymaf.MafFrame.plot_snvcls` method.
+* Deprecate method :meth:`pymaf.MafFrame.plot_snvcls`.
 * Add new argument ``hue_order`` to :meth:`pyvcf.VcfFrame.plot_hist` method.
 * Update aesthetic aspect of :meth:`pymaf.MafFrame.plot_oncoplot` method.
 * Add new argument ``width`` to :meth:`pymaf.MafFrame.plot_tmb` method.
@@ -90,10 +111,10 @@ Changelog
 * Add :meth:`pymaf.MafFrame.filter_annot` method.
 * Add :meth:`pymaf.AnnFrame.sorted_samples` method.
 * Fix bug in :meth:`pymaf.MafFrame.to_frame` method.
-* Deprecate :meth:`pyvep.filter_lof/clinsig` methods.
+* Deprecate methods :meth:`pyvep.filter_lof/clinsig`.
 * Update :meth:`pymaf.MafFrame.from_vcf` method to extract genotype keys (e.g. DP, AD, AF).
 * Update :command:`bam_slice` and :command:`bam_rename` commands.
-* Deprecate :meth:`pybam.rename` method.
+* Deprecate method :meth:`pybam.rename`.
 
 0.14.0 (2021-06-20)
 -------------------
@@ -146,7 +167,7 @@ Changelog
 * Add new method :meth:`pymaf.MafFrame.to_string`.
 * Update :command:`maf_oncoplt` command.
 * Add new method :meth:`pyvcf.VcfFrame.filter_qual`.
-* Deprecate :meth:`pymaf.plot_legend` method and add :meth:`pymaf.legend_handles` method.
+* Deprecate method :meth:`pymaf.plot_legend` and add :meth:`pymaf.legend_handles` method.
 * Add new methods :meth:`pymaf.AnnFrame.legend_handles/plot_annot`.
 * Add new method :meth:`pyvcf.VcfFrame.expand`.
 * Rename methods :meth:`pyvcf.gt_missing/haspolyp` to :meth:`pyvcf.gt_miss/polyp`.
@@ -157,10 +178,10 @@ Changelog
 ------------------
 
 * Add new submodule ``pymaf``.
-* Deprecate :meth:`pyvcf.read_file` method and add :meth:`pyvcf.VcfFrame.from_file` method.
-* Deprecate :meth:`pybed.read_file` method and add :meth:`pybed.BedFrame.from_file` method.
-* Deprecate :meth:`pyfq.read_file` method and add :meth:`pyfq.FqFrame.from_file` method.
-* Deprecate :meth:`pycov.read_file` method and add :meth:`pycov.CovFrame.from_file` method.
+* Deprecate method :meth:`pyvcf.read_file` and add :meth:`pyvcf.VcfFrame.from_file` method.
+* Deprecate method :meth:`pybed.read_file` and add :meth:`pybed.BedFrame.from_file` method.
+* Deprecate method :meth:`pyfq.read_file` and add :meth:`pyfq.FqFrame.from_file` method.
+* Deprecate method :meth:`pycov.read_file` and add :meth:`pycov.CovFrame.from_file` method.
 * Add new method :meth:`common.parse_region`.
 * Add new commands :command:`maf_oncoplt/vcf2maf`.
 
@@ -184,7 +205,7 @@ Changelog
 * Update :meth:`pyvep.filter_clinsig` method.
 * Update :meth:`pyvep.filter_impact` method.
 * Add ``as_nan`` argument to :meth:`pyvcf.VcfFrame.markmiss_ad/af/dp` methods.
-* Deprecate :meth:`pyvcf.update` method.
+* Deprecate method :meth:`pyvcf.update`.
 * Add new methods :meth:`pyvcf.row_updateinfo/parseinfo`.
 * The ``fuc`` package is now available on `Bioconda <https://anaconda.org/bioconda/fuc>`__.
 
@@ -211,22 +232,22 @@ Changelog
 
 * Update Read the Docs.
 * Add new methods to :class:`pyvcf.VcfFrame` class.
-* :issue:`6`: Add ``sphinx.ext.linkcode`` extension to Read the Docs.
+* :issue:`6`: Add new extension ``sphinx.ext.linkcode`` to Read the Docs.
 
 0.3.2 (2021-04-30)
 ------------------
 
 * Rename ``snpeff`` submodule to ``pysnpeff``.
 * Add new submodule ``pyvep``.
 * Update :class:`pyvcf.VcfFrame` class.
-* Add ``autodocsumm`` extension to Read the Docs.
+* Add new extension ``autodocsumm`` to Read the Docs.
 * Add contents to Read the Docs.
 
 0.2.0 (2021-04-26)
 ------------------
 
 * :issue:`2`: Fix Read the Docs automodule not working properly.
-* :issue:`3`: Add ``sphinx-issues`` extension to Read the Docs.
+* :issue:`3`: Add new extension ``sphinx-issues`` to Read the Docs.
 * Rename submodules ``BedFrame``, ``FastqFrame``, and ``VcfFrame`` to ``pybed``, ``pyfq``, and ``pyvcf``, respectively.
 * Add new methods to ``pyvcf`` submodule.
 * Add new methods to :class:`pyvcf.VcfFrame` class.

README.rst

@@ -28,7 +28,7 @@ Introduction
 
 The main goal of the fuc package (pronounced "eff-you-see") is to wrap some of the most **f**\ requently **u**\ sed **c**\ ommands in the field of bioinformatics into one place.
 
-The package supports both command line interface (CLI) and application programming interface (API) whose documentations are available at the `Read the Docs <https://sbslee-fuc.readthedocs.io/en/latest/>`_.
+The package is written in Python, and supports both command line interface (CLI) and application programming interface (API) whose documentations are available at the `Read the Docs <https://sbslee-fuc.readthedocs.io/en/latest/>`_.
 
 Currently, fuc can be used to analyze, summarize, visualize, and manipulate the following file formats:
 
@@ -97,7 +97,7 @@ Finally, you can clone the GitHub repository and then install fuc locally:
 
 The nice thing about this approach is that you will have access to development versions that are not available in Anaconda or PyPI. For example, you can access a development branch with the ``git checkout`` command. When you do this, please make sure your environment already has all the dependencies installed.
 
-Getting Help
+Getting help
 ============
 
 For detailed documentations on the fuc package's CLI and API, please refer to the `Read the Docs <https://sbslee-fuc.readthedocs.io/en/latest/>`_.
@@ -111,30 +111,32 @@ For getting help on the fuc CLI:
    
    positional arguments:
      COMMAND
-       bam-head     [BAM] Print the header of a SAM/BAM/CRAM file.
-       bam-index    [BAM] Index a SAM/BAM/CRAM file.
-       bam-rename   [BAM] Rename the samples in a SAM/BAM/CRAM file.
-       bam-slice    [BAM] Slice a SAM/BAM/CRAM file.
-       bed-intxn    [BED] Find the intersection of two or more BED files.
-       bed-sum      [BED] Summarize a BED file.
-       fq-count     [FASTQ] Count sequence reads in FASTQ files.
-       fq-sum       [FASTQ] Summarize a FASTQ file.
-       fuc-compf    [FUC] Compare the contents of two files.
-       fuc-demux    [FUC] Parse the Reports directory from bcl2fastq.
-       fuc-exist    [FUC] Check whether certain files exist.
-       fuc-find     [FUC] Find all filenames matching a specified pattern recursively.
-       maf-maf2vcf  [MAF] Convert a MAF file to a VCF file.
-       maf-oncoplt  [MAF] Create an oncoplot with a MAF file.
-       maf-sumplt   [MAF] Create a summary plot with a MAF file.
-       maf-vcf2maf  [MAF] Convert a VCF file to a MAF file.
-       tbl-merge    [TABLE] Merge two table files.
-       tbl-sum      [TABLE] Summarize a table file.
-       vcf-filter   [VCF] Filter a VCF file.
-       vcf-merge    [VCF] Merge two or more VCF files.
-       vcf-rename   [VCF] Rename the samples in a VCF file.
-       vcf-slice    [VCF] Slice a VCF file.
-       vcf-vcf2bed  [VCF] Convert a VCF file to a BED file.
-       vcf-vep      [VCF] Filter a VCF file annotated by Ensembl VEP.
+       bam-depth    Compute read depth from SAM/BAM/CRAM files.
+       bam-head     Print the header of a SAM/BAM/CRAM file.
+       bam-index    Index a SAM/BAM/CRAM file.
+       bam-rename   Rename the samples in a SAM/BAM/CRAM file.
+       bam-slice    Slice a SAM/BAM/CRAM file.
+       bed-intxn    Find the intersection of two or more BED files.
+       bed-sum      Summarize a BED file.
+       fq-count     Count sequence reads in FASTQ files.
+       fq-sum       Summarize a FASTQ file.
+       fuc-undetm   Compute top unknown barcodes using undertermined FASTQ from bcl2fastq.
+       fuc-compf    Compare the contents of two files.
+       fuc-demux    Parse the Reports directory from bcl2fastq.
+       fuc-exist    Check whether certain files exist.
+       fuc-find     Find all filenames matching a specified pattern recursively.
+       maf-maf2vcf  Convert a MAF file to a VCF file.
+       maf-oncoplt  Create an oncoplot with a MAF file.
+       maf-sumplt   Create a summary plot with a MAF file.
+       maf-vcf2maf  Convert a VCF file to a MAF file.
+       tbl-merge    Merge two table files.
+       tbl-sum      Summarize a table file.
+       vcf-filter   Filter a VCF file.
+       vcf-merge    Merge two or more VCF files.
+       vcf-rename   Rename the samples in a VCF file.
+       vcf-slice    Slice a VCF file.
+       vcf-vcf2bed  Convert a VCF file to a BED file.
+       vcf-vep      Filter a VCF file annotated by Ensembl VEP.
    
    optional arguments:
      -h, --help     Show this help message and exit.
@@ -166,7 +168,7 @@ For getting help on a specific submodule (e.g. pyvcf):
    >>> from fuc import pyvcf
    >>> help(pyvcf)
 
-CLI Examples
+CLI examples
 ============
 
 **SAM/BAM/CRAM**
@@ -247,7 +249,7 @@ To filter a VCF file annotated by Ensembl VEP:
 
    $ fuc vcf-vep in.vcf 'SYMBOL == "TP53"' > out.vcf
 
-API Examples
+API examples
 ============
 
 **BAM**
@@ -306,13 +308,13 @@ To create various figures for normal-tumor analysis:
     >>> from fuc import common, pyvcf
     >>> common.load_dataset('pyvcf')
     >>> vf = pyvcf.VcfFrame.from_file('~/fuc-data/pyvcf/normal-tumor.vcf')
-    >>> af = pyvcf.AnnFrame.from_file('~/fuc-data/pyvcf/normal-tumor-annot.tsv', 'Sample')
+    >>> af = pyvcf.AnnFrame.from_file('~/fuc-data/pyvcf/normal-tumor-annot.tsv', sample_col='Sample')
     >>> normal = af.df[af.df.Tissue == 'Normal'].index
     >>> tumor = af.df[af.df.Tissue == 'Tumor'].index
     >>> fig, [[ax1, ax2], [ax3, ax4]] = plt.subplots(2, 2, figsize=(10, 10))
     >>> vf.plot_tmb(ax=ax1)
-    >>> vf.plot_tmb(ax=ax2, af=af, hue='Tissue')
-    >>> vf.plot_hist('DP', ax=ax3, af=af, hue='Tissue')
+    >>> vf.plot_tmb(ax=ax2, af=af, group_col='Tissue')
+    >>> vf.plot_hist('DP', ax=ax3, af=af, group_col='Tissue')
     >>> vf.plot_regplot(normal, tumor, ax=ax4)
     >>> plt.tight_layout()
 

docs/cli.rst

@@ -18,30 +18,32 @@ For getting help on the fuc CLI:
    
    positional arguments:
      COMMAND
-       bam-head     [BAM] Print the header of a SAM/BAM/CRAM file.
-       bam-index    [BAM] Index a SAM/BAM/CRAM file.
-       bam-rename   [BAM] Rename the samples in a SAM/BAM/CRAM file.
-       bam-slice    [BAM] Slice a SAM/BAM/CRAM file.
-       bed-intxn    [BED] Find the intersection of two or more BED files.
-       bed-sum      [BED] Summarize a BED file.
-       fq-count     [FASTQ] Count sequence reads in FASTQ files.
-       fq-sum       [FASTQ] Summarize a FASTQ file.
-       fuc-compf    [FUC] Compare the contents of two files.
-       fuc-demux    [FUC] Parse the Reports directory from bcl2fastq.
-       fuc-exist    [FUC] Check whether certain files exist.
-       fuc-find     [FUC] Find all filenames matching a specified pattern recursively.
-       maf-maf2vcf  [MAF] Convert a MAF file to a VCF file.
-       maf-oncoplt  [MAF] Create an oncoplot with a MAF file.
-       maf-sumplt   [MAF] Create a summary plot with a MAF file.
-       maf-vcf2maf  [MAF] Convert a VCF file to a MAF file.
-       tbl-merge    [TABLE] Merge two table files.
-       tbl-sum      [TABLE] Summarize a table file.
-       vcf-filter   [VCF] Filter a VCF file.
-       vcf-merge    [VCF] Merge two or more VCF files.
-       vcf-rename   [VCF] Rename the samples in a VCF file.
-       vcf-slice    [VCF] Slice a VCF file.
-       vcf-vcf2bed  [VCF] Convert a VCF file to a BED file.
-       vcf-vep      [VCF] Filter a VCF file annotated by Ensembl VEP.
+       bam-depth    Compute read depth from SAM/BAM/CRAM files.
+       bam-head     Print the header of a SAM/BAM/CRAM file.
+       bam-index    Index a SAM/BAM/CRAM file.
+       bam-rename   Rename the samples in a SAM/BAM/CRAM file.
+       bam-slice    Slice a SAM/BAM/CRAM file.
+       bed-intxn    Find the intersection of two or more BED files.
+       bed-sum      Summarize a BED file.
+       fq-count     Count sequence reads in FASTQ files.
+       fq-sum       Summarize a FASTQ file.
+       fuc-undetm   Compute top unknown barcodes using undertermined FASTQ from bcl2fastq.
+       fuc-compf    Compare the contents of two files.
+       fuc-demux    Parse the Reports directory from bcl2fastq.
+       fuc-exist    Check whether certain files exist.
+       fuc-find     Find all filenames matching a specified pattern recursively.
+       maf-maf2vcf  Convert a MAF file to a VCF file.
+       maf-oncoplt  Create an oncoplot with a MAF file.
+       maf-sumplt   Create a summary plot with a MAF file.
+       maf-vcf2maf  Convert a VCF file to a MAF file.
+       tbl-merge    Merge two table files.
+       tbl-sum      Summarize a table file.
+       vcf-filter   Filter a VCF file.
+       vcf-merge    Merge two or more VCF files.
+       vcf-rename   Rename the samples in a VCF file.
+       vcf-slice    Slice a VCF file.
+       vcf-vcf2bed  Convert a VCF file to a BED file.
+       vcf-vep      Filter a VCF file annotated by Ensembl VEP.
    
    optional arguments:
      -h, --help     Show this help message and exit.
@@ -53,6 +55,31 @@ For getting help on a specific command (e.g. vcf-merge):
 
    $ fuc vcf-merge -h
 
+bam-depth
+=========
+
+.. code-block:: text
+
+   $ fuc bam-depth -h
+   usage: fuc bam-depth [-h] [--bam PATH [PATH ...]] [--fn PATH] [--bed PATH]
+                        [--region TEXT]
+   
+   This command will compute read depth from the input SAM/BAM/CRAM files.
+   
+   Either the '--bam' or '--fn' argument must be provided, but not both.
+   
+   Usage examples:
+     $ fuc bam-depth --bam 1.bam 2.bam --bed in.bed > out.tsv
+     $ fuc bam-depth --fn bam.list --region chr1:100-200 > out.tsv
+   
+   Optional arguments:
+     -h, --help            Show this help message and exit.
+     --bam PATH [PATH ...]
+                           One or more input files.
+     --fn PATH             File containing one input filename per line.
+     --bed PATH            BED file.
+     --region TEXT         Only report depth in specified region ('chrom:start-end').
+
 bam-head
 ========
 
@@ -230,6 +257,26 @@ fq-sum
    Optional arguments:
      -h, --help  Show this help message and exit.
 
+fuc-undetm
+==========
+
+.. code-block:: text
+
+   $ fuc fuc-undetm -h
+   usage: fuc fuc-undetm [-h] [--count INT] fastq
+   
+   This command will compute top unknown barcodes using undertermined FASTQ from the bcl2fastq or bcl2fastq2 prograrm.
+   
+   Usage examples:
+     $ fuc fuc-undetm Undetermined_S0_R1_001.fastq.gz
+   
+   Positional arguments:
+     fastq        Undertermined FASTQ (zipped or unzipped).
+   
+   Optional arguments:
+     -h, --help   Show this help message and exit.
+     --count INT  Number of top unknown barcodes to return (default: 30).
+
 fuc-compf
 =========
 
@@ -256,25 +303,27 @@ fuc-demux
 .. code-block:: text
 
    $ fuc fuc-demux -h
-   usage: fuc fuc-demux [-h] reports_dir output_dir
+   usage: fuc fuc-demux [-h] [--sheet PATH] reports output
    
    This command will parse the Reports directory from the bcl2fastq or bcl2fastq2 prograrm.
    
-   In the output directory, the command will create four files:
+   After creating the output directory, the command will write the following files:
      - flowcell_summary.csv
      - lane_summary.csv
      - top_unknown_barcodes.csv
      - reports.pdf
    
    Usage examples:
-     $ fuc fuc-demux reports_dir output_dir
+     $ fuc fuc-demux Reports output
+     $ fuc fuc-demux Reports output --sheet SampleSheet.csv
    
    Positional arguments:
-     reports_dir  Reports directory.
-     output_dir   Output directory.
+     reports       Reports directory.
+     output        Output directory (will be created).
    
    Optional arguments:
-     -h, --help   Show this help message and exit.
+     -h, --help    Show this help message and exit.
+     --sheet PATH  SampleSheet.csv file. When provided, samples in the lane_summary.csv file will be sorted in the same order as in the SampleSheet.csv file.
 
 fuc-exist
 =========
@@ -456,9 +505,8 @@ tbl-merge
                         left right
    
    This command will merge two table files using one or more shared columns.
-   It essentially wraps the 'pandas.DataFrame.merge' method from the pandas
-   package. For details on the merging algorithms, please visit the method's
-   documentation page.
+   
+   The command essentially wraps the 'pandas.DataFrame.merge' method from the pandas package. For details on the merging algorithms, please visit the method's documentation page.
    
    Usage examples:
      $ fuc tbl-merge left.tsv right.tsv > merged.tsv