Merge pull request #7 from bcgsc/release/v5.1.0

Release/v5.1.0

.dockerignore

@@ -1,2 +1,7 @@
 node_modules
 npm-debug.log
+snakemake_logs
+snakemake_data
+logs
+.snakemake
+venv

.eslintrc.json

@@ -101,7 +101,7 @@
             "error",
             "always"
         ],
-        "sort-keys": "warn",
+        "sort-keys": ["warn", "asc", {"caseSensitive": true, "natural": false}],
         "sort-keys-fix/sort-keys-fix": "warn",
         "space-in-parens": [
             "error",

.github/workflows/npm-test.yml

@@ -9,7 +9,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        node: [10]
+        node: [10, 12, 14]
     name: node-${{ matrix.node }}
     steps:
       - uses: actions/checkout@v2

.gitignore

@@ -2,9 +2,11 @@
 .nyc_output
 coverage
 node_modules/
+node_modules.bk
 .orientdb_history
 dist
 *png
+!docs/*png
 *gz
 *zip
 *tar
@@ -32,7 +34,14 @@ logs
 pm2_logs
 errorLog-*
 .~*
-data/asco.json
 data/ligand_actions.json
 *.log
 tmp
+
+# snakemake stuff
+.snakemake
+venv
+snakemake_data
+snakemake_logs
+dag.svg
+dag.png

README.md

@@ -1,6 +1,6 @@
 # GraphKB Loader
 
-![build](https://github.com/bcgsc/pori_graphkb_loader/workflows/build/badge.svg?branch=master)
+![build](https://github.com/bcgsc/pori_graphkb_loader/workflows/build/badge.svg?branch=master) ![Docker Image Version (latest semver)](https://img.shields.io/docker/v/bcgsc/pori-graphkb-loader?label=docker%20image) ![node versions](https://img.shields.io/badge/node-10%20%7C%2012%20%7C%2014-blue)
 
 This package is used to import content from a variety of sources into GraphKB using the API.
 
@@ -12,6 +12,7 @@ This package is used to import content from a variety of sources into GraphKB us
   - [Creating a new Loader](#creating-a-new-loader)
     - [API Loaders](#api-loaders)
     - [File Loaders](#file-loaders)
+- [Initializing GraphKB Content](#initializing-graphkb-content)
 
 Automatic Import modules are provided for a variety of input sources. To Start importing external data, first the GraphKB API
 must already be running. Then the command line interface can be used for upload. Get the help menu
@@ -21,6 +22,12 @@ detailing the commands and required inputs as follows
 node bin/load.js -- --help
 ```
 
+or using docker
+
+```bash
+docker run bcgsc/pori-graphkb-loader --help
+```
+
 ## Loaders
 
 ### Ontologies
@@ -50,7 +57,8 @@ node bin/load.js -- --help
 - [DGIdb](./src/dgidb)
 - [DoCM](./src/docm)
 - [OncoKB](./src/oncokb)
-- [tcgaFusions](./src/tcgaFusions)
+- [PMC4468049](./src/PMC4468049)
+- [PMC4232638](./src/PMC4232638)
 
 ## Guidelines for Developers
 
@@ -59,8 +67,8 @@ node bin/load.js -- --help
 Clone the repository
 
 ```bash
-git clone https://svn.bcgsc.ca/bitbucket/scm/dat/knowledgebase_importer.git
-cd knowledgebase_importer
+git clone https://github.com/bcgsc/pori_graphkb_loader.git
+cd pori_graphkb_loader
 git checkout develop
 ```
 
@@ -113,3 +121,39 @@ export is called `uploadFile` and accepts an additional argument. For example se
  */
 const uploadFile = async ({ filename, conn }) => {
 ```
+
+## Initializing GraphKB Content
+
+For convenience, a snakemake workflow is included to run all available loaders in an optimal order
+to initialize the content in a new instance of GraphKB. This is done via python snakemake. To set
+up snakemake in a virtual environment run the following
+
+```bash
+python3 -m venv venv
+source venv/bin/activate
+pip install -U pip setuptools
+pip install snakemake
+```
+
+Then the workflow can be run as follows (single core by default but can be adjusted depending on
+your server settings)
+
+```bash
+snakemake -j 1
+```
+
+![default workflow](./docs/basic_workflow.png)
+
+The COSMIC and DrugBank options require licensing and are therefore not run by default. If you have
+a license to use them then you can include one or both of them by providing email and password
+as config parameters
+
+```bash
+snakemake -j 1 \
+  --config drugbank_email="YOUR EMAIL" \
+  drugbank_password="YOUR PASSWORD" \
+  cosmic_email="YOUR EMAIL" \
+  cosmic_password="YOUR PASSWORD"
+```
+
+![full workflow](./docs/full_workflow.png)