adapter refactoring (#21)

* new: initial cleanup commit * new: from_graph & from_collections * temporary: test script using fraud_dump data * Update arangoDB_networkx_adapter.py * checkpoint * Delete imdb.py * cleanup & increased abstraction * Update README.md * Update main.py * Update main.py * Update dgl_arangoDB_networkx_adapter.py * Update main.py * checkpoint * checkpoint * more progress * Update main.py * cleanup * update: from_arango_graph() & from_collections() abstraction * cleanup * checkpoint * temporary nuke: dgl (post-ML-sync discussion) * cleanup * Update README.md * checkpoint * second checkpoint * new: imdb data dump * new: preparing tests directory * update: setup * checkpoint pre-ML-sync * cleanup * Update README.md * bump version to 1.0.0 * leaving jupyter notebooks alone for now (will be addressed in separate PR) * Update README.md * Update arangoDB_networkx_adapter.py * Update oasis.py * delete creds.data & oasis.py to keep things consistent, will pull from from https://github.com/arangodb/interactive_tutorials/tree/oasis_connector * Update arangorestore * refactor user responsibility * Delete arangorestore * new: overrides dependency For python 3.6/3.7 support * new: create_arangodb_graph with overwrite mode * cleanup * Update README.md * Update requirements.txt * Update setup.py * cleanup * Update ArangoDB_NetworkxAdapter.ipynb * Update setup.py * Update ArangoDB_NetworkxAdapter.ipynb * Create build.yml * Update test_arangoDB_networkx_adapter.py * Update build.yml * Update build.yml * Update build.yml * new: arangorestore exec for running tests * Delete docker-compose.yml switching to oasis * cleanup * cleanup * Update conftest.py * Update conftest.py * Update build.yml * Update conftest.py * Update build.yml * Update build.yml * Update build.yml * Update build.yml * bump * Update conftest.py * Update conftest.py * Update conftest.py * Update conftest.py * bump * cleanup * cleanup * Update .gitignore * fix: typos * fix: _prepare method names * new: file renaming, adbnx_controller, notebook changes * Update ArangoDB_NetworkxAdapter.ipynb * fix: notebook typo * update: identify & keyify edge implementations * Update ArangoDB_NetworkxAdapter.ipynb * Update build.yml * Update build.yml * Update build.yml * Update build.yml * Update build.yml * Update build.yml * Update build.yml * new: badges * Update adbnx_adapter.py * Update build.yml * Update build.yml * attempt to break pytest * break test again * pass tests * Update build.yml * Update build.yml * Update build.yml * Update build.yml * Update build.yml * python 3.10 support, cleanup * Update README.md * Update README.md * cleanup * new: karate_nx_graph unit test * Update README.md * cleanup: test_create_arangodb_graph * cleanup test & notebook * fix: test_full_cycle_from_networkx
arangoml · Nov 17, 2021 · a93fec4 · a93fec4
1 parent 3b73e51
commit a93fec4
Show file tree

Hide file tree

Showing 30 changed files with 1,597 additions and 487 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -0,0 +1,32 @@
+name: build
+on: [push]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: ./adbnx_adapter
+    strategy:
+      matrix:
+        python: ['3.6', '3.7', '3.8', '3.9', '3.10']
+    name: Python ${{ matrix.python }}
+    env:
+      COVERALLS_REPO_TOKEN: ${{secrets.COVERALLS_REPO_TOKEN}}
+      GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python }}
+      - name: Lint with Black
+        uses: psf/black@stable
+        with:
+          options: "--check --verbose --diff --color"
+          src: "adbnx_adapter"
+      - name: Install dependencies
+        run: pip install -e . pytest pytest-cov coveralls
+      - name: Run pytest
+        run: |
+          pytest --cov=adbnx_adapter --cov-report term-missing -v --color=yes --no-cov-on-fail --code-highlight=yes
+          coveralls
diff --git a/.gitignore b/.gitignore
@@ -9,4 +9,4 @@ adbnx_adapter/dist/
 # Remove the build directory from repo
 adbnx_adapter/build/
 adbnx_adapter/*.egg-info
-
+.vscode
diff --git a/README.md b/README.md
@@ -1,61 +1,45 @@
 # ArangoDB-Networkx Adapter
+[![build](https://github.com/arangoml/networkx-adapter/actions/workflows/build.yml/badge.svg)](https://github.com/arangoml/networkx-adapter/actions/workflows/build.yml)
+[![Coverage Status](https://coveralls.io/repos/github/arangoml/networkx-adapter/badge.svg)](https://coveralls.io/github/arangoml/networkx-adapter)
 
-<center>
-<img src="examples/assets/logos/networkx_logo.svg" width=50% >
-</center>
-<center>
-<img src="examples/assets/logos/ArangoDB_logo.png" width=50% >
-</center>
-
-The ArangoDB-Networkx Adapter export Graphs from ArangoDB, a multi-model Graph Database into NetworkX, the swiss army knife for graph analysis with python.
-
-
-##  Quickstart
-
-To get started quickly you just use this setup free jupyter notebook: <a href="https://colab.research.google.com/github/arangoml/networkx-adapter/blob/master/examples/ArangoDB_NetworkxAdapter.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
-
-To get started in custom code:
-```bash
-pip install adbnx_adapter networkx matplotlib python-arango
-```
+[![PyPI version badge](https://img.shields.io/pypi/v/adbnx-adapter)](https://pypi.org/project/adbnx-adapter/)
+[![Python versions badge](https://img.shields.io/pypi/pyversions/adbnx-adapter)](https://github.com/arangoml/networkx-adapter)
 
-``` python
-import networkx as nx
-from adbnx_adapter.arangoDB_networkx_adapter import ArangoDB_Networkx_Adapter
+[![License](https://img.shields.io/github/license/arangoml/networkx-adapter)](https://github.com/arangoml/networkx-adapter/blob/master/LICENSE)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![Downloads per month](https://img.shields.io/pypi/dm/adbnx-adapter)](https://pypi.org/project/adbnx-adapter/)
 
-# Specify the connection to the ArangoDB Database
-con = {'dbName': 'YOURDBNAME',
- 'username': 'YOURUSERNAME',
- 'password': 'YOURPASSOWRD',
- 'hostname': 'instance.arangodb.cloud',
- 'port': 8529}
-
-# Create Adapter instance
-ma = ArangoDB_Networkx_Adapter(conn = con)
-
-# Specify attributes to be imported
-attributes = { 'vertexCollections':
-                                  {'account': {'Balance', 'account_type', 'customer_id', 'rank'}},\
-                               'edgeCollections' :
-                                  {'accountHolder': {'_from', '_to'},\
-                                   'transaction': {'_from', '_to'}}}
-
-# Export networkX graph                                  
-g = ma.create_networkx_graph(graph_name = 'FraudDetection',  graph_attributes = attributes)
+<center>
+    <span>
+        <img src="examples/assets/logos/ArangoDB_logo.png" width=45% >
+        <img src="examples/assets/logos/networkx_logo.svg" width=45% >
+    </span>
+</center>
+<br></br>
 
-# You can also provide valid Python-Arango AQL query options to the command above, like such:
-# g = ma.create_networkx_graph(graph_name = 'FraudDetection',  graph_attributes = attributes, ttl=1000, stream=True)
 
-# Use networkX
-nx.draw(g, with_labels=True)
-```
+The ArangoDB-Networkx Adapter exports Graphs from ArangoDB, a multi-model Graph Database, into NetworkX, the swiss army knife for graph analysis with python, and vice-versa.
 
-# Introduction
+## About NetworkX
 
 Networkx is a commonly used tool for analysis of network-data. If your analytics use cases require the use of all your graph data, for example, to summarize graph structure, or answer global path traversal queries, then using the ArangoDB Pregel API is recommended. If your analysis pertains to a subgraph, then you may be interested in getting the Networkx representation of the subgraph for one of the following reasons:
 
     1. An algorithm for your use case is available in Networkx.
     2. A library that you want to use for your use case works with Networkx Graphs as input.
 
+##  Quickstart
+
+Get Started on Colab: <a href="https://colab.research.google.com/github/arangoml/networkx-adapter/blob/master/examples/ArangoDB_NetworkxAdapter.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>
+
+##  Development & Testing
+
+Prerequisite: `arangorestore` must be installed
 
-Check the DGL folder for an implementation of a Networkx-Adapter for the Deep Graph Library.
+1. `git clone https://github.com/arangoml/networkx-adapter.git`
+2. `cd networkx-adapter`
+3. `python -m venv .venv`
+4. `source .venv/bin/activate` (MacOS) or `.venv/scripts/activate` (Windows)
+5. `cd adbnx_adapter`
+6. `pip install -e . pytest`
+7. `pytest`
+    * If you encounter `ModuleNotFoundError`, try closing & relaunching your virtual environment by running `deactivate` in your terminal & restarting from Step 4.
diff --git a/adbnx_adapter/.gitignore b/adbnx_adapter/.gitignore
@@ -6,3 +6,4 @@
 */.ipynb_checkpoints/*
 #OSX DS_Store
 **/*.DS_Store
+.venv
diff --git a/adbnx_adapter/MANIFEST.in b/adbnx_adapter/MANIFEST.in
@@ -1 +1 @@
- 
+graft adbnx_adapter
diff --git a/adbnx_adapter/__init__.py b/adbnx_adapter/__init__.py
diff --git a/adbnx_adapter/adbnx_adapter/abc.py b/adbnx_adapter/adbnx_adapter/abc.py
@@ -0,0 +1,107 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""
+Created on Thu Mar 26 11:38:31 2020
+
+@author: Rajiv Sambasivan
+@author: Joerg Schad
+@author: Anthony Mahanna
+"""
+
+from abc import ABC
+
+
+class ADBNX_Adapter(ABC):
+    def __init__(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def create_networkx_graph(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def create_networkx_graph_from_arangodb_collections(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def create_networkx_graph_from_arangodb_graph(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def create_arangodb_graph(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def __validate_attributes(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def __fetch_arangodb_docs(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def __insert_arangodb_vertex(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def __insert_arangodb_edge(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def __insert_networkx_node(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def __insert_networkx_edge(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    @property
+    def CONNECTION_ATRIBS(self):
+        return {"hostname", "username", "password", "dbName"}
+
+    @property
+    def GRAPH_ATRIBS(self):
+        return {"vertexCollections", "edgeCollections"}
+
+
+class ADBNX_Controller(ABC):
+    def __init__(self):
+        raise NotImplementedError()  # pragma: no cover
+
+    def _prepare_adb_vertex(self, vertex: dict, collection: str):
+        raise NotImplementedError()  # pragma: no cover
+
+    def _prepare_adb_edge(self, edge: dict, collection: str):
+        raise NotImplementedError()  # pragma: no cover
+
+    def _identify_nx_node(self, id, node: dict, overwrite: bool) -> str:
+        raise NotImplementedError()  # pragma: no cover
+
+    def _identify_nx_edge(
+        self, edge: dict, from_node: dict, to_node: dict, overwrite: bool
+    ) -> str:
+        raise NotImplementedError()  # pragma: no cover
+
+    def _keyify_nx_node(self, id, node: dict, collection: str, overwrite: bool) -> str:
+        raise NotImplementedError()  # pragma: no cover
+
+    def _keyify_nx_edge(
+        self,
+        edge: dict,
+        from_node: dict,
+        to_node: dict,
+        collection: str,
+        overwrite: bool,
+    ):
+        raise NotImplementedError()  # pragma: no cover
+
+    @property
+    def VALID_KEY_CHARS(self):
+        return {
+            "_",
+            "-",
+            ":",
+            ".",
+            "@",
+            "(",
+            ")",
+            "+",
+            ",",
+            "=",
+            ";",
+            "$",
+            "!",
+            "*",
+            "'",
+            "%",
+        }