Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: CLI tool for visualizing plans #8

Merged
merged 17 commits into from
Dec 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .eslintrc.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
module.exports = {
parser: '@typescript-eslint/parser',
plugins: ['@typescript-eslint', 'prettier', 'import'],
ignorePatterns: ['node_modules/', 'dist/', 'generated/'],
extends: ['plugin:@typescript-eslint/recommended', 'prettier'],
rules: {
'@typescript-eslint/no-explicit-any': 'off',
'prettier/prettier': 'error',
'import/order': [
'error',
{
'groups': [['builtin', 'external'], ['internal', 'parent', 'sibling', 'index']],
'newlines-between': 'always',
},
],
},
};
12 changes: 0 additions & 12 deletions .eslintrc.js

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Lint

on: [push]
on: [push, pull_request]

jobs:
ESLint:
Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: substraitJS

on: [push, pull_request]

jobs:
test:
runs-on: ubuntu-latest

strategy:
matrix:
node-version: [18]

steps:
- uses: actions/checkout@v3

- name: Initialize and update submodules
run: |
git submodule update --init --recursive

- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version: 18
cache: "npm"

- name: Install dependencies
run: npm ci

- name: Run tests
run: npm run test

26 changes: 25 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

# susbtrait-js
Typescript typings for Substrait specifications. Also provides a parser for translating substrait plans.
Typescript typings for Substrait specifications. Also provides a parser for translating substrait plans and a CLI tool to visualize substrait plans.

Substrait is cross-language specification for data compute operations, composed primarily of:
1. a formal specification
Expand Down Expand Up @@ -29,6 +29,30 @@ Installation includes the following steps:
npm ci
npm run build
```

## Visualization
substrait-JS provides a CLI tool for exporting graph visualization of substrait JSON and binary plans. The tool currently uses the `--experimental-specifier-resolution` flag for module resolution. The visualization functions are the required helper methods for the [substrait-fiddle](https://github.com/voltrondata/substrait-fiddle) tool.

The tool uses [viz-js](https://github.com/mdaines/viz-js) for rendering plots on JSDOM using [Graphviz](https://graphviz.org/)'s DOT language.

Plot generation includes the following steps:
```
// The tool currently requires NodeJS version 18.0.0 for operation
source ~/.nvm/nvm.sh
nvm use 18.0.0

substrait -p ../plan.json -o <output path>
```

## Testing
Jest is used for testing the visualization functionality of the library. To run the test, we need
to use `nvm v18.0.0`
```
source ~/.nvm/nvm.sh
nvm use 18.0.0

npm run test
```

## License
[Apache-2.0 license](https://www.apache.org/licenses/LICENSE-2.0)
Loading