Base 91
Similar to Base 64 but uses more characters resulting in smaller strings.
diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/404.html b/404.html new file mode 100644 index 00000000..c64f6d85 --- /dev/null +++ b/404.html @@ -0,0 +1,21 @@ + + +
+ + +@hpcc-js/wasm • Docs
@hpcc-js/wasm • Docs
@hpcc-js/wasm • Docs
Base 91 WASM library, similar to Base 64 but uses more characters resulting in smaller strings.
See Base91 for more details.
import { Base91 } from "@hpcc-js/wasm/base91";
+
+const base91 = await Base91.load();
+
+const encoded_data = await base91.encode(data);
+const decoded_data = await base91.decode(encoded_data);
static
load():Promise
<Base91
>
Compiles and instantiates the raw wasm.
INFO
In general WebAssembly compilation is disallowed on the main thread if the buffer size is larger than 4KB, hence forcing load
to be asynchronous;
Promise
<Base91
>
A promise to an instance of the Base91 class.
static
unload():void
Unloades the compiled wasm instance.
void
version():
string
string
The Base91 c++ version
encode(
data
):string
• data: Uint8Array
Data to encode.
string
string containing the Base 91 encoded data
decode(
base91Str
):Uint8Array
• base91Str: string
encoded string
Uint8Array
origonal data
@hpcc-js/wasm • Docs
@hpcc-js/wasm • Docs
DuckDB WASM library, a in-process SQL OLAP Database Management System..
See DuckDB for more details.
import { DuckDB } from "@hpcc-js/wasm/duckdb";
+
+let duckdb = await DuckDB.load();
+const c = await duckdb.db.connect();
+
+const data = [
+ { "col1": 1, "col2": "foo" },
+ { "col1": 2, "col2": "bar" },
+];
+await duckdb.db.registerFileText("rows.json", JSON.stringify(data));
+await c.insertJSONFromPath('rows.json', { name: 'rows' });
+
+const arrowResult = await c.query("SELECT * FROM read_json_auto('rows.json')");
+const result = arrowResult.toArray().map((row) => row.toJSON());
+expect(result.length).to.equal(data.length);
+for (let i = 0; i < result.length; i++) {
+ expect(result[i].col2).to.equal(data[i].col2);
+}
+
+c.close();
db:
AsyncDuckDB
static
load():Promise
<DuckDB
>
Compiles and instantiates the raw wasm.
INFO
In general WebAssembly compilation is disallowed on the main thread if the buffer size is larger than 4KB, hence forcing load
to be asynchronous;
Promise
<DuckDB
>
A promise to an instance of the DuckDB class.
static
unload():void
Unloades the compiled wasm instance.
void
version():
string
string
The DuckDB version
@hpcc-js/wasm • Docs
@hpcc-js/wasm • Docs
Expat XML parser WASM library, provides a simplified wrapper around the Expat XML Parser library.
See libexpat.github.io for c++ details.
import { Expat } from "@hpcc-js/wasm/expat";
+
+const expat = await Expat.load();
+
+const xml = \` \\
+ <root>
+ <child xxx="yyy">content</child>
+ </root>
+\`;
+
+const callback = {
+ startElement(tag, attrs) { console.log("start", tag, attrs); },
+ endElement(tag) { console.log("end", tag); },
+ characterData(content) { console.log("characterData", content); }
+};
+
+expat.parse(xml, callback);
static
load():Promise
<Expat
>
Compiles and instantiates the raw wasm.
INFO
In general WebAssembly compilation is disallowed on the main thread if the buffer size is larger than 4KB, hence forcing load
to be asynchronous;
Promise
<Expat
>
A promise to an instance of the Expat class.
static
unload():void
Unloades the compiled wasm instance.
void
version():
string
string
The Expat c++ version
parse(
xml
,callback
):boolean
Parses the XML with suitable callbacks.
TIP
The IParser.characterData callback method can get called several times for a single tag element.
• xml: string
string containing XML
• callback: IParser
Callback interface
boolean
true
|false
if the XML parse succeeds.
@hpcc-js/wasm • Docs
new StackElement(
tag
,attrs
):StackElement
• tag: string
• attrs: Attributes
readonly
tag:string
readonly
attrs:Attributes
get
content():string
string
appendContent(
content
):void
• content: string
void
@hpcc-js/wasm • Docs
new StackParser():
StackParser
parse(
xml
):Promise
<boolean
>
• xml: string
Promise
<boolean
>
top():
StackElement
startElement(
tag
,attrs
):StackElement
• tag: string
• attrs: Attributes
endElement(
tag
):StackElement
• tag: string
characterData(
content
):void
• content: string
void
@hpcc-js/wasm • Docs
startElement(
tag
,attrs
):void
• tag: string
• attrs: Attributes
void
endElement(
tag
):void
• tag: string
void
characterData(
content
):void
• content: string
void
@hpcc-js/wasm • Docs
Attributes:
object
[key
: string
]: string
By default @hpcc-js/wasm is a modern JavaScript Module (ESM) package, for convenience it also includes Universal Module Definition (UMD) bundles which can be loaded in older browser / build environments.
The simplest way to include this project is via NPM:
npm install --save @hpcc-js/wasm
It can then be referenced within your source code:
import { Base91, Graphviz, Zstd } from "@hpcc-js/wasm";
+
+// Graphviz ---
+async function dot2svg() {
+ const graphviz = await Graphviz.load();
+ console.log("svg: ", graphviz.dot('digraph G { Hello -> World }'));
+}
+
+dot2svg();
+
+// Base91 + Zstd ---
+const text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus. Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed, dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper congue, euismod non, mi. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed non risus. Suspendisse lectus tortor, dignissim sit amet, adipiscing nec, ultricies sed, dolor. Cras elementum ultrices diam. Maecenas ligula massa, varius a, semper congue, euismod non, mi.";
+const data = new TextEncoder().encode(text);
+
+async function compressDecompress() {
+ const zstd = await Zstd.load();
+ const compressed_data = zstd.compress(data);
+ const base91 = await Base91.load();
+ const base91Str = base91.encode(compressed_data);
+
+ const compressed_data2 = base91.decode(base91Str);
+ const data2 = zstd.decompress(compressed_data2);
+ const text2 = new TextDecoder().decode(data2);
+
+ console.log("Text Length: ", text.length);
+ console.log("Compressed Length: ", compressed_data.length);
+ console.log("Base91 Length: ", base91Str.length);
+ console.log("Passed: ", text === text2);
+}
+
+compressDecompress();
Alternatively the @hpcc-js/wasm package can be imported directly within the html page, using a NPM CDN server like unpkg or jsdelivr.
For modern browsers and import
:
<script type="module">
+ import { Graphviz } from "https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/graphviz.js";
+
+ const graphviz = await Graphviz.load();
+ const dot = "digraph G { Hello -> World }";
+ const svg = graphviz.dot(dot);
+ const div = document.getElementById("placeholder");
+ div.innerHTML = graphviz.layout(dot, "svg", "dot");
+</script>
For legacy environments you can load the UMD packages:
<script src="https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/graphviz.umd.js"></script>
+<script>
+ var hpccWasm = window["@hpcc-js/wasm"];
+ hpccWasm.Graphviz.load().then(graphviz => {
+ var dot = "digraph G { Hello -> World }";
+ var svg = graphviz.dot(dot);
+ var div = document.getElementById("placeholder");
+ div.innerHTML = graphviz.layout(dot, "svg", "dot");
+ });
+</script>
To call dot-wasm
without installing:
npx -p @hpcc-js/wasm dot-wasm [options] fileOrDot
To install the global command dot-wasm
via NPM:
npm install --global @hpcc-js/wasm
Usage:
Usage: dot-wasm [options] fileOrDot
+
+Options:
+ --version Show version number [boolean]
+ -K, --layout Set layout engine (circo | dot | fdp | sfdp | neato | osage
+ | patchwork | twopi | nop | nop2). By default, dot is used.
+ -T, --format Set output language to one of the supported formats (svg,
+ dot, json, dot_json, xdot_json, plain, plain-ext). By
+ default, svg is produced.
+ -n, --neato-no-op Sets no-op flag in neato.
+ "-n 1" assumes neato nodes have already been positioned and
+ all nodes have a pos attribute giving the positions. It
+ then performs an optional adjustment to remove node-node
+ overlap, depending on the value of the overlap attribute,
+ computes the edge layouts, depending on the value of the
+ splines attribute, and emits the graph in the appropriate
+ format.
+ "-n 2" Use node positions as specified, with no adjustment
+ to remove node-node overlaps, and use any edge layouts
+ already specified by the pos attribute. neato computes an
+ edge layout for any edge that does not have a pos
+ attribute. As usual, edge layout is guided by the splines
+ attribute.
+ -y, --invert-y By default, the coordinate system used in generic output
+ formats, such as attributed dot, extended dot, plain and
+ plain-ext, is the standard cartesian system with the origin
+ in the lower left corner, and with increasing y coordinates
+ as points move from bottom to top. If the -y flag is used,
+ the coordinate system is inverted, so that increasing
+ values of y correspond to movement from top to bottom.
+ -v Echo GraphViz library version
+ -h, --help Show help [boolean]
+
+Examples:
+ dot-wasm -K neato -T xdot ./input.dot Execute NEATO layout and outputs XDOT
+ format.
@hpcc-js/wasm • Docs
@hpcc-js/wasm • Docs
The Graphviz layout algorithms take descriptions of graphs in a simple text language, and make diagrams in useful formats, such as images and SVG for web pages or display in an interactive graph browser.
Graphviz has many useful features for concrete diagrams, such as options for colors, fonts, tabular node layouts, line styles, hyperlinks, and custom shapes.
See graphviz.org for more details.
import { Graphviz } from "@hpcc-js/wasm/graphviz";
+
+const graphviz = await Graphviz.load();
+
+const dot = "digraph G { Hello -> World }";
+const svg = graphviz.dot(dot);
static
load():Promise
<Graphviz
>
Compiles and instantiates the raw wasm.
INFO
In general WebAssembly compilation is disallowed on the main thread if the buffer size is larger than 4KB, hence forcing load
to be asynchronous;
Promise
<Graphviz
>
A promise to an instance of the Graphviz class.
static
unload():void
Unloades the compiled wasm instance.
void
version():
string
string
The Graphviz c++ version
layout(
dotSource
,outputFormat
,layoutEngine
,options
?):string
Performs layout for the supplied dotSource, see The DOT Language for specification.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• layoutEngine: Engine
= "dot"
The type of layout to perform.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
acyclic(
dotSource
,doWrite
,verbose
):object
acyclic is a filter that takes a directed graph as input and outputs a copy of the graph with sufficient edges reversed to make the graph acyclic. The reversed edge inherits all of the attributes of the original edge. The optional file argument specifies where the input graph is stored; by default.
• dotSource: string
Required - graph definition in DOT language
• doWrite: boolean
= false
Enable output is produced, though the return value will indicate whether the graph is acyclic or not.
• verbose: boolean
= false
Print information about whether the file is acyclic, has a cycle or is undirected.
object
{ acyclic: boolean, num_rev: number, outFile: string }
acyclic
will be true if a cycle was found, num_rev
will contain the number of reversed edges and outFile
will (optionally) contain the output.
acyclic:
boolean
num_rev:
number
outFile:
string
tred(
dotSource
,verbose
,printRemovedEdges
):object
tred computes the transitive reduction of directed graphs, and prints the resulting graphs to standard output. This removes edges implied by transitivity. Nodes and subgraphs are not otherwise affected. The ‘‘meaning’’ and validity of the reduced graphs is application dependent. tred is particularly useful as a preprocessor to dot to reduce clutter in dense layouts. Undirected graphs are silently ignored.
• dotSource: string
Required - graph definition in DOT language
• verbose: boolean
= false
Print additional information.
• printRemovedEdges: boolean
= false
Print information about removed edges.
object
{ out: string, err: string }
.
out:
string
err:
string
unflatten(
dotSource
,maxMinlen
,do_fans
,chainLimit
):string
unflatten is a preprocessor to dot that is used to improve the aspect ratio of graphs having many leaves or disconnected nodes. The usual layout for such a graph is generally very wide or tall. unflatten inserts invisible edges or adjusts the minlen on edges to improve layout compaction.
• dotSource: string
Required - graph definition in DOT language
• maxMinlen: number
= 0
The minimum length of leaf edges is staggered between 1 and len (a small integer).
• do_fans: boolean
= false
Enables the staggering of the -maxMinlen option to fanout nodes whose indegree and outdegree are both 1. This helps with structures such as a -> {w x y } -> b. This option only works if the -maxMinlen flag is set.
• chainLimit: number
= 0
Form disconnected nodes into chains of up to len nodes.
string
A string containing the "unflattened" dotSource.
circo(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the circo layout, is equivalent to layout(dotSource, outputFormat, "circo");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
dot(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the dot layout, is equivalent to layout(dotSource, outputFormat, "dot");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
fdp(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the fdp layout, is equivalent to layout(dotSource, outputFormat, "fdp");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
sfdp(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the sfdp layout, is equivalent to layout(dotSource, outputFormat, "sfdp");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
neato(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the neato layout, is equivalent to layout(dotSource, outputFormat, "neato");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
osage(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the osage layout, is equivalent to layout(dotSource, outputFormat, "osage");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
patchwork(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the patchwork layout, is equivalent to layout(dotSource, outputFormat, "patchwork");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
twopi(
dotSource
,outputFormat
,options
?):string
Convenience function that performs the twopi layout, is equivalent to layout(dotSource, outputFormat, "twopi");
.
• dotSource: string
Required - graph definition in DOT language
• outputFormat: Format
= "svg"
The format of the result.
• options?: Options
Advanced Options for images, files, yInvert and nop.
string
A string containing the calculated layout in the format specified by outputFormat
nop(
dotSource
):string
Convenience function that performs the nop layout, is equivalent to layout(dotSource, "dot", "nop");
.
• dotSource: string
Required - graph definition in DOT language
string
A string containing the "pretty printed" dotSource.
nop2(
dotSource
):string
Convenience function that performs the nop2 layout, is equivalent to layout(dotSource, "dot", "nop2");
.
• dotSource: string
Required - graph definition in DOT language
string
A string containing the "pretty printed" dotSource.
@hpcc-js/wasm • Docs
path:
string
data:
string
@hpcc-js/wasm • Docs
Example: Passing a web hosted Image to GraphViz:
import { Graphviz } from "@hpcc-js/wasm/graphviz";
+
+const graphviz = await Graphviz.load();
+const svg = graphviz.layout('digraph { a[image="https://.../image.png"]; }', "svg", "dot", {
+ images: [{
+ path: "https://.../image.png",
+ width: "272px",
+ height: "92px"
+ }]
+});
+document.getElementById("placeholder").innerHTML = svg;
path:
string
Full URL to image
width:
string
height:
string
@hpcc-js/wasm • Docs
optional
images:Image
[]
optional
files:File
[]
optional
yInvert:boolean
optional
nop:number
@hpcc-js/wasm • Docs
Engine:
"circo"
|"dot"
|"fdp"
|"sfdp"
|"neato"
|"osage"
|"patchwork"
|"twopi"
|"nop"
|"nop2"
Various algorithms for projecting abstract graphs into a space for visualization. See Layout Engines for more details.
@hpcc-js/wasm • Docs
Format:
"svg"
|"dot"
|"json"
|"dot_json"
|"xdot_json"
|"plain"
|"plain-ext"
Various graphic and data formats for end user, web, documents and other applications. See Output Formats for more information.
@hpcc-js/wasm • Docs
@hpcc-js/wasm • Docs
Base class to simplify moving data into and out of Wasm memory.
@hpcc-js/wasm • Docs
ptr:
number
size:
number
@hpcc-js/wasm • Docs
PTR:
number
@hpcc-js/wasm • Docs
@hpcc-js/wasm • Docs
The Zstandard WASM library, provides a simplified wrapper around the Zstandard c++ library.
See Zstandard for more details.
import { Zstd } from "@hpcc-js/wasm/zstd";
+
+const zstd = await Zstd.load();
+
+// Generate some "data"
+const data = new Uint8Array(Array.from({ length: 100000 }, (_, i) => i % 256));
+
+const compressed_data = zstd.compress(data);
+const decompressed_data = zstd.decompress(compressed_data);
static
load():Promise
<Zstd
>
Compiles and instantiates the raw wasm.
INFO
In general WebAssembly compilation is disallowed on the main thread if the buffer size is larger than 4KB, hence forcing load
to be asynchronous;
Promise
<Zstd
>
A promise to an instance of the Zstd class.
static
unload():void
Unloades the compiled wasm instance.
void
version():
string
string
The Zstd c++ version
compress(
data
,compressionLevel
):Uint8Array
• data: Uint8Array
Data to be compressed
• compressionLevel: number
= ...
Compression v Speed tradeoff, when omitted it will default to zstd.defaultCLevel()
which is currently 3.
Uint8Array
Compressed data.
TIP
A note on compressionLevel: The library supports regular compression levels from 1 up o 22. Levels >= 20, should be used with caution, as they require more memory. The library also offers negative compression levels, which extend the range of speed vs. ratio preferences. The lower the level, the faster the speed (at the cost of compression).
decompress(
compressedData
):Uint8Array
• compressedData: Uint8Array
Data to be compressed
Uint8Array
Uncompressed data.
defaultCLevel():
number
number
Default compression level (see notes above above).
minCLevel():
number
number
maxCLevel():
number
number