Skip to content

Commit

Permalink
docs: update
Browse files Browse the repository at this point in the history
  • Loading branch information
kmaasrud committed May 18, 2022
1 parent 5f12578 commit 2e359cf
Showing 1 changed file with 47 additions and 20 deletions.
67 changes: 47 additions & 20 deletions public/docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: mdzk documentation
---


This document will go through how mdzk operates and how you can use it in your workflows. Currently, it is quite limited, as mdzk is still under heavy development. If you have any suggestions to things that should be added, [open an issue on our GitHub](https://github.com/mdzk-rs/mdzk/issues/new) describing it and mark it with the `documentation` label. Any contributions are of course welcome.

---
Expand Down Expand Up @@ -38,19 +39,47 @@ An mdzk binary for your system will now be available in `./target/release`.

# Core concepts

At it's core, mdzk operates on a *vault* with *notes* containing optional *internal links*. Let's unpack these terms:
At it's core, mdzk creates a *vault* of *notes* containing *arcs*. Let's unpack these terms:

- A **vault** is a [directed graph](https://en.wikipedia.org/wiki/Directed_graph).
- A **vault** is a [simple directed graph](https://en.wikipedia.org/wiki/Directed_graph).
- **Notes** are the nodes in a vault.
- **Internal links** create edges connecting each note.
- An **arc** is a directed edge connecting two notes.

## Vault

Mathematically we can define a vault as the tuple $V = (N, A)$, where

- $N = \{n_i\}$ is a set of [notes](#note);
- $A \subseteq \{(n_i, n_j)$ where $(n_i, n_j)\in N^2\}$ is a set of [arcs](#arc).[^arcs-def]

mdzk represents a vault in memory as an [adjacency matrix](https://mathworld.wolfram.com/AdjacencyMatrix.html). This gives us very fast lookup times on arcs, and simplifies finding inversed arcs (also called [backlinks](https://en.wikipedia.org/wiki/Backlink)).

[^arcs-def]: Note that this definition allows loops, meaning notes can link to themselves. This does not provide any real practical benefits, but it simplifies how we model the vault and gives some flexibility in linkage.

In the filesystem, a vault is simply a directory with a bunch of Markdown files. You may structure this directory however you want and include any other filetypes - mdzk does not care. Something to keep in mind: mdzk does not have any notion of a directory hierarchy. You may bury a note in a subdirectory of a subdirectory of a subdirectory if you want, mdzk will still handle it the same way as a note placed in the root.

## Note

In the filesystem, a vault is represented by directory, the notes being Markdown-files. mdzk takes a directory as input, recursively finds any Markdown-files in it and loads each of them as a node in the graph. It then parses every file and establishes edges between the nodes based on the links contained in it.
A note is a node in a vault, represented by a single Markdown file inside the vault's source directory. mdzk recursively finds all Markdown files in a directory and creates notes from them. Notes also contain some metadata:

The *vault* data structure is a very efficient and useful concept in dealing with notes and their relation together. mdzk models a vault as an [adjacency matrix](https://mathworld.wolfram.com/AdjacencyMatrix.html) and does custom hashing on each note to obtain unique note IDs. This means that we get incredibly fast lookups on note adjacencies.
- An [ID](#note-ids);
- The original content of the source file;
- The processed content of the source file;
- A relative path from the vault root;
- An optional date;
- Optional tags.

## Vaults
Since mdzk only processes front matter and wiki-style links, there is no limitation on what Markdown extensions you can use in a note's content. However, mdzk adheres to [CommonMark](https://commonmark.org/), so links inside e.g. code blocks and raw HTML will not get touched. If you are unfamiliar with CommonMark, I strongly recommend you check out [this quick guide](https://commonmark.org/help/) to get the gist of it, and stick to it's conventions where possible. We also support [tables](https://github.github.com/gfm/#tables-extension-), [task lists](https://github.github.com/gfm/#task-list-items-extension-), [strikethrough](https://github.github.com/gfm/#strikethrough-extension-) and [footnotes](https://talk.commonmark.org/t/how-should-footnotes-behave/1106).

As a user, you can think of a vault simply as a directory with a bunch of Markdown-files. You may structure this directory however you want and include any other filetypes - mdzk does not care. The major difference to keep in mind, is that a vault does not include the notion of any hierarchy. You may bury a note in a subdirectory of a subdirectory of a subdirectory if you want, mdzk will still handle it the same way as a note placed in the root.
### Note IDs

Each note is uniquely identified by a 16-digit hexidecimal ID (64 bits). This ID is gained from the [Fowler-Noll-Vo hash](https://en.wikipedia.org/wiki/Fowler%E2%80%93Noll%E2%80%93Vo_hash_function) of the relative path from the vault root to the note's source file. This means that you can change the title, metadata and content of your note as much as you want without changing it's ID - as long as the filepath remains the same. As a user, you will likely never need to care about a note's ID; it is only a way for mdzk and external programs to handle uniqueness in cases like name changes or note's with the same title.

## Arc

An arc is created by writing a [wiki-style](https://www.mediawiki.org/wiki/Help:Links#Internal_links) link inside a note, namely by surrounding a note identifier by double square brackets like this: `[[link to a note]]`. The note identifier can be both the title of a note or it's filename.

---

Consider the vault with the following tree:

Expand All @@ -68,11 +97,7 @@ Producing a visualization of this vault, might look like this:
<!-- Using http://bl.ocks.org/rkirsling/5001347 to produce illustrative graphs atm. -->
![](images/vaults_01.png){ width=50% }

As you can see, the paths were totally ignored; *[internal links](#internal-links) are the only constructs that define the vault structure*. The filepaths are still stored as metadata, so custom workflows using directory hierarchies can still be made if one wants.

## Notes

## Internal links
As you can see, the paths were totally ignored; *[arcs](#arc) define the vault structure, not filepaths*. Paths are still stored as metadata, so custom workflows using directory hierarchies can still be made if one wants.


---
Expand Down Expand Up @@ -106,16 +131,18 @@ As you can see, the paths were totally ignored; *[internal links](#internal-link

// The content of the note. All internal links are
// converted to CommonMark and the front matter
// is stripped.
// is stripped
"content": string,

// A set of note IDs, each representing an
// outgoing internal link
"links": [string],

// A set of note IDs, each representing an
// incoming internal link
"backlinks": [string]
"arcs": {
// An array of note IDs that are the
// destination of an arc starting at the
// current note
"outgoing": [string],
// An array of note IDs where the current
// note is the destination
"incoming": [string]
},
}
],
// Not yet implemented: "tags": []
Expand Down

0 comments on commit 2e359cf

Please sign in to comment.