* Overhauling literate file processing. There is now one class respon…

…sible for handling the parsing and rendering of literate files and for creation of fragment maps. Different parts use the new central fragment repository. This has been done with future enhancements in mind. * Literate program split up into several documents. TOC, chapter linking and similar features have become important, but for now accept that browsing the literate program isn't the easiest. Something to be address better through #7, #10 and #11. * Diagnostics don't repeat unnecessarily.
jesterKing · Jan 7, 2022 · a4e3af1 · a4e3af1
1 parent 2d0cecf
commit a4e3af1
Show file tree

Hide file tree

Showing 15 changed files with 2,178 additions and 1,509 deletions.
diff --git a/.vscodeignore b/.vscodeignore
@@ -10,3 +10,9 @@ vsc-extension-quickstart.md
 **/.eslintrc.json
 **/*.map
 **/*.ts
+*.vsix
+literate/*.literate
+literate/*.html
+*.literate
+*.html
+out/extension.js
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,17 @@
 # Change Log
 
+## 0.4.1
+
+* Overhauling literate file processing. There is now one class responsible for
+  handling the parsing and rendering of literate files and for creation of
+  fragment maps. Different parts use the new central fragment repository. This
+  has been done with future enhancements in mind.
+* Literate program split up into several documents. TOC, chapter linking and
+  similar features have become important, but for now accept that browsing the
+  literate program isn't the easiest. Something to be address better through #7,
+  #10 and #11.
+* Diagnostics don't repeat unnecessarily.
+
 ## 0.4.0
 
 * Add hovers when mousing over fragment usage or fragment mention.

diff --git a/index.html b/index.html
@@ -7,6 +7,9 @@ <h1>Literate Programming</h1>
 <p>A Visual Studio Code extension for writing literate programs.</p>
 <ul>
 <li><a href="literate/literate.html">Literate Programming</a>, the main extension program</li>
+<li><a href="literate/fragment_explorer.html">Fragment Explorer</a></li>
+<li><a href="literate/code_completion.html">Code Completion</a></li>
+<li><a href="literate/hovers.html">Hovers</a></li>
 <li><a href="literate/grabber.html">Grabber</a>, plug-in code used to grab the MarkdownIt
 parser state</li>
 </ul>

diff --git a/index.literate b/index.literate
@@ -3,5 +3,8 @@
 A Visual Studio Code extension for writing literate programs.
 
 * [Literate Programming](literate/literate.html), the main extension program
+* [Fragment Explorer](literate/fragment_explorer.html)
+* [Code Completion](literate/code_completion.html)
+* [Hovers](literate/hovers.html)
 * [Grabber](literate/grabber.html), plug-in code used to grab the MarkdownIt
   parser state
diff --git a/literate/code_completion.html b/literate/code_completion.html
@@ -0,0 +1,95 @@
+<html>
+    <head>
+      <link rel="stylesheet" type="text/css" href="./style.css">
+    </head>
+    <body>
+    <h1>Code completion</h1>
+<p>A simple implementation to provide code completion will help authors writing
+their literate programs. Having possible tag names suggested will help
+decreasing the cognitive load of remembering all code fragment names in a
+literate project. This project itself has well over 50 fragments, and having to
+remember them by name is not easy.</p>
+<p>Until there is a good <strong>literate</strong> file type integration with Visual Studio Code
+we'll be relying on the built-in <strong>Markdown</strong> functionality.</p>
+<pre><code><span class="hljs-keyword">const</span> completionItemProvider =
+  vscode.<span class="hljs-property">languages</span>.<span class="hljs-title function_">registerCompletionItemProvider</span>(<span class="hljs-string">&#x27;markdown&#x27;</span>, {
+    &lt;&lt;implement provide completion items&gt;&gt;
+}, <span class="hljs-string">&#x27;&lt;&#x27;</span>);
+context.<span class="hljs-property">subscriptions</span>.<span class="hljs-title function_">push</span>(completionItemProvider);
+</code></pre>
+<h2>Providing completion items</h2>
+<p>The completion item provider will generate a <code>CompletionItem</code> for each fragment
+we currently know of. Although the provider gets passed in the <code>TextDocument</code>
+for which it was triggered we will present fragments from the entire project.</p>
+<pre><code><span class="hljs-keyword">async</span> <span class="hljs-title function_">provideCompletionItems</span>(<span class="hljs-params">
+  <span class="hljs-variable language_">document</span> : vscode.TextDocument,
+  ..._
+</span>)
+{
+</code></pre>
+<p>After setting up the necessary variables with
+<code>&lt;&lt;setup variables for providing completion items&gt;&gt;</code> we figure out to which
+workspace folder the current <code>TextDocument</code>. If no workspace folder can be
+determined we return an empty array. This can happen with an unsaved new file,
+or when documents were opened that are not part of the workspace.</p>
+<pre><code>  &lt;&lt;setup variables <span class="hljs-keyword">for</span> providing completion items&gt;&gt;
+  &lt;&lt;get workspace <span class="hljs-keyword">for</span> <span class="hljs-title class_">TextDocument</span>&gt;&gt;
+</code></pre>
+<p>After the workspace folder has been determined we can gather all fragments in
+our project.</p>
+<pre><code>  &lt;&lt;get fragments <span class="hljs-keyword">for</span> completion items&gt;&gt;
+</code></pre>
+<p>Finally we generate the completion items into the array <code>completionItems</code> that
+we return when done.</p>
+<pre><code>  &lt;&lt;<span class="hljs-keyword">for</span> each fragment create a completion item&gt;&gt;
+  <span class="hljs-keyword">return</span> completionItems;
+}
+</code></pre>
+<h3>Setting up variables</h3>
+<p>Completion items are going to be collected in an <code>Array&lt;CompletionItem&gt;</code>.</p>
+<pre><code><span class="hljs-keyword">let</span> completionItems : <span class="hljs-title class_">Array</span>&lt;vscode.<span class="hljs-property">CompletionItem</span>&gt; =
+    <span class="hljs-keyword">new</span> <span class="hljs-title class_">Array</span>&lt;vscode.<span class="hljs-property">CompletionItem</span>&gt;();
+</code></pre>
+<h3>Workspace folder for TextDocument</h3>
+<p>Determining the workspace folder for the given TextDocument is done by creating
+relative paths from each workspace folder to the document. If the path does not
+start with <code>..</code> we found the workspace folder where the document is from.</p>
+<p>If no workspace folders were found, or if the TextDocument did not have a
+workspace folder we essentially end up returning an empty array from the
+completion item provider.</p>
+<pre><code><span class="hljs-keyword">const</span> workspaceFolder : vscode.<span class="hljs-property">WorkspaceFolder</span> | <span class="hljs-literal">undefined</span> = <span class="hljs-title function_">determineWorkspaceFolder</span>(<span class="hljs-variable language_">document</span>);
+<span class="hljs-keyword">if</span>(!workspaceFolder) { <span class="hljs-keyword">return</span> []; }
+</code></pre>
+<h3>Retrieving fragments of project</h3>
+<p>Code completion item providers run essentially on document changes. The
+<code>FragmentRepository</code> in most cases handles processing of <strong>literate</strong> files
+automatically, but it skips that when a change is caused by typing <code>&lt;</code>, the
+opening chevron. That means we need to ensure <strong>literate</strong> files are processed
+before getting the fragment map for our workspace folder.</p>
+<pre><code><span class="hljs-keyword">await</span> theOneRepository.<span class="hljs-title function_">processLiterateFiles</span>(workspaceFolder);
+<span class="hljs-keyword">let</span> fragments = theOneRepository.<span class="hljs-title function_">getFragments</span>(workspaceFolder).<span class="hljs-property">map</span>;
+</code></pre>
+<h3>Creating the CompletionItems</h3>
+<p>With all fragments in the map we iterate over all the keys. For each key we
+fetch the corresponding <code>FragmentInformation</code>. Now we can create the
+<code>CompletionItem</code> with the <code>fragmentName</code> as its content.</p>
+<p>Further the fragment code is set to be the detail of the completion item. This
+will provide a tooltip with the code fragment readable, so that it is easy to
+understand what fragment is currently highlighted in the completion list.</p>
+<p>Finally the set the completion item kind to <code>Reference</code> so that we get a nice
+icon in the completion list pop-up.</p>
+<pre><code><span class="hljs-keyword">for</span>(<span class="hljs-keyword">const</span> fragmentName <span class="hljs-keyword">of</span> fragments.<span class="hljs-title function_">keys</span>())
+{
+  <span class="hljs-keyword">const</span> fragment : <span class="hljs-title class_">FragmentInformation</span> | <span class="hljs-literal">undefined</span> = fragments.<span class="hljs-title function_">get</span>(fragmentName);
+  <span class="hljs-keyword">if</span>(!fragment) {
+    <span class="hljs-keyword">continue</span>;
+  }
+  <span class="hljs-keyword">const</span> fragmentCompletion = <span class="hljs-keyword">new</span> vscode.<span class="hljs-title class_">CompletionItem</span>(fragmentName);
+  fragmentCompletion.<span class="hljs-property">detail</span> = fragment.<span class="hljs-property">code</span>;
+  fragmentCompletion.<span class="hljs-property">kind</span> = vscode.<span class="hljs-property">CompletionItemKind</span>.<span class="hljs-property">Reference</span>;
+  completionItems.<span class="hljs-title function_">push</span>(fragmentCompletion);
+}
+</code></pre>
+
+    </body>
+  </html>
diff --git a/literate/code_completion.literate b/literate/code_completion.literate
@@ -0,0 +1,123 @@
+# Code completion
+
+A simple implementation to provide code completion will help authors writing
+their literate programs. Having possible tag names suggested will help
+decreasing the cognitive load of remembering all code fragment names in a
+literate project. This project itself has well over 50 fragments, and having to
+remember them by name is not easy.
+
+Until there is a good **literate** file type integration with Visual Studio Code
+we'll be relying on the built-in **Markdown** functionality.
+
+``` ts : <<register completion item provider>>=
+const completionItemProvider =
+  vscode.languages.registerCompletionItemProvider('markdown', {
+    <<implement provide completion items>>
+}, '<');
+context.subscriptions.push(completionItemProvider);
+```
+
+## Providing completion items
+
+The completion item provider will generate a `CompletionItem` for each fragment
+we currently know of. Although the provider gets passed in the `TextDocument`
+for which it was triggered we will present fragments from the entire project.
+
+``` ts : <<implement provide completion items>>=
+async provideCompletionItems(
+  document : vscode.TextDocument,
+  ..._
+)
+{
+```
+
+After setting up the necessary variables with
+`<<setup variables for providing completion items>>` we figure out to which
+workspace folder the current `TextDocument`. If no workspace folder can be
+determined we return an empty array. This can happen with an unsaved new file,
+or when documents were opened that are not part of the workspace.
+
+``` ts : <<implement provide completion items>>=+
+  <<setup variables for providing completion items>>
+  <<get workspace for TextDocument>>
+```
+
+After the workspace folder has been determined we can gather all fragments in
+our project.
+
+``` ts : <<implement provide completion items>>=+
+  <<get fragments for completion items>>
+```
+
+Finally we generate the completion items into the array `completionItems` that
+we return when done.
+
+``` ts : <<implement provide completion items>>=+
+  <<for each fragment create a completion item>>
+  return completionItems;
+}
+```
+
+### Setting up variables
+
+Completion items are going to be collected in an `Array<CompletionItem>`.
+
+``` ts : <<setup variables for providing completion items>>=
+let completionItems : Array<vscode.CompletionItem> =
+    new Array<vscode.CompletionItem>();
+```
+
+### Workspace folder for TextDocument
+
+Determining the workspace folder for the given TextDocument is done by creating
+relative paths from each workspace folder to the document. If the path does not
+start with `..` we found the workspace folder where the document is from.
+
+If no workspace folders were found, or if the TextDocument did not have a
+workspace folder we essentially end up returning an empty array from the
+completion item provider.
+
+``` ts : <<get workspace for TextDocument>>=
+const workspaceFolder : vscode.WorkspaceFolder | undefined = determineWorkspaceFolder(document);
+if(!workspaceFolder) { return []; }
+```
+
+### Retrieving fragments of project
+
+Code completion item providers run essentially on document changes. The
+`FragmentRepository` in most cases handles processing of **literate** files
+automatically, but it skips that when a change is caused by typing `<`, the
+opening chevron. That means we need to ensure **literate** files are processed
+before getting the fragment map for our workspace folder.
+
+``` ts : <<get fragments for completion items>>=
+await theOneRepository.processLiterateFiles(workspaceFolder);
+let fragments = theOneRepository.getFragments(workspaceFolder).map;
+```
+
+### Creating the CompletionItems
+
+With all fragments in the map we iterate over all the keys. For each key we
+fetch the corresponding `FragmentInformation`. Now we can create the
+`CompletionItem` with the `fragmentName` as its content.
+
+Further the fragment code is set to be the detail of the completion item. This
+will provide a tooltip with the code fragment readable, so that it is easy to
+understand what fragment is currently highlighted in the completion list.
+
+Finally the set the completion item kind to `Reference` so that we get a nice
+icon in the completion list pop-up.
+
+``` ts : <<for each fragment create a completion item>>=
+for(const fragmentName of fragments.keys())
+{
+  const fragment : FragmentInformation | undefined = fragments.get(fragmentName);
+  if(!fragment) {
+    continue;
+  }
+  const fragmentCompletion = new vscode.CompletionItem(fragmentName);
+  fragmentCompletion.detail = fragment.code;
+  fragmentCompletion.kind = vscode.CompletionItemKind.Reference;
+  completionItems.push(fragmentCompletion);
+}
+```