optimisation step

docs/OPERATORS.md

@@ -27,9 +27,9 @@ This is clearly unacceptable.
 > macro definition and make sure those names are looked up in the scope
 > of the macro definition instead of the scope where the macro was used.
 
-This offers hope, if we can re-work the macro system to be hygenic by default,
-then the parser instead of generating
-`addition(a, b)` for `a + b` could instead generate:
+This offers hope, if we can re-work the macro system to be hygenic by
+default, then the parser instead of generating `addition(a, b)` for `a +
+b` could instead generate:
 
 ```
 macro gensym$1(a, b) { addition(a, b) }
@@ -38,21 +38,23 @@ macro gensym$1(a, b) { addition(a, b) }
 at the point of the operator declaration, and generate `gensym$1(a, b)`
 when `a + b` is subsequently encountered.
 
-Firstly I now think the use of a `$` prefix to indicate a gensym in a macro
-is not the best idea. Instead the lambda conversion should identify bound
-`let` variables and replace them automatically. That also frees up `$` as a
-potentially useful user-defined prefix operator.
+Firstly I now think the use of a `$` prefix to indicate a gensym in a
+macro is not the best idea. Instead the lambda conversion should identify
+bound `let` variables and replace them automatically. That also frees up
+`$` as a potentially useful user-defined prefix operator.
 
-The bigger problem is that we can't continue to do naiive macro expansion during
-the lambda conversion step, or we'd be back where we started with
-`addition(a, b)` referring to whatever `addition` happens to be the current
-definition.
+The bigger problem is that we can't continue to do naiive macro expansion
+during the lambda conversion step, or we'd be back where we started with
+`addition(a, b)` referring to whatever `addition` happens to be the
+current definition.
 
-We may have to revert to the scheme definition of a macro: pass the arguments
-unevaluated to the macro, evaluate the macro body, then re-evaluate the result.
+We may have to revert to the scheme definition of a macro: pass the
+arguments unevaluated to the macro, evaluate the macro body, then
+re-evaluate the result.
 
-But we really don't want to have the macro evaluated like that, because F♮ is not
-homoiconic, "evaluating the macro body" can only mean substitution.
+But we really don't want to have the macro evaluated like that,
+because F♮ is not homoiconic, "evaluating the macro body" can only
+mean substitution.
 
 What if the arguments to macros were wrapped in a closure?
 
@@ -62,16 +64,18 @@ macro AND(a, b) { if (a) { b } else { false } } => fn AND(a, b) { if (a()) { b()
 AND(a, b) => AND(fn () { a }, fn () { b })
 ```
 
-That would definately work, though it won't be quite as efficient. It solves both
-local scoping rules, since `AND` is now a normal function then free variables in the
-body are evaluated in the context of the function definition, and variables in the
-argument expressions are evaluated in the calling context.
+That would definately work, though it won't be quite as efficient. It
+solves both local scoping rules, since `AND` is now a normal function then
+free variables in the body are evaluated in the context of the function
+definition, and variables in the argument expressions are evaluated in
+the calling context.
 
-Got that working, and we're also handling local shadowing of arguments so they don't
-get wrapped in an invocation unless they are the same lexical variable.
+> Got that working, and we're also handling local shadowing of arguments
+so they don't get wrapped in an invocation unless they are the same
+lexical variable.
 
-One little unnecssary inefficiency needs to be addressed. If one macro calls another,
-for example
+One little unnecssary inefficiency needs to be addressed. If one macro
+calls another, for example
 
 ```
 macro NAND(a, b) { NOT(AND(a, b)) }
@@ -109,6 +113,6 @@ pattern if the argument is being modified, for example if a macro
 called another with it's argument modified in some way then the pattern
 i.e. `fn() { a() + 1 }` would be necessary.
 
-Got option 1 working, but no need for extra types, just inspect the
+> Got option 1 working, but no need for extra types, just inspect the
 thunk during macro conversion, if it has no arguments and just contains
 a symbol that would otherwise be invoked then return the symbol.

docs/generated/ast.md

@@ -18,8 +18,6 @@ AstDefinitions --definition--> AstDefinition
 AstDefinitions --next--> AstDefinitions
 AstDefine --symbol--> HashSymbol
 AstDefine --expression--> AstExpression
-AstGensymDefine --basename--> HashSymbol
-AstGensymDefine --expression--> AstExpression
 AstAlias --name--> HashSymbol
 AstAlias --type--> AstType
 AstExprAlias --name--> HashSymbol
@@ -89,7 +87,6 @@ AstTypeConstructorArgs --map--> AstTypeMap
 AstLookupOrSymbol --symbol--> HashSymbol
 AstLookupOrSymbol --lookup--> AstLookupSymbol
 AstDefinition --define--> AstDefine
-AstDefinition --gensymDefine--> AstGensymDefine
 AstDefinition --typeDef--> AstTypeDef
 AstDefinition --macro--> AstDefMacro
 AstDefinition --alias--> AstAlias
@@ -114,7 +111,6 @@ AstExpression --alias--> AstExprAlias
 AstExpression --funCall--> AstFunCall
 AstExpression --lookup--> AstLookup
 AstExpression --symbol--> HashSymbol
-AstExpression --gensym--> HashSymbol
 AstExpression --number--> MaybeBigInt
 AstExpression --character--> character
 AstExpression --fun--> AstCompositeFunction

docs/generated/lambda.md

@@ -5,15 +5,12 @@ Plain lambda structures generated by lambda conversion.
 ```mermaid
 flowchart TD
 LamMacroTable --entries--> entries
-LamGenSymTable --entries--> entries
+LamMacroArgsTable --entries--> entries
 LamInfoTable --entries--> entries
 LamAliasTable --entries--> entries
 LamExpTable --entries--> entries
 LamLam --args--> LamVarList
 LamLam --exp--> LamExp
-LamMacro --args--> LamVarList
-LamMacro --exp--> LamExp
-LamMacro --env--> LamContext
 LamVarList --var--> HashSymbol
 LamVarList --next--> LamVarList
 LamPrimApp --type--> LamPrimOp
@@ -74,7 +71,6 @@ LamLetRec --nbindings--> int
 LamLetRec --bindings--> LamLetRecBindings
 LamLetRec --body--> LamExp
 LamLetRecBindings --var--> HashSymbol
-LamLetRecBindings --isGenSym--> bool
 LamLetRecBindings --val--> LamExp
 LamLetRecBindings --next--> LamLetRecBindings
 LamContext --frame--> LamInfoTable
@@ -117,7 +113,6 @@ LamTypeConstructorInfo --index--> int
 LamExp --namespaces--> LamNamespaceArray
 LamExp --lam--> LamLam
 LamExp --var--> HashSymbol
-LamExp --gensym--> HashSymbol
 LamExp --stdint--> int
 LamExp --biginteger--> MaybeBigInt
 LamExp --prim--> LamPrimApp

docs/generated/pratt.md

@@ -31,6 +31,7 @@ PrattParser --namespaces--> PrattIntTable
 PrattParser --lexer--> PrattLexer
 PrattParser --trie--> PrattTrie
 PrattParser --panicMode--> bool
+PrattParser --isPreamble--> bool
 PrattParser --next--> PrattParser
 PrattRecord --symbol--> HashSymbol
 PrattRecord --prefixOp--> PrattOp
@@ -50,6 +51,7 @@ PrattNumberState["enum PrattNumberState"]
 PrattStringState["enum PrattStringState"]
 PrattFixity["enum PrattFixity"]
 PrattUTF8["PrattUTF8[]"] --entries--> uchar
+PrattParsers["PrattParsers[]"] --entries--> PrattParser
 PrattUnicode["PrattUnicode[]"] --entries--> character
 PrattValueVal
 PrattValueType