Docs arrears

qi-doc/scribblings/field-guide.scrbl

@@ -31,7 +31,37 @@ A journeyman of one's craft -- a woodworker, electrician, or a plumber, say -- a
 
 @subsection{Be Intentional About Effects}
 
-Qi encourages a style that avoids "accidental" effects. A flow should either be pure (that is, it should be free of "side effects" such as printing to the screen or writing to a file), or its entire purpose should be to fulfill a side effect. It is considered inadvisable to have a function with sane inputs and outputs (resembling a pure function) that also performs a side effect. It would be better to decouple the effect from the rest of your function (@seclink["Use_Small_Building_Blocks"]{splitting it into smaller functions}, as necessary) and perform the effect explicitly via the @racket[effect] form, or otherwise escape from Qi using something like @racket[esc] (note that @seclink["Identifiers"]{function identifiers} used in a flow context are implicitly @racket[esc]aped) in order to perform the effect. This will ensure that there are no surprises with regard to @seclink["Order_of_Effects"]{order of effects}.
+Qi encourages a style that avoids "accidental" effects.
+
+In functional programming, "effects" refer to anything that the function does that is not captured in its inputs and outputs. This could include things like printing to the screen, writing to a file, or mutating a global variable.
+
+A @tech{flow} should either be pure (that is, free of such side effects), or its entire purpose should be to fulfill a side effect. It is considered inadvisable to have a function with sane inputs and outputs (resembling a pure function) that also performs a side effect. It would be better to decouple the effect from the rest of your function (@seclink["Use_Small_Building_Blocks"]{splitting it into smaller functions}, as necessary) and perform the effect explicitly via the @racket[effect] form, or otherwise escape from Qi using something like @racket[esc] (note that @seclink["Identifiers"]{function identifiers} used in a flow context are implicitly @racket[esc]aped) in order to perform the effect. This will ensure that there are no surprises with regard to @seclink["Order_of_Effects"]{order of effects}.
+
+For example, say that we wish to perform a simple numeric transformation on an input number, but also print the intermediate values to the screen. We might do it this way:
+
+@examples[
+    #:eval eval-for-docs
+    #:label #f
+    (define (my-square v)
+      (displayln v)
+      (sqr v))
+
+    (define (my-add1 v)
+      (displayln v)
+      (add1 v))
+
+    (~> (3) my-square my-add1)
+]
+
+This is considered poor style since we've mixed pure functions with implicit effects. Instead, following the above guideline, we would write it this way:
+
+@examples[
+    #:eval eval-for-docs
+    #:label #f
+	(~> (3) (ε displayln sqr) (ε displayln add1))
+]
+
+This uses the pure functions @racket[sqr] and @racket[add1], extracting the effectful @racket[displayln] as an explicit @racket[effect].
 
 @section{Debugging}
 
@@ -172,9 +202,16 @@ Methodical use of @racket[gen] together with the @seclink["Using_a_Probe"]{probe
 ;   in: lambda
 }
 
-@bold{Meaning}: The Racket interpreter received syntax, in this case simply "lambda", that it considers to be invalid. Note that if it received something it didn't know anything about, it would say "undefined" rather than "bad syntax." Bad syntax indicates known syntax used in an incorrect way.
+@bold{Meaning}: The expander (@seclink["It_s_Languages_All_the_Way_Down"]{either the Racket or Qi expander}) received syntax, in this case simply "lambda", that it considers to be invalid. Note that if it received something it didn't know anything about, it would say "undefined" rather than "bad syntax." Bad syntax indicates known syntax used in an incorrect way.
 
-@bold{Common example}: A Racket expression has not been properly escaped within a Qi context. For instance, @racket[(flow (lambda (x) x))] is invalid because the wrapped expression is Racket rather than Qi. To fix this, use @racket[esc], as in @racket[(flow (esc (lambda (x) x)))].
+@bold{Common example}: A Racket expression has not been properly escaped within a Qi context. For instance, @racket[(☯ (lambda (x) x))] is invalid because the wrapped expression is Racket rather than Qi. To fix this, use @racket[esc], as in @racket[(☯ (esc (lambda (x) x)))].
+
+@codeblock{
+; not: bad syntax
+;   in: not
+}
+
+@bold{Common example}: Similar to the previous one, a Racket expression has not been properly escaped within a Qi context, but in a special case where the Racket expression has the same name as a Qi form. In this instance, you may have used @racket[(☯ not)] expecting to invoke Racket's @racket[not] function, since @seclink["Identifiers"]{function identifiers may be used as flows directly} without needing to be escaped. But as Qi has a @racket[not] form as well, Qi's expander first attempts to match this against legitimate use of Qi's @racket[not], which fails, since this expects a flow as an argument and cannot be used in identifier form. To fix this in general, use an explicit @racket[esc], as in @racket[(☯ (esc not))]. In this specific case, you could also use Qi's @racket[(☯ NOT)] instead.
 
 @bold{Common example}: Trying to use a Racket macro (rather than a function), or a macro from another DSL, as a @tech{flow} without first registering it via @racket[define-qi-foreign-syntaxes]. In general, Qi expects flows to be functions unless otherwise explicitly signaled.
 
@@ -356,16 +393,18 @@ So in general, use mutable values with caution. Such values can be useful as sid
 
 @subsubsection{Order of Effects}
 
- Qi flows may exhibit a different order of effects (in the functional programming sense) than equivalent Racket functions.
+ Qi @tech{flows} may exhibit a different order of effects (in the @seclink["Be_Intentional_About_Effects"]{functional programming sense}) than equivalent Racket functions.
 
 Consider the Racket expression: @racket[(map sqr (filter odd? (list 1 2 3 4 5)))]. As this invokes @racket[odd?] on all of the elements of the input list, followed by @racket[sqr] on all of the elements of the intermediate list, if we imagine that @racket[odd?] and @racket[sqr] print their inputs as a side effect before producing their results, then executing this program would print the numbers in the sequence @racket[1,2,3,4,5,1,3,5].
 
-The equivalent Qi flow is @racket[(~> ((list 1 2 3 4 5)) (filter odd?) (map sqr))]. As this sequence is @seclink["Don_t_Stop_Me_Now"]{"deforested" by Qi's compiler} to avoid multiple passes over the data and the memory overhead of intermediate representations, it invokes the functions in sequence @emph{on each element} rather than @emph{on all of the elements of each list in turn}. The printed sequence with Qi would be @racket[1,1,2,3,3,4,5,5].
+The equivalent Qi flow is @racket[(~> ((list 1 2 3 4 5)) (filter odd?) (map sqr))]. As this sequence is @seclink["Don_t_Stop_Me_Now"]{deforested by Qi's compiler} to avoid multiple passes over the data and the memory overhead of intermediate representations, it invokes the functions in sequence @emph{on each element} rather than @emph{on all of the elements of each list in turn}. The printed sequence with Qi would be @racket[1,1,2,3,3,4,5,5].
 
 Yet, either implementation produces the same output: @racket[(list 1 9 25)].
 
 So, to reiterate, while the output of Qi flows will be the same as the output of equivalent Racket expressions, they may nevertheless exhibit a different order of effects.
 
+If you'd like to ensure a particular order of effects, use @racket[effect] at the appropriate points in your flow. If you'd like to use Racket's order of effects, define your flow using @racket[esc] (although this would lose any Qi compiler optimizations).
+
 @section{Effectively Using Feedback Loops}
 
 @racket[feedback] is Qi's most powerful looping form, useful for arbitrary recursion. As it encourages quite a different way of thinking than Racket's usual looping forms do, here are some tips on "grokking" it.
@@ -463,9 +502,9 @@ Using this approach, you would need to register each such foreign macro using @r
 
 @subsection{Bindings are an Alternative to Nonlinearity}
 
-In some cases, we'd prefer to think of a nonlinear @tech{flow} as a linear sequence on a subset of arguments that happens to need the remainder of the arguments somewhere down the line. In such cases, it is advisable to employ bindings so that the flow can be defined on this subset of them and employ the remainder by name.
+In some cases, we'd prefer to think of a nonlinear @tech{flow} as a linear sequence on a subset of arguments that happens to need the remainder of the arguments somewhere down the line. In such cases, it is advisable to employ @seclink["Binding"]{bindings} so that the flow can be defined on this subset of them and employ the remainder by name.
 
-For example, these are equivalent:
+For example, for a function called @racket[make-document] accepting two arguments that are the name of the document and a file object, these implementations are equivalent:
 
 @codeblock{
   (define-flow make-document
@@ -477,8 +516,8 @@ For example, these are equivalent:
 }
 
 @codeblock{
-  (define (make-document name file)
-    (~>> (file)
+  (define-flow make-document
+    (~>> (== (as name) _)
          file-contents
          (parse-result document/p)
          △

qi-doc/scribblings/forms.scrbl

@@ -216,7 +216,7 @@ The core syntax of the Qi language. These forms may be used in any @tech{flow}.
   @defidform[NOT]
   @defidform[!]
 )]{
-  A Boolean NOT gate, this negates the input.
+  A Boolean NOT gate, this negates the input. This is equivalent to Racket's @racket[not].
 
 @examples[
     #:eval eval-for-docs
@@ -819,7 +819,7 @@ A form of generalized @racket[sieve], passing all the inputs that satisfy each
 @section{Binding}
 
 @defform[(as v ...)]{
-  A @tech{flow} that binds an identifier @racket[v] to the input value. If there are many input values, than there should be as many identifiers as there are inputs.
+  A @tech{flow} that binds an identifier @racket[v] to the input value. If there are many input values, than there should be as many identifiers as there are inputs. Aside from introducing bindings, this flow produces no output.
 
 @examples[
     #:eval eval-for-docs
@@ -836,32 +836,60 @@ A form of generalized @racket[sieve], passing all the inputs that satisfy each
 
 @subsection{Variable Scope}
 
-In general, bindings are scoped to the @emph{outermost} threading form (as the first example above shows), and may be referenced downstream. We will use @racket[(gen v)] as an example of a flow referencing a binding, to illustrate variable scope.
+We will use @racket[(gen v)] as an example of a flow referencing a binding, to illustrate variable scope.
 
-@codeblock{(~> 5 (as v) (gen v))}
+In general, bindings are scoped to the @emph{outermost} threading form, and may be referenced downstream.
 
-... produces @racket[5].
+@examples[
+    #:eval eval-for-docs
+    #:label #f
+    (~> (5) (as v) (gen v))
+    (~> (5) (-< (~> sqr (as v))
+                _) (gen v))
+]
 
 A @racket[tee] junction binds downstream flows in a containing threading form, with later tines shadowing earlier tines.
 
-@codeblock{(~> (-< (~> 5 (as v)) (~> 6 (as v))) (gen v))}
-
-... produces @racket[6].
+@examples[
+    #:eval eval-for-docs
+    #:label #f
+    (~> () (-< (~> 5 (as v))
+               (~> 6 (as v))) (gen v))
+]
 
 A @racket[relay] binds downstream flows in a containing threading form, with later tines shadowing earlier tines.
 
-@codeblock{(~> (gen 5 6) (== (as v) (as v)) (gen v))}
-
-... produces @racket[6].
+@examples[
+    #:eval eval-for-docs
+    #:label #f
+    (~> (5 6)
+        (== (as v)
+            (as v))
+        (gen v))
+]
 
 In an @racket[if] conditional form, variables bound in the condition bind the consequent and alternative flows, and do not bind downstream flows.
 
-@codeblock{(if (~> ... (as v) ...) (gen v) (gen v))}
+@examples[
+    #:eval eval-for-docs
+    #:label #f
+    (on ("Ferdinand")
+      (if (-< (~> string-titlecase (as name))
+              (string-suffix? "cat"))
+          (gen name)
+          (gen (~a name " the Cat"))))
+]
 
 Analogously, in a @racket[switch], variables bound in each condition bind the corresponding consequent flow.
 
-@codeblock{(switch [(~> ... (as v) ...) (gen v)]
-                   [(~> ... (as v) ...) (gen v)])}
+@examples[
+    #:eval eval-for-docs
+    #:label #f
+    (switch ("Ferdinand the cat")
+      [(-< (~> string-titlecase (as name))
+           (string-suffix? "cat")) (gen name)]
+      [else "dog"])
+]
 
 As @racket[switch] compiles to @racket[if], technically, earlier conditions bind all later switch clauses (and are shadowed by them), but this is considered an incidental implementation detail. Like @racket[if], @racket[switch] bindings are unavailable downstream.
 

qi-doc/scribblings/intro.scrbl

@@ -73,7 +73,7 @@ For macros, we cannot use them naively as @tech{flows} because macros expect all
 
 The threading library also provides numerous shorthands for common cases, many of which don't have equivalents in Qi -- if you'd like to have these, please @hyperlink["https://github.com/drym-org/qi/issues/"]{create an issue} on the source repo to register your interest.
 
-Finally, by virtue of having an optimizing compiler, Qi also offers performance benefits in some cases, including for use of sequences of standard functional operations on lists like @racket[map] and @racket[filter], which in Qi avoid constructing intermediate representations along the way to generating the final result.
+Finally, by virtue of having an @seclink["It_s_Languages_All_the_Way_Down"]{optimizing compiler}, Qi also offers performance benefits in some cases, including for use of sequences of standard functional operations on lists like @racket[map] and @racket[filter], which in Qi @seclink["Don_t_Stop_Me_Now"]{avoid constructing intermediate representations} along the way to generating the final result.
 
 @close-eval[eval-for-docs]
 @(set! eval-for-docs #f)

qi-doc/scribblings/macros.scrbl

@@ -14,7 +14,7 @@
 
 Qi may be extended in much the same way as Racket -- using @tech/reference{macros}. Qi macros are indistinguishable from built-in Qi forms during the macro expansion phase, just as user-defined Racket macros are indistinguishable from macros that are part of the Racket language. This allows us to have the same syntactic freedom with Qi as we are used to with Racket, from being able to @seclink["Adding_New_Language_Features"]{add new language features} to implementing @seclink["Writing_Languages_in_Qi"]{entire new languages} in Qi.
 
-This "first class" macro extensibility of Qi follows the general approach described in @hyperlink["https://dl.acm.org/doi/abs/10.1145/3428297"]{Macros for Domain-Specific Languages (Ballantyne et. al.)}.
+For more on how this is accomplished under the hood, see @secref["It_s_Languages_All_the_Way_Down"].
 
 @table-of-contents[]