Merge pull request #186 from countvajhula/update-docs-for-deforestable

Update docs for deforestable

qi-doc/scribblings/forms.scrbl

@@ -14,10 +14,100 @@
 
 @title[#:tag "Qi_Forms"]{The Qi Language}
 
-The core syntax of the Qi language. These forms may be used in any @tech{flow}. Flows may be specified in Racket via the @seclink["Language_Interface"]{language interface}.
+The syntax and semantics of the Qi language. Qi @tech{flows} may be described using these forms and @seclink["Embedding_a_Hosted_Language"]{embedded} into Racket using the @seclink["Language_Interface"]{language interface}.
 
 @table-of-contents[]
 
+@section{Syntax}
+
+The syntax of a language is most economically and clearly expressed using a grammar, in the form of "nonterminal" symbols along with production rules expressing the syntax that is entailed in positions marked by those symbols. We may thus take the single starting symbol in such a grammar to formally designate the entire syntax of the language.
+
+The symbol @racket[expr] is typically used in this sense to indicate a Racket nonterminal position in the syntax -- that is, a position that expects a Racket expression. Analogously, we use the identifier @deftech{@racket[floe]} (pronounced "flow-e," for "flow expression") to refer to the Qi nonterminal, i.e. a position expecting Qi syntax.
+
+The full surface syntax of Qi is given below. Note that this expands to a @seclink["The_Qi_Core_Language"]{smaller core language} before being @seclink["It_s_Languages_All_the_Way_Down"]{compiled to Racket}. It does not include the @seclink["List_Operations"]{list-oriented forms}, which may be added via @racket[(require qi/list)].
+
+@racketgrammar*[
+[floe @#,seclink["The_Qi_Core_Language"]{core-form}
+      macro-form]
+[macro-form △
+            ▽
+            (one-of? expr ...)
+            (and% floe ...)
+            (or% floe ...)
+            AND
+            &
+            OR
+            ∥
+            NOR
+            NAND
+            XNOR
+            any?
+            all?
+            none?
+            inverter
+            ⏚
+            (~> floe ...)
+            (~>> floe ...)
+            (thread-right floe ...)
+            X
+            crossover
+            ==
+            (== floe ...)
+            (==* floe ...)
+            (relay* floe ...)
+            -<
+            (-< floe ...)
+            count
+            1>
+            2>
+            3>
+            4>
+            5>
+            6>
+            7>
+            8>
+            9>
+            (bundle (index ...) floe floe)
+            (when floe floe)
+            (unless floe floe)
+            switch
+            (switch switch-expr ...)
+            (switch (% floe) switch-expr ...)
+            (switch (divert floe) switch-expr ...)
+            (gate floe)
+            ><
+            (>< floe)
+            (ε floe floe)
+            (effect floe floe)
+            apply
+            (expr expr ... __ expr ...)
+            (expr expr ... _ expr ...)
+            (expr expr ...)
+            literal
+            identifier]
+[literal boolean
+         char
+         string
+         bytes
+         number
+         regexp
+         byte-regexp
+         vector-literal
+         box-literal
+         prefab-literal
+         (@#,racket[quote] value)
+         (quasiquote value)
+         (quote-syntax value)
+         (syntax value)]
+[expr a-racket-expression]
+[index exact-positive-integer?]
+[nat exact-nonnegative-integer?]
+[switch-expr [floe floe]
+             [floe (=> floe)]
+             [else floe]]
+[identifier a-racket-identifier]
+[value a-racket-value]]
+
 @section{Basic}
 
 @defidform[_]{
@@ -528,9 +618,9 @@ A form of generalized @racket[sieve], passing all the inputs that satisfy each
                [(switch maybe-divert-expr switch-expr ...)]
                 ([maybe-divert-expr (divert condition-gate-flow consequent-gate-flow)
                                     (% condition-gate-flow consequent-gate-flow)]
-                 [switch-expr [flow-expr flow-expr]
-                              [flow-expr (=> flow-expr)]
-                              [else flow-expr]])]{
+                 [switch-expr [floe floe]
+                              [floe (=> floe)]
+                              [else floe]])]{
   The @tech{flow} analogue of @racket[cond], this is a dispatcher where the condition and consequent expressions are all flows which operate on the switch inputs.
 
   Typically, each of the component flows -- conditions and consequents both -- receives all of the original inputs to the @racket[switch]. This can be changed by using a @racket[divert] clause, which takes two flow arguments, the first of whose outputs go to all of the condition flows, and the second of whose outputs go to all of the consequent flows. This can be useful in cases where multiple values flow, but only some of them are predicated upon, and others (or all of them) inform the actions to be taken. Using @racket[(divert _ _)] is equivalent to not using it. @racket[%] is a symbolic alias for @racket[divert] -- parse it visually not as the percentage sign, but as a convenient way to depict a "floodgate" diverting values down different channels.

qi-doc/scribblings/interface.scrbl

@@ -22,140 +22,8 @@ The core entry-point to Qi from the host language is the form @racket[☯]. In a
 @subsection{Core}
 
 @deftogether[(
-@defform*/subs[[(☯ flow-expr)]
-               ([flow-expr (code:line)
-                           _
-                           (gen expr ...)
-                           △
-                           sep
-                           ▽
-                           collect
-                           (esc expr)
-                           (clos flow-expr)
-                           (as identifier ...)
-                           (one-of? expr ...)
-                           (all flow-expr)
-                           (any flow-expr)
-                           (none flow-expr)
-                           (and flow-expr ...)
-                           (or flow-expr ...)
-                           (not flow-expr)
-                           (and% flow-expr ...)
-                           (or% flow-expr ...)
-                           NOT
-                           !
-                           AND
-                           &
-                           OR
-                           ∥
-                           NOR
-                           NAND
-                           XOR
-                           XNOR
-                           any?
-                           all?
-                           none?
-                           inverter
-                           ⏚
-                           ground
-                           (~> flow-expr ...)
-                           (thread flow-expr ...)
-                           (~>> flow-expr ...)
-                           (thread-right flow-expr ...)
-                           X
-                           crossover
-                           ==
-                           (== flow-expr ...)
-                           relay
-                           (relay flow-expr ...)
-                           (==* flow-expr ...)
-                           (relay* flow-expr ...)
-                           -<
-                           (-< flow-expr ...)
-                           tee
-                           (tee flow-expr ...)
-                           fanout
-                           (fanout nat)
-                           feedback
-                           (feedback nat flow-expr)
-                           (feedback nat (then flow-expr) flow-expr)
-                           (feedback (while flow-expr) flow-expr)
-                           (feedback (while flow-expr) (then flow-expr) flow-expr)
-                           count
-                           1>
-                           2>
-                           3>
-                           4>
-                           5>
-                           6>
-                           7>
-                           8>
-                           9>
-                           (select index ...)
-                           (block index ...)
-                           (bundle (index ...) flow-expr flow-expr)
-                           (group nat flow-expr flow-expr)
-                           sieve
-                           (sieve flow-expr flow-expr flow-expr)
-                           (partition [flow-expr flow-expr] ...)
-                           (if flow-expr flow-expr)
-                           (if flow-expr flow-expr flow-expr)
-                           (when flow-expr flow-expr)
-                           (unless flow-expr flow-expr)
-                           switch
-                           (switch switch-expr ...)
-                           (switch (% flow-expr) switch-expr ...)
-                           (switch (divert flow-expr) switch-expr ...)
-                           (gate flow-expr)
-                           ><
-                           (>< flow-expr)
-                           amp
-                           (amp flow-expr)
-                           pass
-                           (pass flow-expr)
-                           <<
-                           (<< flow-expr)
-                           (<< flow-expr flow-expr)
-                           >>
-                           (>> flow-expr)
-                           (>> flow-expr flow-expr)
-                           (loop flow-expr)
-                           (loop flow-expr flow-expr)
-                           (loop flow-expr flow-expr flow-expr)
-                           (loop flow-expr flow-expr flow-expr flow-expr)
-                           (loop2 flow-expr flow-expr flow-expr)
-                           (ε flow-expr flow-expr)
-                           (effect flow-expr flow-expr)
-                           apply
-                           (qi:* expr ...)
-                           (expr expr ... __ expr ...)
-                           (expr expr ... _ expr ...)
-                           (expr expr ...)
-                           literal
-                           identifier]
-                [literal boolean
-                         char
-                         string
-                         bytes
-                         number
-                         regexp
-                         byte-regexp
-                         vector-literal
-                         box-literal
-                         prefab-literal
-                         (quote value)
-                         (quasiquote value)
-                         (quote-syntax value)
-                         (syntax value)]
-                [expr a-racket-expression]
-                [index exact-positive-integer?]
-                [nat exact-nonnegative-integer?]
-                [switch-expr [flow-expr flow-expr]
-                             [flow-expr (=> flow-expr)]
-                             [else flow-expr]]
-                [identifier a-racket-identifier]
-                [value a-racket-value])]
-  @defform[(flow flow-expr)]
+  @defform[(☯ @#,tech{floe})]
+  @defform[(flow @#,tech{floe})]
   )]{
   Define a @tech{flow} by using the various @seclink["Qi_Forms"]{forms} of the Qi language.
 
@@ -173,14 +41,14 @@ See @secref["Flowing_with_the_Flow"] for ways to enter the @racket[☯] symbol i
   ]
 }
 
-@defform[(on (arg ...) flow-expr)]{
+@defform[(on (arg ...) @#,tech{floe})]{
   Define and execute a @tech{flow} with the inputs named in advance.
 
   This is a way to pass inputs to a flow that is an alternative to the usual function invocation syntax (i.e. an alternative to simply invoking the flow with arguments). It may be preferable in certain cases, since the inputs are named at the beginning rather than at the end.
 
   In the respect that it both defines as well as invokes the flow, it has the same relationship to @racket[☯] as @racket[let] has to @racket[lambda], and can be used in analogous ways.
 
-  Equivalent to @racket[((☯ flow-expr) arg ...)].
+  Equivalent to @racket[((☯ @#,tech{floe}) arg ...)].
 
 @examples[
     #:eval eval-for-docs
@@ -194,8 +62,8 @@ See @secref["Flowing_with_the_Flow"] for ways to enter the @racket[☯] symbol i
 @subsection{Threading}
 
 @deftogether[(
-@defform[(~> (args ...) flow-expr ...)]
-@defform[(~>> (args ...) flow-expr ...)]
+@defform[(~> (args ...) @#,tech{floe} ...)]
+@defform[(~>> (args ...) @#,tech{floe} ...)]
 )]{
   These @emph{Racket} forms leverage the identically-named @emph{Qi} forms to thread inputs through a sequence of @tech{flows}. @racket[~>] threads arguments in the first position by default, while @racket[~>>] uses the last position, but in either case the positions can instead be explicitly indicated by using @racket[_] or @racket[___].
 
@@ -207,7 +75,7 @@ See @secref["Flowing_with_the_Flow"] for ways to enter the @racket[☯] symbol i
 
   In the respect that these both define as well as invoke the flow, they have the same relationship to @racket[☯] as @racket[let] has to @racket[lambda], and can be used in analogous ways.
 
-  Equivalent to @racket[((☯ (~> flow-expr ...)) args ...)].
+  Equivalent to @racket[((☯ (~> @#,tech{floe} ...)) args ...)].
 
   See also: @secref["Relationship_to_the_Threading_Macro"].
 
@@ -250,11 +118,11 @@ Each of the @racket[predicate] and @racket[consequent] expressions is a @tech{fl
   These anonymous function forms may be used in cases where you need to explicitly @emph{name} the arguments for some reason. Otherwise, in most cases, just use @racket[☯] directly instead as it produces a function while avoiding the extraneous layer of bindings.
 
 @deftogether[(
-  @defform[(flow-lambda args flow-expr)]
-  @defform[(flow-λ args flow-expr)]
-  @defform[(π args flow-expr)]
+  @defform[(flow-lambda args @#,tech{floe})]
+  @defform[(flow-λ args @#,tech{floe})]
+  @defform[(π args @#,tech{floe})]
 )]{
-  Similiar to @racket[lambda] but constrained to the flow language. This is exactly equivalent to @racket[(lambda args (on (args) flow-expr))] except that the keywords only introduce bindings, and aren't part of the values that are fed into @racket[flow-expr]. @racket[flow-λ] and @racket[π] are aliases for @racket[flow-lambda]. The present form mainly finds its use internally in @racket[define-flow], and in most cases you should use @racket[☯] directly.
+  Similiar to @racket[lambda] but constrained to the flow language. This is exactly equivalent to @racket[(lambda args (on (args) @#,tech{floe}))] except that the keywords only introduce bindings, and aren't part of the values that are fed into @tech{floe}. @racket[flow-λ] and @racket[π] are aliases for @racket[flow-lambda]. The present form mainly finds its use internally in @racket[define-flow], and in most cases you should use @racket[☯] directly.
 
 @examples[
     #:eval eval-for-docs
@@ -284,7 +152,7 @@ Each of the @racket[predicate] and @racket[consequent] expressions is a @tech{fl
              ...
              [else consequent ...])]
 )]{
-  Similar to @racket[lambda] but constrained to be a flow-based dispatcher. This is exactly equivalent to @racket[(lambda args (switch (args) maybe-divert-clause [predicate consequent ...] ... [else consequent ...]))] except that the keywords only introduce bindings, and aren't part of the values that are fed into @racket[flow-expr]. @racket[switch-λ] and @racket[λ01] are aliases for @racket[switch-lambda].
+  Similar to @racket[lambda] but constrained to be a flow-based dispatcher. This is exactly equivalent to @racket[(lambda args (switch (args) maybe-divert-clause [predicate consequent ...] ... [else consequent ...]))] except that the keywords only introduce bindings, and aren't part of the values that are fed into @tech{floe}. @racket[switch-λ] and @racket[λ01] are aliases for @racket[switch-lambda].
 
 @examples[
     #:eval eval-for-docs
@@ -305,10 +173,10 @@ Each of the @racket[predicate] and @racket[consequent] expressions is a @tech{fl
 The following definition forms may be used in place of the usual general-purpose @racket[define] form when defining @tech{flows}.
 
 @deftogether[(
-  @defform[(define-flow name flow-expr)]
+  @defform[(define-flow name @#,tech{floe})]
   @defform[#:link-target? #f
-           (define-flow (head args) flow-expr)])]{
-  Similiar to the function form of @racket[define] but constrained to the flow language. This is exactly equivalent to @racket[(define head (flow-lambda args flow-expr))].
+           (define-flow (head args) @#,tech{floe})])]{
+  Similiar to the function form of @racket[define] but constrained to the flow language. This is exactly equivalent to @racket[(define head (flow-lambda args @#,tech{floe}))].
 }
 
 @deftogether[(

qi-doc/scribblings/list-operations.scrbl

@@ -1,20 +1,40 @@
 #lang scribble/doc
 @require[scribble/manual
-	 (for-label racket/list
-	 	    racket/base)]
+         (for-label racket/list
+                    racket/base)]
 
 @title{List Operations}
 
 @defmodule[qi/list]
 
-This module defines bindings that can leverage stream fusion /
-deforestation optimization when found in succession within a
-flow. When not part of optimized flow, their behavior is identical to
-the bindings of the same name from @racketmodname[racket/base] and
-@racketmodname[racket/list].
-
-The bindings are categorized based on their intended usage inside the
-deforested pipeline.
+This module defines functional list operations analogous to those in
+@racketmodname[racket/base] and @racketmodname[racket/list], except
+that these forms support @tech{flows} in higher-order function
+positions and leverage the @seclink["Don_t_Stop_Me_Now"]{stream fusion
+/ deforestation} optimization to avoid constructing intermediate
+representations along the way to computing the result.
+
+The forms in this module extend the syntax of the @seclink["The_Qi_Core_Language"]{core Qi language}. This extended syntax is given below:
+
+@racketgrammar*[
+[floe (map floe)
+      (filter floe)
+      (filter-map floe)
+      (foldl floe expr)
+      (foldr floe expr)
+      (range expr expr expr)
+      (take expr)
+      car
+      cadr
+      caddr
+      cadddr
+      (list-ref expr)
+      length
+      empty?
+      null?]]
+
+The operations are categorized based on their role in the deforested
+pipeline.
 
 @section{Producers}