-
Notifications
You must be signed in to change notification settings - Fork 0
jtlc and JXL Features
The jtlc is designed around the two-stage model of execution: first, the template, represented as an abstract syntax tree, is compiled into a Javascript function with a call to dojox.jtlc.compile( template, language, options ). The returned evaluator function may then be executed as many times as necessary on different inputs. Structure of the input and output data for the template depends on the language; JXL treats both as anonymous (i.e. without a custom constructor) Javascript objects and arrays in keeping with the philosophy of JSON.
Note that a JXL program — and, strictly speaking, any useful jtlc input in general — cannot be represented in JSON, because many of its nodes are objects with non-trivial constructors and prototypes.
JXL functionality is easiest to describe in terms of a dataflow execution model where values are generated by the leaves of the tree and flow towards the root of the tree through functional nodes to end up in sinks. The root of the tree serves as a top-level sink, the value of which becomes the return value of the entire template (i.e. compiled function).
A JXL subtree may have one of two distinct modes of execution — singleton or iterative — determined unambiguously from the context in which the subtree appears. Some of the primitives (tags) may function in either mode, others require a specific mode. The context is defined by certain tags, that control mode of execution for their child subtrees. Some of these tags (group, each) may be thought of as looping constructs.
A subtree appearing in a singleton context produces a single value. Every node in such a tree behaves as a function — insofar as its execution has no side effects. The entire JXL template is implicitly in singleton context with the final return statement of the compiled function as the sink.
Iterative contexts are equivalent to loops over the entire sequence of values generated by a single node (typically the bottom-left corner of the JXL subtree in question, as the trees are usually drawn). The sink above the subtree is responsible for the ultimate disposal of processed values — usually pushing them into an array, though it is also possible to populate dictionaries or aggregate the sequence into a single value. Sinks may even have multiple subtrees connected to them, some of them iterative and others singletons.
Value generation is achieved by running a query on the input data or, in simpler cases, enumerating a specific array or dictionary contained within.
Yet another context-dependent parameter is current input: the root of the data hierarchy that current subtree operates upon. Current input is set to the value of the first argument passed into the call to compiled template but tags such as group and each may locally change it for certain of their child subtrees. Note that compiled templates may accept and operate upon multiple arguments.
Access to the value associated with current input is gained via a leaf tag current which also serves as a generator in iterative contexts, assuming that current input is associated with an array. Most tags default to using current when no explicit arguments are provided.
JXL contains a facility (tag expr) for embedding parameterized Javascript expressions into the template. Their use and parameter substitution syntax is very similar to that of JSON Query. Note however that actual JSON Queries embeded into a JXL program compile into function bodies of their own while expr tags inject code directly into the compiled template.
At its core, the compilation by jtlc is performed in a parent-first recursive descent of the input tree. All tag nodes provide a compile() method which is called as a visitor on the object encapsulating the compiler state: current output, expression stack, dictionaries for local and global variables etc.; tags are free to inject additional properties into it as they see fit. Additional tags can generally be created without any impact on the rest of the compiler and don’t have to be specially registered with jtlc. The language description object (second argument of the dojox.jtlc.compile()) determines how the literal values — strings, numbers, functions and objects without the compile method — are interpreted and carries global settings that certain tags may depend upon; the compiler state uses this object as a prototype. It is expected that constructors for language descriptions accept a bag of properties to be mixed into the instance providing an additional shortcut for customization.
The primary reason behind the two-stage execution model is performance: compiled templates should generally compare well to handwritten Javascript code in the efficiency of execution. Care is taken to avoid or minimize spurious copying, use efficient loop constructs, evaluate complex subexpressions once etc. All data structures maintained during the execution of the template remain under direct control of the template’s programmer: the compiler introduces only local variables and no additional arrays or objects. Turning on optional argument checks may lead to introduction of function calls to contexts where default settings produce only inline code: choosing uncompromising performance vs. timely detection of errors in input data is a tradeoff left to the user of the library.
In order to minimize re-evaluation or even replication of complex fragments, the compiler maintains a dictionary of “global variables”: in fact, those variables are injected into a closure around the resulting compiled function so that their values are effectively evaluated only once during the compilation. For example, all JSON Queries used by the template become compiled functions referenced from within the closure.
An extra optimization step is provided after the compilation is finished, it is performed on the resulting Javascript code still in string representation. Any tag may register an optimization callback with the compiler instance in order to transform the string that’s about to become the body of the compiled template. These callbacks usually remove redundancies that can be pinpointed with a regex search. Surprisingly enough, this simple procedure proves very effective in reducing spurious copying and compacting the code.