v2.0.0

This is the compiler for Yarn Spinner. If you want to use Yarn Spinner in a Unity game, please see the releases page for Yarn Spinner for Unity! If you're not sure what this means, check the documentation.

Yarn Spinner is made possible by your generous patronage. Please consider supporting Yarn Spinner's development by becoming a patron!

Yarn Spinner 2.0

Yarn Spinner 2.0 is a major new release, and contains a large number of new features and improvements.

New syntax for jumping to a different node.

• We have added a <<jump Destination>> command, which replaces the [[Destination]] jump syntax.

• Accordingly, the [[Destination]] and [[Option|Destination]] syntax has been removed from the language.
• Instead of using [[Option|Destination]] syntax, combine the new <<jump Destination>> command with shortcut -> options instead. For example:

// Before
Kim: You want a bagel?
[[Yes, please!|GiveBagel]]
[[No, thanks!|DontWantBagel]]

// After
Kim: You want a bagel?
-> Yes, please!
  <<jump GiveBagel>>
-> No, thanks!
  <<jump DontWantBagel>>

• The old syntax was inherited from the original Yarn language, which itself inherited it from Twine.
  - We removed it for four reasons:
  1. it conflated jumps and options, which are very different operations, with too-similar syntax
  2. the Option-destination syntax for declaring options involved the management of non-obvious state (that is, if an option statement was inside an if branch that was never executed, it was not presented, and the runtime needed to keep track of that)
  3. it was not obvious that options accumulated and were only presented at the end of the node
  4. finally, shortcut options provide a cleaner way to present the same behaviour.
• No change to the bytecode is made here; these changes only affect the compiler.

Automatic upgrader for Yarn Spinner 1.0 variables.

• An automatic upgrader has been added that attempts to determine the types of variables in Yarn Spinner 1.0, and generates <<declare>> statements for variables.

• This upgrader infers the type of a variable based on the values that are assigned to it, and the values of expressions that it participates in.
  • If the upgrader cannot determine the type of a variable, it generates a declaration of the form <<declare $variable_name as undefined>>. The word undefined is not a valid type in Yarn Spinner, which means that these declarations will cause an error in compilation (which is a signal to the developer that the script needs to be manually updated.)
  • For example: given the following script:

<<set $const_string = "foo">>
<<set $const_number = 2>>
<<set $const_bool = true>>

• The upgrader will generate the following variable declarations:

    <<declare $const_string = "" as string>>
    <<declare $const_number = 0 as number>>
    <<declare $const_bool = false as bool>>

• The upgrader is able to make use of type even when it appears later in the program, and is able to make inferences about type using indirect information.

// These variables are participating in expressions that include
// variables we've derived the type for earlier in this program, so they
// will be bound to that type
{$derived_expr_const_string + $const_string}
{$derived_expr_const_number + $const_number}
{$derived_expr_const_bool && $const_bool}

// These variables are participating in expressions that include
// variables that we define a type for later in this program. They will
// also be bound to that type.
{$derived_expr_const_string_late + $const_string_late}
{$derived_expr_const_number_late + $const_number_late}
{$derived_expr_const_bool_late && $const_bool_late}

<<set $const_string_late = "yes">>
<<set $const_number_late = 1>>
<<set $const_bool_late = true>>

• The upgrader will also make in-line changes to any if or elseif statements where the expression is determined to use a number rather than a bool will be rewritten so that the expression evaluates to a bool:

// Define some variables whose type is known before the expressions are
// hit
<<set $some_num_var = 1>>
<<set $some_other_num_var = 1>>

// This will be converted to a bool expression
<<if $some_num_var>>
<<elseif $some_other_num_var>>
<<endif>>

* Will be rewritten to:

<<elseif $some_other_num_var != 0>>
<<endif>>

You can use characters that the parser uses in scripts!

• Characters can now be escaped in lines and options.

• The \ character can be used to write characters that the parser would otherwise use.
• The following characters can be escaped: { } < > # / \
  • The / and < characters don't usually need to be escaped if they're appearing on their own (they're only meaningful when they appear in pairs), but this allows you to escape things like commands and comments.

Identifiers now support a wider range of characters.

This includes most multilingual letters and numbers, as well as symbols and emoji.

Made line conditions control the IsAvailable flag on options that are sent to the game.

• This change was made in order to allow games to conditionally present, but disallow, options that the player can't choose. For example, consider the following script:

TD-110: Let me see your identification.
-> Of course... um totally not General Kenobi and the son of Darth Vader.
    Luke: Wait, what?!
    TD-110: Promotion Time!
-> You don't need to see his identification. <<if $learnt_mind_trick is true>>
    TD-110: We don't need to see his identification.

• If the variable $learnt_mind_trick is false, a game may want to show the option but not allow the player to select it (i.e., show that this option could have been chosen if they'd learned how to do a mind trick.)
  • In previous versions of Yarn Spinner, if a line condition failed, the entire option was not delivered to the game. With this change, all options are delivered, and the OptionSet.Option.IsAvailable variable contains false if the condition was not met, and true if it was (or was not present.)
  • It's entirely up to the game to decide what to do with this information. To re-create the behaviour from previous Yarn Spinner versions, simply don't show any options whose IsAvailable value is false.

Variable declarations are now automatically determined, where possible

• If a variable is not declared (i.e. it doesn't have a <<declare>> statement), the compiler will now attempt to infer its declaration.
• When a variable doesn't have a declaration, the compiler will try to figure out the type based on how the variable is being used. It will always try to figure out the single type that the variable must be; if it's ambiguous, or no information is available at all, it will report an error, and you will have to add a declaration.

Variable declaration descriptions use comments

• Declarations have their descriptions set using a triple-slash (///) comment:

/// The number of coins the player has
<<declare $coins = 0>>

• These documentation comments can be before a declaration, or on the same line as a declaration:

<<declare $player_likes_dogs = true>> /// Whether the player likes dogs or not

• Multiple-line documentation comments are also supported:

/// Whether these are the droids that the 
/// guards are looking for.
<<declare $are_the_droids_we're_looking_for = false>>

A new type system has been added.

• The type-checking system in Yarn Spinner now supports types with supertypes and methods. This change has no significant impact on users writing Yarn scripts, but it enables the development of more advanced language features.
  • The main impact on users of this library (such as, for example, Yarn Spinner for Unity) is that the Yarn.Type enumeration has been removed, and is now replaced with the Yarn.IType interface and the BuiltinTypes class.
  • The type checker no longer hard-codes which operations can be run on which types; this decision is now determined by the types themselves.

Better Error Messages

The Compiler will no longer throw a ParseException, TypeException or CompilerException when an error is encountered during compilation. Instead, CompilationResult.Diagnostics contains a collection of Diagnostic objects, which represent errors, warnings, or other diagnostic information related to the compiled program.

• This change was implemented so that if multiple problems can be detected in a program, they can all be reported at once, rather than the compiler stopping at the first one.
• This also allows the compiler to issue non-fatal diagnostic messages, like warnings, that do not prevent the script from being compiled, but might indicate a problem with the code.
• Exceptions will continue to be thrown if the compiler encounters an internal error (in other words, if Yarn Spinner itself has a bug.)
  • If an error is encountered during compilation, CompilationResult.Program will be null.
  • This change means that compilation failures will not cause Compiler.Compile() to throw an exception; code that was previously using a try...catch to detect problems will need to be rewritten to check the CompilationResult.Diagnostics property to find the actual problem.

Fixes and Smaller Changes

• Fixed a crash in LineParser if a null input was provided to it.
• Fixed a crash in FormatFunctionUpgrader (which upgrades v1 Yarn scripts to v2) if an invalid format format function was encountered.
• Variable declaration upgrader now generates .yarnproject files, not .yarnprogram files.
• Line tagger now adds line tags before any // comment in the line.
• Dialogue: LogErrorMessage and LogDebugMessage now perform null-checks before being invoked.
• Utility.GenerateYarnFileWithDeclarations now generates files that use triple-slash (///) comments.
• Fixed a bug where expressions inside an if statement or elseif statement would not be type-checked.
• The keywords enum, endenum and case are now reserved.
• The type-conversion functions, string, number and bool, are no longer built-in special-case functions; they are now regular built-in functions that take a value of Any type.
• The lexer no longer uses semantic predicates when lexing the TEXT rule, which reduces the amount of C# code present in the grammar file.
• Markup can now be escaped, using the \ character:

\[b\]hello\[/b\]
// will appear to the user as "[b]hello[/b]", and will not 
// be treated as markup

• Dialogue.SetSelectedOption can now be called within the options handler itself.
  • If you do this, the Dialogue will continue executing after the options handler returns, and you do not need to call Continue.
• The compiler now generates better error messages for syntax errors. For example, given the following code (note the lack of an <<endif>> at the end):

<<if $has_key>>
  Guard: You found the key! Let me unlock the door.

The compiler will produce the following error message:

Expected an <<endif>> to match the <<if>> statement on line 1

• The compiler's new error messages now also report additional information about the context of a syntax error. For example, given the following code:

<<if hasCompletedObjective("find_key" >>
  // error! we forgot to add an ')'!
<<endif>>

The compiler will produce the following error message:

Unexpected ">>" while reading a function call

• VirtualMachine.executionState has been renamed to VirtualMachine.CurrentExecutionState.
• It is now a compiler error if the same line ID is used on more than one line.
• Dialogue.VariableStorage is now public.
• The ParseException, TypeException and CompilerException classes have been removed.