rhai/doc/src/engine/custom-syntax.md

Extend Rhai with Custom Syntax

{{#include ../links.md}}

For the ultimate advantageous, there is a built-in facility to extend the Rhai language with custom-defined syntax.

But before going off to define the next weird statement type, heed this warning:

Don't Do It™

Stick with standard language syntax as much as possible.

Having to learn Rhai is bad enough, no sane user would ever want to learn yet another obscure language syntax just to do something.

Try to use [custom operators] first. Defining a custom syntax should be considered a last resort.

Where This Might Be Useful

• Where an operation is used a LOT and a custom syntax saves a lot of typing.

• Where a custom syntax significantly simplifies the code and significantly enhances understanding of the code's intent.

• Where certain logic cannot be easily encapsulated inside a function. This is usually the case where closures are required, because Rhai does not have closures.

• Where you just want to confuse your user and make their lives miserable, because you can.

Step One - Start With internals

Since a custom syntax taps deeply into the AST and evaluation process of the Engine, the [internals] feature must be on in order to expose these necessary internal data structures.

Beware that Rhai internal data structures are volatile and may change without warning.

Caveat emptor.

Step Two - Design The Syntax

A custom syntax is simply a list of symbols.

These symbol types can be used:

• Standard keywords

• Standard operators.

• Reserved symbols.

• Identifiers following the [variable] naming rules.

• $expr$ - any valid expression, statement or statement block.

• $block$ - any valid statement block (i.e. must be enclosed by '{' .. '}').

• $ident$ - any [variable] name.

The First Symbol Must be a Keyword

There is no specific limit on the combination and sequencing of each symbol type, except the first symbol which must be a [custom keyword].

It cannot be a built-in keyword.

However, it may be a built-in keyword that has been [disabled][disable keywords and operators].

The First Symbol Must be Unique

Rhai uses the first symbol as a clue to parse custom syntax.

Therefore, at any one time, there can only be one custom syntax starting with each unique symbol.

Any new custom syntax definition using the same first symbol simply overwrites the previous one.

Example

exec $ident$ <- $expr$ : $block$

The above syntax is made up of a stream of symbols:

Position	Input	Symbol	Description
1		exec	custom keyword
2	1	$ident$	a variable name
3		<-	the left-arrow symbol (which is a reserved symbol in Rhai).
4	2	$expr$	an expression, which may be enclosed with { .. }, or not.
5		:	the colon symbol
6	3	$block$	a statement block, which must be enclosed with { .. }.

This syntax matches the following sample code and generates three inputs (one for each non-keyword):

// Assuming the 'exec' custom syntax implementation declares the variable 'hello':
let x = exec hello <- foo(1, 2) : {
            hello += bar(hello);
            baz(hello);
        };

print(x);       // variable 'x'  has a value returned by the custom syntax

print(hello);   // variable declared by a custom syntax persists!

Step Three - Implementation

Any custom syntax must include an implementation of it.

Function Signature

The function signature of an implementation is:

Fn(
    engine: &Engine,
    scope: &mut Scope,
    mods: &mut Imports,
    state: &mut State,
    lib: &Module,
    this_ptr: &mut Option<&mut Dynamic>,
    inputs: &[Expression],
    level: usize
) -> Result<Dynamic, Box<EvalAltResult>>

where:

• engine : &Engine - reference to the current [Engine].
• scope : &mut Scope - mutable reference to the current [Scope]; variables can be added to it.
• mods : &mut Imports - mutable reference to the current collection of imported [Module]'s; do not touch.
• state : &mut State - mutable reference to the current evaluation state; do not touch.
• lib : &Module - reference to the current collection of script-defined functions.
• this_ptr : &mut Option<&mut Dynamic> - mutable reference to the current binding of the this pointer; do not touch.
• inputs : &[Expression] - a list of input expression trees.
• level : usize - the current function call level.

There are a lot of parameters, most of which should not be touched or Bad Things Happen™. They represent the running content of a script evaluation and should simply be passed straight-through the the [Engine].

Access Arguments

The most important argument is inputs where the matched identifiers ($ident$), expressions/statements ($expr$) and statement blocks (`block) are provided.

To access a particular argument, use the following patterns:

Argument type	Pattern (n = slot in inputs)	Result type	Description
$ident$	inputs[n].get_variable_name().unwrap()	&str	name of a variable
$expr$	inputs.get(n).unwrap()	Expression	an expression tree
$block$	inputs.get(n).unwrap()	Expression	an expression tree

Evaluate an Expression Tree

Use the engine::eval_expression_tree method to evaluate an expression tree.

let expr = inputs.get(0).unwrap();
let result = engine.eval_expression_tree(scope, mods, state, lib, this_ptr, expr, level)?;

As can be seem above, most arguments are simply passed straight-through to engine::eval_expression_tree.

Declare Variables

New variables maybe declared (usually with a variable name that is passed in via `ident).

It can simply be pushed into the [scope].

However, beware that all new variables must be declared prior to evaluating any expression tree. In other words, any scope.push(...) calls must come before any engine::eval_expression_tree(...) calls.

let var_name = inputs[0].get_variable_name().unwrap().to_string();
let expr = inputs.get(1).unwrap();

scope.push(var_name, 0 as INT);     // do this BEFORE engine.eval_expression_tree!

let result = engine.eval_expression_tree(scope, mods, state, lib, this_ptr, expr, level)?;

Step Four - Register the Custom Syntax

Use Engine::register_custom_syntax to register a custom syntax.

Again, beware that the first symbol must be unique. If there already exists a custom syntax starting with that symbol, the previous syntax will be overwritten.

The syntax is passed simply as a slice of &str.

// Custom syntax implementation
fn implementation_func(
    engine: &Engine,
    scope: &mut Scope,
    mods: &mut Imports,
    state: &mut State,
    lib: &Module,
    this_ptr: &mut Option<&mut Dynamic>,
    inputs: &[Expression],
    level: usize
) -> Result<Dynamic, Box<EvalAltResult>> {
    let var_name = inputs[0].get_variable_name().unwrap().to_string();
    let stmt = inputs.get(1).unwrap();
    let condition = inputs.get(2).unwrap();

    // Push one new variable into the 'scope' BEFORE 'eval_expression_tree'
    scope.push(var_name, 0 as INT);

    loop {
        // Evaluate the statement block
        engine.eval_expression_tree(scope, mods, state, lib, this_ptr, stmt, level)?;

        // Evaluate the condition expression
        let stop = !engine
            .eval_expression_tree(scope, mods, state, lib, this_ptr, condition, level)?
            .as_bool()
            .map_err(|_| EvalAltResult::ErrorBooleanArgMismatch(
                "do-while".into(), expr.position()
            ))?;

        if stop {
            break;
        }
    }

    Ok(().into())
}

// Register the custom syntax (sample): do |x| -> { x += 1 } while x < 0;
engine.register_custom_syntax(
    &[ "do", "|", "$ident$", "|", "->", "$block$", "while", "$expr$" ], // the custom syntax
    1,  // the number of new variables declared within this custom syntax
    implementation_func
)?;

Step Five - Disable Unneeded Statement Types

When a DSL needs a custom syntax, most likely than not it is extremely specialized. Therefore, many statement types actually may not make sense under the same usage scenario.

So, while at it, better [disable][disable keywords and operators] those built-in keywords and operators that should not be used by the user. The would leave only the bare minimum language surface exposed, together with the custom syntax that is tailor-designed for the scenario.

A keyword or operator that is disabled can still be used in a custom syntax.

In an extreme case, it is possible to disable every keyword in the language, leaving only custom syntax (plus possibly expressions). But again, Don't Do It™ - unless you are certain of what you're doing.

Step Six - Document

For custom syntax, documentation is crucial.

Make sure there are lots of examples for users to follow.

Step Seven - Profit!