Merge pull request #318 from schungx/master

Ready 0.19.9
This commit is contained in:
Stephen Chung 2020-12-30 15:20:03 +08:00 committed by GitHub
commit f58d7937b5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
61 changed files with 301 additions and 176 deletions

View File

@ -33,6 +33,7 @@ Enhancements
* `Engine::register_static_module` now supports sub-module paths (e.g. `foo::bar::baz`).
* `Engine::register_custom_operator` now accepts reserved symbols.
* `Engine::register_custom_operator` now returns an error if given a precedence of zero.
* The examples `repl` and `rhai_runner` are moved into `bin` and renamed `rhai-repl` and `rhai-run` respectively.
Version 0.19.8

View File

@ -1,6 +1,6 @@
[package]
name = "rhai_codegen"
version = "0.3.0"
version = "0.3.1"
edition = "2018"
authors = ["jhwgh1968"]
description = "Procedural macro support package for Rhai, a scripting language for Rust"

View File

@ -1,5 +1,5 @@
Procedural Macros for Plugins
============================
=============================
This crate holds procedural macros for code generation, supporting the plugins system
for [Rhai](https://github.com/jonathandturner/rhai).

View File

@ -16,17 +16,18 @@ The Rhai Scripting Language
2. [Minimal](start/builds/minimal.md)
3. [no-std](start/builds/no-std.md)
4. [WebAssembly (WASM)](start/builds/wasm.md)
5. [Examples](start/examples/index.md)
5. [Packaged Utilities](start/bin.md)
6. [Examples](start/examples/index.md)
1. [Rust](start/examples/rust.md)
2. [Scripts](start/examples/scripts.md)
3. [Using the `Engine`](engine/index.md)
1. [Hello World in Rhai - Evaluate a Script](engine/hello-world.md)
1. [Hello World in Rhai – Evaluate a Script](engine/hello-world.md)
2. [Compile to AST for Repeated Evaluations](engine/compile.md)
3. [Call a Rhai Function from Rust](engine/call-fn.md)
4. [Create a Rust Closure from a Rhai Function](engine/func.md)
5. [Evaluate Expressions Only](engine/expressions.md)
6. [Raw Engine](engine/raw.md)
7. [Scope - Initializing and Maintaining State](engine/scope.md)
7. [Scope – Initializing and Maintaining State](engine/scope.md)
8. [Engine Configuration Options](engine/options.md)
4. [Extend Rhai with Rust](rust/index.md)
1. [Traits](rust/traits.md)
@ -34,22 +35,22 @@ The Rhai Scripting Language
1. [String Parameters in Rust Functions](rust/strings.md)
3. [Register a Generic Rust Function](rust/generic.md)
4. [Register a Fallible Rust Function](rust/fallible.md)
6. [Override a Built-in Function](rust/override.md)
7. [Operator Overloading](rust/operators.md)
8. [Register any Rust Type and its Methods](rust/custom.md)
5. [Override a Built-in Function](rust/override.md)
6. [Operator Overloading](rust/operators.md)
7. [Register any Rust Type and its Methods](rust/custom.md)
1. [Property Getters and Setters](rust/getters-setters.md)
2. [Indexers](rust/indexers.md)
3. [Disable Custom Types](rust/disable-custom.md)
4. [Printing Custom Types](rust/print-custom.md)
9. [Modules](rust/modules/index.md)
8. [Modules](rust/modules/index.md)
1. [Create from Rust](rust/modules/create.md)
2. [Create from AST](rust/modules/ast.md)
3. [Module Resolvers](rust/modules/resolvers.md)
1. [Custom Module Resolvers](rust/modules/imp-resolver.md)
10. [Plugins](plugins/index.md)
9. [Plugins](plugins/index.md)
1. [Export a Rust Module](plugins/module.md)
2. [Export a Rust Function](plugins/function.md)
11. [Packages](rust/packages/index.md)
10. [Packages](rust/packages/index.md)
1. [Built-in Packages](rust/packages/builtin.md)
2. [Custom Packages](rust/packages/create.md)
5. [Rhai Language Reference](language/index.md)

View File

@ -11,12 +11,12 @@ Easy
* Tight integration with native Rust [functions] and [types][custom types] including [getters/setters],
[methods][custom type] and [indexers].
* Freely pass Rust variables/constants into a script via an external [`Scope`] - all clonable Rust types are supported seamlessly
* Freely pass Rust variables/constants into a script via an external [`Scope`] – all clonable Rust types are supported seamlessly
without the need to implement any special trait.
* Easily [call a script-defined function]({{rootUrl}}/engine/call-fn.md) from Rust.
* Very few additional dependencies - right now only [`smallvec`](https://crates.io/crates/smallvec/) plus crates for procedural macros;
* Very few additional dependencies – right now only [`smallvec`](https://crates.io/crates/smallvec/) plus crates for procedural macros;
for [`no-std`] and `WASM` builds, a number of additional dependencies are pulled in to provide for missing functionalities.
* [Plugins] system powered by procedural macros simplifies custom API development.
@ -52,7 +52,7 @@ Safe
* Relatively little `unsafe` code (yes there are some for performance reasons).
* Sand-boxed - the scripting [`Engine`], if declared immutable, cannot mutate the containing environment unless
* Sand-boxed – the scripting [`Engine`], if declared immutable, cannot mutate the containing environment unless
[explicitly permitted]({{rootUrl}}/patterns/control.md).
Rugged

View File

@ -29,7 +29,7 @@ which is an embedded scripting language for C++.
Originally it was intended to be a scripting language similar to **JavaScript**.
With java being a kind of hot beverage, the new language was named after
another hot beverage - **Chai**, which is the word for "tea" in many world languages
another hot beverage – **Chai**, which is the word for "tea" in many world languages
and, in particular, a popular kind of milk tea consumed in India.
Later, when the novel implementation technique behind ChaiScript was ported from C++ to Rust,

View File

@ -10,24 +10,24 @@ It doesn't attempt to be a new language. For example:
* **No traits**... so it is also not Rust. Do your Rusty stuff in Rust.
* **No structures/records/tuples** - define your types in Rust instead; Rhai can seamlessly work with _any Rust type_.
* **No structures/records/tuples** – define your types in Rust instead; Rhai can seamlessly work with _any Rust type_.
There is, however, a built-in [object map] type which is adequate for most uses.
It is possible to simulate [object-oriented programming (OOP)][OOP] by storing [function pointers]
or [closures] in [object map] properties, turning them into _methods_.
* **No first-class functions** - Code your functions in Rust instead, and register them with Rhai.
* **No first-class functions** – Code your functions in Rust instead, and register them with Rhai.
There is, however, support for simple [function pointers] to allow runtime dispatch by function name.
* **No garbage collection** - this should be expected, so...
* **No garbage collection** – this should be expected, so...
* **No first-class closures** - do your closure magic in Rust instead: [turn a Rhai scripted function into a Rust closure]({{rootUrl}}/engine/call-fn.md).
* **No first-class closures** – do your closure magic in Rust instead: [turn a Rhai scripted function into a Rust closure]({{rootUrl}}/engine/call-fn.md).
There is, however, support for simulated [closures] via [currying] a [function pointer] with
captured shared variables.
* **No byte-codes/JIT** - Rhai has an optimized AST-walking interpreter which is fast enough for most casual
* **No byte-codes/JIT** – Rhai has an optimized AST-walking interpreter which is fast enough for most casual
usage scenarios. Essential AST data structures are packed and kept together to maximize cache friendliness.
Functions are dispatched based on pre-calculated hashes and accessing variables are mostly through pre-calculated
@ -41,7 +41,7 @@ It doesn't attempt to be a new language. For example:
Still, the purpose of Rhai is not to be super _fast_, but to make it as easy and versatile as possible to
integrate with native Rust applications.
* **No formal language grammar** - Rhai uses a hand-coded lexer, a hand-coded top-down recursive-descent parser
* **No formal language grammar** – Rhai uses a hand-coded lexer, a hand-coded top-down recursive-descent parser
for statements, and a hand-coded Pratt parser for expressions.
This lack of formalism allows the _tokenizer_ and _parser_ themselves to be exposed as services in order

View File

@ -7,30 +7,30 @@ Related Resources
Online Resources for Rhai
-------------------------
* [GitHub](https://github.com/jonathandturner/rhai) - Home repository
* [GitHub](https://github.com/jonathandturner/rhai) – Home repository
* [`crates.io`](https://crates.io/crates/rhai) - Rhai crate
* [`crates.io`](https://crates.io/crates/rhai) – Rhai crate
* [`DOCS.RS`](https://docs.rs/rhai) - Rhai API documentation
* [`DOCS.RS`](https://docs.rs/rhai) – Rhai API documentation
* [`LIB.RS`](https://lib.rs/crates/rhai) - Rhai library info
* [`LIB.RS`](https://lib.rs/crates/rhai) – Rhai library info
* [Discord Chat](https://discord.gg/HquqbYFcZ9) - Rhai channel
* [Discord Chat](https://discord.gg/HquqbYFcZ9) – Rhai channel
* [Reddit](https://www.reddit.com/r/Rhai) - Rhai community
* [Reddit](https://www.reddit.com/r/Rhai) – Rhai community
External Tools
--------------
* [Online Playground][playground] - Run Rhai scripts directly from an editor in the browser
* [Online Playground][playground] – Run Rhai scripts directly from an editor in the browser
* [`rhai-doc`] - Rhai script documentation tool
* [`rhai-doc`] – Rhai script documentation tool
Other Cool Projects
-------------------
* [ChaiScript](http://chaiscript.com) - A strong inspiration for Rhai. An embedded scripting language for C++.
* [ChaiScript](http://chaiscript.com) – A strong inspiration for Rhai. An embedded scripting language for C++.
* Check out the list of [scripting languages for Rust](https://github.com/rust-unofficial/awesome-rust#scripting) on [awesome-rust](https://github.com/rust-unofficial/awesome-rust)

View File

@ -3,7 +3,7 @@ Calling Rhai Functions from Rust
{{#include ../links.md}}
Rhai also allows working _backwards_ from the other direction - i.e. calling a Rhai-scripted function
Rhai also allows working _backwards_ from the other direction – i.e. calling a Rhai-scripted function
from Rust via `Engine::call_fn`.
Functions declared with `private` are hidden and cannot be called from Rust (see also [modules]).
@ -56,8 +56,8 @@ let result: () = engine.call_fn(&mut scope, &ast, "hidden", ())?;
```
Low-Level API - `Engine::call_fn_dynamic`
----------------------------------------
Low-Level API – `Engine::call_fn_dynamic`
----------------------------------------------
For more control, construct all arguments as `Dynamic` values and use `Engine::call_fn_dynamic`, passing it
anything that implements `AsMut<Dynamic>` (such as a simple array or a `Vec<Dynamic>`):

View File

@ -33,8 +33,8 @@ Where This Might Be Useful
* Where you just want to confuse your user and make their lives miserable, because you can.
Step One - Design The Syntax
---------------------------
Step One &ndash; Design The Syntax
---------------------------------
A custom syntax is simply a list of symbols.
@ -48,11 +48,11 @@ These symbol types can be used:
* Identifiers following the [variable] naming rules.
* `$expr$` - any valid expression, statement or statement block.
* `$expr$` &ndash; any valid expression, statement or statement block.
* `$block$` - any valid statement block (i.e. must be enclosed by `'{'` .. `'}'`).
* `$block$` &ndash; any valid statement block (i.e. must be enclosed by `'{'` .. `'}'`).
* `$ident$` - any [variable] name.
* `$ident$` &ndash; any [variable] name.
### The First Symbol Must be an Identifier
@ -103,8 +103,8 @@ print(hello); // variable declared by a custom syntax persists!
```
Step Two - Implementation
-------------------------
Step Two &ndash; Implementation
------------------------------
Any custom syntax must include an _implementation_ of it.
@ -176,8 +176,8 @@ let result = context.eval_expression_tree(expression)?;
```
Step Three - Register the Custom Syntax
--------------------------------------
Step Three &ndash; Register the Custom Syntax
--------------------------------------------
Use `Engine::register_custom_syntax` to register a custom syntax.
@ -244,8 +244,8 @@ exec |x| -> { x += 1 } while x < 0;
```
Step Four - Disable Unneeded Statement Types
-------------------------------------------
Step Four &ndash; Disable Unneeded Statement Types
-------------------------------------------------
When a DSL needs a custom syntax, most likely than not it is extremely specialized.
Therefore, many statement types actually may not make sense under the same usage scenario.
@ -258,24 +258,24 @@ the scenario.
A keyword or operator that is disabled can still be used in a custom syntax.
In an extreme case, it is possible to disable _every_ keyword in the language, leaving only
custom syntax (plus possibly expressions). But again, Don't Do It™ - unless you are certain
custom syntax (plus possibly expressions). But again, Don't Do It™ &ndash; unless you are certain
of what you're doing.
Step Five - Document
--------------------
Step Five &ndash; Document
-------------------------
For custom syntax, documentation is crucial.
Make sure there are _lots_ of examples for users to follow.
Step Six - Profit!
------------------
Step Six &ndash; Profit!
------------------------
Really Advanced - Custom Parsers
-------------------------------
Really Advanced &ndash; Custom Parsers
-------------------------------------
Sometimes it is desirable to have multiple custom syntax starting with the
same symbol. This is especially common for _command-style_ syntax where the
@ -396,7 +396,7 @@ Most strings are [`ImmutableString`][string]'s so it is usually more efficient t
The return value is `Result<Option<ImmutableString>, ParseError>` where:
| Value | Description |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Ok(None)` | parsing complete and there are no more symbols to match |
| `Ok(Some(symbol))` | the next symbol to match, which can also be `$expr$`, `$ident$` or `$block$` |
| `Err(ParseError)` | error that is reflected back to the [`Engine`] - normally `ParseError(ParseErrorType::BadInput(LexError::ImproperSymbol(message)), Position::NONE)` to indicate that there is a syntax error, but it can be any `ParseError`. |
| `Err(ParseError)` | error that is reflected back to the [`Engine`] &ndash; normally `ParseError(ParseErrorType::BadInput(LexError::ImproperSymbol(message)), Position::NONE)` to indicate that there is a syntax error, but it can be any `ParseError`. |

View File

@ -11,7 +11,7 @@ In these cases, use the `Engine::compile_expression` and `Engine::eval_expressio
let result = engine.eval_expression::<i64>("2 + (10 + 10) * 2")?;
```
When evaluating _expressions_, no full-blown statement (e.g. `if`, `while`, `for`, `fn`) - not even variable assignment -
When evaluating _expressions_, no full-blown statement (e.g. `if`, `while`, `for`, `fn`) &ndash; not even variable assignment &ndash;
is supported and will be considered parse errors when encountered.
[Closures] and [anonymous functions] are also not supported because in the background they compile to functions.

View File

@ -43,7 +43,7 @@ Return Type
The type parameter for `Engine::eval` is used to specify the type of the return value,
which _must_ match the actual type or an error is returned. Rhai is very strict here.
There are two ways to specify the return type - _turbofish_ notation, or type inference.
There are two ways to specify the return type &ndash; _turbofish_ notation, or type inference.
Use [`Dynamic`] for uncertain return types.

View File

@ -37,12 +37,12 @@ A function registered under the name `foo` with three parameters and unknown ret
> `foo(_, _, _)`
An operator function - again, unknown parameters and return type.
An operator function &ndash; again, unknown parameters and return type.
Notice that function names do not need to be valid identifiers.
> `+(_, _)`
A [property setter][getters/setters] - again, unknown parameters and return type.
A [property setter][getters/setters] &ndash; again, unknown parameters and return type.
Notice that function names do not need to be valid identifiers.
In this case, the first parameter should be `&mut T` of the custom type and the return value is `()`:

View File

@ -5,7 +5,7 @@ Optimization Levels
There are three levels of optimization: `None`, `Simple` and `Full`.
* `None` is obvious - no optimization on the AST is performed.
* `None` is obvious &ndash; no optimization on the AST is performed.
* `Simple` (default) performs only relatively _safe_ optimizations without causing side-effects
(i.e. it only relies on static analysis and [built-in operators] for constant [standard types],

View File

@ -17,7 +17,7 @@ if true { // condition always true
foo(42) // <- the above optimizes to this
```
If the original script were evaluated instead, it would have been an error - the variable `hello` does not exist,
If the original script were evaluated instead, it would have been an error &ndash; the variable `hello` does not exist,
so the script would have been terminated at that point with an error return.
In fact, any errors inside a statement that has been eliminated will silently _disappear_:

View File

@ -6,7 +6,7 @@ Volatility Considerations for Full Optimization Level
Even if a custom function does not mutate state nor cause side-effects, it may still be _volatile_,
i.e. it _depends_ on the external environment and is not _pure_.
A perfect example is a function that gets the current time - obviously each run will return a different value!
A perfect example is a function that gets the current time &ndash; obviously each run will return a different value!
The optimizer, when using [`OptimizationLevel::Full`], will _merrily assume_ that all functions are _pure_,
so when it finds constant arguments (or none) it eagerly executes the function call and replaces it with the result.

View File

@ -1,4 +1,4 @@
`Scope` - Initializing and Maintaining State
`Scope` &ndash; Initializing and Maintaining State
=================================================
{{#include ../links.md}}
@ -37,7 +37,7 @@ scope
// First invocation
engine.eval_with_scope::<()>(&mut scope, r"
let x = 4 + 5 - y + z + MY_NUMBER + s.len;
let x = 4 + 5 &ndash; y + z + MY_NUMBER + s.len;
y = 1;
")?;
@ -46,7 +46,7 @@ let result = engine.eval_with_scope::<i64>(&mut scope, "x")?;
println!("result: {}", result); // prints 1102
// Variable y is changed in the script - read it with 'get_value'
// Variable y is changed in the script &ndash; read it with 'get_value'
assert_eq!(scope.get_value::<i64>("y").expect("variable y should exist"), 1);
// We can modify scope variables directly with 'set_value'

View File

@ -61,11 +61,11 @@ r"
```
Caveat - Constants Can be Modified via Rust
------------------------------------------
Caveat &ndash; Constants Can be Modified via Rust
------------------------------------------------
A custom type stored as a constant cannot be modified via script, but _can_ be modified via
a registered Rust function that takes a first `&mut` parameter - because there is no way for
a registered Rust function that takes a first `&mut` parameter &ndash; because there is no way for
Rhai to know whether the Rust function modifies its argument!
```rust

View File

@ -32,7 +32,7 @@ switch type_of(mystery) {
Functions Returning `Dynamic`
----------------------------
In Rust, sometimes a `Dynamic` forms part of a returned value - a good example is an [array]
In Rust, sometimes a `Dynamic` forms part of a returned value &ndash; a good example is an [array]
which contains `Dynamic` elements, or an [object map] which contains `Dynamic` property values.
To get the _real_ values, the actual value types _must_ be known in advance.

View File

@ -50,8 +50,8 @@ fn anon_fn_1002() { print this.data; }
```
WARNING - NOT Real Closures
--------------------------
WARNING &ndash; NOT Real Closures
--------------------------------
Remember: anonymous functions, though having the same syntax as Rust _closures_, are themselves
**not** real closures.

View File

@ -13,8 +13,8 @@ access to the calling environment.
When a function accesses a variable that is not defined within that function's scope,
it raises an evaluation error.
It is possible, through a special syntax, to capture the calling scope - i.e. the scope
that makes the function call - and access variables defined there.
It is possible, through a special syntax, to capture the calling scope &ndash; i.e. the scope
that makes the function call &ndash; and access variables defined there.
```rust
fn foo(y) { // function accesses 'x' and 'y', but 'x' is not defined

View File

@ -10,7 +10,7 @@ Since [anonymous functions] de-sugar to standard function definitions, they reta
Rhai functions, including being _pure_, having no access to external variables.
The anonymous function syntax, however, automatically _captures_ variables that are not defined within
the current scope, but are defined in the external scope - i.e. the scope where the anonymous function
the current scope, but are defined in the external scope &ndash; i.e. the scope where the anonymous function
is created.
Variables that are accessible during the time the [anonymous function] is created can be captured,
@ -83,14 +83,14 @@ for f in funcs {
```
Therefore - Be Careful to Prevent Data Races
-------------------------------------------
Therefore &ndash; Be Careful to Prevent Data Races
-------------------------------------------------
Rust does not have data races, but that doesn't mean Rhai doesn't.
Avoid performing a method call on a captured shared variable (which essentially takes a
mutable reference to the shared object) while using that same variable as a parameter
in the method call - this is a sure-fire way to generate a data race error.
in the method call &ndash; this is a sure-fire way to generate a data race error.
If a shared value is used as the `this` pointer in a method call to a closure function,
then the same shared value _must not_ be captured inside that function, or a data race
@ -143,7 +143,7 @@ The actual implementation of closures de-sugars to:
1. Keeping track of what variables are accessed inside the anonymous function,
2. If a variable is not defined within the anonymous function's scope, it is looked up _outside_ the function and
in the current execution scope - where the anonymous function is created.
in the current execution scope &ndash; where the anonymous function is created.
3. The variable is added to the parameters list of the anonymous function, at the front.
@ -158,13 +158,13 @@ The actual implementation of closures de-sugars to:
### Q: Why are closures implemented as automatic currying?
In concept, a closure _closes_ over captured variables from the outer scope - that's why
In concept, a closure _closes_ over captured variables from the outer scope &ndash; that's why
they are called _closures_. When this happen, a typical language implementation hoists
those variables that are captured away from the stack frame and into heap-allocated storage.
This is because those variables may be needed after the stack frame goes away.
These heap-allocated captured variables only go away when all the closures that need them
are finished with them. A garbage collector makes this trivial to implement - they are
are finished with them. A garbage collector makes this trivial to implement &ndash; they are
automatically collected as soon as all closures needing them are destroyed.
In Rust, this can be done by reference counting instead, with the potential pitfall of creating

View File

@ -63,7 +63,7 @@ There is one _global_ namespace for every [`Engine`], which includes (in the fol
Anywhere in a Rhai script, when a function call is made, the function is searched within the
global namespace, in the above search order.
Therefore, function calls in Rhai are _late_ bound - meaning that the function called cannot be
Therefore, function calls in Rhai are _late_ bound &ndash; meaning that the function called cannot be
determined or guaranteed and there is no way to _lock down_ the function being called.
This aspect is very similar to JavaScript before ES6 modules.

View File

@ -125,8 +125,8 @@ x == 500; // 'x' is NOT changed!
```
`this` - Simulating an Object Method
-----------------------------------
`this` &ndash; Simulating an Object Method
-----------------------------------------
Script-defined functions can also be called in method-call style.
When this happens, the keyword '`this`' binds to the object in the method call and can be changed.

View File

@ -6,7 +6,7 @@ Parse an Object Map from JSON
The syntax for an [object map] is extremely similar to the JSON representation of a object hash,
with the exception of `null` values which can technically be mapped to [`()`].
A valid JSON string does not start with a hash character `#` while a Rhai [object map] does - that's the major difference!
A valid JSON string does not start with a hash character `#` while a Rhai [object map] does &ndash; that's the major difference!
Use the `Engine::parse_json` method to parse a piece of JSON into an object map.
The JSON text must represent a single object hash (i.e. must be wrapped within "`{ .. }`")

View File

@ -60,7 +60,7 @@ Boolean operators
| <code>\|\|</code> | boolean _OR_ | yes |
| <code>\|</code> | boolean _OR_ | no |
Double boolean operators `&&` and `||` _short-circuit_ - meaning that the second operand will not be evaluated
Double boolean operators `&&` and `||` _short-circuit_ &ndash; meaning that the second operand will not be evaluated
if the first one already proves the condition wrong.
Single boolean operators `&` and `|` always evaluate both operands.

View File

@ -15,7 +15,7 @@ Unlike functions defined in script (for which all arguments are passed by _value
native Rust functions may mutate the object (or the first argument if called in normal function call style).
However, sometimes it is not as straight-forward, and methods called in function-call style may end up
not muting the object - see the example below. Therefore, it is best to always use method-call style.
not muting the object &ndash; see the example below. Therefore, it is best to always use method-call style.
Custom types, properties and methods can be disabled via the [`no_object`] feature.
@ -59,7 +59,7 @@ The following table illustrates the differences:
Using a `&mut` first parameter is highly encouraged when using types that are expensive to clone,
even when the intention is not to mutate that argument, because it avoids cloning that argument value.
Even when a function is never intended to be a method - for example an operator,
Even when a function is never intended to be a method &ndash; for example an operator,
it is still sometimes beneficial to make it method-like (i.e. with a first `&mut` parameter)
if the first parameter is not modified.

View File

@ -78,7 +78,7 @@ for x in range(0, 1000) {
Recursive Imports
----------------
Beware of _import cycles_ - i.e. recursively loading the same module. This is a sure-fire way to
Beware of _import cycles_ &ndash; i.e. recursively loading the same module. This is a sure-fire way to
cause a stack overflow in the [`Engine`], unless stopped by setting a limit for [maximum number of modules].
For instance, importing itself always causes an infinite recursion:

View File

@ -10,7 +10,7 @@ The default system integer type (also aliased to `INT`) is `i64`. It can be turn
Floating-point numbers are also supported if not disabled with [`no_float`]. The default system floating-point type is `i64`
(also aliased to `FLOAT`). It can be turned into `f32` via the [`f32_float`] feature.
'`_`' separators can be added freely and are ignored within a number - except at the very beginning or right after
'`_`' separators can be added freely and are ignored within a number &ndash; except at the very beginning or right after
a decimal point ('`.`').
| Format | Type |

View File

@ -150,7 +150,7 @@ In order not to affect the speed of accessing properties in an object map, new p
property access.
A property [getter][getters/setters] function registered via `Engine::register_get`, for example,
for a `Map` will never be found - instead, the property will be looked up in the object map.
for a `Map` will never be found &ndash; instead, the property will be looked up in the object map.
Therefore, _method-call_ notation must be used for built-in properties:

View File

@ -4,7 +4,7 @@ Function Overloading
{{#include ../links.md}}
[Functions] defined in script can be _overloaded_ by _arity_ (i.e. they are resolved purely upon the function's _name_
and _number_ of parameters, but not parameter _types_ since all parameters are the same type - [`Dynamic`]).
and _number_ of parameters, but not parameter _types_ since all parameters are the same type &ndash; [`Dynamic`]).
New definitions _overwrite_ previous definitions of the same name and number of parameters.

View File

@ -24,7 +24,7 @@ An `ImmutableString` does not change and can be shared.
Modifying an `ImmutableString` causes it first to be cloned, and then the modification made to the copy.
### **IMPORTANT** - Avoid `String` Parameters
### **IMPORTANT** &ndash; Avoid `String` Parameters
`ImmutableString` should be used in place of `String` for function parameters because using
`String` is very inefficient (the `String` argument is cloned during every call).

View File

@ -76,7 +76,7 @@ for more details).
Difference From `if`-`else if` Chain
-------------------------------------
-----------------------------------
Although a `switch` expression looks _almost_ the same as an `if`-`else if` chain,
there are subtle differences between the two.
@ -98,8 +98,8 @@ efficient, but it also means that [overloading][operator overloading]
the `==` operator will have no effect.
Therefore, in environments where it is desirable to [overload][operator overloading]
the `==` operator - though it is difficult to think of valid scenarios where you'd want
`1 == 1` to return something other than `true` - avoid using the `switch` expression.
the `==` operator &ndash; though it is difficult to think of valid scenarios where you'd want
`1 == 1` to return something other than `true` &ndash; avoid using the `switch` expression.
### Efficiency

View File

@ -102,7 +102,7 @@ Non-Catchable Exceptions
Some exceptions _cannot_ be caught:
* Syntax error during parsing
* System error - e.g. script file not found
* System error &ndash; e.g. script file not found
* Script evaluation metrics over [safety limits]({{rootUrl}}/safety/index.md)
* Function calls nesting exceeding [maximum call stack depth]
* Script evaluation manually terminated

View File

@ -31,9 +31,9 @@ Custom Types
`type_of()` a [custom type] returns:
* if registered via `Engine::register_type_with_name` - the registered name
* if registered via `Engine::register_type_with_name` &ndash; the registered name
* if registered via `Engine::register_type` - the full Rust path name
* if registered via `Engine::register_type` &ndash; the full Rust path name
```rust
struct TestStruct1;

View File

@ -6,7 +6,7 @@ Variables
Valid Names
-----------
Variables in Rhai follow normal C naming rules - must contain only ASCII letters, digits and underscores '`_`',
Variables in Rhai follow normal C naming rules &ndash; must contain only ASCII letters, digits and underscores '`_`',
and cannot start with a digit.
For example: '`_c3po`' and '`r2d2`' are valid variable names, but '`3abc`' is not.

View File

@ -228,7 +228,7 @@ let x = switch [value.type, value.field_0, value.field_1] {
```
Usually, a helper method returns an array of values that can uniquely determine
the switch case based on actual usage requirements - which means that it probably
the switch case based on actual usage requirements &ndash; which means that it probably
skips fields that contain data instead of discriminants.
Then `switch` is used to very quickly match through a large number of array shapes

View File

@ -45,9 +45,9 @@ Multiple Instantiations of Rhai Within The Same Project
The trick is to differentiate between multiple identical copies of Rhai, each having
a different [features] set, by their _sources_:
* Different versions from [`crates.io`](https://crates.io/crates/rhai/) - The official crate.
* Different versions from [`crates.io`](https://crates.io/crates/rhai/) &ndash; The official crate.
* Different releases from [`GitHub`](https://github.com/jonathandturner/rhai) - Crate source on GitHub.
* Different releases from [`GitHub`](https://github.com/jonathandturner/rhai) &ndash; Crate source on GitHub.
* Forked copy of [https://github.com/jonathandturner/rhai](https://github.com/jonathandturner/rhai) on GitHub.
@ -73,14 +73,14 @@ If more than four different instantiations of Rhai is necessary (why?), create m
or GitHub forks or branches.
Caveat - No Way To Avoid Dependency Conflicts
--------------------------------------------
Caveat &ndash; No Way To Avoid Dependency Conflicts
--------------------------------------------------
Unfortunately, pulling in Rhai from different sources do not resolve the problem of
[features] conflict between dependencies. Even overriding `crates.io` via the `[patch]` manifest
section doesn't work - all dependencies will eventually find the only one copy.
section doesn't work &ndash; all dependencies will eventually find the only one copy.
What is necessary - multiple copies of Rhai, one for each dependent crate that requires it,
What is necessary &ndash; multiple copies of Rhai, one for each dependent crate that requires it,
together with their _unique_ [features] set intact. In other words, turning off Cargo's
crate merging feature _just for Rhai_.

View File

@ -24,7 +24,7 @@ Macros
Apply `#[export_fn]` onto a function defined at _module level_ to convert it into a Rhai plugin function.
The function cannot be nested inside another function - it can only be defined directly under a module.
The function cannot be nested inside another function &ndash; it can only be defined directly under a module.
To register the plugin function, simply call `register_exported_fn!`. The name of the function can be
any text string, so it is possible to register _overloaded_ functions as well as operators.

View File

@ -32,7 +32,7 @@ This Rust module can then be registered into an [`Engine`] as a normal [module].
This is done via the `exported_module!` macro.
The macro `combine_with_exported_module!` can be used to _combine_ all the functions
and variables into an existing [module], _flattening_ the namespace - i.e. all sub-modules
and variables into an existing [module], _flattening_ the namespace &ndash; i.e. all sub-modules
are eliminated and their contents promoted to the top level. This is typical for
developing [custom packages].

View File

@ -16,7 +16,7 @@ prepare a Rhai script for this purpose as well as to control which functions/var
When given an [`AST`], it is first evaluated, then the following items are exposed as members of the
new [module]:
* Global variables - all variables exported via the `export` statement (those not exported remain hidden).
* Global variables &ndash; all variables exported via the `export` statement (those not exported remain hidden).
* Functions not specifically marked `private`.

View File

@ -19,8 +19,8 @@ Manually creating a [module] is possible via the `Module` API.
For the complete `Module` API, refer to the [documentation](https://docs.rs/rhai/{{version}}/rhai/struct.Module.html) online.
Use Case 1 - Make the `Module` Globally Available
------------------------------------------------
Use Case 1 &ndash; Make the `Module` Globally Available
------------------------------------------------------
`Engine::register_global_module` registers a shared [module] into the _global_ namespace.
@ -62,8 +62,8 @@ engine.register_fn("inc", |x: i64| x + 1);
engine.eval::<i64>("inc(41)")? == 42; // no need to import module
```
Use Case 2 - Make the `Module` a Static Module
---------------------------------------------
Use Case 2 &ndash; Make the `Module` a Static Module
---------------------------------------------------
`Engine::register_static_module` registers a [module] and under a specific module namespace.
@ -122,8 +122,8 @@ engine.eval::<i64>("let x = 41; inc(x)")? == 42;
```
Use Case 3 - Make the `Module` Dynamically Loadable
--------------------------------------------------
Use Case 3 &ndash; Make the `Module` Dynamically Loadable
--------------------------------------------------------
In order to dynamically load a custom module, there must be a [module resolver] which serves
the module when loaded via `import` statements.

View File

@ -100,8 +100,8 @@ When there is a mutable reference to the `this` object (i.e. the first argument)
there can be no other immutable references to `args`, otherwise the Rust borrow checker will complain.
Example - Passing a Callback to a Rust Function
----------------------------------------------
Example &ndash; Passing a Callback to a Rust Function
----------------------------------------------------
The low-level API is useful when there is a need to interact with the scripting [`Engine`]
within a function.
@ -147,8 +147,8 @@ let result = engine.eval::<i64>(
```
TL;DR - Why `read_lock` and `write_lock`
---------------------------------------
TL;DR &ndash; Why `read_lock` and `write_lock`
---------------------------------------------
The `Dynamic` API that casts it to a reference to a particular data type is `read_lock`
(for an immutable reference) and `write_lock` (for a mutable reference).

View File

@ -108,7 +108,7 @@ let x: MyStruct = from_dynamic(&result)?;
Cannot Deserialize Shared Values
-------------------------------
A [`Dynamic`] containing a _shared_ value cannot be deserialized - i.e. it will give a type error.
A [`Dynamic`] containing a _shared_ value cannot be deserialized &ndash; i.e. it will give a type error.
Use `Dynamic::flatten` to obtain a cloned copy before deserialization
(if the value is not shared, it is simply returned and not cloned).

View File

@ -9,7 +9,7 @@ Avoid `String`
As must as possible, avoid using `String` parameters in functions.
Each `String` argument is cloned during every single call to that function - and the copy
Each `String` argument is cloned during every single call to that function &ndash; and the copy
immediately thrown away right after the call.
Needless to say, it is _extremely_ inefficient to use `String` parameters.

View File

@ -29,7 +29,7 @@ engine.set_max_operations(0); // allow unlimited operations
What Does One _Operation_ Mean
-----------------------------
The concept of one single _operation_ in Rhai is volatile - it roughly equals one expression node,
The concept of one single _operation_ in Rhai is volatile &ndash; it roughly equals one expression node,
loading one variable/constant, one operator call, one iteration of a loop, or one function call etc.
with sub-expressions, statements and function calls executed inside these contexts accumulated on top.

View File

@ -1,5 +1,5 @@
Sand-Boxing - Block Access to External Data
==========================================
Sand-Boxing &ndash; Block Access to External Data
================================================
{{#include ../links.md}}

24
doc/src/start/bin.md Normal file
View File

@ -0,0 +1,24 @@
Packaged Utilities
==================
{{#include ../links.md}}
A number of Rhai-driven utility programs can be found in the `src/bin` directory:
| Utility program | Description |
| :-----------------------------------------------: | ----------------------------------------------------------- |
| [`rhai-repl`]({{repoTree}}/examples/rhai-repl.rs) | a simple REPL, interactively evaluate statements from stdin |
| [`rhai-run`]({{repoTree}}/examples/rhai-run.rs) | runs each filename passed to it as a Rhai script |
`rhai-repl` is particularly useful &ndash; it allows one to interactively try out Rhai's
language features in a standard REPL (**R**ead-**E**val-**P**rint **L**oop).
Running a Utility Program
-------------------------
Utilities can be run with the following command:
```bash
cargo run --bin {program_name}
```

View File

@ -6,7 +6,7 @@ Minimal Build
Configuration
-------------
In order to compile a _minimal_ build - i.e. a build optimized for size - perhaps for `no-std` embedded targets or for
In order to compile a _minimal_ build &ndash; i.e. a build optimized for size &ndash; perhaps for `no-std` embedded targets or for
compiling to [WASM], it is essential that the correct linker flags are used in `cargo.toml`:
```toml

View File

@ -9,7 +9,7 @@ Some features are for performance. For example, using [`only_i32`] or [`only_i6
Use Only One Integer Type
------------------------
If only a single integer type is needed in scripts - most of the time this is the case - it is best to avoid registering
If only a single integer type is needed in scripts &ndash; most of the time this is the case &ndash; it is best to avoid registering
lots of functions related to other integer types that will never be used. As a result, [`Engine`] creation will be faster
because fewer functions need to be loaded.
@ -19,7 +19,7 @@ The [`only_i32`] and [`only_i64`] features disable all integer types except `i32
Use Only 32-Bit Numbers
----------------------
If only 32-bit integers are needed - again, most of the time this is the case - turn on [`only_i32`].
If only 32-bit integers are needed &ndash; again, most of the time this is the case &ndash; turn on [`only_i32`].
Under this feature, only `i32` is supported as a built-in integer type and no others.
On 64-bit targets this may not gain much, but on certain 32-bit targets this improves performance

View File

@ -7,7 +7,7 @@ It is possible to use Rhai when compiling to WebAssembly (WASM).
This yields a scripting engine (and language) that can be run in a standard web browser.
Why you would _want_ to is another matter... as there is already a nice, fast, complete scripting language
for the the common WASM environment (i.e. a browser) - and it is called JavaScript.
for the the common WASM environment (i.e. a browser) &ndash; and it is called JavaScript.
But anyhow, do it because you _can_!
@ -40,8 +40,8 @@ Some Rhai functionalities are not necessary in a WASM environment, so the follow
are typically used for a WASM build:
| Feature | Description |
| :-----------: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`unchecked`] | When a WASM module panics, it doesn't crash the entire web app; however this also disables [maximum number of operations] and [progress] tracking so a script can still run indefinitely - the web app must terminate it itself. |
| :-----------: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`unchecked`] | When a WASM module panics, it doesn't crash the entire web app; however this also disables [maximum number of operations] and [progress] tracking so a script can still run indefinitely &ndash; the web app must terminate it itself. |
| [`only_i32`] | WASM supports 32-bit and 64-bit integers, but most scripts will only need 32-bit. |
| [`f32_float`] | WASM supports 32-bit single-precision and 64-bit double-precision floating-point numbers, but single-precision is usually fine for most uses. |
| [`no_module`] | A WASM module cannot load modules from the file system, so usually this is not needed, but the savings are minimal; alternatively, a custom [module resolver] can be provided that loads other Rhai scripts. |

View File

@ -11,15 +11,10 @@ A number of examples can be found in the `examples` directory:
| [`custom_types_and_methods`]({{repoTree}}/examples/custom_types_and_methods.rs) | shows how to register a custom Rust type and methods for it |
| [`hello`]({{repoTree}}/examples/hello.rs) | simple example that evaluates an expression and prints the result |
| [`reuse_scope`]({{repoTree}}/examples/reuse_scope.rs) | evaluates two pieces of code in separate runs, but using a common [`Scope`] |
| [`rhai-repl`]({{repoTree}}/examples/rhai-repl.rs) | a simple REPL, interactively evaluate statements from stdin |
| [`rhai-run`]({{repoTree}}/examples/rhai-run.rs) | runs each filename passed to it as a Rhai script |
| [`serde`]({{repoTree}}/examples/serde.rs) | example to serialize and deserialize Rust types with [`serde`](https://crates.io/crates/serde).<br/>The [`serde`] feature is required to run |
| [`simple_fn`]({{repoTree}}/examples/simple_fn.rs) | shows how to register a simple function |
| [`strings`]({{repoTree}}/examples/strings.rs) | shows different ways to register functions taking string arguments |
The `rhai-repl` example is a particularly good one as it allows one to interactively try out Rhai's
language features in a standard REPL (**R**ead-**E**val-**P**rint **L**oop).
Running Examples
----------------

View File

@ -46,8 +46,8 @@ The following scripts are for benchmarking the speed of Rhai:
Running Example Scripts
----------------------
The [`rhai-run`](../examples/rust.md) example can be used to run the scripts:
The [`rhai-run`](../bin.md) utility can be used to run Rhai scripts:
```bash
cargo run --example rhai-run scripts/any_script.rhai
cargo run --bin rhai-run scripts/any_script.rhai
```

View File

@ -38,7 +38,7 @@ Example
The `Cargo.toml` configuration below turns on these six features:
* `sync` (everything `Send + Sync`)
* `unchecked` (disable all checking - should not be used with untrusted user scripts)
* `unchecked` (disable all checking &ndash; should not be used with untrusted user scripts)
* `only_i32` (only 32-bit signed integers)
* `no_float` (no floating point numbers)
* `no_module` (no loading external [modules])
@ -56,12 +56,12 @@ nor loading external [modules].
This configuration is perfect for an expression parser in a 32-bit embedded system without floating-point hardware.
Caveat - Features Are Not Additive
---------------------------------
Caveat &ndash; Features Are Not Additive
---------------------------------------
Most Rhai features are not strictly _additive_ - i.e. they do not only add optional functionalities.
Most Rhai features are not strictly _additive_ &ndash; i.e. they do not only add optional functionalities.
In fact, most features are _subtractive_ - i.e. they _remove_ functionalities.
In fact, most features are _subtractive_ &ndash; i.e. they _remove_ functionalities.
There is a reason for this design, because the _lack_ of a language feature by itself is a feature.

View File

@ -1,7 +1,7 @@
Sample Applications
===================
Sample applications written in Rhai.
Sample applications that use the Rhai scripting engine.
How to Run

12
src/bin/README.md Normal file
View File

@ -0,0 +1,12 @@
Rhai Tools
==========
Tools written in Rhai.
How to Run
----------
```bash
cargo run --bin sample_app_to_run
```

View File

@ -1,10 +1,16 @@
use rhai::{Dynamic, Engine, EvalAltResult, Scope, AST};
use rhai::{Dynamic, Engine, EvalAltResult, Module, Scope, AST};
#[cfg(not(feature = "no_optimize"))]
use rhai::OptimizationLevel;
use std::io::{stdin, stdout, Write};
use std::{
env,
fs::File,
io::{stdin, stdout, Read, Write},
process::exit,
};
/// Pretty-print error.
fn print_error(input: &str, err: EvalAltResult) {
let lines: Vec<_> = input.trim().split('\n').collect();
let pos = err.position();
@ -19,16 +25,17 @@ fn print_error(input: &str, err: EvalAltResult) {
"".to_string()
};
// Print error
// Print error position
let pos_text = format!(" ({})", pos);
if pos.is_none() {
// No position
println!("{}", err);
} else {
// Specific position
// Specific position - print line text
println!("{}{}", line_no, lines[pos.line().unwrap() - 1]);
// Display position marker
println!(
"{0:>1$} {2}",
"^",
@ -38,6 +45,7 @@ fn print_error(input: &str, err: EvalAltResult) {
}
}
/// Print help text.
fn print_help() {
println!("help => print this help");
println!("quit, exit => quit");
@ -52,6 +60,63 @@ fn print_help() {
fn main() {
let mut engine = Engine::new();
println!("Rhai REPL tool");
println!("==============");
print_help();
// Load init scripts
let mut contents = String::new();
let mut has_init_scripts = false;
for filename in env::args().skip(1) {
{
contents.clear();
let mut f = match File::open(&filename) {
Err(err) => {
eprintln!("Error reading script file: {}\n{}", filename, err);
exit(1);
}
Ok(f) => f,
};
if let Err(err) = f.read_to_string(&mut contents) {
println!("Error reading script file: {}\n{}", filename, err);
exit(1);
}
}
let module = match engine
.compile(&contents)
.map_err(|err| err.into())
.and_then(|ast| Module::eval_ast_as_new(Default::default(), &ast, &engine))
{
Err(err) => {
eprintln!("{:=<1$}", "", filename.len());
eprintln!("{}", filename);
eprintln!("{:=<1$}", "", filename.len());
eprintln!("");
print_error(&contents, *err);
exit(1);
}
Ok(m) => m,
};
engine.register_global_module(module.into());
has_init_scripts = true;
println!("Script '{}' loaded.", filename);
}
if has_init_scripts {
println!();
}
// Setup Engine
#[cfg(not(feature = "no_optimize"))]
engine.set_optimization_level(OptimizationLevel::None);
@ -62,9 +127,7 @@ fn main() {
let mut ast_u: AST = Default::default();
let mut ast: AST = Default::default();
println!("Rhai REPL tool");
println!("==============");
print_help();
// REPL loop
'main_loop: loop {
print!("rhai-repl> ");

View File

@ -37,6 +37,8 @@ fn eprint_error(input: &str, err: EvalAltResult) {
}
fn main() {
let mut contents = String::new();
for filename in env::args().skip(1) {
let mut engine = Engine::new();
@ -51,7 +53,7 @@ fn main() {
Ok(f) => f,
};
let mut contents = String::new();
contents.clear();
if let Err(err) = f.read_to_string(&mut contents) {
eprintln!("Error reading script file: {}\n{}", filename, err);

View File

@ -109,13 +109,13 @@ impl<'e, 's, 'a, 'm, 'pm> NativeCallContext<'e, 's, 'a, 'm, 'pm> {
pub fn new_with_all_fields(
engine: &'e Engine,
source: &'s Option<ImmutableString>,
mods: &'a mut Imports,
imports: &'a mut Imports,
lib: &'m impl AsRef<[&'pm Module]>,
) -> Self {
Self {
engine,
source: source.as_ref().map(|s| s.as_str()),
mods: Some(mods),
mods: Some(imports),
lib: lib.as_ref(),
}
}

View File

@ -295,13 +295,27 @@ impl fmt::Display for ParseError {
impl From<ParseErrorType> for Box<EvalAltResult> {
#[inline(always)]
fn from(err: ParseErrorType) -> Self {
Box::new(EvalAltResult::ErrorParsing(err, Position::NONE))
Box::new(err.into())
}
}
impl From<ParseErrorType> for EvalAltResult {
#[inline(always)]
fn from(err: ParseErrorType) -> Self {
EvalAltResult::ErrorParsing(err, Position::NONE)
}
}
impl From<ParseError> for Box<EvalAltResult> {
#[inline(always)]
fn from(err: ParseError) -> Self {
Box::new(EvalAltResult::ErrorParsing(*err.0, err.1))
Box::new(err.into())
}
}
impl From<ParseError> for EvalAltResult {
#[inline(always)]
fn from(err: ParseError) -> Self {
EvalAltResult::ErrorParsing(*err.0, err.1)
}
}

View File

@ -1685,18 +1685,18 @@ pub struct TokenIterator<'a, 'e> {
/// Input character stream.
stream: MultiInputsStream<'a>,
/// A processor function that maps a token to another.
map: fn(Token) -> Token,
map: Option<fn(Token) -> Token>,
}
impl<'a> Iterator for TokenIterator<'a, '_> {
type Item = (Token, Position);
fn next(&mut self) -> Option<Self::Item> {
let token = match get_next_token(&mut self.stream, &mut self.state, &mut self.pos) {
let (token, pos) = match get_next_token(&mut self.stream, &mut self.state, &mut self.pos) {
// {EOF}
None => None,
None => return None,
// Reserved keyword/symbol
Some((Token::Reserved(s), pos)) => Some((match
Some((Token::Reserved(s), pos)) => (match
(s.as_str(), self.engine.custom_keywords.contains_key(&s))
{
("===", false) => Token::LexError(LERR::ImproperSymbol(s,
@ -1736,16 +1736,16 @@ impl<'a> Iterator for TokenIterator<'a, '_> {
},
// Reserved keyword/operator that is not custom.
(_, false) => Token::Reserved(s),
}, pos)),
}, pos),
// Custom keyword
Some((Token::Identifier(s), pos)) if self.engine.custom_keywords.contains_key(&s) => {
Some((Token::Custom(s), pos))
(Token::Custom(s), pos)
}
// Custom standard keyword/symbol - must be disabled
Some((token, pos)) if self.engine.custom_keywords.contains_key(token.syntax().as_ref()) => {
if self.engine.disabled_symbols.contains(token.syntax().as_ref()) {
// Disabled standard keyword/symbol
Some((Token::Custom(token.syntax().into()), pos))
(Token::Custom(token.syntax().into()), pos)
} else {
// Active standard keyword - should never be a custom keyword!
unreachable!("{:?} is an active keyword", token)
@ -1753,16 +1753,20 @@ impl<'a> Iterator for TokenIterator<'a, '_> {
}
// Disabled symbol
Some((token, pos)) if self.engine.disabled_symbols.contains(token.syntax().as_ref()) => {
Some((Token::Reserved(token.syntax().into()), pos))
(Token::Reserved(token.syntax().into()), pos)
}
// Normal symbol
r => r,
Some(r) => r,
};
match token {
None => None,
Some((token, pos)) => Some(((self.map)(token), pos)),
}
// Run the mapper, if any
let token = if let Some(map) = self.map {
map(token)
} else {
token
};
Some((token, pos))
}
}
@ -1773,14 +1777,23 @@ impl Engine {
&'e self,
input: impl IntoIterator<Item = &'a &'a str>,
) -> TokenIterator<'a, 'e> {
self.lex_with_map(input, |f| f)
self.lex_raw(input, None)
}
/// Tokenize an input text stream with a mapping function.
#[inline]
#[inline(always)]
pub fn lex_with_map<'a, 'e>(
&'e self,
input: impl IntoIterator<Item = &'a &'a str>,
map: fn(Token) -> Token,
) -> TokenIterator<'a, 'e> {
self.lex_raw(input, Some(map))
}
/// Tokenize an input text stream with an optional mapping function.
#[inline]
fn lex_raw<'a, 'e>(
&'e self,
input: impl IntoIterator<Item = &'a &'a str>,
map: Option<fn(Token) -> Token>,
) -> TokenIterator<'a, 'e> {
TokenIterator {
engine: self,