kjuulh/rhai
1
0
Fork 0
Code Issues 1 Pull Requests 1 Packages Projects Releases Wiki Activity

Add custom operators.

Browse Source
This commit is contained in:
Stephen Chung 2020-07-05 17:41:45 +08:00
parent 936a3ff44a
commit e390dd73e6
13 changed files with 326 additions and 139 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -34,7 +34,7 @@ Features
* Rugged - protected against malicious attacks (such as [stack-overflow](https://schungx.github.io/rhai/safety/max-call-stack.html), [over-sized data](https://schungx.github.io/rhai/safety/max-string-size.html), and [runaway scripts](https://schungx.github.io/rhai/safety/max-operations.html) etc.) that may come from untrusted third-party user-land scripts. * Rugged - protected against malicious attacks (such as [stack-overflow](https://schungx.github.io/rhai/safety/max-call-stack.html), [over-sized data](https://schungx.github.io/rhai/safety/max-string-size.html), and [runaway scripts](https://schungx.github.io/rhai/safety/max-operations.html) etc.) that may come from untrusted third-party user-land scripts.
* Track script evaluation [progress](https://schungx.github.io/rhai/safety/progress.html) and manually terminate a script run. * Track script evaluation [progress](https://schungx.github.io/rhai/safety/progress.html) and manually terminate a script run.
* [Function overloading](https://schungx.github.io/rhai/language/overload.html). * [Function overloading](https://schungx.github.io/rhai/language/overload.html).
* [Operator overloading](https://schungx.github.io/rhai/rust/operators.html). * [Operator overloading](https://schungx.github.io/rhai/rust/operators.html) and [custom operators](https://schungx.github.io/rhai/engine/custom-op.html).
* Dynamic dispatch via [function pointers](https://schungx.github.io/rhai/language/fn-ptr.html). * Dynamic dispatch via [function pointers](https://schungx.github.io/rhai/language/fn-ptr.html).
* Some support for [object-oriented programming (OOP)](https://schungx.github.io/rhai/language/oop.html). * Some support for [object-oriented programming (OOP)](https://schungx.github.io/rhai/language/oop.html).
* Organize code base with dynamically-loadable [modules](https://schungx.github.io/rhai/language/modules.html). * Organize code base with dynamically-loadable [modules](https://schungx.github.io/rhai/language/modules.html).

6
RELEASES.md
View File

@ -6,8 +6,9 @@ Version 0.17.0
This version adds: This version adds:
* [`serde`](https://crates.io/crates/serde) support for working with `Dynamic` values (particularly _object maps_) * [`serde`](https://crates.io/crates/serde) support for working with `Dynamic` values (particularly _object maps_).
* Ability to surgically disable keywords and/or operators in the language * Ability to surgically disable keywords and/or operators in the language.
* Ability to define custom operators (which must be valid identifiers).
Breaking changes Breaking changes
---------------- ----------------
@ -20,6 +21,7 @@ New features
* New `serde` feature to allow serializating/deserializating to/from `Dynamic` values using [`serde`](https://crates.io/crates/serde). * New `serde` feature to allow serializating/deserializating to/from `Dynamic` values using [`serde`](https://crates.io/crates/serde).
This is particularly useful when converting a Rust `struct` to a `Dynamic` _object map_ and back. This is particularly useful when converting a Rust `struct` to a `Dynamic` _object map_ and back.
* `Engine::disable_symbol` to surgically disable keywords and/or operators. * `Engine::disable_symbol` to surgically disable keywords and/or operators.
* `Engine::register_custom_operator` to define a custom operator.
Version 0.16.1 Version 0.16.1

19
doc/src/SUMMARY.md
View File

@ -95,18 +95,19 @@ The Rhai Scripting Language
8. [Maximum Call Stack Depth](safety/max-call-stack.md) 8. [Maximum Call Stack Depth](safety/max-call-stack.md)
9. [Maximum Statement Depth](safety/max-stmt-depth.md) 9. [Maximum Statement Depth](safety/max-stmt-depth.md)
8. [Advanced Topics](advanced.md) 8. [Advanced Topics](advanced.md)
1. [Disable Keywords and/or Operators](engine/disable.md) 1. [Object-Oriented Programming (OOP)](language/oop.md)
2. [Object-Oriented Programming (OOP)](language/oop.md) 2. [Serialization/Deserialization of `Dynamic` with `serde`](rust/serde.md)
3. [Serialization/Deserialization of `Dynamic` with `serde`](rust/serde.md) 3. [Script Optimization](engine/optimize/index.md)
4. [Script Optimization](engine/optimize/index.md)
1. [Optimization Levels](engine/optimize/optimize-levels.md) 1. [Optimization Levels](engine/optimize/optimize-levels.md)
2. [Re-Optimize an AST](engine/optimize/reoptimize.md) 2. [Re-Optimize an AST](engine/optimize/reoptimize.md)
3. [Eager Function Evaluation](engine/optimize/eager.md) 3. [Eager Function Evaluation](engine/optimize/eager.md)
4. [Side-Effect Considerations](engine/optimize/side-effects.md) 4. [Side-Effect Considerations](engine/optimize/side-effects.md)
5. [Volatility Considerations](engine/optimize/volatility.md) 5. [Volatility Considerations](engine/optimize/volatility.md)
6. [Subtle Semantic Changes](engine/optimize/semantics.md) 6. [Subtle Semantic Changes](engine/optimize/semantics.md)
5. [Eval Statement](language/eval.md) 4. [Disable Keywords and/or Operators](engine/disable.md)
9. [Appendix](appendix/index.md) 5. [Custom Operators](engine/custom-op.md)
6. [Keywords](appendix/keywords.md) 6. [Eval Statement](language/eval.md)
7. [Operators](appendix/operators.md) 9. [Appendix](appendix/index.md)
8. [Literals](appendix/literals.md) 1. [Keywords](appendix/keywords.md)
2. [Operators](appendix/operators.md)
3. [Literals](appendix/literals.md)

2
doc/src/about/features.md
View File

@ -66,3 +66,5 @@ Flexible
* Supports [most build targets](targets.md) including `no-std` and [WASM]. * Supports [most build targets](targets.md) including `no-std` and [WASM].
* Surgically [disable keywords and operators] to restrict the language. * Surgically [disable keywords and operators] to restrict the language.
* [Custom operators].

50
doc/src/appendix/operators.md
View File

@ -3,28 +3,28 @@ Operators
{{#include ../links.md}} {{#include ../links.md}}
| Operator | Description | Binary? | | Operator | Description | Binary? | Binding direction |
| :---------------: | ------------------------------ | :-----: | | :---------------: | ------------------------------ | :-----: | :---------------: |
| `+` | Add | Yes | | `+` | Add | Yes | Left |
| `-` | Subtract, Minus | Yes/No | | `-` | Subtract, Minus | Yes/No | Left |
| `*` | Multiply | Yes | | `*` | Multiply | Yes | Left |
| `/` | Divide | Yes | | `/` | Divide | Yes | Left |
| `%` | Modulo | Yes | | `%` | Modulo | Yes | Left |
| `~` | Power | Yes | | `~` | Power | Yes | Left |
| `>>` | Right bit-shift | Yes | | `>>` | Right bit-shift | Yes | Left |
| `<<` | Left bit-shift | Yes | | `<<` | Left bit-shift | Yes | Left |
| `&` | Bit-wise _And_, Boolean _And_ | Yes | | `&` | Bit-wise _And_, Boolean _And_ | Yes | Left |
| <code>\|</code> | Bit-wise _Or_, Boolean _Or_ | Yes | | <code>\|</code> | Bit-wise _Or_, Boolean _Or_ | Yes | Left |
| `^` | Bit-wise _Xor_ | Yes | | `^` | Bit-wise _Xor_ | Yes | Left |
| `==` | Equals to | Yes | | `==` | Equals to | Yes | Left |
| `~=` | Not equals to | Yes | | `~=` | Not equals to | Yes | Left |
| `>` | Greater than | Yes | | `>` | Greater than | Yes | Left |
| `>=` | Greater than or equals to | Yes | | `>=` | Greater than or equals to | Yes | Left |
| `<` | Less than | Yes | | `<` | Less than | Yes | Left |
| `<=` | Less than or equals to | Yes | | `<=` | Less than or equals to | Yes | Left |
| `>=` | Greater than or equals to | Yes | | `>=` | Greater than or equals to | Yes | Left |
| `&&` | Boolean _And_ (short-circuits) | Yes | | `&&` | Boolean _And_ (short-circuits) | Yes | Left |
| <code>\|\|</code> | Boolean _Or_ (short-circuits) | Yes | | <code>\|\|</code> | Boolean _Or_ (short-circuits) | Yes | Left |
| `!` | Boolean _Not_ | No | | `!` | Boolean _Not_ | No | Left |
| `[` .. `]` | Indexing | Yes | | `[` .. `]` | Indexing | Yes | Right |
| `.` | Property access, Method call | Yes | | `.` | Property access, Method call | Yes | Right |

105
doc/src/engine/custom-op.md Normal file
View File

@ -0,0 +1,105 @@
Custom Operators
================
{{#include ../links.md}}
For use as a DSL (Domain-Specific Languages), it is sometimes more convenient to augment Rhai with
customized operators performing specific logic.
`Engine::register_custom_operator` registers a keyword as a custom operator.
Example
-------
```rust
use rhai::{Engine, RegisterFn};
let mut engine = Engine::new();
// Register a custom operator called 'foo' and give it
// a precedence of 140 (i.e. between +|- and *|/)
engine.register_custom_operator("foo", 140).unwrap();
// Register the implementation of the customer operator as a function
engine.register_fn("foo", |x: i64, y: i64| (x * y) - (x + y));
// The custom operator can be used in expressions
let result = engine.eval_expression::<i64>("1 + 2 * 3 foo 4 - 5 / 6")?;
// ^ custom operator
// The above is equivalent to: 1 + ((2 * 3) foo 4) - (5 / 6)
result == 15;
```
Alternatives to a Custom Operator
--------------------------------
Custom operators are merely _syntactic sugar_. They map directly to registered functions.
Therefore, the following are equivalent (assuming `foo` has been registered as a custom operator):
```rust
1 + 2 * 3 foo 4 - 5 / 6 // use custom operator
1 + foo(2 * 3, 4) - 5 / 6 // use function call
```
A script using custom operators can always be pre-processed, via a pre-processor application,
into a syntax that uses the corresponding function calls.
Using `Engine::register_custom_operator` merely enables a convenient short-cut.
Must Follow Variable Naming
--------------------------
All custom operators must be _identifiers_ that follow the same naming rules as [variables].
```rust
engine.register_custom_operator("foo", 20); // 'foo' is a valid custom operator
engine.register_custom_operator("=>", 30); // <- error: '=>' is not a valid custom operator
```
Binary Operators Only
---------------------
All custom operators must be _binary_ (i.e. they take two operands).
_Unary_ custom operators are not supported.
```rust
engine.register_custom_operator("foo", 140).unwrap();
engine.register_fn("foo", |x: i64| x * x);
engine.eval::<i64>("1 + 2 * 3 foo 4 - 5 / 6")?; // error: function 'foo (i64, i64)' not found
```
Operator Precedence
-------------------
All operators in Rhai has a _precedence_ indicating how tightly they bind.
The following _precedence table_ show the built-in precedence of standard Rhai operators:
| Category | Operators | Precedence (0-255) |
| ------------------- | :-------------------------------------------------------------------------------------: | :----------------: |
| Assignments | `=`, `+=`, `-=`, `*=`, `/=`, `~=`, `%=`,<br/>`<<=`, `>>=`, `&=`, <code>\|=</code>, `^=` | 0 |
| Logic and bit masks | <code>\|\|</code>, <code>\|</code>, `^` | 30 |
| Logic and bit masks | `&`, `&&` | 60 |
| Comparisons | `==`, `!=`, `>`, `>=`, `<`, `<=` | 90 |
| | `in` | 110 |
| Arithmetic | `+`, `-` | 130 |
| Arithmetic | `*`, `/`, `~` | 160 |
| Bit-shifts | `<<`, `>>` | 190 |
| Arithmetic | `%` | 210 |
| Object | `.` _(binds to right)_ | 240 |
| _Others_ | | 0 |
A higher precedence binds more tightly than a lower precedence, so `*` and `/` binds before `+` and `-` etc.
When registering a custom operator, the operator's precedence must also be provided.

2
doc/src/links.md
View File

@ -104,3 +104,5 @@
[`OptimizationLevel::None`]: {{rootUrl}}/engine/optimize/optimize-levels.md [`OptimizationLevel::None`]: {{rootUrl}}/engine/optimize/optimize-levels.md
[disable keywords and operators]: {{rootUrl}}/engine/disable.md [disable keywords and operators]: {{rootUrl}}/engine/disable.md
[custom operator]: {{rootUrl}}/engine/custom-op.md
[custom operators]: {{rootUrl}}/engine/custom-op.md

10
src/api.rs
View File

@ -553,7 +553,7 @@ impl Engine {
scripts: &[&str], scripts: &[&str],
optimization_level: OptimizationLevel, optimization_level: OptimizationLevel,
) -> Result<AST, ParseError> { ) -> Result<AST, ParseError> {
let stream = lex(scripts, self.max_string_size, self.disable_tokens.as_ref()); let stream = lex(scripts, self);
self.parse(&mut stream.peekable(), scope, optimization_level) self.parse(&mut stream.peekable(), scope, optimization_level)
} }
@ -678,7 +678,7 @@ impl Engine {
// Trims the JSON string and add a '#' in front // Trims the JSON string and add a '#' in front
let scripts = ["#", json.trim()]; let scripts = ["#", json.trim()];
let stream = lex(&scripts, self.max_string_size, self.disable_tokens.as_ref()); let stream = lex(&scripts, self);
let ast = let ast =
self.parse_global_expr(&mut stream.peekable(), &scope, OptimizationLevel::None)?; self.parse_global_expr(&mut stream.peekable(), &scope, OptimizationLevel::None)?;
@ -759,7 +759,7 @@ impl Engine {
script: &str, script: &str,
) -> Result<AST, ParseError> { ) -> Result<AST, ParseError> {
let scripts = [script]; let scripts = [script];
let stream = lex(&scripts, self.max_string_size, self.disable_tokens.as_ref()); let stream = lex(&scripts, self);
{ {
let mut peekable = stream.peekable(); let mut peekable = stream.peekable();
self.parse_global_expr(&mut peekable, scope, self.optimization_level) self.parse_global_expr(&mut peekable, scope, self.optimization_level)
@ -914,7 +914,7 @@ impl Engine {
script: &str, script: &str,
) -> Result<T, Box<EvalAltResult>> { ) -> Result<T, Box<EvalAltResult>> {
let scripts = [script]; let scripts = [script];
let stream = lex(&scripts, self.max_string_size, self.disable_tokens.as_ref()); let stream = lex(&scripts, self);
// No need to optimize a lone expression // No need to optimize a lone expression
let ast = self.parse_global_expr(&mut stream.peekable(), scope, OptimizationLevel::None)?; let ast = self.parse_global_expr(&mut stream.peekable(), scope, OptimizationLevel::None)?;
@ -1047,7 +1047,7 @@ impl Engine {
script: &str, script: &str,
) -> Result<(), Box<EvalAltResult>> { ) -> Result<(), Box<EvalAltResult>> {
let scripts = [script]; let scripts = [script];
let stream = lex(&scripts, self.max_string_size, self.disable_tokens.as_ref()); let stream = lex(&scripts, self);
let ast = self.parse(&mut stream.peekable(), scope, self.optimization_level)?; let ast = self.parse(&mut stream.peekable(), scope, self.optimization_level)?;
self.consume_ast_with_scope(scope, &ast) self.consume_ast_with_scope(scope, &ast)
} }

12
src/engine.rs
View File

@ -268,8 +268,10 @@ pub struct Engine {
/// A hashmap mapping type names to pretty-print names. /// A hashmap mapping type names to pretty-print names.
pub(crate) type_names: Option<HashMap<String, String>>, pub(crate) type_names: Option<HashMap<String, String>>,
/// A hash-set containing tokens to disable. /// A hashset containing symbols to disable.
pub(crate) disable_tokens: Option<HashSet<String>>, pub(crate) disabled_symbols: Option<HashSet<String>>,
/// A hashset containing custom keywords and precedence to recognize.
pub(crate) custom_keywords: Option<HashMap<String, u8>>,
/// Callback closure for implementing the `print` command. /// Callback closure for implementing the `print` command.
pub(crate) print: Callback<str, ()>, pub(crate) print: Callback<str, ()>,
@ -317,7 +319,8 @@ impl Default for Engine {
module_resolver: None, module_resolver: None,
type_names: None, type_names: None,
disable_tokens: None, disabled_symbols: None,
custom_keywords: None,
// default print/debug implementations // default print/debug implementations
print: Box::new(default_print), print: Box::new(default_print),
@ -497,7 +500,8 @@ impl Engine {
module_resolver: None, module_resolver: None,
type_names: None, type_names: None,
disable_tokens: None, disabled_symbols: None,
custom_keywords: None,
print: Box::new(|_| {}), print: Box::new(|_| {}),
debug: Box::new(|_| {}), debug: Box::new(|_| {}),

83
src/parser.rs
View File

@ -332,36 +332,26 @@ pub enum ReturnType {
Exception, Exception,
} }
#[derive(Debug, Clone, Eq, PartialEq, Hash, Default)] #[derive(Clone)]
struct ParseState { struct ParseState<'e> {
/// Reference to the scripting `Engine`.
engine: &'e Engine,
/// Encapsulates a local stack with variable names to simulate an actual runtime scope. /// Encapsulates a local stack with variable names to simulate an actual runtime scope.
pub stack: Vec<(String, ScopeEntryType)>, stack: Vec<(String, ScopeEntryType)>,
/// Encapsulates a local stack with variable names to simulate an actual runtime scope. /// Encapsulates a local stack with variable names to simulate an actual runtime scope.
pub modules: Vec<String>, modules: Vec<String>,
/// Maximum levels of expression nesting. /// Maximum levels of expression nesting.
pub max_expr_depth: usize, max_expr_depth: usize,
/// Maximum length of a string.
pub max_string_size: usize,
/// Maximum length of an array.
pub max_array_size: usize,
/// Maximum number of properties in a map.
pub max_map_size: usize,
} }
impl ParseState { impl<'e> ParseState<'e> {
/// Create a new `ParseState`. /// Create a new `ParseState`.
pub fn new( pub fn new(engine: &'e Engine, max_expr_depth: usize) -> Self {
max_expr_depth: usize,
max_string_size: usize,
max_array_size: usize,
max_map_size: usize,
) -> Self {
Self { Self {
engine,
max_expr_depth, max_expr_depth,
max_string_size, stack: Default::default(),
max_array_size, modules: Default::default(),
max_map_size,
..Default::default()
} }
} }
/// Find a variable by name in the `ParseState`, searching in reverse. /// Find a variable by name in the `ParseState`, searching in reverse.
@ -1206,10 +1196,10 @@ fn parse_array_literal(
let mut arr = StaticVec::new(); let mut arr = StaticVec::new();
while !input.peek().unwrap().0.is_eof() { while !input.peek().unwrap().0.is_eof() {
if state.max_array_size > 0 && arr.len() >= state.max_array_size { if state.engine.max_array_size > 0 && arr.len() >= state.engine.max_array_size {
return Err(PERR::LiteralTooLarge( return Err(PERR::LiteralTooLarge(
"Size of array literal".to_string(), "Size of array literal".to_string(),
state.max_array_size, state.engine.max_array_size,
) )
.into_err(input.peek().unwrap().1)); .into_err(input.peek().unwrap().1));
} }
@ -1306,10 +1296,10 @@ fn parse_map_literal(
} }
}; };
if state.max_map_size > 0 && map.len() >= state.max_map_size { if state.engine.max_map_size > 0 && map.len() >= state.engine.max_map_size {
return Err(PERR::LiteralTooLarge( return Err(PERR::LiteralTooLarge(
"Number of properties in object map literal".to_string(), "Number of properties in object map literal".to_string(),
state.max_map_size, state.engine.max_map_size,
) )
.into_err(input.peek().unwrap().1)); .into_err(input.peek().unwrap().1));
} }
@ -1866,7 +1856,8 @@ fn parse_binary_op(
loop { loop {
let (current_op, _) = input.peek().unwrap(); let (current_op, _) = input.peek().unwrap();
let precedence = current_op.precedence(); let custom = state.engine.custom_keywords.as_ref();
let precedence = current_op.precedence(custom);
let bind_right = current_op.is_bind_right(); let bind_right = current_op.is_bind_right();
// Bind left to the parent lhs expression if precedence is higher // Bind left to the parent lhs expression if precedence is higher
@ -1879,7 +1870,7 @@ fn parse_binary_op(
let rhs = parse_unary(input, state, settings)?; let rhs = parse_unary(input, state, settings)?;
let next_precedence = input.peek().unwrap().0.precedence(); let next_precedence = input.peek().unwrap().0.precedence(custom);
// Bind to right if the next operator has higher precedence // Bind to right if the next operator has higher precedence
// If same precedence, then check if the operator binds right // If same precedence, then check if the operator binds right
@ -1949,6 +1940,19 @@ fn parse_binary_op(
make_dot_expr(current_lhs, rhs, pos)? make_dot_expr(current_lhs, rhs, pos)?
} }
Token::Custom(s)
if state
.engine
.custom_keywords
.as_ref()
.map(|c| c.contains_key(&s))
.unwrap_or(false) =>
{
// Accept non-native functions for custom operators
let op = (op.0, false, op.2);
Expr::FnCall(Box::new((op, None, hash, args, None)))
}
op_token => return Err(PERR::UnknownOperator(op_token.into()).into_err(pos)), op_token => return Err(PERR::UnknownOperator(op_token.into()).into_err(pos)),
}; };
} }
@ -2467,7 +2471,7 @@ fn parse_fn(
settings.ensure_level_within_max_limit(state.max_expr_depth)?; settings.ensure_level_within_max_limit(state.max_expr_depth)?;
let name = match input.next().unwrap() { let name = match input.next().unwrap() {
(Token::Identifier(s), _) => s, (Token::Identifier(s), _) | (Token::Custom(s), _) => s,
(_, pos) => return Err(PERR::FnMissingName.into_err(pos)), (_, pos) => return Err(PERR::FnMissingName.into_err(pos)),
}; };
@ -2555,12 +2559,7 @@ impl Engine {
scope: &Scope, scope: &Scope,
optimization_level: OptimizationLevel, optimization_level: OptimizationLevel,
) -> Result<AST, ParseError> { ) -> Result<AST, ParseError> {
let mut state = ParseState::new( let mut state = ParseState::new(self, self.max_expr_depth);
self.max_expr_depth,
self.max_string_size,
self.max_array_size,
self.max_map_size,
);
let settings = ParseSettings { let settings = ParseSettings {
allow_if_expr: false, allow_if_expr: false,
allow_stmt_expr: false, allow_stmt_expr: false,
@ -2596,12 +2595,7 @@ impl Engine {
) -> Result<(Vec<Stmt>, Vec<ScriptFnDef>), ParseError> { ) -> Result<(Vec<Stmt>, Vec<ScriptFnDef>), ParseError> {
let mut statements = Vec::<Stmt>::new(); let mut statements = Vec::<Stmt>::new();
let mut functions = HashMap::<u64, ScriptFnDef, _>::with_hasher(StraightHasherBuilder); let mut functions = HashMap::<u64, ScriptFnDef, _>::with_hasher(StraightHasherBuilder);
let mut state = ParseState::new( let mut state = ParseState::new(self, self.max_expr_depth);
self.max_expr_depth,
self.max_string_size,
self.max_array_size,
self.max_map_size,
);
while !input.peek().unwrap().0.is_eof() { while !input.peek().unwrap().0.is_eof() {
// Collect all the function definitions // Collect all the function definitions
@ -2615,12 +2609,7 @@ impl Engine {
match input.peek().unwrap() { match input.peek().unwrap() {
(Token::Fn, pos) => { (Token::Fn, pos) => {
let mut state = ParseState::new( let mut state = ParseState::new(self, self.max_function_expr_depth);
self.max_function_expr_depth,
self.max_string_size,
self.max_array_size,
self.max_map_size,
);
let settings = ParseSettings { let settings = ParseSettings {
allow_if_expr: true, allow_if_expr: true,
allow_stmt_expr: true, allow_stmt_expr: true,

57
src/settings.rs
View File

@ -2,6 +2,7 @@ use crate::engine::Engine;
use crate::module::ModuleResolver; use crate::module::ModuleResolver;
use crate::optimize::OptimizationLevel; use crate::optimize::OptimizationLevel;
use crate::packages::PackageLibrary; use crate::packages::PackageLibrary;
use crate::token::is_valid_identifier;
impl Engine { impl Engine {
/// Load a new package into the `Engine`. /// Load a new package into the `Engine`.
@ -194,10 +195,60 @@ impl Engine {
/// # } /// # }
/// ``` /// ```
pub fn disable_symbol(&mut self, symbol: &str) { pub fn disable_symbol(&mut self, symbol: &str) {
if self.disable_tokens.is_none() { if self.disabled_symbols.is_none() {
self.disable_tokens = Some(Default::default()); self.disabled_symbols = Some(Default::default());
} }
self.disable_tokens.as_mut().unwrap().insert(symbol.into()); self.disabled_symbols
.as_mut()
.unwrap()
.insert(symbol.into());
}
/// Register a custom operator into the language.
///
/// The operator must be a valid identifier (i.e. it cannot be a symbol).
///
/// # Examples
///
/// ```rust
/// # fn main() -> Result<(), Box<rhai::EvalAltResult>> {
/// use rhai::{Engine, RegisterFn};
///
/// let mut engine = Engine::new();
///
/// // Register a custom operator called 'foo' and give it
/// // a precedence of 140 (i.e. between +|- and *|/).
/// engine.register_custom_operator("foo", 140).unwrap();
///
/// // Register a binary function named 'foo'
/// engine.register_fn("foo", |x: i64, y: i64| (x * y) - (x + y));
///
/// assert_eq!(
/// engine.eval_expression::<i64>("1 + 2 * 3 foo 4 - 5 / 6")?,
/// 15
/// );
/// # Ok(())
/// # }
/// ```
pub fn register_custom_operator(
&mut self,
keyword: &str,
precedence: u8,
) -> Result<(), String> {
if !is_valid_identifier(keyword.chars()) {
return Err(format!("not a valid identifier: '{}'", keyword).into());
}
if self.custom_keywords.is_none() {
self.custom_keywords = Some(Default::default());
}
self.custom_keywords
.as_mut()
.unwrap()
.insert(keyword.into(), precedence);
Ok(())
} }
} }

86
src/token.rs
View File

@ -1,5 +1,6 @@
//! Main module defining the lexer and parser. //! Main module defining the lexer and parser.
use crate::engine::Engine;
use crate::error::LexError; use crate::error::LexError;
use crate::parser::INT; use crate::parser::INT;
use crate::utils::StaticVec; use crate::utils::StaticVec;
@ -11,7 +12,7 @@ use crate::stdlib::{
borrow::Cow, borrow::Cow,
boxed::Box, boxed::Box,
char, char,
collections::HashSet, collections::{HashMap, HashSet},
fmt, fmt,
iter::Peekable, iter::Peekable,
str::{Chars, FromStr}, str::{Chars, FromStr},
@ -212,6 +213,7 @@ pub enum Token {
As, As,
LexError(Box<LexError>), LexError(Box<LexError>),
Comment(String), Comment(String),
Custom(String),
EOF, EOF,
} }
@ -224,12 +226,13 @@ impl Token {
IntegerConstant(i) => i.to_string().into(), IntegerConstant(i) => i.to_string().into(),
#[cfg(not(feature = "no_float"))] #[cfg(not(feature = "no_float"))]
FloatConstant(f) => f.to_string().into(), FloatConstant(f) => f.to_string().into(),
Identifier(s) => s.clone().into(), StringConstant(_) => "string".into(),
CharConstant(c) => c.to_string().into(), CharConstant(c) => c.to_string().into(),
Identifier(s) => s.clone().into(),
Custom(s) => s.clone().into(),
LexError(err) => err.to_string().into(), LexError(err) => err.to_string().into(),
token => match token { token => match token {
StringConstant(_) => "string",
LeftBrace => "{", LeftBrace => "{",
RightBrace => "}", RightBrace => "}",
LeftParen => "(", LeftParen => "(",
@ -324,9 +327,9 @@ impl Token {
match self { match self {
LexError(_) | LexError(_) |
LeftBrace | // (+expr) - is unary LeftBrace | // {+expr} - is unary
// RightBrace | {expr} - expr not unary & is closing // RightBrace | {expr} - expr not unary & is closing
LeftParen | // {-expr} - is unary LeftParen | // (-expr) - is unary
// RightParen | (expr) - expr not unary & is closing // RightParen | (expr) - expr not unary & is closing
LeftBracket | // [-expr] - is unary LeftBracket | // [-expr] - is unary
// RightBracket | [expr] - expr not unary & is closing // RightBracket | [expr] - expr not unary & is closing
@ -371,14 +374,14 @@ impl Token {
Throw | Throw |
PowerOf | PowerOf |
In | In |
PowerOfAssign => true, PowerOfAssign => true,
_ => false, _ => false,
} }
} }
/// Get the precedence number of the token. /// Get the precedence number of the token.
pub fn precedence(&self) -> u8 { pub fn precedence(&self, custom: Option<&HashMap<String, u8>>) -> u8 {
use Token::*; use Token::*;
match self { match self {
@ -387,24 +390,27 @@ impl Token {
| RightShiftAssign | AndAssign | OrAssign | XOrAssign | ModuloAssign | RightShiftAssign | AndAssign | OrAssign | XOrAssign | ModuloAssign
| PowerOfAssign => 0, | PowerOfAssign => 0,
Or | XOr | Pipe => 40, Or | XOr | Pipe => 30,
And | Ampersand => 50, And | Ampersand => 60,
LessThan | LessThanEqualsTo | GreaterThan | GreaterThanEqualsTo | EqualsTo LessThan | LessThanEqualsTo | GreaterThan | GreaterThanEqualsTo | EqualsTo
| NotEqualsTo => 60, | NotEqualsTo => 90,
In => 70, In => 110,
Plus | Minus => 80, Plus | Minus => 130,
Divide | Multiply | PowerOf => 90, Divide | Multiply | PowerOf => 160,
LeftShift | RightShift => 100, LeftShift | RightShift => 190,
Modulo => 110, Modulo => 210,
Period => 120, Period => 240,
// Custom operators
Custom(s) => custom.map_or(0, |c| *c.get(s).unwrap()),
_ => 0, _ => 0,
} }
@ -1211,9 +1217,9 @@ impl InputStream for MultiInputsStream<'_> {
} }
/// An iterator on a `Token` stream. /// An iterator on a `Token` stream.
pub struct TokenIterator<'a, 't> { pub struct TokenIterator<'a, 'e> {
/// Disable certain tokens. /// Reference to the scripting `Engine`.
pub disable_tokens: Option<&'t HashSet<String>>, engine: &'e Engine,
/// Current state. /// Current state.
state: TokenizeState, state: TokenizeState,
/// Current position. /// Current position.
@ -1226,15 +1232,15 @@ impl<'a> Iterator for TokenIterator<'a, '_> {
type Item = (Token, Position); type Item = (Token, Position);
fn next(&mut self) -> Option<Self::Item> { fn next(&mut self) -> Option<Self::Item> {
match get_next_token(&mut self.stream, &mut self.state, &mut self.pos) { match (
None => None, get_next_token(&mut self.stream, &mut self.state, &mut self.pos),
r @ Some(_) if self.disable_tokens.is_none() => r, self.engine.disabled_symbols.as_ref(),
Some((token, pos)) self.engine.custom_keywords.as_ref(),
if token.is_operator() ) {
&& self (None, _, _) => None,
.disable_tokens (r @ Some(_), None, None) => r,
.unwrap() (Some((token, pos)), Some(disabled), _)
.contains(token.syntax().as_ref()) => if token.is_operator() && disabled.contains(token.syntax().as_ref()) =>
{ {
// Convert disallowed operators into lex errors // Convert disallowed operators into lex errors
Some(( Some((
@ -1242,31 +1248,27 @@ impl<'a> Iterator for TokenIterator<'a, '_> {
pos, pos,
)) ))
} }
Some((token, pos)) (Some((token, pos)), Some(disabled), _)
if token.is_keyword() if token.is_keyword() && disabled.contains(token.syntax().as_ref()) =>
&& self
.disable_tokens
.unwrap()
.contains(token.syntax().as_ref()) =>
{ {
// Convert disallowed keywords into identifiers // Convert disallowed keywords into identifiers
Some((Token::Identifier(token.syntax().into()), pos)) Some((Token::Identifier(token.syntax().into()), pos))
} }
r => r, (Some((Token::Identifier(s), pos)), _, Some(custom)) if custom.contains_key(&s) => {
// Convert custom keywords
Some((Token::Custom(s), pos))
}
(r, _, _) => r,
} }
} }
} }
/// Tokenize an input text stream. /// Tokenize an input text stream.
pub fn lex<'a, 't>( pub fn lex<'a, 'e>(input: &'a [&'a str], engine: &'e Engine) -> TokenIterator<'a, 'e> {
input: &'a [&'a str],
max_string_size: usize,
disable_tokens: Option<&'t HashSet<String>>,
) -> TokenIterator<'a, 't> {
TokenIterator { TokenIterator {
disable_tokens, engine,
state: TokenizeState { state: TokenizeState {
max_string_size, max_string_size: engine.max_string_size,
non_unary: false, non_unary: false,
comment_level: 0, comment_level: 0,
end_with_none: false, end_with_none: false,

31
tests/tokens.rs
View File

@ -1,4 +1,4 @@
use rhai::{Engine, ParseErrorType}; use rhai::{Engine, EvalAltResult, ParseErrorType, RegisterFn, INT};
#[test] #[test]
fn test_tokens_disabled() { fn test_tokens_disabled() {
@ -18,3 +18,32 @@ fn test_tokens_disabled() {
ParseErrorType::BadInput(ref s) if s == "Unexpected '+='" ParseErrorType::BadInput(ref s) if s == "Unexpected '+='"
)); ));
} }
#[test]
fn test_tokens_custom_operator() -> Result<(), Box<EvalAltResult>> {
let mut engine = Engine::new();
// Register a custom operator called `foo` and give it
// a precedence of 140 (i.e. between +|- and *|/).
engine.register_custom_operator("foo", 140).unwrap();
// Register a binary function named `foo`
engine.register_fn("foo", |x: INT, y: INT| (x * y) - (x + y));
assert_eq!(
engine.eval_expression::<INT>("1 + 2 * 3 foo 4 - 5 / 6")?,
15
);
assert_eq!(
engine.eval::<INT>(
r"
fn foo(x, y) { y - x }
1 + 2 * 3 foo 4 - 5 / 6
"
)?,
-1
);
Ok(())
}