kjuulh/rhai
1
0
Fork 0
Code Issues 1 Pull Requests 1 Packages Projects Releases Wiki Activity

Enable custom syntax without internals.

Browse Source
This commit is contained in:
Stephen Chung 2020-07-22 17:05:13 +08:00
parent 35374f5b3b
commit abf66850f6
8 changed files with 98 additions and 131 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
RELEASES.md
View File

@ -15,6 +15,7 @@ New features
* Disallow many keywords as variables, such as `print`, `eval`, `call`, `this` etc.
* `x.call(f, ...)` allows binding `x` to `this` for the function referenced by the function pointer `f`.
* Anonymous functions in the syntax of a closure, e.g. `|x, y, z| x + y - z`.
* Custom syntax now works even without the `internals` feature.
Breaking changes
----------------
@ -48,7 +49,7 @@ Breaking changes
New features
------------
* New `serde` feature to allow serializating/deserializating to/from `Dynamic` values using [`serde`](https://crates.io/crates/serde).
* New `serde` feature to allow serializing/deserializing to/from `Dynamic` values using [`serde`](https://crates.io/crates/serde).
This is particularly useful when converting a Rust `struct` to a `Dynamic` _object map_ and back.
* `Engine::disable_symbol` to surgically disable keywords and/or operators.
* `Engine::register_custom_operator` to define a custom operator.

37
doc/src/engine/custom-syntax.md
View File

@ -33,18 +33,7 @@ Where This Might Be Useful
* Where you just want to confuse your user and make their lives miserable, because you can.
Step One - Start With `internals`
--------------------------------
Since a custom syntax taps deeply into the `AST` and evaluation process of the `Engine`,
the [`internals`] feature must be on in order to expose these necessary internal data structures.
Beware that Rhai internal data structures are _volatile_ and may change without warning.
Caveat emptor.
Step Two - Design The Syntax
Step One - Design The Syntax
---------------------------
A custom syntax is simply a list of symbols.
@ -116,8 +105,8 @@ print(hello); // variable declared by a custom syntax persists!
```
Step Three - Implementation
--------------------------
Step Two - Implementation
-------------------------
Any custom syntax must include an _implementation_ of it.
@ -164,8 +153,6 @@ let expr = inputs.get(0).unwrap();
let result = engine.eval_expression_tree(context, scope, expr)?;
```
As can be seem above, most arguments are simply passed straight-through to `engine::eval_expression_tree`.
### Declare Variables
New variables maybe declared (usually with a variable name that is passed in via `$ident$).
@ -179,14 +166,14 @@ In other words, any `scope.push(...)` calls must come _before_ any `engine::eval
let var_name = inputs[0].get_variable_name().unwrap().to_string();
let expr = inputs.get(1).unwrap();
scope.push(var_name, 0 as INT); // do this BEFORE engine.eval_expression_tree!
scope.push(var_name, 0 as INT); // do this BEFORE 'engine.eval_expression_tree'!
let result = engine.eval_expression_tree(scope, mods, state, lib, this_ptr, expr, level)?;
let result = engine.eval_expression_tree(context, scope, expr)?;
```
Step Four - Register the Custom Syntax
-------------------------------------
Step Three - Register the Custom Syntax
--------------------------------------
Use `Engine::register_custom_syntax` to register a custom syntax.
@ -236,7 +223,7 @@ engine.register_custom_syntax(
```
Step Five - Disable Unneeded Statement Types
Step Four - Disable Unneeded Statement Types
-------------------------------------------
When a DSL needs a custom syntax, most likely than not it is extremely specialized.
@ -254,13 +241,13 @@ custom syntax (plus possibly expressions). But again, Don't Do It™ - unless y
of what you're doing.
Step Six - Document
-------------------
Step Five - Document
--------------------
For custom syntax, documentation is crucial.
Make sure there are _lots_ of examples for users to follow.
Step Seven - Profit!
--------------------
Step Six - Profit!
------------------

31
doc/src/engine/dsl.md
View File

@ -11,7 +11,7 @@ Expressions Only
In many DSL scenarios, only evaluation of expressions is needed.
The `Engine::eval_expression_XXX`[`eval_expression`] API can be used to restrict
The [`Engine::eval_expression_XXX`][`eval_expression`] API can be used to restrict
a script to expressions only.
@ -21,8 +21,7 @@ Disable Keywords and/or Operators
In some DSL scenarios, it is necessary to further restrict the language to exclude certain
language features that are not necessary or dangerous to the application.
For example, a DSL may disable the `while` loop altogether while keeping all other statement
types intact.
For example, a DSL may disable the `while` loop while keeping all other statement types intact.
It is possible, in Rhai, to surgically [disable keywords and operators].
@ -54,31 +53,29 @@ Custom Syntax
For advanced DSL scenarios, it is possible to define entire expression [_syntax_][custom syntax] -
essentially custom statement types.
The [`internals`] feature is needed to be able to define [custom syntax] in Rhai.
For example, the following is a SQL like syntax for some obscure DSL operation:
For example, the following is a SQL-like syntax for some obscure DSL operation:
```rust
let table = [..., ..., ..., ...];
// Syntax = "calculate" $ident$ $ident$ "from" $expr$ "->" $ident$ ":" $expr$
// Syntax = calculate $ident$ $ident$ from $expr$ -> $ident$ : $expr$
let total = calculate sum price from table -> row : row.weight > 50;
// Note: There is nothing special about the use of symbols; to make it look exactly like SQL:
// Syntax = "SELECT" $ident$ "(" $ident$ ")" "FROM" $expr$ "AS" $ident$ "WHERE" $expr$
// Note: There is nothing special about those symbols; to make it look exactly like SQL:
// Syntax = SELECT $ident$ ( $ident$ ) FROM $expr$ AS $ident$ WHERE $expr$
let total = SELECT sum(price) FROM table AS row WHERE row.weight > 50;
```
After registering this custom syntax with Rhai, it can be used anywhere inside a script as
a normal expression.
For its evaluation, the callback function will receive the following list of parameters:
For its evaluation, the callback function will receive the following list of inputs:
`exprs[0] = "sum"` - math operator
`exprs[1] = "price"` - field name
`exprs[2] = Expression(table)` - data source
`exprs[3] = "row"` - loop variable name
`exprs[4] = Expression(row.wright > 50)` - expression
* `inputs[0] = "sum"` - math operator
* `inputs[1] = "price"` - field name
* `inputs[2] = Expression(table)` - data source
* `inputs[3] = "row"` - loop variable name
* `inputs[4] = Expression(row.wright > 50)` - filter predicate
The other identified, such as `"select"`, `"from"`, as as as symbols `->` and `:` are
parsed in the order defined within the custom syntax.
Other identifiers, such as `"calculate"`, `"from"`, as well as symbols such as `->` and `:`,
are parsed in the order defined within the custom syntax.

47
src/engine.rs
View File

@ -11,15 +11,13 @@ use crate::parser::{Expr, FnAccess, ImmutableString, ReturnType, ScriptFnDef, St
use crate::r#unsafe::unsafe_cast_var_name_to_lifetime;
use crate::result::EvalAltResult;
use crate::scope::{EntryType as ScopeEntryType, Scope};
use crate::syntax::{CustomSyntax, EvalContext, Expression};
use crate::token::Position;
use crate::utils::StaticVec;
#[cfg(not(feature = "no_float"))]
use crate::parser::FLOAT;
#[cfg(feature = "internals")]
use crate::syntax::{CustomSyntax, EvalContext};
use crate::stdlib::{
any::{type_name, TypeId},
borrow::Cow,
@ -88,45 +86,10 @@ pub const FN_SET: &str = "set$";
pub const FN_IDX_GET: &str = "index$get$";
pub const FN_IDX_SET: &str = "index$set$";
pub const FN_ANONYMOUS: &str = "anon$";
#[cfg(feature = "internals")]
pub const MARKER_EXPR: &str = "$expr$";
#[cfg(feature = "internals")]
pub const MARKER_BLOCK: &str = "$block$";
#[cfg(feature = "internals")]
pub const MARKER_IDENT: &str = "$ident$";
#[cfg(feature = "internals")]
#[derive(Debug, Clone, Hash)]
pub struct Expression<'a>(&'a Expr);
#[cfg(feature = "internals")]
impl<'a> From<&'a Expr> for Expression<'a> {
fn from(expr: &'a Expr) -> Self {
Self(expr)
}
}
#[cfg(feature = "internals")]
impl Expression<'_> {
/// If this expression is a variable name, return it. Otherwise `None`.
#[cfg(feature = "internals")]
pub fn get_variable_name(&self) -> Option<&str> {
match self.0 {
Expr::Variable(x) => Some((x.0).0.as_str()),
_ => None,
}
}
/// Get the expression.
pub(crate) fn expr(&self) -> &Expr {
&self.0
}
/// Get the position of this expression.
pub fn position(&self) -> Position {
self.0.position()
}
}
/// A type specifying the method of chaining.
#[derive(Debug, Clone, Copy, Eq, PartialEq, Hash)]
enum ChainType {
@ -316,7 +279,6 @@ pub struct Engine {
/// A hashset containing custom keywords and precedence to recognize.
pub(crate) custom_keywords: Option<HashMap<String, u8>>,
/// Custom syntax.
#[cfg(feature = "internals")]
pub(crate) custom_syntax: Option<HashMap<String, CustomSyntax>>,
/// Callback closure for implementing the `print` command.
@ -376,8 +338,6 @@ impl Default for Engine {
type_names: None,
disabled_symbols: None,
custom_keywords: None,
#[cfg(feature = "internals")]
custom_syntax: None,
// default print/debug implementations
@ -605,8 +565,6 @@ impl Engine {
type_names: None,
disabled_symbols: None,
custom_keywords: None,
#[cfg(feature = "internals")]
custom_syntax: None,
print: Box::new(|_| {}),
@ -1734,8 +1692,6 @@ impl Engine {
/// ## WARNING - Low Level API
///
/// This function is very low level. It evaluates an expression from an AST.
#[cfg(feature = "internals")]
#[deprecated(note = "this method is volatile and may change")]
pub fn eval_expression_tree(
&self,
context: &mut EvalContext,
@ -2215,7 +2171,6 @@ impl Engine {
Expr::False(_) => Ok(false.into()),
Expr::Unit(_) => Ok(().into()),
#[cfg(feature = "internals")]
Expr::Custom(x) => {
let func = (x.0).1.as_ref();
let ep: StaticVec<_> = (x.0).0.iter().map(|e| e.into()).collect();

8
src/lib.rs
View File

@ -91,7 +91,6 @@ mod scope;
mod serde;
mod settings;
mod stdlib;
#[cfg(feature = "internals")]
mod syntax;
mod token;
mod r#unsafe;
@ -106,6 +105,7 @@ pub use module::Module;
pub use parser::{ImmutableString, AST, INT};
pub use result::EvalAltResult;
pub use scope::Scope;
pub use syntax::{EvalContext, Expression};
pub use token::Position;
pub use utils::calc_fn_spec as calc_fn_hash;
@ -169,11 +169,7 @@ pub use parser::{CustomExpr, Expr, ReturnType, ScriptFnDef, Stmt};
#[cfg(feature = "internals")]
#[deprecated(note = "this type is volatile and may change")]
pub use engine::{Expression, Imports, State as EvalState};
#[cfg(feature = "internals")]
#[deprecated(note = "this type is volatile and may change")]
pub use syntax::EvalContext;
pub use engine::{Imports, State as EvalState};
#[cfg(feature = "internals")]
#[deprecated(note = "this type is volatile and may change")]

25
src/parser.rs
View File

@ -2,23 +2,19 @@
use crate::any::{Dynamic, Union};
use crate::calc_fn_hash;
use crate::engine::{make_getter, make_setter, Engine, FN_ANONYMOUS, KEYWORD_THIS};
use crate::engine::{
make_getter, make_setter, Engine, FN_ANONYMOUS, KEYWORD_THIS, MARKER_BLOCK, MARKER_EXPR,
MARKER_IDENT,
};
use crate::error::{LexError, ParseError, ParseErrorType};
use crate::fn_native::Shared;
use crate::module::{Module, ModuleRef};
use crate::optimize::{optimize_into_ast, OptimizationLevel};
use crate::scope::{EntryType as ScopeEntryType, Scope};
use crate::syntax::FnCustomSyntaxEval;
use crate::token::{is_valid_identifier, Position, Token, TokenStream};
use crate::utils::{StaticVec, StraightHasherBuilder};
#[cfg(feature = "internals")]
use crate::engine::{MARKER_BLOCK, MARKER_EXPR, MARKER_IDENT};
#[cfg(feature = "internals")]
use crate::fn_native::Shared;
#[cfg(feature = "internals")]
use crate::syntax::FnCustomSyntaxEval;
use crate::stdlib::{
borrow::Cow,
boxed::Box,
@ -587,17 +583,14 @@ impl Stmt {
}
#[derive(Clone)]
#[cfg(feature = "internals")]
pub struct CustomExpr(pub StaticVec<Expr>, pub Shared<FnCustomSyntaxEval>);
#[cfg(feature = "internals")]
impl fmt::Debug for CustomExpr {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
fmt::Debug::fmt(&self.0, f)
}
}
#[cfg(feature = "internals")]
impl Hash for CustomExpr {
fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
self.0.hash(state);
@ -683,7 +676,6 @@ pub enum Expr {
/// ()
Unit(Position),
/// Custom syntax
#[cfg(feature = "internals")]
Custom(Box<(CustomExpr, Position)>),
}
@ -781,7 +773,6 @@ impl Expr {
Self::Dot(x) | Self::Index(x) => x.0.position(),
#[cfg(feature = "internals")]
Self::Custom(x) => x.1,
}
}
@ -816,8 +807,6 @@ impl Expr {
Self::Assignment(x) => x.3 = new_pos,
Self::Dot(x) => x.2 = new_pos,
Self::Index(x) => x.2 = new_pos,
#[cfg(feature = "internals")]
Self::Custom(x) => x.1 = new_pos,
}
@ -925,7 +914,6 @@ impl Expr {
_ => false,
},
#[cfg(feature = "internals")]
Self::Custom(_) => false,
}
}
@ -2148,7 +2136,6 @@ fn parse_expr(
settings.ensure_level_within_max_limit(state.max_expr_depth)?;
// Check if it is a custom syntax.
#[cfg(feature = "internals")]
if let Some(ref custom) = state.engine.custom_syntax {
let (token, pos) = input.peek().unwrap();
let token_pos = *pos;

71
src/syntax.rs
View File

@ -1,14 +1,13 @@
//! Module containing implementation for custom syntax.
#![cfg(feature = "internals")]
use crate::any::Dynamic;
use crate::engine::{Engine, Expression, Imports, State, MARKER_BLOCK, MARKER_EXPR, MARKER_IDENT};
use crate::error::LexError;
use crate::engine::{Engine, Imports, State, MARKER_BLOCK, MARKER_EXPR, MARKER_IDENT};
use crate::error::{LexError, ParseError, ParseErrorType};
use crate::fn_native::{SendSync, Shared};
use crate::module::Module;
use crate::parser::Expr;
use crate::result::EvalAltResult;
use crate::scope::Scope;
use crate::token::{is_valid_identifier, Token};
use crate::token::{is_valid_identifier, Position, Token};
use crate::utils::StaticVec;
use crate::stdlib::{
@ -32,6 +31,33 @@ pub type FnCustomSyntaxEval = dyn Fn(&Engine, &mut EvalContext, &mut Scope, &[Ex
+ Send
+ Sync;
#[derive(Debug, Clone, Hash)]
pub struct Expression<'a>(&'a Expr);
impl<'a> From<&'a Expr> for Expression<'a> {
fn from(expr: &'a Expr) -> Self {
Self(expr)
}
}
impl Expression<'_> {
/// If this expression is a variable name, return it. Otherwise `None`.
pub fn get_variable_name(&self) -> Option<&str> {
match self.0 {
Expr::Variable(x) => Some((x.0).0.as_str()),
_ => None,
}
}
/// Get the expression.
pub(crate) fn expr(&self) -> &Expr {
&self.0
}
/// Get the position of this expression.
pub fn position(&self) -> Position {
self.0.position()
}
}
#[derive(Clone)]
pub struct CustomSyntax {
pub segments: StaticVec<String>,
@ -45,6 +71,7 @@ impl fmt::Debug for CustomSyntax {
}
}
#[derive(Debug)]
pub struct EvalContext<'a, 'b: 'a, 's, 'm, 't, 'd: 't> {
pub(crate) mods: &'a mut Imports<'b>,
pub(crate) state: &'s mut State,
@ -56,7 +83,7 @@ pub struct EvalContext<'a, 'b: 'a, 's, 'm, 't, 'd: 't> {
impl Engine {
pub fn register_custom_syntax<S: AsRef<str> + ToString>(
&mut self,
value: &[S],
keywords: &[S],
scope_delta: isize,
func: impl Fn(
&Engine,
@ -66,15 +93,18 @@ impl Engine {
) -> Result<Dynamic, Box<EvalAltResult>>
+ SendSync
+ 'static,
) -> Result<&mut Self, Box<LexError>> {
if value.is_empty() {
return Err(Box::new(LexError::ImproperSymbol("".to_string())));
}
) -> Result<&mut Self, Box<ParseError>> {
let mut segments: StaticVec<_> = Default::default();
for s in value {
let seg = match s.as_ref() {
for s in keywords {
let s = s.as_ref().trim();
// skip empty keywords
if s.is_empty() {
continue;
}
let seg = match s {
// Markers not in first position
MARKER_EXPR | MARKER_BLOCK | MARKER_IDENT if !segments.is_empty() => s.to_string(),
// Standard symbols not in first position
@ -115,12 +145,25 @@ impl Engine {
s.into()
}
// Anything else is an error
_ => return Err(Box::new(LexError::ImproperSymbol(s.to_string()))),
_ => {
return Err(LexError::ImproperSymbol(format!(
"Improper symbol for custom syntax: '{}'",
s
))
.into_err(Position::none())
.into());
}
};
segments.push(seg);
}
// If the syntax has no keywords, just ignore the registration
if segments.is_empty() {
return Ok(self);
}
// Remove the first keyword as the discriminator
let key = segments.remove(0);
let syntax = CustomSyntax {

7
tests/syntax.rs
View File

@ -1,5 +1,6 @@
#![cfg(feature = "internals")]
use rhai::{Engine, EvalAltResult, EvalContext, Expression, LexError, ParseErrorType, Scope, INT};
use rhai::{
Engine, EvalAltResult, EvalContext, Expression, ParseError, ParseErrorType, Scope, INT,
};
#[test]
fn test_custom_syntax() -> Result<(), Box<EvalAltResult>> {
@ -72,7 +73,7 @@ fn test_custom_syntax() -> Result<(), Box<EvalAltResult>> {
// The first symbol must be an identifier
assert!(matches!(
*engine.register_custom_syntax(&["!"], 0, |_, _, _, _| Ok(().into())).expect_err("should error"),
LexError::ImproperSymbol(s) if s == "!"
ParseError(err, _) if *err == ParseErrorType::BadInput("Improper symbol for custom syntax: '!'".to_string())
));
Ok(())