diff --git a/RELEASES.md b/RELEASES.md index 6248f124..bdc86fd0 100644 --- a/RELEASES.md +++ b/RELEASES.md @@ -15,6 +15,7 @@ New features * Disallow many keywords as variables, such as `print`, `eval`, `call`, `this` etc. * `x.call(f, ...)` allows binding `x` to `this` for the function referenced by the function pointer `f`. * Anonymous functions in the syntax of a closure, e.g. `|x, y, z| x + y - z`. +* Custom syntax now works even without the `internals` feature. Breaking changes ---------------- @@ -48,7 +49,7 @@ Breaking changes New features ------------ -* New `serde` feature to allow serializating/deserializating to/from `Dynamic` values using [`serde`](https://crates.io/crates/serde). +* New `serde` feature to allow serializing/deserializing to/from `Dynamic` values using [`serde`](https://crates.io/crates/serde). This is particularly useful when converting a Rust `struct` to a `Dynamic` _object map_ and back. * `Engine::disable_symbol` to surgically disable keywords and/or operators. * `Engine::register_custom_operator` to define a custom operator. diff --git a/doc/src/engine/custom-syntax.md b/doc/src/engine/custom-syntax.md index f97e4203..89fdf792 100644 --- a/doc/src/engine/custom-syntax.md +++ b/doc/src/engine/custom-syntax.md @@ -33,18 +33,7 @@ Where This Might Be Useful * Where you just want to confuse your user and make their lives miserable, because you can. -Step One - Start With `internals` --------------------------------- - -Since a custom syntax taps deeply into the `AST` and evaluation process of the `Engine`, -the [`internals`] feature must be on in order to expose these necessary internal data structures. - -Beware that Rhai internal data structures are _volatile_ and may change without warning. - -Caveat emptor. - - -Step Two - Design The Syntax +Step One - Design The Syntax --------------------------- A custom syntax is simply a list of symbols. @@ -116,8 +105,8 @@ print(hello); // variable declared by a custom syntax persists! ``` -Step Three - Implementation --------------------------- +Step Two - Implementation +------------------------- Any custom syntax must include an _implementation_ of it. @@ -164,8 +153,6 @@ let expr = inputs.get(0).unwrap(); let result = engine.eval_expression_tree(context, scope, expr)?; ``` -As can be seem above, most arguments are simply passed straight-through to `engine::eval_expression_tree`. - ### Declare Variables New variables maybe declared (usually with a variable name that is passed in via `$ident$). @@ -179,14 +166,14 @@ In other words, any `scope.push(...)` calls must come _before_ any `engine::eval let var_name = inputs[0].get_variable_name().unwrap().to_string(); let expr = inputs.get(1).unwrap(); -scope.push(var_name, 0 as INT); // do this BEFORE engine.eval_expression_tree! +scope.push(var_name, 0 as INT); // do this BEFORE 'engine.eval_expression_tree'! -let result = engine.eval_expression_tree(scope, mods, state, lib, this_ptr, expr, level)?; +let result = engine.eval_expression_tree(context, scope, expr)?; ``` -Step Four - Register the Custom Syntax -------------------------------------- +Step Three - Register the Custom Syntax +-------------------------------------- Use `Engine::register_custom_syntax` to register a custom syntax. @@ -236,7 +223,7 @@ engine.register_custom_syntax( ``` -Step Five - Disable Unneeded Statement Types +Step Four - Disable Unneeded Statement Types ------------------------------------------- When a DSL needs a custom syntax, most likely than not it is extremely specialized. @@ -254,13 +241,13 @@ custom syntax (plus possibly expressions). But again, Don't Do It™ - unless y of what you're doing. -Step Six - Document -------------------- +Step Five - Document +-------------------- For custom syntax, documentation is crucial. Make sure there are _lots_ of examples for users to follow. -Step Seven - Profit! --------------------- +Step Six - Profit! +------------------ diff --git a/doc/src/engine/dsl.md b/doc/src/engine/dsl.md index 3585936b..5b381d4a 100644 --- a/doc/src/engine/dsl.md +++ b/doc/src/engine/dsl.md @@ -11,7 +11,7 @@ Expressions Only In many DSL scenarios, only evaluation of expressions is needed. -The `Engine::eval_expression_XXX`[`eval_expression`] API can be used to restrict +The [`Engine::eval_expression_XXX`][`eval_expression`] API can be used to restrict a script to expressions only. @@ -21,8 +21,7 @@ Disable Keywords and/or Operators In some DSL scenarios, it is necessary to further restrict the language to exclude certain language features that are not necessary or dangerous to the application. -For example, a DSL may disable the `while` loop altogether while keeping all other statement -types intact. +For example, a DSL may disable the `while` loop while keeping all other statement types intact. It is possible, in Rhai, to surgically [disable keywords and operators]. @@ -54,31 +53,29 @@ Custom Syntax For advanced DSL scenarios, it is possible to define entire expression [_syntax_][custom syntax] - essentially custom statement types. -The [`internals`] feature is needed to be able to define [custom syntax] in Rhai. - -For example, the following is a SQL like syntax for some obscure DSL operation: +For example, the following is a SQL-like syntax for some obscure DSL operation: ```rust let table = [..., ..., ..., ...]; -// Syntax = "calculate" $ident$ $ident$ "from" $expr$ "->" $ident$ ":" $expr$ +// Syntax = calculate $ident$ $ident$ from $expr$ -> $ident$ : $expr$ let total = calculate sum price from table -> row : row.weight > 50; -// Note: There is nothing special about the use of symbols; to make it look exactly like SQL: -// Syntax = "SELECT" $ident$ "(" $ident$ ")" "FROM" $expr$ "AS" $ident$ "WHERE" $expr$ +// Note: There is nothing special about those symbols; to make it look exactly like SQL: +// Syntax = SELECT $ident$ ( $ident$ ) FROM $expr$ AS $ident$ WHERE $expr$ let total = SELECT sum(price) FROM table AS row WHERE row.weight > 50; ``` After registering this custom syntax with Rhai, it can be used anywhere inside a script as a normal expression. -For its evaluation, the callback function will receive the following list of parameters: +For its evaluation, the callback function will receive the following list of inputs: -`exprs[0] = "sum"` - math operator -`exprs[1] = "price"` - field name -`exprs[2] = Expression(table)` - data source -`exprs[3] = "row"` - loop variable name -`exprs[4] = Expression(row.wright > 50)` - expression +* `inputs[0] = "sum"` - math operator +* `inputs[1] = "price"` - field name +* `inputs[2] = Expression(table)` - data source +* `inputs[3] = "row"` - loop variable name +* `inputs[4] = Expression(row.wright > 50)` - filter predicate -The other identified, such as `"select"`, `"from"`, as as as symbols `->` and `:` are -parsed in the order defined within the custom syntax. +Other identifiers, such as `"calculate"`, `"from"`, as well as symbols such as `->` and `:`, +are parsed in the order defined within the custom syntax. diff --git a/src/engine.rs b/src/engine.rs index ffd1a3eb..010949f1 100644 --- a/src/engine.rs +++ b/src/engine.rs @@ -11,15 +11,13 @@ use crate::parser::{Expr, FnAccess, ImmutableString, ReturnType, ScriptFnDef, St use crate::r#unsafe::unsafe_cast_var_name_to_lifetime; use crate::result::EvalAltResult; use crate::scope::{EntryType as ScopeEntryType, Scope}; +use crate::syntax::{CustomSyntax, EvalContext, Expression}; use crate::token::Position; use crate::utils::StaticVec; #[cfg(not(feature = "no_float"))] use crate::parser::FLOAT; -#[cfg(feature = "internals")] -use crate::syntax::{CustomSyntax, EvalContext}; - use crate::stdlib::{ any::{type_name, TypeId}, borrow::Cow, @@ -88,45 +86,10 @@ pub const FN_SET: &str = "set$"; pub const FN_IDX_GET: &str = "index$get$"; pub const FN_IDX_SET: &str = "index$set$"; pub const FN_ANONYMOUS: &str = "anon$"; - -#[cfg(feature = "internals")] pub const MARKER_EXPR: &str = "$expr$"; -#[cfg(feature = "internals")] pub const MARKER_BLOCK: &str = "$block$"; -#[cfg(feature = "internals")] pub const MARKER_IDENT: &str = "$ident$"; -#[cfg(feature = "internals")] -#[derive(Debug, Clone, Hash)] -pub struct Expression<'a>(&'a Expr); - -#[cfg(feature = "internals")] -impl<'a> From<&'a Expr> for Expression<'a> { - fn from(expr: &'a Expr) -> Self { - Self(expr) - } -} - -#[cfg(feature = "internals")] -impl Expression<'_> { - /// If this expression is a variable name, return it. Otherwise `None`. - #[cfg(feature = "internals")] - pub fn get_variable_name(&self) -> Option<&str> { - match self.0 { - Expr::Variable(x) => Some((x.0).0.as_str()), - _ => None, - } - } - /// Get the expression. - pub(crate) fn expr(&self) -> &Expr { - &self.0 - } - /// Get the position of this expression. - pub fn position(&self) -> Position { - self.0.position() - } -} - /// A type specifying the method of chaining. #[derive(Debug, Clone, Copy, Eq, PartialEq, Hash)] enum ChainType { @@ -316,7 +279,6 @@ pub struct Engine { /// A hashset containing custom keywords and precedence to recognize. pub(crate) custom_keywords: Option>, /// Custom syntax. - #[cfg(feature = "internals")] pub(crate) custom_syntax: Option>, /// Callback closure for implementing the `print` command. @@ -376,8 +338,6 @@ impl Default for Engine { type_names: None, disabled_symbols: None, custom_keywords: None, - - #[cfg(feature = "internals")] custom_syntax: None, // default print/debug implementations @@ -605,8 +565,6 @@ impl Engine { type_names: None, disabled_symbols: None, custom_keywords: None, - - #[cfg(feature = "internals")] custom_syntax: None, print: Box::new(|_| {}), @@ -1734,8 +1692,6 @@ impl Engine { /// ## WARNING - Low Level API /// /// This function is very low level. It evaluates an expression from an AST. - #[cfg(feature = "internals")] - #[deprecated(note = "this method is volatile and may change")] pub fn eval_expression_tree( &self, context: &mut EvalContext, @@ -2215,7 +2171,6 @@ impl Engine { Expr::False(_) => Ok(false.into()), Expr::Unit(_) => Ok(().into()), - #[cfg(feature = "internals")] Expr::Custom(x) => { let func = (x.0).1.as_ref(); let ep: StaticVec<_> = (x.0).0.iter().map(|e| e.into()).collect(); diff --git a/src/lib.rs b/src/lib.rs index 8bebc0a1..fb25ff4d 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -91,7 +91,6 @@ mod scope; mod serde; mod settings; mod stdlib; -#[cfg(feature = "internals")] mod syntax; mod token; mod r#unsafe; @@ -106,6 +105,7 @@ pub use module::Module; pub use parser::{ImmutableString, AST, INT}; pub use result::EvalAltResult; pub use scope::Scope; +pub use syntax::{EvalContext, Expression}; pub use token::Position; pub use utils::calc_fn_spec as calc_fn_hash; @@ -169,11 +169,7 @@ pub use parser::{CustomExpr, Expr, ReturnType, ScriptFnDef, Stmt}; #[cfg(feature = "internals")] #[deprecated(note = "this type is volatile and may change")] -pub use engine::{Expression, Imports, State as EvalState}; - -#[cfg(feature = "internals")] -#[deprecated(note = "this type is volatile and may change")] -pub use syntax::EvalContext; +pub use engine::{Imports, State as EvalState}; #[cfg(feature = "internals")] #[deprecated(note = "this type is volatile and may change")] diff --git a/src/parser.rs b/src/parser.rs index b3ed0756..b6a3fa08 100644 --- a/src/parser.rs +++ b/src/parser.rs @@ -2,23 +2,19 @@ use crate::any::{Dynamic, Union}; use crate::calc_fn_hash; -use crate::engine::{make_getter, make_setter, Engine, FN_ANONYMOUS, KEYWORD_THIS}; +use crate::engine::{ + make_getter, make_setter, Engine, FN_ANONYMOUS, KEYWORD_THIS, MARKER_BLOCK, MARKER_EXPR, + MARKER_IDENT, +}; use crate::error::{LexError, ParseError, ParseErrorType}; +use crate::fn_native::Shared; use crate::module::{Module, ModuleRef}; use crate::optimize::{optimize_into_ast, OptimizationLevel}; use crate::scope::{EntryType as ScopeEntryType, Scope}; +use crate::syntax::FnCustomSyntaxEval; use crate::token::{is_valid_identifier, Position, Token, TokenStream}; use crate::utils::{StaticVec, StraightHasherBuilder}; -#[cfg(feature = "internals")] -use crate::engine::{MARKER_BLOCK, MARKER_EXPR, MARKER_IDENT}; - -#[cfg(feature = "internals")] -use crate::fn_native::Shared; - -#[cfg(feature = "internals")] -use crate::syntax::FnCustomSyntaxEval; - use crate::stdlib::{ borrow::Cow, boxed::Box, @@ -587,17 +583,14 @@ impl Stmt { } #[derive(Clone)] -#[cfg(feature = "internals")] pub struct CustomExpr(pub StaticVec, pub Shared); -#[cfg(feature = "internals")] impl fmt::Debug for CustomExpr { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { fmt::Debug::fmt(&self.0, f) } } -#[cfg(feature = "internals")] impl Hash for CustomExpr { fn hash(&self, state: &mut H) { self.0.hash(state); @@ -683,7 +676,6 @@ pub enum Expr { /// () Unit(Position), /// Custom syntax - #[cfg(feature = "internals")] Custom(Box<(CustomExpr, Position)>), } @@ -781,7 +773,6 @@ impl Expr { Self::Dot(x) | Self::Index(x) => x.0.position(), - #[cfg(feature = "internals")] Self::Custom(x) => x.1, } } @@ -816,8 +807,6 @@ impl Expr { Self::Assignment(x) => x.3 = new_pos, Self::Dot(x) => x.2 = new_pos, Self::Index(x) => x.2 = new_pos, - - #[cfg(feature = "internals")] Self::Custom(x) => x.1 = new_pos, } @@ -925,7 +914,6 @@ impl Expr { _ => false, }, - #[cfg(feature = "internals")] Self::Custom(_) => false, } } @@ -2148,7 +2136,6 @@ fn parse_expr( settings.ensure_level_within_max_limit(state.max_expr_depth)?; // Check if it is a custom syntax. - #[cfg(feature = "internals")] if let Some(ref custom) = state.engine.custom_syntax { let (token, pos) = input.peek().unwrap(); let token_pos = *pos; diff --git a/src/syntax.rs b/src/syntax.rs index 78bf94ff..33b0c9e4 100644 --- a/src/syntax.rs +++ b/src/syntax.rs @@ -1,14 +1,13 @@ //! Module containing implementation for custom syntax. -#![cfg(feature = "internals")] - use crate::any::Dynamic; -use crate::engine::{Engine, Expression, Imports, State, MARKER_BLOCK, MARKER_EXPR, MARKER_IDENT}; -use crate::error::LexError; +use crate::engine::{Engine, Imports, State, MARKER_BLOCK, MARKER_EXPR, MARKER_IDENT}; +use crate::error::{LexError, ParseError, ParseErrorType}; use crate::fn_native::{SendSync, Shared}; use crate::module::Module; +use crate::parser::Expr; use crate::result::EvalAltResult; use crate::scope::Scope; -use crate::token::{is_valid_identifier, Token}; +use crate::token::{is_valid_identifier, Position, Token}; use crate::utils::StaticVec; use crate::stdlib::{ @@ -32,6 +31,33 @@ pub type FnCustomSyntaxEval = dyn Fn(&Engine, &mut EvalContext, &mut Scope, &[Ex + Send + Sync; +#[derive(Debug, Clone, Hash)] +pub struct Expression<'a>(&'a Expr); + +impl<'a> From<&'a Expr> for Expression<'a> { + fn from(expr: &'a Expr) -> Self { + Self(expr) + } +} + +impl Expression<'_> { + /// If this expression is a variable name, return it. Otherwise `None`. + pub fn get_variable_name(&self) -> Option<&str> { + match self.0 { + Expr::Variable(x) => Some((x.0).0.as_str()), + _ => None, + } + } + /// Get the expression. + pub(crate) fn expr(&self) -> &Expr { + &self.0 + } + /// Get the position of this expression. + pub fn position(&self) -> Position { + self.0.position() + } +} + #[derive(Clone)] pub struct CustomSyntax { pub segments: StaticVec, @@ -45,6 +71,7 @@ impl fmt::Debug for CustomSyntax { } } +#[derive(Debug)] pub struct EvalContext<'a, 'b: 'a, 's, 'm, 't, 'd: 't> { pub(crate) mods: &'a mut Imports<'b>, pub(crate) state: &'s mut State, @@ -56,7 +83,7 @@ pub struct EvalContext<'a, 'b: 'a, 's, 'm, 't, 'd: 't> { impl Engine { pub fn register_custom_syntax + ToString>( &mut self, - value: &[S], + keywords: &[S], scope_delta: isize, func: impl Fn( &Engine, @@ -66,15 +93,18 @@ impl Engine { ) -> Result> + SendSync + 'static, - ) -> Result<&mut Self, Box> { - if value.is_empty() { - return Err(Box::new(LexError::ImproperSymbol("".to_string()))); - } - + ) -> Result<&mut Self, Box> { let mut segments: StaticVec<_> = Default::default(); - for s in value { - let seg = match s.as_ref() { + for s in keywords { + let s = s.as_ref().trim(); + + // skip empty keywords + if s.is_empty() { + continue; + } + + let seg = match s { // Markers not in first position MARKER_EXPR | MARKER_BLOCK | MARKER_IDENT if !segments.is_empty() => s.to_string(), // Standard symbols not in first position @@ -115,12 +145,25 @@ impl Engine { s.into() } // Anything else is an error - _ => return Err(Box::new(LexError::ImproperSymbol(s.to_string()))), + _ => { + return Err(LexError::ImproperSymbol(format!( + "Improper symbol for custom syntax: '{}'", + s + )) + .into_err(Position::none()) + .into()); + } }; segments.push(seg); } + // If the syntax has no keywords, just ignore the registration + if segments.is_empty() { + return Ok(self); + } + + // Remove the first keyword as the discriminator let key = segments.remove(0); let syntax = CustomSyntax { diff --git a/tests/syntax.rs b/tests/syntax.rs index 11074c6a..b5a8d7d9 100644 --- a/tests/syntax.rs +++ b/tests/syntax.rs @@ -1,5 +1,6 @@ -#![cfg(feature = "internals")] -use rhai::{Engine, EvalAltResult, EvalContext, Expression, LexError, ParseErrorType, Scope, INT}; +use rhai::{ + Engine, EvalAltResult, EvalContext, Expression, ParseError, ParseErrorType, Scope, INT, +}; #[test] fn test_custom_syntax() -> Result<(), Box> { @@ -72,7 +73,7 @@ fn test_custom_syntax() -> Result<(), Box> { // The first symbol must be an identifier assert!(matches!( *engine.register_custom_syntax(&["!"], 0, |_, _, _, _| Ok(().into())).expect_err("should error"), - LexError::ImproperSymbol(s) if s == "!" + ParseError(err, _) if *err == ParseErrorType::BadInput("Improper symbol for custom syntax: '!'".to_string()) )); Ok(())