Limit expression/statement nesting depths.

This commit is contained in:
Stephen Chung 2020-05-18 19:32:22 +08:00
parent f4a528a88a
commit 1824dced69
7 changed files with 513 additions and 198 deletions

View File

@ -24,7 +24,7 @@ Rhai's current features set:
one single source file, all with names starting with `"unsafe_"`). one single source file, all with names starting with `"unsafe_"`).
* Re-entrant scripting [`Engine`] can be made `Send + Sync` (via the [`sync`] feature). * Re-entrant scripting [`Engine`] can be made `Send + Sync` (via the [`sync`] feature).
* Sand-boxed - the scripting [`Engine`], if declared immutable, cannot mutate the containing environment without explicit permission. * Sand-boxed - the scripting [`Engine`], if declared immutable, cannot mutate the containing environment without explicit permission.
* Rugged (protection against [stack-overflow](#maximum-stack-depth) and [runaway scripts](#maximum-number-of-operations) etc.). * Rugged (protection against [stack-overflow](#maximum-call-stack-depth) and [runaway scripts](#maximum-number-of-operations) etc.).
* Track script evaluation [progress](#tracking-progress) and manually terminate a script run. * Track script evaluation [progress](#tracking-progress) and manually terminate a script run.
* [`no-std`](#optional-features) support. * [`no-std`](#optional-features) support.
* [Function overloading](#function-overloading). * [Function overloading](#function-overloading).
@ -1067,11 +1067,12 @@ Engine configuration options
--------------------------- ---------------------------
| Method | Description | | Method | Description |
| ------------------------ | ---------------------------------------------------------------------------------------- | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `set_optimization_level` | Set the amount of script _optimizations_ performed. See [`script optimization`]. | | `set_optimization_level` | Set the amount of script _optimizations_ performed. See [script optimization]. |
| `set_max_call_levels` | Set the maximum number of function call levels (default 50) to avoid infinite recursion. | | `set_max_expr_depths` | Set the maximum nesting levels of an expression/statement. See [maximum statement depth](#maximum-statement-depth). |
| `set_max_call_levels` | Set the maximum number of function call levels (default 50) to avoid infinite recursion. See [maximum call stack depth](#maximum-call-stack-depth). |
[`script optimization`]: #script-optimization | `set_max_operations` | Set the maximum number of _operations_ that a script is allowed to consume. See [maximum number of operations](#maximum-number-of-operations). |
| `set_max_modules` | Set the maximum number of [modules] that a script is allowed to load. See [maximum number of modules](#maximum-number-of-modules). |
------- -------
@ -2267,9 +2268,12 @@ so that it does not consume more resources that it is allowed to.
The most important resources to watch out for are: The most important resources to watch out for are:
* **Memory**: A malignant script may continuously grow an [array] or [object map] until all memory is consumed. * **Memory**: A malignant script may continuously grow an [array] or [object map] until all memory is consumed.
It may also create a large [array] or [objecct map] literal that exhausts all memory during parsing.
* **CPU**: A malignant script may run an infinite tight loop that consumes all CPU cycles. * **CPU**: A malignant script may run an infinite tight loop that consumes all CPU cycles.
* **Time**: A malignant script may run indefinitely, thereby blocking the calling system which is waiting for a result. * **Time**: A malignant script may run indefinitely, thereby blocking the calling system which is waiting for a result.
* **Stack**: A malignant script may attempt an infinite recursive call that exhausts the call stack. * **Stack**: A malignant script may attempt an infinite recursive call that exhausts the call stack.
Alternatively, it may create a degenerated deep expression with so many levels that the parser exhausts the call stack
when parsing the expression; or even deeply-nested statement blocks, if nested deep enough.
* **Overflows**: A malignant script may deliberately cause numeric over-flows and/or under-flows, divide by zero, and/or * **Overflows**: A malignant script may deliberately cause numeric over-flows and/or under-flows, divide by zero, and/or
create bad floating-point representations, in order to crash the system. create bad floating-point representations, in order to crash the system.
* **Files**: A malignant script may continuously [`import`] an external module within an infinite loop, * **Files**: A malignant script may continuously [`import`] an external module within an infinite loop,
@ -2341,10 +2345,15 @@ engine.set_max_modules(5); // allow loading only up to 5 module
engine.set_max_modules(0); // allow unlimited modules engine.set_max_modules(0); // allow unlimited modules
``` ```
### Maximum stack depth ### Maximum call stack depth
Rhai by default limits function calls to a maximum depth of 256 levels (28 levels in debug build). Rhai by default limits function calls to a maximum depth of 128 levels (8 levels in debug build).
This limit may be changed via the `Engine::set_max_call_levels` method. This limit may be changed via the `Engine::set_max_call_levels` method.
When setting this limit, care must be also taken to the evaluation depth of each _statement_
within the function. It is entirely possible for a malignant script to embed an recursive call deep
inside a nested expression or statement block (see [maximum statement depth](#maximum-statement-depth)).
The limit can be disabled via the [`unchecked`] feature for higher performance The limit can be disabled via the [`unchecked`] feature for higher performance
(but higher risks as well). (but higher risks as well).
@ -2358,12 +2367,57 @@ engine.set_max_call_levels(0); // allow no function calls at all (m
A script exceeding the maximum call stack depth will terminate with an error result. A script exceeding the maximum call stack depth will terminate with an error result.
### Maximum statement depth
Rhai by default limits statements and expressions nesting to a maximum depth of 128
(which should be plenty) when they are at _global_ level, but only a depth of 32
when they are within function bodies. For debug builds, these limits are set further
downwards to 32 and 16 respectively.
That is because it is possible to overflow the [`Engine`]'s stack when it tries to
recursively parse an extremely deeply-nested code stream.
```rust
// The following, if long enough, can easily cause stack overflow during parsing.
let a = (1+(1+(1+(1+(1+(1+(1+(1+(1+(1+(...)+1)))))))))));
```
This limit may be changed via the `Engine::set_max_expr_depths` method. There are two limits to set,
one for the maximum depth at global level, and the other for function bodies.
```rust
let mut engine = Engine::new();
engine.set_max_expr_depths(50, 5); // allow nesting up to 50 layers of expressions/statements
// at global level, but only 5 inside functions
```
Beware that there may be multiple layers for a simple language construct, even though it may correspond
to only one AST node. That is because the Rhai _parser_ internally runs a recursive chain of function calls
and it is important that a malignant script does not panic the parser in the first place.
Functions are placed under stricter limits because of the multiplicative effect of recursion.
A script can effectively call itself while deep inside an expression chain within the function body,
thereby overflowing the stack even when the level of recursion is within limit.
Make sure that `C x ( 5 + F ) + S` layered calls do not cause a stack overflow, where:
* `C` = maximum call stack depth,
* `F` = maximum statement depth for functions,
* `S` = maximum statement depth at global level.
A script exceeding the maximum nesting depths will terminate with a parsing error.
The malignant `AST` will not be able to get past parsing in the first place.
The limits can be disabled via the [`unchecked`] feature for higher performance
(but higher risks as well).
### Checked arithmetic ### Checked arithmetic
All arithmetic calculations in Rhai are _checked_, meaning that the script terminates with an error whenever By default, all arithmetic calculations in Rhai are _checked_, meaning that the script terminates
it detects a numeric over-flow/under-flow condition or an invalid floating-point operation, instead of with an error whenever it detects a numeric over-flow/under-flow condition or an invalid
crashing the entire system. This checking can be turned off via the [`unchecked`] feature for higher performance floating-point operation, instead of crashing the entire system. This checking can be turned off
(but higher risks as well). via the [`unchecked`] feature for higher performance (but higher risks as well).
### Blocking access to external data ### Blocking access to external data
@ -2383,6 +2437,8 @@ let engine = engine; // shadow the variable so that 'engi
Script optimization Script optimization
=================== ===================
[script optimization]: #script-optimization
Rhai includes an _optimizer_ that tries to optimize a script after parsing. Rhai includes an _optimizer_ that tries to optimize a script after parsing.
This can reduce resource utilization and increase execution speed. This can reduce resource utilization and increase execution speed.
Script optimization can be turned off via the [`no_optimize`] feature. Script optimization can be turned off via the [`no_optimize`] feature.

View File

@ -1,6 +1,15 @@
Rhai Release Notes Rhai Release Notes
================== ==================
Version 0.15.0
==============
New features
------------
* Set limits on maximum level of nesting expressions and statements to avoid panics during parsing.
Version 0.14.1 Version 0.14.1
============== ==============

View File

@ -444,7 +444,14 @@ impl Engine {
optimization_level: OptimizationLevel, optimization_level: OptimizationLevel,
) -> Result<AST, Box<ParseError>> { ) -> Result<AST, Box<ParseError>> {
let stream = lex(scripts); let stream = lex(scripts);
parse(&mut stream.peekable(), self, scope, optimization_level)
parse(
&mut stream.peekable(),
self,
scope,
optimization_level,
(self.max_expr_depth, self.max_function_expr_depth),
)
} }
/// Read the contents of a file into a string. /// Read the contents of a file into a string.
@ -571,6 +578,7 @@ impl Engine {
self, self,
&scope, &scope,
OptimizationLevel::None, OptimizationLevel::None,
self.max_expr_depth,
)?; )?;
// Handle null - map to () // Handle null - map to ()
@ -654,7 +662,13 @@ impl Engine {
{ {
let mut peekable = stream.peekable(); let mut peekable = stream.peekable();
parse_global_expr(&mut peekable, self, scope, self.optimization_level) parse_global_expr(
&mut peekable,
self,
scope,
self.optimization_level,
self.max_expr_depth,
)
} }
} }
@ -805,8 +819,14 @@ impl Engine {
) -> Result<T, Box<EvalAltResult>> { ) -> Result<T, Box<EvalAltResult>> {
let scripts = [script]; let scripts = [script];
let stream = lex(&scripts); let stream = lex(&scripts);
// Since the AST will be thrown away afterwards, don't bother to optimize it
let ast = parse_global_expr(&mut stream.peekable(), self, scope, OptimizationLevel::None)?; let ast = parse_global_expr(
&mut stream.peekable(),
self,
scope,
self.optimization_level,
self.max_expr_depth,
)?;
self.eval_ast_with_scope(scope, &ast) self.eval_ast_with_scope(scope, &ast)
} }
@ -931,8 +951,13 @@ impl Engine {
let scripts = [script]; let scripts = [script];
let stream = lex(&scripts); let stream = lex(&scripts);
// Since the AST will be thrown away afterwards, don't bother to optimize it let ast = parse(
let ast = parse(&mut stream.peekable(), self, scope, OptimizationLevel::None)?; &mut stream.peekable(),
self,
scope,
self.optimization_level,
(self.max_expr_depth, self.max_function_expr_depth),
)?;
self.consume_ast_with_scope(scope, &ast) self.consume_ast_with_scope(scope, &ast)
} }

View File

@ -49,14 +49,30 @@ pub type Map = HashMap<String, Dynamic>;
#[cfg(not(feature = "unchecked"))] #[cfg(not(feature = "unchecked"))]
#[cfg(debug_assertions)] #[cfg(debug_assertions)]
pub const MAX_CALL_STACK_DEPTH: usize = 28; pub const MAX_CALL_STACK_DEPTH: usize = 8;
#[cfg(not(feature = "unchecked"))]
#[cfg(debug_assertions)]
pub const MAX_EXPR_DEPTH: usize = 32;
#[cfg(not(feature = "unchecked"))]
#[cfg(debug_assertions)]
pub const MAX_FUNCTION_EXPR_DEPTH: usize = 16;
#[cfg(not(feature = "unchecked"))] #[cfg(not(feature = "unchecked"))]
#[cfg(not(debug_assertions))] #[cfg(not(debug_assertions))]
pub const MAX_CALL_STACK_DEPTH: usize = 256; pub const MAX_CALL_STACK_DEPTH: usize = 128;
#[cfg(not(feature = "unchecked"))]
#[cfg(not(debug_assertions))]
pub const MAX_EXPR_DEPTH: usize = 128;
#[cfg(not(feature = "unchecked"))]
#[cfg(not(debug_assertions))]
pub const MAX_FUNCTION_EXPR_DEPTH: usize = 32;
#[cfg(feature = "unchecked")] #[cfg(feature = "unchecked")]
pub const MAX_CALL_STACK_DEPTH: usize = usize::MAX; pub const MAX_CALL_STACK_DEPTH: usize = usize::MAX;
#[cfg(feature = "unchecked")]
pub const MAX_EXPR_DEPTH: usize = usize::MAX;
#[cfg(feature = "unchecked")]
pub const MAX_FUNCTION_EXPR_DEPTH: usize = usize::MAX;
pub const KEYWORD_PRINT: &str = "print"; pub const KEYWORD_PRINT: &str = "print";
pub const KEYWORD_DEBUG: &str = "debug"; pub const KEYWORD_DEBUG: &str = "debug";
@ -338,8 +354,12 @@ pub struct Engine {
pub(crate) optimization_level: OptimizationLevel, pub(crate) optimization_level: OptimizationLevel,
/// Maximum levels of call-stack to prevent infinite recursion. /// Maximum levels of call-stack to prevent infinite recursion.
/// ///
/// Defaults to 28 for debug builds and 256 for non-debug builds. /// Defaults to 8 for debug builds and 128 for non-debug builds.
pub(crate) max_call_stack_depth: usize, pub(crate) max_call_stack_depth: usize,
/// Maximum depth of statements/expressions at global level.
pub(crate) max_expr_depth: usize,
/// Maximum depth of statements/expressions in functions.
pub(crate) max_function_expr_depth: usize,
/// Maximum number of operations allowed to run. /// Maximum number of operations allowed to run.
pub(crate) max_operations: Option<NonZeroU64>, pub(crate) max_operations: Option<NonZeroU64>,
/// Maximum number of modules allowed to load. /// Maximum number of modules allowed to load.
@ -382,6 +402,8 @@ impl Default for Engine {
optimization_level: OptimizationLevel::Full, optimization_level: OptimizationLevel::Full,
max_call_stack_depth: MAX_CALL_STACK_DEPTH, max_call_stack_depth: MAX_CALL_STACK_DEPTH,
max_expr_depth: MAX_EXPR_DEPTH,
max_function_expr_depth: MAX_FUNCTION_EXPR_DEPTH,
max_operations: None, max_operations: None,
max_modules: None, max_modules: None,
}; };
@ -523,6 +545,8 @@ impl Engine {
optimization_level: OptimizationLevel::Full, optimization_level: OptimizationLevel::Full,
max_call_stack_depth: MAX_CALL_STACK_DEPTH, max_call_stack_depth: MAX_CALL_STACK_DEPTH,
max_expr_depth: MAX_EXPR_DEPTH,
max_function_expr_depth: MAX_FUNCTION_EXPR_DEPTH,
max_operations: None, max_operations: None,
max_modules: None, max_modules: None,
} }
@ -574,6 +598,13 @@ impl Engine {
self.max_modules = NonZeroU64::new(modules); self.max_modules = NonZeroU64::new(modules);
} }
/// Set the depth limits for expressions/statements.
#[cfg(not(feature = "unchecked"))]
pub fn set_max_expr_depths(&mut self, max_expr_depth: usize, max_function_expr_depth: usize) {
self.max_expr_depth = max_expr_depth;
self.max_function_expr_depth = max_function_expr_depth;
}
/// Set the module resolution service used by the `Engine`. /// Set the module resolution service used by the `Engine`.
/// ///
/// Not available under the `no_module` feature. /// Not available under the `no_module` feature.

View File

@ -110,6 +110,10 @@ pub enum ParseErrorType {
AssignmentToCopy, AssignmentToCopy,
/// Assignment to an a constant variable. /// Assignment to an a constant variable.
AssignmentToConstant(String), AssignmentToConstant(String),
/// Expression exceeding the maximum levels of complexity.
///
/// Never appears under the `unchecked` feature.
ExprTooDeep,
/// Break statement not inside a loop. /// Break statement not inside a loop.
LoopBreak, LoopBreak,
} }
@ -158,7 +162,8 @@ impl ParseError {
ParseErrorType::DuplicatedExport(_) => "Duplicated variable/function in export statement", ParseErrorType::DuplicatedExport(_) => "Duplicated variable/function in export statement",
ParseErrorType::WrongExport => "Export statement can only appear at global level", ParseErrorType::WrongExport => "Export statement can only appear at global level",
ParseErrorType::AssignmentToCopy => "Only a copy of the value is change with this assignment", ParseErrorType::AssignmentToCopy => "Only a copy of the value is change with this assignment",
ParseErrorType::AssignmentToConstant(_) => "Cannot assign to a constant value.", ParseErrorType::AssignmentToConstant(_) => "Cannot assign to a constant value",
ParseErrorType::ExprTooDeep => "Expression exceeds maximum complexity",
ParseErrorType::LoopBreak => "Break statement should only be used inside a loop" ParseErrorType::LoopBreak => "Break statement should only be used inside a loop"
} }
} }

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long