kjuulh/rhai
1
0
Fork 0
Code Issues 1 Pull Requests 1 Packages Projects Releases Wiki Activity

Merge pull request #382 from schungx/master

Browse Source 
SmartString for identifiers, plus literal strings.
This commit is contained in:
Stephen Chung 2021-03-30 09:31:58 +08:00 committed by GitHub
commit 5730b0a26a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
34 changed files with 628 additions and 385 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
CHANGELOG.md
View File

@ -16,7 +16,14 @@ an object map is small.
`HashMap` and `BTreeMap` have almost identical public API's so this change is unlikely to break
existing code.
Im addition, all function signature/metadata methods are now grouped under the umbrella `metadata` feature.
[`SmartString`](https://crates.io/crates/smartstring) is used to store identifiers (which tends to
be short, fewer than 23 characters, and ASCII-based) because they can usually be stored inline.
`Map` keys now also use [`SmartString`](https://crates.io/crates/smartstring).
In addition, there is now support for line continuation in strings (put `\` at the end of line) as
well as multi-line literal strings (wrapped by back-ticks: <code>\`...\`</code>).
Finally, all function signature/metadata methods are now grouped under the umbrella `metadata` feature.
This avoids spending precious resources maintaining metadata for functions for the vast majority of
use cases where such information is not required.
@ -24,7 +31,6 @@ use cases where such information is not required.
Breaking changes
----------------
* `Map` is now an alias to `BTreeMap` instead of `HashMap` because most object maps hold few properties.
* The traits `RegisterFn` and `RegisterResultFn` are removed. `Engine::register_fn` and `Engine::register_result_fn` are now implemented directly on `Engine`.
* `FnPtr::call_dynamic` now takes `&NativeCallContext` instead of consuming it.
* All `Module::set_fn_XXX` methods are removed, in favor of `Module::set_native_fn`.
@ -34,6 +40,14 @@ Breaking changes
* The _reflections_ API such as `Engine::gen_fn_signatures`, `Module::update_fn_metadata` etc. are put under the `metadata` feature gate.
* The shebang `#!` is now a reserved symbol.
* Shebangs at the very beginning of script files are skipped when loading them.
* [`smartstring`](https://crates.io/crates/smartstring) is used for identifiers by default. Currently, a PR branch is pulled because it breaks on `no-std` builds. The official crate will be used once `smartstring` is fixed to support `no-std`.
* `Map` is now an alias to `BTreeMap<SmartString, Dynamic>` instead of `HashMap` because most object maps hold few properties.
New features
------------
* Line continuation (via `\`) and multi-line literal strings (wrapped with <code>\`</code>) support are added.
* Rhai scripts can now start with a shebang `#!` which is ignored.
Enhancements
------------
@ -41,7 +55,6 @@ Enhancements
* Replaced all `HashMap` usage with `BTreeMap` for better performance because collections in Rhai are tiny.
* `Engine::register_result_fn` no longer requires the successful return type to be `Dynamic`. It can now be any clonable type.
* `#[rhai_fn(return_raw)]` can now return `Result<T, Box<EvalAltResult>>` where `T` is any clonable type instead of `Result<Dynamic, Box<EvalAltResult>>`.
* Rhai scripts can now start with a shebang `#!`.
Version 0.19.14

3
Cargo.toml
View File

@ -19,6 +19,8 @@ categories = ["no-std", "embedded", "wasm", "parser-implementations"]
smallvec = { version = "1.6", default-features = false, features = ["union"] }
ahash = { version = "0.7", default-features = false }
num-traits = { version = "0.2", default_features = false }
#smartstring = { version = "0.2.6" }
smartstring = { git = "https://github.com/okready/smartstring", branch = "fix-no_std-builds", default_features = false }
rhai_codegen = { version = "0.3.4", path = "codegen", features = ["metadata"] }
[features]
@ -40,6 +42,7 @@ internals = [] # expose internal data structures
unicode-xid-ident = ["unicode-xid"] # allow Unicode Standard Annex #31 for identifiers.
metadata = ["serde_json"] # enable exporting functions metadata
no_smartstring = [] # set Identifier=ImmutableString
no_std = ["smallvec/union", "num-traits/libm", "core-error", "libm", "ahash/compile-time-rng"]
# compiling for WASM

2
README.md
View File

@ -35,7 +35,7 @@ Standard features
* Built-in support for most common [data types](https://rhai.rs/book/language/values-and-types.html) including booleans, [integers](https://rhai.rs/book/language/numbers.html), [floating-point numbers](https://rhai.rs/book/language/numbers.html) (including [`Decimal`](https://crates.io/crates/rust_decimal)), [strings](https://rhai.rs/book/language/strings-chars.html), [Unicode characters](https://rhai.rs/book/language/strings-chars.html), [arrays](https://rhai.rs/book/language/arrays.html) and [object maps](https://rhai.rs/book/language/object-maps.html).
* Easily [call a script-defined function](https://rhai.rs/book/engine/call-fn.html) from Rust.
* Relatively little `unsafe` code (yes there are some for performance reasons).
* Few dependencies (currently only [`smallvec`](https://crates.io/crates/smallvec), [`num-traits`](https://crates.io/crates/num-traits) and [`ahash`](https://crates.io/crates/ahash)).
* Few dependencies (currently only [`smallvec`](https://crates.io/crates/smallvec), [`num-traits`](https://crates.io/crates/num-traits), [`ahash`](https://crates.io/crates/ahash)) and [`smartstring`](https://crates.io/crates/smartstring)).
* Re-entrant scripting engine can be made `Send + Sync` (via the `sync` feature).
* Compile once to [AST](https://rhai.rs/book/engine/compile.html) form for repeated evaluations.
* Scripts are [optimized](https://rhai.rs/book/engine/optimize.html) (useful for template-based machine-generated scripts).

2
codegen/Cargo.toml
View File

@ -21,5 +21,5 @@ trybuild = "1"
[dependencies]
proc-macro2 = "1"
syn = { version = "1", features = ["full", "parsing", "printing", "proc-macro", "extra-traits"] }
syn = { version = "1.0", features = ["full", "parsing", "printing", "proc-macro", "extra-traits"] }
quote = "1"

6
codegen/src/rhai_module.rs
View File

@ -52,7 +52,8 @@ pub fn generate_body(
.collect();
add_mod_blocks.push(
syn::parse2::<syn::ExprBlock>(quote! {
#(#cfg_attrs)* {
{
#(#cfg_attrs)*
m.set_sub_module(#exported_name, self::#module_name::rhai_module_generate());
}
})
@ -60,7 +61,8 @@ pub fn generate_body(
);
set_flattened_mod_blocks.push(
syn::parse2::<syn::ExprBlock>(quote! {
#(#cfg_attrs)* {
{
#(#cfg_attrs)*
self::#module_name::rhai_generate_into_module(m, flatten);
}
})

6
codegen/src/test/module.rs
View File

@ -1276,11 +1276,13 @@ mod generate_tests {
#[allow(unused_mut)]
pub fn rhai_generate_into_module(m: &mut Module, flatten: bool) {
if flatten {
#[cfg(not(feature = "no_float"))] {
{
#[cfg(not(feature = "no_float"))]
self::it_is::rhai_generate_into_module(m, flatten);
}
} else {
#[cfg(not(feature = "no_float"))] {
{
#[cfg(not(feature = "no_float"))]
m.set_sub_module("it_is", self::it_is::rhai_module_generate());
}
}

2
no_std/no_std_test/Cargo.toml
View File

@ -12,7 +12,7 @@ homepage = "https://github.com/rhaiscript/rhai/tree/no_std/no_std_test"
repository = "https://github.com/rhaiscript/rhai"
[dependencies]
rhai = { path = "../../", features = [ "no_std" ], default_features = false }
rhai = { path = "../../", features = [ "no_std" ] }
wee_alloc = { version = "0.4.5", default_features = false }
[profile.dev]

68
src/ast.rs
View File

@ -4,9 +4,8 @@ use crate::dynamic::{AccessMode, Union};
use crate::fn_native::shared_make_mut;
use crate::module::NamespaceRef;
use crate::stdlib::{
borrow::Cow,
boxed::Box,
collections::{BTreeMap, BTreeSet},
collections::BTreeMap,
fmt,
hash::Hash,
num::NonZeroUsize,
@ -17,7 +16,8 @@ use crate::stdlib::{
};
use crate::token::Token;
use crate::{
Dynamic, FnNamespace, FnPtr, ImmutableString, Module, Position, Shared, StaticVec, INT,
Dynamic, FnNamespace, FnPtr, Identifier, ImmutableString, Module, Position, Shared, StaticVec,
INT,
};
#[cfg(not(feature = "no_float"))]
@ -51,14 +51,14 @@ pub struct ScriptFnDef {
#[cfg(not(feature = "no_module"))]
pub mods: crate::engine::Imports,
/// Function name.
pub name: ImmutableString,
pub name: Identifier,
/// Function access mode.
pub access: FnAccess,
/// Names of function parameters.
pub params: StaticVec<ImmutableString>,
pub params: StaticVec<Identifier>,
/// Access to external variables.
#[cfg(not(feature = "no_closure"))]
pub externals: BTreeSet<ImmutableString>,
pub externals: crate::stdlib::collections::BTreeSet<Identifier>,
/// Function doc-comments (if any).
pub comments: StaticVec<String>,
}
@ -145,7 +145,7 @@ impl<'a> Into<ScriptFnMetadata<'a>> for &'a ScriptFnDef {
#[derive(Debug, Clone)]
pub struct AST {
/// Source of the [`AST`].
source: Option<ImmutableString>,
source: Option<Identifier>,
/// Global statements.
body: StmtBlock,
/// Script-defined functions.
@ -191,7 +191,7 @@ impl AST {
pub fn new_with_source(
statements: impl IntoIterator<Item = Stmt>,
functions: impl Into<Shared<Module>>,
source: impl Into<ImmutableString>,
source: impl Into<Identifier>,
) -> Self {
Self {
source: Some(source.into()),
@ -211,12 +211,12 @@ impl AST {
}
/// Clone the source, if any.
#[inline(always)]
pub(crate) fn clone_source(&self) -> Option<ImmutableString> {
pub(crate) fn clone_source(&self) -> Option<Identifier> {
self.source.clone()
}
/// Set the source.
#[inline(always)]
pub fn set_source(&mut self, source: impl Into<ImmutableString>) -> &mut Self {
pub fn set_source(&mut self, source: impl Into<Identifier>) -> &mut Self {
self.source = Some(source.into());
if let Some(module) = Shared::get_mut(&mut self.functions) {
@ -757,7 +757,7 @@ impl AsRef<Module> for AST {
}
}
/// _(INTERNALS)_ An identifier containing an [immutable string][ImmutableString] name and a [position][Position].
/// _(INTERNALS)_ An identifier containing a name and a [position][Position].
/// Exported under the `internals` feature only.
///
/// # Volatile Data Structure
@ -766,7 +766,7 @@ impl AsRef<Module> for AST {
#[derive(Clone, Eq, PartialEq, Hash)]
pub struct Ident {
/// Identifier name.
pub name: ImmutableString,
pub name: Identifier,
/// Declaration position.
pub pos: Position,
}
@ -870,11 +870,11 @@ pub enum Stmt {
/// `do` `{` stmt `}` `while`|`until` expr
Do(Box<StmtBlock>, Expr, bool, Position),
/// `for` id `in` expr `{` stmt `}`
For(Expr, Box<(String, StmtBlock)>, Position),
For(Expr, Box<(Identifier, StmtBlock)>, Position),
/// \[`export`\] `let` id `=` expr
Let(Expr, Ident, bool, Position),
Let(Expr, Box<Ident>, bool, Position),
/// \[`export`\] `const` id `=` expr
Const(Expr, Ident, bool, Position),
Const(Expr, Box<Ident>, bool, Position),
/// expr op`=` expr
Assignment(Box<(Expr, Expr, Option<OpAssignment>)>, Position),
/// `{` stmt`;` ... `}`
@ -895,13 +895,13 @@ pub enum Stmt {
Return(ReturnType, Option<Expr>, Position),
/// `import` expr `as` var
#[cfg(not(feature = "no_module"))]
Import(Expr, Option<Ident>, Position),
Import(Expr, Option<Box<Ident>>, Position),
/// `export` var `as` var `,` ...
#[cfg(not(feature = "no_module"))]
Export(Vec<(Ident, Option<Ident>)>, Position),
/// Convert a variable to shared.
#[cfg(not(feature = "no_closure"))]
Share(Ident),
Share(Box<Ident>),
}
impl Default for Stmt {
@ -1257,7 +1257,7 @@ pub struct CustomExpr {
/// List of keywords.
pub keywords: StaticVec<Expr>,
/// List of tokens actually parsed.
pub tokens: Vec<ImmutableString>,
pub tokens: Vec<Identifier>,
/// Delta number of variables in the scope.
pub scope_delta: isize,
}
@ -1398,14 +1398,27 @@ pub struct FnCallExpr {
pub hash: FnCallHash,
/// Does this function call capture the parent scope?
pub capture: bool,
/// List of function call arguments.
/// List of function call argument expressions.
pub args: StaticVec<Expr>,
/// List of function call arguments that are constants.
pub constant_args: smallvec::SmallVec<[(Dynamic, Position); 2]>,
/// Namespace of the function, if any. Boxed because it occurs rarely.
pub namespace: Option<NamespaceRef>,
/// Function name.
/// Use [`Cow<'static, str>`][Cow] because a lot of operators (e.g. `==`, `>=`) are implemented as
/// function calls and the function names are predictable, so no need to allocate a new [`String`].
pub name: Cow<'static, str>,
pub name: Identifier,
}
impl FnCallExpr {
/// Are there no arguments to this function call?
#[inline(always)]
pub fn is_args_empty(&self) -> bool {
self.args.is_empty() && self.constant_args.is_empty()
}
/// Get the number of arguments to this function call.
#[inline(always)]
pub fn num_args(&self) -> usize {
self.args.len() + self.constant_args.len()
}
}
/// A type that wraps a [`FLOAT`] and implements [`Hash`].
@ -1468,6 +1481,7 @@ impl fmt::Display for FloatWrapper {
#[inline(always)]
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::Float;
let abs = self.0.abs();
@ -1534,7 +1548,7 @@ pub enum Expr {
Array(Box<StaticVec<Expr>>, Position),
/// #{ name:expr, ... }
Map(
Box<(StaticVec<(Ident, Expr)>, BTreeMap<ImmutableString, Dynamic>)>,
Box<(StaticVec<(Ident, Expr)>, BTreeMap<Identifier, Dynamic>)>,
Position,
),
/// ()
@ -1542,7 +1556,7 @@ pub enum Expr {
/// Variable access - (optional index, optional (hash, modules), variable name)
Variable(Box<(Option<NonZeroUsize>, Option<(u64, NamespaceRef)>, Ident)>),
/// Property access - ((getter, hash), (setter, hash), prop)
Property(Box<((ImmutableString, u64), (ImmutableString, u64), Ident)>),
Property(Box<((Identifier, u64), (Identifier, u64), Ident)>),
/// { [statement][Stmt] ... }
Stmt(Box<StmtBlock>),
/// func `(` expr `,` ... `)`
@ -1597,7 +1611,7 @@ impl Expr {
Self::Map(x, _) if self.is_constant() => {
let mut map = x.1.clone();
x.0.iter().for_each(|(k, v)| {
*map.get_mut(&k.name).unwrap() = v.get_constant_value().unwrap()
*map.get_mut(k.name.as_str()).unwrap() = v.get_constant_value().unwrap()
});
Dynamic(Union::Map(Box::new(map), AccessMode::ReadOnly))
}
@ -1854,8 +1868,8 @@ mod tests {
assert_eq!(size_of::<Position>(), 4);
assert_eq!(size_of::<ast::Expr>(), 16);
assert_eq!(size_of::<Option<ast::Expr>>(), 16);
assert_eq!(size_of::<ast::Stmt>(), 40);
assert_eq!(size_of::<Option<ast::Stmt>>(), 40);
assert_eq!(size_of::<ast::Stmt>(), 32);
assert_eq!(size_of::<Option<ast::Stmt>>(), 32);
assert_eq!(size_of::<FnPtr>(), 80);
assert_eq!(size_of::<Scope>(), 288);
assert_eq!(size_of::<LexError>(), 56);

19
src/dynamic.rs
View File

@ -1667,6 +1667,19 @@ impl<S: Into<ImmutableString>> From<S> for Dynamic {
Self(Union::Str(value.into(), AccessMode::ReadWrite))
}
}
impl From<&ImmutableString> for Dynamic {
#[inline(always)]
fn from(value: &ImmutableString) -> Self {
value.clone().into()
}
}
#[cfg(not(feature = "no_smartstring"))]
impl From<&crate::Identifier> for Dynamic {
#[inline(always)]
fn from(value: &crate::Identifier) -> Self {
crate::stdlib::string::ToString::to_string(value).into()
}
}
#[cfg(not(feature = "no_index"))]
impl<T: Variant + Clone> From<crate::stdlib::vec::Vec<T>> for Dynamic {
#[inline(always)]
@ -1699,7 +1712,7 @@ impl<T: Variant + Clone> crate::stdlib::iter::FromIterator<T> for Dynamic {
}
#[cfg(not(feature = "no_object"))]
#[cfg(not(feature = "no_std"))]
impl<K: Into<ImmutableString>, T: Variant + Clone> From<crate::stdlib::collections::HashMap<K, T>>
impl<K: Into<crate::Identifier>, T: Variant + Clone> From<crate::stdlib::collections::HashMap<K, T>>
for Dynamic
{
#[inline(always)]
@ -1716,8 +1729,8 @@ impl<K: Into<ImmutableString>, T: Variant + Clone> From<crate::stdlib::collectio
}
}
#[cfg(not(feature = "no_object"))]
impl<K: Into<ImmutableString>, T: Variant + Clone> From<crate::stdlib::collections::BTreeMap<K, T>>
for Dynamic
impl<K: Into<crate::Identifier>, T: Variant + Clone>
From<crate::stdlib::collections::BTreeMap<K, T>> for Dynamic
{
#[inline(always)]
fn from(value: crate::stdlib::collections::BTreeMap<K, T>) -> Self {

72
src/engine.rs
View File

@ -25,8 +25,8 @@ use crate::stdlib::{
use crate::syntax::CustomSyntax;
use crate::utils::get_hasher;
use crate::{
Dynamic, EvalAltResult, FnPtr, ImmutableString, Module, Position, RhaiResult, Scope, Shared,
StaticVec,
Dynamic, EvalAltResult, FnPtr, Identifier, ImmutableString, Module, Position, RhaiResult,
Scope, Shared, StaticVec,
};
#[cfg(not(feature = "no_index"))]
@ -50,7 +50,7 @@ pub type Precedence = NonZeroU8;
// the module name will live beyond the AST of the eval script text.
// The best we can do is a shared reference.
#[derive(Debug, Clone, Default)]
pub struct Imports(StaticVec<ImmutableString>, StaticVec<Shared<Module>>);
pub struct Imports(StaticVec<Identifier>, StaticVec<Shared<Module>>);
impl Imports {
/// Get the length of this stack of imported [modules][Module].
@ -79,7 +79,7 @@ impl Imports {
}
/// Push an imported [modules][Module] onto the stack.
#[inline(always)]
pub fn push(&mut self, name: impl Into<ImmutableString>, module: impl Into<Shared<Module>>) {
pub fn push(&mut self, name: impl Into<Identifier>, module: impl Into<Shared<Module>>) {
self.0.push(name.into());
self.1.push(module.into());
}
@ -102,18 +102,18 @@ impl Imports {
/// Get an iterator to this stack of imported [modules][Module] in reverse order.
#[allow(dead_code)]
#[inline(always)]
pub(crate) fn iter_raw(&self) -> impl Iterator<Item = (&ImmutableString, &Shared<Module>)> {
pub(crate) fn iter_raw(&self) -> impl Iterator<Item = (&Identifier, &Shared<Module>)> {
self.0.iter().rev().zip(self.1.iter().rev())
}
/// Get an iterator to this stack of imported [modules][Module] in forward order.
#[allow(dead_code)]
#[inline(always)]
pub(crate) fn scan_raw(&self) -> impl Iterator<Item = (&ImmutableString, &Shared<Module>)> {
pub(crate) fn scan_raw(&self) -> impl Iterator<Item = (&Identifier, &Shared<Module>)> {
self.0.iter().zip(self.1.iter())
}
/// Get a consuming iterator to this stack of imported [modules][Module] in reverse order.
#[inline(always)]
pub fn into_iter(self) -> impl Iterator<Item = (ImmutableString, Shared<Module>)> {
pub fn into_iter(self) -> impl Iterator<Item = (Identifier, Shared<Module>)> {
self.0.into_iter().rev().zip(self.1.into_iter().rev())
}
/// Does the specified function hash key exist in this stack of imported [modules][Module]?
@ -124,7 +124,7 @@ impl Imports {
}
/// Get specified function via its hash key.
#[inline(always)]
pub fn get_fn(&self, hash: u64) -> Option<(&CallableFunction, Option<&ImmutableString>)> {
pub fn get_fn(&self, hash: u64) -> Option<(&CallableFunction, Option<&Identifier>)> {
self.1
.iter()
.rev()
@ -491,7 +491,7 @@ pub struct FnResolutionCacheEntry {
/// Function.
pub func: CallableFunction,
/// Optional source.
pub source: Option<ImmutableString>,
pub source: Option<Identifier>,
}
/// A function resolution cache.
@ -506,7 +506,7 @@ pub type FnResolutionCache = BTreeMap<u64, Option<FnResolutionCacheEntry>>;
#[derive(Debug, Clone, Default)]
pub struct State {
/// Source of the current context.
pub source: Option<ImmutableString>,
pub source: Option<Identifier>,
/// Normally, access to variables are parsed with a relative offset into the scope to avoid a lookup.
/// In some situation, e.g. after running an `eval` statement, subsequent offsets become mis-aligned.
/// When that happens, this flag is turned on to force a scope lookup by name.
@ -703,7 +703,7 @@ pub struct Engine {
/// A collection of all modules loaded into the global namespace of the Engine.
pub(crate) global_modules: StaticVec<Shared<Module>>,
/// A collection of all sub-modules directly loaded into the Engine.
pub(crate) global_sub_modules: BTreeMap<ImmutableString, Shared<Module>>,
pub(crate) global_sub_modules: BTreeMap<Identifier, Shared<Module>>,
/// A module resolution service.
#[cfg(not(feature = "no_module"))]
@ -717,7 +717,7 @@ pub struct Engine {
/// A map containing custom keywords and precedence to recognize.
pub(crate) custom_keywords: BTreeMap<String, Option<Precedence>>,
/// Custom syntax.
pub(crate) custom_syntax: BTreeMap<ImmutableString, CustomSyntax>,
pub(crate) custom_syntax: BTreeMap<Identifier, CustomSyntax>,
/// Callback closure for resolving variable access.
pub(crate) resolve_var: Option<OnVarCallback>,
@ -1183,7 +1183,7 @@ impl Engine {
// {xxx:map}.id op= ???
Expr::Property(x) if target_val.is::<Map>() && new_val.is_some() => {
let Ident { name, pos, .. } = &x.2;
let index = name.clone().into();
let index = name.into();
let val = self.get_indexed_mut(
mods, state, lib, target_val, index, *pos, true, is_ref, false, level,
)?;
@ -1196,7 +1196,7 @@ impl Engine {
// {xxx:map}.id
Expr::Property(x) if target_val.is::<Map>() => {
let Ident { name, pos, .. } = &x.2;
let index = name.clone().into();
let index = name.into();
let val = self.get_indexed_mut(
mods, state, lib, target_val, index, *pos, false, is_ref, false, level,
)?;
@ -1231,7 +1231,7 @@ impl Engine {
let mut val = match &x.lhs {
Expr::Property(p) => {
let Ident { name, pos, .. } = &p.2;
let index = name.clone().into();
let index = name.into();
self.get_indexed_mut(
mods, state, lib, target_val, index, *pos, false, is_ref, true,
level,
@ -1440,16 +1440,21 @@ impl Engine {
Expr::FnCall(x, _) if parent_chain_type == ChainType::Dot && x.namespace.is_none() => {
let mut arg_positions: StaticVec<_> = Default::default();
let arg_values = x
let mut arg_values = x
.args
.iter()
.inspect(|arg_expr| arg_positions.push(arg_expr.position()))
.map(|arg_expr| {
arg_positions.push(arg_expr.position());
self.eval_expr(scope, mods, state, lib, this_ptr, arg_expr, level)
.map(Dynamic::flatten)
})
.collect::<Result<StaticVec<_>, _>>()?;
x.constant_args
.iter()
.inspect(|(_, pos)| arg_positions.push(*pos))
.for_each(|(v, _)| arg_values.push(v.clone()));
idx_values.push((arg_values, arg_positions).into());
}
Expr::FnCall(_, _) if parent_chain_type == ChainType::Dot => {
@ -1475,16 +1480,21 @@ impl Engine {
{
let mut arg_positions: StaticVec<_> = Default::default();
let arg_values = x
let mut arg_values = x
.args
.iter()
.inspect(|arg_expr| arg_positions.push(arg_expr.position()))
.map(|arg_expr| {
arg_positions.push(arg_expr.position());
self.eval_expr(scope, mods, state, lib, this_ptr, arg_expr, level)
.map(Dynamic::flatten)
})
.collect::<Result<StaticVec<_>, _>>()?;
x.constant_args
.iter()
.inspect(|(_, pos)| arg_positions.push(*pos))
.for_each(|(v, _)| arg_values.push(v.clone()));
(arg_values, arg_positions).into()
}
Expr::FnCall(_, _) if parent_chain_type == ChainType::Dot => {
@ -1563,12 +1573,12 @@ impl Engine {
self.make_type_mismatch_err::<ImmutableString>(idx.type_name(), idx_pos)
})?;
if _create && !map.contains_key(index) {
map.insert(index.clone(), Default::default());
if _create && !map.contains_key(index.as_str()) {
map.insert(index.clone().into(), Default::default());
}
Ok(map
.get_mut(index)
.get_mut(index.as_str())
.map(Target::from)
.unwrap_or_else(|| Target::from(())))
}
@ -1686,7 +1696,7 @@ impl Engine {
Expr::Map(x, _) => {
let mut map = x.1.clone();
for (Ident { name: key, .. }, expr) in &x.0 {
*map.get_mut(key).unwrap() = self
*map.get_mut(key.as_str()).unwrap() = self
.eval_expr(scope, mods, state, lib, this_ptr, expr, level)?
.flatten();
}
@ -1700,10 +1710,12 @@ impl Engine {
capture,
hash,
args,
constant_args: c_args,
..
} = x.as_ref();
self.make_function_call(
scope, mods, state, lib, this_ptr, name, args, *hash, *pos, *capture, level,
scope, mods, state, lib, this_ptr, name, args, c_args, *hash, *pos, *capture,
level,
)
}
@ -1714,12 +1726,14 @@ impl Engine {
namespace,
hash,
args,
constant_args: c_args,
..
} = x.as_ref();
let namespace = namespace.as_ref();
let hash = hash.native_hash();
self.make_qualified_function_call(
scope, mods, state, lib, this_ptr, namespace, name, args, hash, *pos, level,
scope, mods, state, lib, this_ptr, namespace, name, args, c_args, hash, *pos,
level,
)
}
@ -2195,7 +2209,7 @@ impl Engine {
if let Some(func) = func {
// Add the loop variable
let var_name: Cow<'_, str> = if state.is_global() {
name.clone().into()
name.to_string().into()
} else {
unsafe_cast_var_name_to_lifetime(name).into()
};
@ -2286,7 +2300,7 @@ impl Engine {
err_map.insert("message".into(), err.to_string().into());
if let Some(ref source) = state.source {
err_map.insert("source".into(), source.clone().into());
err_map.insert("source".into(), source.into());
}
if err_pos.is_none() {
@ -2368,8 +2382,8 @@ impl Engine {
}
// Let/const statement
Stmt::Let(expr, Ident { name, .. }, export, _)
| Stmt::Const(expr, Ident { name, .. }, export, _) => {
Stmt::Let(expr, x, export, _) | Stmt::Const(expr, x, export, _) => {
let name = &x.name;
let entry_type = match stmt {
Stmt::Let(_, _, _, _) => AccessMode::ReadWrite,
Stmt::Const(_, _, _, _) => AccessMode::ReadOnly,

36
src/engine_api.rs
View File

@ -8,11 +8,10 @@ use crate::optimize::OptimizationLevel;
use crate::stdlib::{
any::{type_name, TypeId},
boxed::Box,
format,
string::String,
};
use crate::{
scope::Scope, Dynamic, Engine, EvalAltResult, FnAccess, FnNamespace, ImmutableString, Module,
scope::Scope, Dynamic, Engine, EvalAltResult, FnAccess, FnNamespace, Identifier, Module,
NativeCallContext, ParseError, Position, RhaiResult, Shared, AST,
};
@ -53,7 +52,7 @@ impl Engine {
#[inline]
pub fn register_fn<N, A, F>(&mut self, name: N, func: F) -> &mut Self
where
N: AsRef<str> + Into<ImmutableString>,
N: AsRef<str> + Into<Identifier>,
F: RegisterNativeFunction<A, ()>,
{
let param_types = F::param_types();
@ -61,7 +60,7 @@ impl Engine {
#[cfg(feature = "metadata")]
let mut param_type_names: crate::StaticVec<_> = F::param_names()
.iter()
.map(|ty| format!("_: {}", self.map_type_name(ty)))
.map(|ty| crate::stdlib::format!("_: {}", self.map_type_name(ty)))
.collect();
#[cfg(feature = "metadata")]
@ -113,7 +112,7 @@ impl Engine {
#[inline]
pub fn register_result_fn<N, A, F, R>(&mut self, name: N, func: F) -> &mut Self
where
N: AsRef<str> + Into<ImmutableString>,
N: AsRef<str> + Into<Identifier>,
F: RegisterNativeFunction<A, Result<R, Box<EvalAltResult>>>,
{
let param_types = F::param_types();
@ -121,7 +120,7 @@ impl Engine {
#[cfg(feature = "metadata")]
let param_type_names: crate::StaticVec<_> = F::param_names()
.iter()
.map(|ty| format!("_: {}", self.map_type_name(ty)))
.map(|ty| crate::stdlib::format!("_: {}", self.map_type_name(ty)))
.chain(crate::stdlib::iter::once(
self.map_type_name(F::return_type_name()).into(),
))
@ -170,7 +169,7 @@ impl Engine {
+ 'static,
) -> &mut Self
where
N: AsRef<str> + Into<ImmutableString>,
N: AsRef<str> + Into<Identifier>,
T: Variant + Clone,
{
self.global_namespace.set_raw_fn(
@ -901,12 +900,12 @@ impl Engine {
#[cfg(not(feature = "no_module"))]
pub fn register_static_module(
&mut self,
name: impl AsRef<str> + Into<ImmutableString>,
name: impl AsRef<str> + Into<Identifier>,
module: Shared<Module>,
) -> &mut Self {
fn register_static_module_raw(
root: &mut crate::stdlib::collections::BTreeMap<crate::ImmutableString, Shared<Module>>,
name: impl AsRef<str> + Into<ImmutableString>,
root: &mut crate::stdlib::collections::BTreeMap<Identifier, Shared<Module>>,
name: impl AsRef<str> + Into<Identifier>,
module: Shared<Module>,
) {
let separator = crate::token::Token::DoubleColon.syntax();
@ -952,7 +951,7 @@ impl Engine {
#[deprecated(since = "0.19.9", note = "use `register_static_module` instead")]
pub fn register_module(
&mut self,
name: impl AsRef<str> + Into<ImmutableString>,
name: impl AsRef<str> + Into<Identifier>,
module: impl Into<Shared<Module>>,
) -> &mut Self {
self.register_static_module(name, module.into())
@ -1045,14 +1044,14 @@ impl Engine {
fn collect_imports(
ast: &AST,
resolver: &StaticModuleResolver,
imports: &mut BTreeSet<ImmutableString>,
imports: &mut BTreeSet<Identifier>,
) {
ast.walk(&mut |path| match path.last().unwrap() {
// Collect all `import` statements with a string constant path
ASTNode::Stmt(Stmt::Import(Expr::StringConstant(s, _), _, _))
if !resolver.contains_path(s) && !imports.contains(s) =>
if !resolver.contains_path(s) && !imports.contains(s.as_str()) =>
{
imports.insert(s.clone());
imports.insert(s.clone().into());
true
}
_ => true,
@ -1167,7 +1166,7 @@ impl Engine {
let mut f = crate::stdlib::fs::File::open(path.clone()).map_err(|err| {
EvalAltResult::ErrorSystem(
format!("Cannot open script file '{}'", path.to_string_lossy()),
crate::stdlib::format!("Cannot open script file '{}'", path.to_string_lossy()),
err.into(),
)
})?;
@ -1176,7 +1175,7 @@ impl Engine {
f.read_to_string(&mut contents).map_err(|err| {
EvalAltResult::ErrorSystem(
format!("Cannot read script file '{}'", path.to_string_lossy()),
crate::stdlib::format!("Cannot read script file '{}'", path.to_string_lossy()),
err.into(),
)
})?;
@ -1991,7 +1990,10 @@ impl Engine {
signatures.extend(self.global_namespace.gen_fn_signatures());
self.global_sub_modules.iter().for_each(|(name, m)| {
signatures.extend(m.gen_fn_signatures().map(|f| format!("{}::{}", name, f)))
signatures.extend(
m.gen_fn_signatures()
.map(|f| crate::stdlib::format!("{}::{}", name, f)),
)
});
if include_packages {

21
src/fn_builtin.rs
View File

@ -11,10 +11,6 @@ use crate::FLOAT;
#[cfg(feature = "decimal")]
use rust_decimal::Decimal;
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::float::Float;
/// Is the type a numeric type?
#[inline(always)]
fn is_numeric(type_id: TypeId) -> bool {
@ -45,6 +41,10 @@ pub fn get_builtin_binary_op_fn(
x: &Dynamic,
y: &Dynamic,
) -> Option<fn(NativeCallContext, &mut FnCallArgs) -> RhaiResult> {
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::Float;
let type1 = x.type_id();
let type2 = y.type_id();
@ -85,6 +85,13 @@ pub fn get_builtin_binary_op_fn(
Ok(x.$func(y).into())
})
};
($xx:ident . $func:ident ( $yy:ident . $yyy:ident () )) => {
return Some(|_, args| {
let x = &*args[0].read_lock::<$xx>().unwrap();
let y = &*args[1].read_lock::<$yy>().unwrap();
Ok(x.$func(y.$yyy()).into())
})
};
($func:ident ( $op:tt )) => {
return Some(|_, args| {
let (x, y) = $func(args);
@ -284,7 +291,7 @@ pub fn get_builtin_binary_op_fn(
use crate::Map;
match op {
OP_CONTAINS => impl_op!(Map.contains_key(ImmutableString)),
OP_CONTAINS => impl_op!(Map.contains_key(ImmutableString.as_str())),
_ => return None,
}
}
@ -415,6 +422,10 @@ pub fn get_builtin_op_assignment_fn(
x: &Dynamic,
y: &Dynamic,
) -> Option<fn(NativeCallContext, &mut FnCallArgs) -> RhaiResult> {
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::Float;
let type1 = x.type_id();
let type2 = y.type_id();

188
src/fn_call.rs
View File

@ -96,6 +96,7 @@ impl Drop for ArgBackup<'_> {
}
}
#[cfg(not(feature = "no_closure"))]
#[inline(always)]
pub fn ensure_no_data_race(
fn_name: &str,
@ -1058,33 +1059,39 @@ impl Engine {
this_ptr: &mut Option<&mut Dynamic>,
fn_name: &str,
args_expr: &[Expr],
constant_args: &[(Dynamic, Position)],
mut hash: FnCallHash,
pos: Position,
capture_scope: bool,
level: usize,
) -> RhaiResult {
let args_expr = args_expr.as_ref();
// Handle call() - Redirect function call
let redirected;
let mut args_expr = args_expr.as_ref();
let mut args_expr = args_expr;
let mut constant_args = constant_args;
let mut total_args = args_expr.len() + constant_args.len();
let mut curry = StaticVec::new();
let mut name = fn_name;
match name {
// Handle call()
KEYWORD_FN_PTR_CALL if args_expr.len() >= 1 => {
let fn_ptr =
self.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[0], level)?;
KEYWORD_FN_PTR_CALL if total_args >= 1 => {
let (arg, arg_pos) = args_expr.get(0).map_or_else(
|| Ok(constant_args[0].clone()),
|arg| {
self.eval_expr(scope, mods, state, lib, this_ptr, arg, level)
.map(|v| (v, arg.position()))
},
)?;
if !fn_ptr.is::<FnPtr>() {
if !arg.is::<FnPtr>() {
return Err(self.make_type_mismatch_err::<FnPtr>(
self.map_type_name(fn_ptr.type_name()),
args_expr[0].position(),
self.map_type_name(arg.type_name()),
arg_pos,
));
}
let fn_ptr = fn_ptr.cast::<FnPtr>();
let fn_ptr = arg.cast::<FnPtr>();
curry.extend(fn_ptr.curry().iter().cloned());
// Redirect function name
@ -1092,10 +1099,15 @@ impl Engine {
name = &redirected;
// Skip the first argument
args_expr = &args_expr.as_ref()[1..];
if !args_expr.is_empty() {
args_expr = &args_expr[1..];
} else {
constant_args = &constant_args[1..];
}
total_args -= 1;
// Recalculate hash
let args_len = args_expr.len() + curry.len();
let args_len = total_args + curry.len();
hash = if !hash.is_native_only() {
FnCallHash::from_script(calc_fn_hash(empty(), name, args_len))
} else {
@ -1103,66 +1115,95 @@ impl Engine {
};
}
// Handle Fn()
KEYWORD_FN_PTR if args_expr.len() == 1 => {
KEYWORD_FN_PTR if total_args == 1 => {
let (arg, arg_pos) = args_expr.get(0).map_or_else(
|| Ok(constant_args[0].clone()),
|arg| {
self.eval_expr(scope, mods, state, lib, this_ptr, arg, level)
.map(|v| (v, arg.position()))
},
)?;
// Fn - only in function call style
return self
.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[0], level)?
return arg
.take_immutable_string()
.map_err(|typ| {
self.make_type_mismatch_err::<ImmutableString>(typ, args_expr[0].position())
})
.map_err(|typ| self.make_type_mismatch_err::<ImmutableString>(typ, arg_pos))
.and_then(|s| FnPtr::try_from(s))
.map(Into::<Dynamic>::into)
.map_err(|err| err.fill_position(args_expr[0].position()));
.map_err(|err| err.fill_position(arg_pos));
}
// Handle curry()
KEYWORD_FN_PTR_CURRY if args_expr.len() > 1 => {
let fn_ptr =
self.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[0], level)?;
KEYWORD_FN_PTR_CURRY if total_args > 1 => {
let (arg, arg_pos) = args_expr.get(0).map_or_else(
|| Ok(constant_args[0].clone()),
|arg| {
self.eval_expr(scope, mods, state, lib, this_ptr, arg, level)
.map(|v| (v, arg.position()))
},
)?;
if !fn_ptr.is::<FnPtr>() {
if !arg.is::<FnPtr>() {
return Err(self.make_type_mismatch_err::<FnPtr>(
self.map_type_name(fn_ptr.type_name()),
args_expr[0].position(),
self.map_type_name(arg.type_name()),
arg_pos,
));
}
let (name, mut fn_curry) = fn_ptr.cast::<FnPtr>().take_data();
let (name, mut fn_curry) = arg.cast::<FnPtr>().take_data();
// Append the new curried arguments to the existing list.
args_expr.iter().skip(1).try_for_each(|expr| {
self.eval_expr(scope, mods, state, lib, this_ptr, expr, level)
.map(|value| fn_curry.push(value))
})?;
if !args_expr.is_empty() {
args_expr.iter().skip(1).try_for_each(|expr| {
self.eval_expr(scope, mods, state, lib, this_ptr, expr, level)
.map(|value| fn_curry.push(value))
})?;
fn_curry.extend(constant_args.iter().map(|(v, _)| v.clone()));
} else {
fn_curry.extend(constant_args.iter().skip(1).map(|(v, _)| v.clone()));
}
return Ok(FnPtr::new_unchecked(name, fn_curry).into());
}
// Handle is_shared()
#[cfg(not(feature = "no_closure"))]
crate::engine::KEYWORD_IS_SHARED if args_expr.len() == 1 => {
let value =
self.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[0], level)?;
return Ok(value.is_shared().into());
crate::engine::KEYWORD_IS_SHARED if total_args == 1 => {
let arg = args_expr.get(0).map_or_else(
|| Ok(constant_args[0].0.clone()),
|arg| self.eval_expr(scope, mods, state, lib, this_ptr, arg, level),
)?;
return Ok(arg.is_shared().into());
}
// Handle is_def_fn()
#[cfg(not(feature = "no_function"))]
crate::engine::KEYWORD_IS_DEF_FN if args_expr.len() == 2 => {
let fn_name = self
.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[0], level)?
crate::engine::KEYWORD_IS_DEF_FN if total_args == 2 => {
let (arg, arg_pos) = if !args_expr.is_empty() {
(
self.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[0], level)?,
args_expr[0].position(),
)
} else {
constant_args[0].clone()
};
let fn_name = arg
.take_immutable_string()
.map_err(|err| {
self.make_type_mismatch_err::<ImmutableString>(err, args_expr[0].position())
})?;
let num_params = self
.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[1], level)?
.map_err(|err| self.make_type_mismatch_err::<ImmutableString>(err, arg_pos))?;
let (arg, arg_pos) = if args_expr.len() > 1 {
(
self.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[1], level)?,
args_expr[1].position(),
)
} else {
constant_args[if args_expr.is_empty() { 1 } else { 0 }].clone()
};
let num_params = arg
.as_int()
.map_err(|err| {
self.make_type_mismatch_err::<crate::INT>(err, args_expr[0].position())
})?;
.map_err(|err| self.make_type_mismatch_err::<crate::INT>(err, arg_pos))?;
return Ok(if num_params < 0 {
Dynamic::FALSE
@ -1174,29 +1215,34 @@ impl Engine {
}
// Handle is_def_var()
KEYWORD_IS_DEF_VAR if args_expr.len() == 1 => {
let var_name = self
.eval_expr(scope, mods, state, lib, this_ptr, &args_expr[0], level)?
KEYWORD_IS_DEF_VAR if total_args == 1 => {
let (arg, arg_pos) = args_expr.get(0).map_or_else(
|| Ok(constant_args[0].clone()),
|arg| {
self.eval_expr(scope, mods, state, lib, this_ptr, arg, level)
.map(|v| (v, arg.position()))
},
)?;
let var_name = arg
.take_immutable_string()
.map_err(|err| {
self.make_type_mismatch_err::<ImmutableString>(err, args_expr[0].position())
})?;
.map_err(|err| self.make_type_mismatch_err::<ImmutableString>(err, arg_pos))?;
return Ok(scope.contains(&var_name).into());
}
// Handle eval()
KEYWORD_EVAL if args_expr.len() == 1 => {
let script_expr = &args_expr[0];
let script_pos = script_expr.position();
KEYWORD_EVAL if total_args == 1 => {
// eval - only in function call style
let prev_len = scope.len();
let script = self
.eval_expr(scope, mods, state, lib, this_ptr, script_expr, level)?
.take_immutable_string()
.map_err(|typ| {
self.make_type_mismatch_err::<ImmutableString>(typ, script_pos)
})?;
let (script, script_pos) = args_expr.get(0).map_or_else(
|| Ok(constant_args[0].clone()),
|script_expr| {
self.eval_expr(scope, mods, state, lib, this_ptr, script_expr, level)
.map(|v| (v, script_expr.position()))
},
)?;
let script = script.take_immutable_string().map_err(|typ| {
self.make_type_mismatch_err::<ImmutableString>(typ, script_pos)
})?;
let result = self.eval_script_expr_in_place(
scope,
mods,
@ -1240,14 +1286,17 @@ impl Engine {
None
};
if args_expr.is_empty() && curry.is_empty() {
if args_expr.is_empty() && constant_args.is_empty() && curry.is_empty() {
// No arguments
args = Default::default();
} else {
// If the first argument is a variable, and there is no curried arguments,
// convert to method-call style in order to leverage potential &mut first argument and
// avoid cloning the value
if curry.is_empty() && args_expr[0].get_variable_access(false).is_some() {
if curry.is_empty()
&& !args_expr.is_empty()
&& args_expr[0].get_variable_access(false).is_some()
{
// func(x, ...) -> x.func(...)
arg_values = args_expr
.iter()
@ -1256,6 +1305,7 @@ impl Engine {
self.eval_expr(scope, mods, state, lib, this_ptr, expr, level)
.map(Dynamic::flatten)
})
.chain(constant_args.iter().map(|(v, _)| Ok(v.clone())))
.collect::<Result<_, _>>()?;
let (mut target, pos) =
@ -1285,6 +1335,7 @@ impl Engine {
self.eval_expr(scope, mods, state, lib, this_ptr, expr, level)
.map(Dynamic::flatten)
})
.chain(constant_args.iter().map(|(v, _)| Ok(v.clone())))
.collect::<Result<_, _>>()?;
args = curry.iter_mut().chain(arg_values.iter_mut()).collect();
@ -1310,25 +1361,24 @@ impl Engine {
namespace: Option<&NamespaceRef>,
fn_name: &str,
args_expr: &[Expr],
constant_args: &[(Dynamic, Position)],
hash: u64,
pos: Position,
level: usize,
) -> RhaiResult {
let args_expr = args_expr.as_ref();
let namespace = namespace.unwrap();
let mut arg_values: StaticVec<_>;
let mut first_arg_value = None;
let mut args: StaticVec<_>;
if args_expr.is_empty() {
if args_expr.is_empty() && constant_args.is_empty() {
// No arguments
args = Default::default();
} else {
// See if the first argument is a variable (not namespace-qualified).
// If so, convert to method-call style in order to leverage potential
// &mut first argument and avoid cloning the value
if args_expr[0].get_variable_access(true).is_some() {
if !args_expr.is_empty() && args_expr[0].get_variable_access(true).is_some() {
// func(x, ...) -> x.func(...)
arg_values = args_expr
.iter()
@ -1342,6 +1392,7 @@ impl Engine {
.map(Dynamic::flatten)
}
})
.chain(constant_args.iter().map(|(v, _)| Ok(v.clone())))
.collect::<Result<_, _>>()?;
// Get target reference to first argument
@ -1368,6 +1419,7 @@ impl Engine {
self.eval_expr(scope, mods, state, lib, this_ptr, expr, level)
.map(Dynamic::flatten)
})
.chain(constant_args.iter().map(|(v, _)| Ok(v.clone())))
.collect::<Result<_, _>>()?;
args = arg_values.iter_mut().collect();

8
src/fn_native.rs
View File

@ -13,8 +13,8 @@ use crate::stdlib::{
};
use crate::token::is_valid_identifier;
use crate::{
calc_fn_hash, Dynamic, Engine, EvalAltResult, EvalContext, ImmutableString, Module, Position,
RhaiResult, StaticVec,
calc_fn_hash, Dynamic, Engine, EvalAltResult, EvalContext, Identifier, ImmutableString, Module,
Position, RhaiResult, StaticVec,
};
/// Trait that maps to `Send + Sync` only under the `sync` feature.
@ -144,9 +144,7 @@ impl<'a> NativeCallContext<'a> {
#[cfg(not(feature = "no_module"))]
#[allow(dead_code)]
#[inline(always)]
pub(crate) fn iter_imports_raw(
&self,
) -> impl Iterator<Item = (&ImmutableString, &Shared<Module>)> {
pub(crate) fn iter_imports_raw(&self) -> impl Iterator<Item = (&Identifier, &Shared<Module>)> {
self.mods.iter().flat_map(|&m| m.iter_raw())
}
/// _(INTERNALS)_ The current set of modules imported via `import` statements.

11
src/lib.rs
View File

@ -135,6 +135,15 @@ pub use syntax::Expression;
pub use token::Position;
pub use utils::ImmutableString;
/// An identifier in Rhai. [`SmartString`](https://crates.io/crates/smartstring) is used because most
/// identifiers are ASCII and short, fewer than 23 characters, so they can be stored inline.
#[cfg(not(feature = "no_smartstring"))]
pub type Identifier = smartstring::SmartString<smartstring::Compact>;
/// An identifier in Rhai.
#[cfg(feature = "no_smartstring")]
pub type Identifier = ImmutableString;
/// A trait to enable registering Rust functions.
/// This trait is no longer needed and will be removed in the future.
#[deprecated(
@ -180,7 +189,7 @@ pub type Array = stdlib::vec::Vec<Dynamic>;
///
/// Not available under `no_object`.
#[cfg(not(feature = "no_object"))]
pub type Map = stdlib::collections::BTreeMap<ImmutableString, Dynamic>;
pub type Map = stdlib::collections::BTreeMap<Identifier, Dynamic>;
#[cfg(not(feature = "no_module"))]
pub use module::ModuleResolver;

44
src/module/mod.rs
View File

@ -16,10 +16,10 @@ use crate::stdlib::{
vec::Vec,
};
use crate::token::Token;
use crate::utils::StringInterner;
use crate::utils::IdentifierBuilder;
use crate::{
calc_fn_hash, calc_fn_params_hash, combine_hashes, Dynamic, EvalAltResult, ImmutableString,
NativeCallContext, Position, Shared, StaticVec,
calc_fn_hash, calc_fn_params_hash, combine_hashes, Dynamic, EvalAltResult, Identifier,
ImmutableString, NativeCallContext, Position, Shared, StaticVec,
};
#[cfg(not(feature = "no_index"))]
@ -55,14 +55,14 @@ pub struct FuncInfo {
/// Function access mode.
pub access: FnAccess,
/// Function name.
pub name: ImmutableString,
pub name: Identifier,
/// Number of parameters.
pub params: usize,
/// Parameter types (if applicable).
pub param_types: StaticVec<TypeId>,
/// Parameter names (if available).
#[cfg(feature = "metadata")]
pub param_names: StaticVec<ImmutableString>,
pub param_names: StaticVec<Identifier>,
}
impl FuncInfo {
@ -128,11 +128,11 @@ fn calc_native_fn_hash<'a>(
#[derive(Clone)]
pub struct Module {
/// ID identifying the module.
id: Option<ImmutableString>,
id: Option<Identifier>,
/// Sub-modules.
modules: BTreeMap<ImmutableString, Shared<Module>>,
modules: BTreeMap<Identifier, Shared<Module>>,
/// [`Module`] variables.
variables: BTreeMap<ImmutableString, Dynamic>,
variables: BTreeMap<Identifier, Dynamic>,
/// Flattened collection of all [`Module`] variables, including those in sub-modules.
all_variables: BTreeMap<u64, Dynamic>,
/// External Rust functions.
@ -149,7 +149,7 @@ pub struct Module {
/// Does the [`Module`] contain indexed functions that have been exposed to the global namespace?
contains_indexed_global_functions: bool,
/// Interned strings
interned_strings: StringInterner,
identifiers: IdentifierBuilder,
}
impl Default for Module {
@ -166,7 +166,7 @@ impl Default for Module {
all_type_iterators: Default::default(),
indexed: false,
contains_indexed_global_functions: false,
interned_strings: Default::default(),
identifiers: Default::default(),
}
}
}
@ -288,7 +288,7 @@ impl Module {
self.id_raw().map(|s| s.as_str())
}
/// Get the ID of the [`Module`] as an [`ImmutableString`], if any.
/// Get the ID of the [`Module`] as an [`Identifier`], if any.
///
/// # Example
///
@ -300,7 +300,7 @@ impl Module {
/// assert_eq!(module.id_raw().map(|s| s.as_str()), Some("hello"));
/// ```
#[inline(always)]
pub fn id_raw(&self) -> Option<&ImmutableString> {
pub fn id_raw(&self) -> Option<&Identifier> {
self.id.as_ref()
}
@ -316,7 +316,7 @@ impl Module {
/// assert_eq!(module.id(), Some("hello"));
/// ```
#[inline(always)]
pub fn set_id<S: Into<ImmutableString>>(&mut self, id: Option<S>) {
pub fn set_id<S: Into<Identifier>>(&mut self, id: Option<S>) {
self.id = id.map(|s| s.into());
}
@ -440,7 +440,7 @@ impl Module {
#[inline(always)]
pub fn set_var(
&mut self,
name: impl Into<ImmutableString>,
name: impl Into<Identifier>,
value: impl Variant + Clone,
) -> &mut Self {
self.variables.insert(name.into(), Dynamic::from(value));
@ -514,7 +514,7 @@ impl Module {
/// Thus the [`Module`] is automatically set to be non-indexed.
#[cfg(not(feature = "no_module"))]
#[inline(always)]
pub(crate) fn sub_modules_mut(&mut self) -> &mut BTreeMap<ImmutableString, Shared<Module>> {
pub(crate) fn sub_modules_mut(&mut self) -> &mut BTreeMap<Identifier, Shared<Module>> {
// We must assume that the user has changed the sub-modules
// (otherwise why take a mutable reference?)
self.all_functions.clear();
@ -577,7 +577,7 @@ impl Module {
#[inline(always)]
pub fn set_sub_module(
&mut self,
name: impl Into<ImmutableString>,
name: impl Into<Identifier>,
sub_module: impl Into<Shared<Module>>,
) -> &mut Self {
self.modules.insert(name.into(), sub_module.into());
@ -622,7 +622,7 @@ impl Module {
pub fn update_fn_metadata(&mut self, hash_fn: u64, arg_names: &[&str]) -> &mut Self {
let param_names = arg_names
.iter()
.map(|&name| self.interned_strings.get(name))
.map(|&name| self.identifiers.get(name))
.collect();
if let Some(f) = self.functions.get_mut(&hash_fn) {
@ -672,7 +672,7 @@ impl Module {
#[inline]
pub fn set_fn(
&mut self,
name: impl AsRef<str> + Into<ImmutableString>,
name: impl AsRef<str> + Into<Identifier>,
namespace: FnNamespace,
access: FnAccess,
_arg_names: Option<&[&str]>,
@ -692,7 +692,7 @@ impl Module {
let param_names = _arg_names
.iter()
.flat_map(|p| p.iter())
.map(|&arg| self.interned_strings.get(arg))
.map(|&arg| self.identifiers.get(arg))
.collect();
let hash_fn = calc_native_fn_hash(empty(), &name, &param_types);
@ -700,7 +700,7 @@ impl Module {
self.functions.insert(
hash_fn,
Box::new(FuncInfo {
name: self.interned_strings.get(name),
name: self.identifiers.get(name),
namespace,
access,
params: param_types.len(),
@ -793,7 +793,7 @@ impl Module {
func: F,
) -> u64
where
N: AsRef<str> + Into<ImmutableString>,
N: AsRef<str> + Into<Identifier>,
T: Variant + Clone,
F: Fn(NativeCallContext, &mut FnCallArgs) -> Result<T, Box<EvalAltResult>>
+ SendSync
@ -838,7 +838,7 @@ impl Module {
#[inline(always)]
pub fn set_native_fn<ARGS, N, T, F>(&mut self, name: N, func: F) -> u64
where
N: AsRef<str> + Into<ImmutableString>,
N: AsRef<str> + Into<Identifier>,
T: Variant + Clone,
F: RegisterNativeFunction<ARGS, Result<T, Box<EvalAltResult>>>,
{

47
src/optimize.rs
View File

@ -1,6 +1,6 @@
//! Module implementing the [`AST`] optimizer.
use crate::ast::{Expr, Ident, Stmt, StmtBlock};
use crate::ast::{Expr, Stmt, StmtBlock};
use crate::dynamic::AccessMode;
use crate::engine::{KEYWORD_DEBUG, KEYWORD_EVAL, KEYWORD_PRINT, KEYWORD_TYPE_OF};
use crate::fn_builtin::get_builtin_binary_op_fn;
@ -214,17 +214,17 @@ fn optimize_stmt_block(
statements.iter_mut().for_each(|stmt| {
match stmt {
// Add constant literals into the state
Stmt::Const(value_expr, Ident { name, .. }, _, _) => {
Stmt::Const(value_expr, x, _, _) => {
optimize_expr(value_expr, state);
if value_expr.is_constant() {
state.push_var(name, AccessMode::ReadOnly, value_expr.clone());
state.push_var(&x.name, AccessMode::ReadOnly, value_expr.clone());
}
}
// Add variables into the state
Stmt::Let(value_expr, Ident { name, pos, .. }, _, _) => {
Stmt::Let(value_expr, x, _, _) => {
optimize_expr(value_expr, state);
state.push_var(name, AccessMode::ReadWrite, Expr::Unit(*pos));
state.push_var(&x.name, AccessMode::ReadWrite, Expr::Unit(x.pos));
}
// Optimize the statement
_ => optimize_stmt(stmt, state, preserve_result),
@ -649,7 +649,7 @@ fn optimize_expr(expr: &mut Expr, state: &mut State) {
// Map literal where everything is pure - promote the indexed item.
// All other items can be thrown away.
state.set_dirty();
*expr = mem::take(&mut m.0).into_iter().find(|(x, _)| x.name == *s)
*expr = mem::take(&mut m.0).into_iter().find(|(x, _)| x.name.as_str() == s.as_str())
.map(|(_, mut expr)| { expr.set_position(*pos); expr })
.unwrap_or_else(|| Expr::Unit(*pos));
}
@ -740,11 +740,14 @@ fn optimize_expr(expr: &mut Expr, state: &mut State) {
Expr::FnCall(x, pos)
if x.namespace.is_none() // Non-qualified
&& state.optimization_level == OptimizationLevel::Simple // simple optimizations
&& x.args.len() == 2 // binary call
&& x.num_args() == 2 // binary call
&& x.args.iter().all(Expr::is_constant) // all arguments are constants
//&& !is_valid_identifier(x.name.chars()) // cannot be scripted
=> {
let mut arg_values: StaticVec<_> = x.args.iter().map(|e| e.get_constant_value().unwrap()).collect();
let mut arg_values: StaticVec<_> = x.args.iter().map(|e| e.get_constant_value().unwrap())
.chain(x.constant_args.iter().map(|(v, _)| v).cloned())
.collect();
let arg_types: StaticVec<_> = arg_values.iter().map(Dynamic::type_id).collect();
// Search for overloaded operators (can override built-in).
@ -764,6 +767,15 @@ fn optimize_expr(expr: &mut Expr, state: &mut State) {
}
x.args.iter_mut().for_each(|a| optimize_expr(a, state));
// Move constant arguments to the right
while x.args.last().map(Expr::is_constant).unwrap_or(false) {
let arg = x.args.pop().unwrap();
let arg_pos = arg.position();
x.constant_args.insert(0, (arg.get_constant_value().unwrap(), arg_pos));
}
x.args.shrink_to_fit();
}
// Eagerly call functions
@ -774,12 +786,14 @@ fn optimize_expr(expr: &mut Expr, state: &mut State) {
=> {
// First search for script-defined functions (can override built-in)
#[cfg(not(feature = "no_function"))]
let has_script_fn = state.lib.iter().any(|&m| m.get_script_fn(x.name.as_ref(), x.args.len()).is_some());
let has_script_fn = state.lib.iter().any(|&m| m.get_script_fn(x.name.as_ref(), x.num_args()).is_some());
#[cfg(feature = "no_function")]
let has_script_fn = false;
if !has_script_fn {
let mut arg_values: StaticVec<_> = x.args.iter().map(|e| e.get_constant_value().unwrap()).collect();
let mut arg_values: StaticVec<_> = x.args.iter().map(|e| e.get_constant_value().unwrap())
.chain(x.constant_args.iter().map(|(v, _)| v).cloned())
.collect();
// Save the typename of the first argument if it is `type_of()`
// This is to avoid `call_args` being passed into the closure
@ -810,7 +824,18 @@ fn optimize_expr(expr: &mut Expr, state: &mut State) {
}
// id(args ..) -> optimize function call arguments
Expr::FnCall(x, _) => x.args.iter_mut().for_each(|a| optimize_expr(a, state)),
Expr::FnCall(x, _) => {
x.args.iter_mut().for_each(|a| optimize_expr(a, state));
// Move constant arguments to the right
while x.args.last().map(Expr::is_constant).unwrap_or(false) {
let arg = x.args.pop().unwrap();
let arg_pos = arg.position();
x.constant_args.insert(0, (arg.get_constant_value().unwrap(), arg_pos));
}
x.args.shrink_to_fit();
}
// constant-name
Expr::Variable(x) if x.1.is_none() && state.find_constant(&x.2.name).is_some() => {

2
src/packages/arithmetic.rs
View File

@ -9,7 +9,7 @@ use crate::FLOAT;
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::float::Float;
use num_traits::Float;
#[inline(always)]
pub fn make_err(msg: impl Into<String>) -> Box<EvalAltResult> {

25
src/packages/fn_basic.rs
View File

@ -34,22 +34,25 @@ mod fn_ptr_functions {
#[cfg(not(feature = "no_index"))]
#[cfg(not(feature = "no_object"))]
fn collect_fn_metadata(ctx: NativeCallContext) -> crate::Array {
use crate::{ast::ScriptFnDef, stdlib::collections::BTreeSet, Array, Map};
use crate::{ast::ScriptFnDef, stdlib::collections::BTreeSet, Array, Identifier, Map};
// Create a metadata record for a function.
fn make_metadata(
dict: &BTreeSet<ImmutableString>,
namespace: Option<ImmutableString>,
dict: &BTreeSet<Identifier>,
namespace: Option<Identifier>,
f: &ScriptFnDef,
) -> Map {
let mut map = Map::new();
if let Some(ns) = namespace {
map.insert(dict.get("namespace").unwrap().clone(), ns.into());
map.insert(dict.get("namespace").unwrap().clone().into(), ns.into());
}
map.insert(dict.get("name").unwrap().clone(), f.name.clone().into());
map.insert(
dict.get("access").unwrap().clone(),
dict.get("name").unwrap().clone().into(),
f.name.clone().into(),
);
map.insert(
dict.get("access").unwrap().clone().into(),
match f.access {
FnAccess::Public => dict.get("public").unwrap().clone(),
FnAccess::Private => dict.get("private").unwrap().clone(),
@ -57,11 +60,11 @@ fn collect_fn_metadata(ctx: NativeCallContext) -> crate::Array {
.into(),
);
map.insert(
dict.get("is_anonymous").unwrap().clone(),
dict.get("is_anonymous").unwrap().clone().into(),
f.name.starts_with(crate::engine::FN_ANONYMOUS).into(),
);
map.insert(
dict.get("params").unwrap().clone(),
dict.get("params").unwrap().clone().into(),
f.params
.iter()
.cloned()
@ -74,7 +77,7 @@ fn collect_fn_metadata(ctx: NativeCallContext) -> crate::Array {
}
// Intern strings
let dict: BTreeSet<ImmutableString> = [
let dict: BTreeSet<Identifier> = [
"namespace",
"name",
"access",
@ -98,8 +101,8 @@ fn collect_fn_metadata(ctx: NativeCallContext) -> crate::Array {
// Recursively scan modules for script-defined functions.
fn scan_module(
list: &mut Array,
dict: &BTreeSet<ImmutableString>,
namespace: ImmutableString,
dict: &BTreeSet<Identifier>,
namespace: Identifier,
module: &Module,
) {
module.iter_script_fn().for_each(|(_, _, _, _, f)| {

4
src/packages/map_basic.rs
View File

@ -15,7 +15,7 @@ def_package!(crate:BasicMapPackage:"Basic object map utilities.", lib, {
mod map_functions {
#[rhai_fn(name = "has", pure)]
pub fn contains(map: &mut Map, prop: ImmutableString) -> bool {
map.contains_key(&prop)
map.contains_key(prop.as_str())
}
#[rhai_fn(pure)]
pub fn len(map: &mut Map) -> INT {
@ -25,7 +25,7 @@ mod map_functions {
map.clear();
}
pub fn remove(map: &mut Map, name: ImmutableString) -> Dynamic {
map.remove(&name).unwrap_or_else(|| ().into())
map.remove(name.as_str()).unwrap_or_else(|| ().into())
}
#[rhai_fn(name = "mixin", name = "+=")]
pub fn mixin(map: &mut Map, map2: Map) {

8
src/packages/math_basic.rs
View File

@ -9,13 +9,13 @@ use crate::FLOAT;
#[cfg(not(feature = "no_float"))]
use crate::result::EvalAltResult;
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::float::Float;
#[cfg(not(feature = "no_float"))]
use crate::stdlib::format;
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::Float;
#[cfg(feature = "decimal")]
use rust_decimal::Decimal;

10
src/packages/string_basic.rs
View File

@ -55,11 +55,12 @@ mod print_debug_functions {
#[cfg(not(feature = "no_float"))]
pub mod float_functions {
#[cfg(feature = "no_std")]
#[cfg(not(feature = "no_float"))]
use num_traits::Float;
#[rhai_fn(name = "print", name = "to_string")]
pub fn print_f64(number: f64) -> ImmutableString {
#[cfg(feature = "no_std")]
use num_traits::Float;
let abs = number.abs();
if abs > 10000000000000.0 || abs < 0.0000000000001 {
format!("{:e}", number).into()
@ -69,9 +70,6 @@ mod print_debug_functions {
}
#[rhai_fn(name = "print", name = "to_string")]
pub fn print_f32(number: f32) -> ImmutableString {
#[cfg(feature = "no_std")]
use num_traits::Float;
let abs = number.abs();
if abs > 10000000000000.0 || abs < 0.0000000000001 {
format!("{:e}", number).into()

197
src/parser.rs
View File

@ -22,9 +22,9 @@ use crate::stdlib::{
};
use crate::syntax::{CustomSyntax, MARKER_BLOCK, MARKER_EXPR, MARKER_IDENT};
use crate::token::{is_keyword_function, is_valid_identifier, Token, TokenStream};
use crate::utils::{get_hasher, StringInterner};
use crate::utils::{get_hasher, IdentifierBuilder};
use crate::{
calc_fn_hash, Dynamic, Engine, ImmutableString, LexError, ParseError, ParseErrorType, Position,
calc_fn_hash, Dynamic, Engine, Identifier, LexError, ParseError, ParseErrorType, Position,
Scope, Shared, StaticVec, AST,
};
@ -44,14 +44,14 @@ struct ParseState<'e> {
/// Reference to the scripting [`Engine`].
engine: &'e Engine,
/// Interned strings.
interned_strings: StringInterner,
interned_strings: IdentifierBuilder,
/// Encapsulates a local stack with variable names to simulate an actual runtime scope.
stack: Vec<(ImmutableString, AccessMode)>,
stack: Vec<(Identifier, AccessMode)>,
/// Size of the local variables stack upon entry of the current block scope.
entry_stack_len: usize,
/// Tracks a list of external variables (variables that are not explicitly declared in the scope).
#[cfg(not(feature = "no_closure"))]
external_vars: BTreeMap<ImmutableString, Position>,
external_vars: BTreeMap<Identifier, Position>,
/// An indicator that disables variable capturing into externals one single time
/// up until the nearest consumed Identifier token.
/// If set to false the next call to `access_var` will not capture the variable.
@ -60,7 +60,7 @@ struct ParseState<'e> {
allow_capture: bool,
/// Encapsulates a local stack with imported [module][crate::Module] names.
#[cfg(not(feature = "no_module"))]
modules: StaticVec<ImmutableString>,
modules: StaticVec<Identifier>,
/// Maximum levels of expression nesting.
#[cfg(not(feature = "unchecked"))]
max_expr_depth: Option<NonZeroUsize>,
@ -166,10 +166,7 @@ impl<'e> ParseState<'e> {
/// Get an interned string, creating one if it is not yet interned.
#[inline(always)]
pub fn get_interned_string(
&mut self,
text: impl AsRef<str> + Into<ImmutableString>,
) -> ImmutableString {
pub fn get_identifier(&mut self, text: impl AsRef<str> + Into<Identifier>) -> Identifier {
self.interned_strings.get(text)
}
}
@ -231,9 +228,9 @@ impl Expr {
match self {
Self::Variable(x) if x.1.is_none() => {
let ident = x.2;
let getter = state.get_interned_string(crate::engine::make_getter(&ident.name));
let getter = state.get_identifier(crate::engine::make_getter(&ident.name));
let hash_get = calc_fn_hash(empty(), &getter, 1);
let setter = state.get_interned_string(crate::engine::make_setter(&ident.name));
let setter = state.get_identifier(crate::engine::make_setter(&ident.name));
let hash_set = calc_fn_hash(empty(), &setter, 2);
Self::Property(Box::new((
@ -310,7 +307,7 @@ fn parse_fn_call(
input: &mut TokenStream,
state: &mut ParseState,
lib: &mut FunctionsLib,
id: ImmutableString,
id: Identifier,
capture: bool,
mut namespace: Option<NamespaceRef>,
settings: ParseSettings,
@ -346,16 +343,18 @@ fn parse_fn_call(
calc_fn_hash(empty(), &id, 0)
};
let hash = if is_valid_identifier(id.chars()) {
FnCallHash::from_script(hash)
} else {
FnCallHash::from_native(hash)
};
return Ok(Expr::FnCall(
Box::new(FnCallExpr {
name: id.to_string().into(),
name: state.get_identifier(id),
capture,
namespace,
hash: if is_valid_identifier(id.chars()) {
FnCallHash::from_script(hash)
} else {
FnCallHash::from_native(hash)
},
hash,
args,
..Default::default()
}),
@ -389,16 +388,18 @@ fn parse_fn_call(
calc_fn_hash(empty(), &id, args.len())
};
let hash = if is_valid_identifier(id.chars()) {
FnCallHash::from_script(hash)
} else {
FnCallHash::from_native(hash)
};
return Ok(Expr::FnCall(
Box::new(FnCallExpr {
name: id.to_string().into(),
name: state.get_identifier(id),
capture,
namespace,
hash: if is_valid_identifier(id.chars()) {
FnCallHash::from_script(hash)
} else {
FnCallHash::from_native(hash)
},
hash,
args,
..Default::default()
}),
@ -664,6 +665,8 @@ fn parse_array_literal(
};
}
arr.shrink_to_fit();
Ok(Expr::Array(Box::new(arr), settings.pos))
}
@ -682,7 +685,7 @@ fn parse_map_literal(
settings.pos = eat_token(input, Token::MapStart);
let mut map: StaticVec<(Ident, Expr)> = Default::default();
let mut template: BTreeMap<ImmutableString, Dynamic> = Default::default();
let mut template: BTreeMap<Identifier, Dynamic> = Default::default();
loop {
const MISSING_RBRACE: &str = "to end this object map literal";
@ -703,7 +706,7 @@ fn parse_map_literal(
let (name, pos) = match input.next().unwrap() {
(Token::Identifier(s), pos) | (Token::StringConstant(s), pos) => {
if map.iter().any(|(p, _)| p.name == &s) {
if map.iter().any(|(p, _)| p.name == s) {
return Err(PERR::DuplicatedProperty(s).into_err(pos));
}
(s, pos)
@ -752,8 +755,8 @@ fn parse_map_literal(
}
let expr = parse_expr(input, state, lib, settings.level_up())?;
let name = state.get_interned_string(name);
template.insert(name.clone(), Default::default());
let name = state.get_identifier(name);
template.insert(name.clone().into(), Default::default());
map.push((Ident { name, pos }, expr));
match input.peek().unwrap() {
@ -778,6 +781,8 @@ fn parse_map_literal(
}
}
map.shrink_to_fit();
Ok(Expr::Map(Box::new((map, template)), settings.pos))
}
@ -931,7 +936,7 @@ fn parse_primary(
Token::IntegerConstant(x) => Expr::IntegerConstant(x, settings.pos),
Token::CharConstant(c) => Expr::CharConstant(c, settings.pos),
Token::StringConstant(s) => {
Expr::StringConstant(state.get_interned_string(s), settings.pos)
Expr::StringConstant(state.get_identifier(s).into(), settings.pos)
}
Token::True => Expr::BoolConstant(true, settings.pos),
Token::False => Expr::BoolConstant(false, settings.pos),
@ -1029,7 +1034,7 @@ fn parse_primary(
state.allow_capture = true;
}
let var_name_def = Ident {
name: state.get_interned_string(s),
name: state.get_identifier(s),
pos: settings.pos,
};
Expr::Variable(Box::new((None, None, var_name_def)))
@ -1043,7 +1048,7 @@ fn parse_primary(
state.allow_capture = true;
}
let var_name_def = Ident {
name: state.get_interned_string(s),
name: state.get_identifier(s),
pos: settings.pos,
};
Expr::Variable(Box::new((None, None, var_name_def)))
@ -1052,7 +1057,7 @@ fn parse_primary(
_ => {
let index = state.access_var(&s, settings.pos);
let var_name_def = Ident {
name: state.get_interned_string(s),
name: state.get_identifier(s),
pos: settings.pos,
};
Expr::Variable(Box::new((index, None, var_name_def)))
@ -1071,7 +1076,7 @@ fn parse_primary(
// Function call is allowed to have reserved keyword
Token::LeftParen | Token::Bang if is_keyword_function(&s) => {
let var_name_def = Ident {
name: state.get_interned_string(s),
name: state.get_identifier(s),
pos: settings.pos,
};
Expr::Variable(Box::new((None, None, var_name_def)))
@ -1079,7 +1084,7 @@ fn parse_primary(
// Access to `this` as a variable is OK within a function scope
_ if s == KEYWORD_THIS && settings.is_function_scope => {
let var_name_def = Ident {
name: state.get_interned_string(s),
name: state.get_identifier(s),
pos: settings.pos,
};
Expr::Variable(Box::new((None, None, var_name_def)))
@ -1172,7 +1177,7 @@ fn parse_primary(
}
let var_name_def = Ident {
name: state.get_interned_string(id2),
name: state.get_identifier(id2),
pos: pos2,
};
Expr::Variable(Box::new((index, namespace, var_name_def)))
@ -1276,14 +1281,13 @@ fn parse_unary(
// Call negative function
expr => {
let op = "-";
let mut args = StaticVec::new();
args.push(expr);
Ok(Expr::FnCall(
Box::new(FnCallExpr {
name: op.into(),
hash: FnCallHash::from_native(calc_fn_hash(empty(), op, 1)),
name: state.get_identifier("-"),
hash: FnCallHash::from_native(calc_fn_hash(empty(), "-", 1)),
args,
..Default::default()
}),
@ -1303,14 +1307,13 @@ fn parse_unary(
// Call plus function
expr => {
let op = "+";
let mut args = StaticVec::new();
args.push(expr);
Ok(Expr::FnCall(
Box::new(FnCallExpr {
name: op.into(),
hash: FnCallHash::from_native(calc_fn_hash(empty(), op, 1)),
name: state.get_identifier("+"),
hash: FnCallHash::from_native(calc_fn_hash(empty(), "+", 1)),
args,
..Default::default()
}),
@ -1326,12 +1329,10 @@ fn parse_unary(
let expr = parse_unary(input, state, lib, settings.level_up())?;
args.push(expr);
let op = "!";
Ok(Expr::FnCall(
Box::new(FnCallExpr {
name: op.into(),
hash: FnCallHash::from_native(calc_fn_hash(empty(), op, 1)),
name: state.get_identifier("!"),
hash: FnCallHash::from_native(calc_fn_hash(empty(), "!", 1)),
args,
..Default::default()
}),
@ -1499,9 +1500,9 @@ fn make_dot_expr(
// lhs.id
(lhs, Expr::Variable(x)) if x.1.is_none() => {
let ident = x.2;
let getter = state.get_interned_string(crate::engine::make_getter(&ident.name));
let getter = state.get_identifier(crate::engine::make_getter(&ident.name));
let hash_get = calc_fn_hash(empty(), &getter, 1);
let setter = state.get_interned_string(crate::engine::make_setter(&ident.name));
let setter = state.get_identifier(crate::engine::make_setter(&ident.name));
let hash_set = calc_fn_hash(empty(), &setter, 2);
let rhs = Expr::Property(Box::new(((getter, hash_get), (setter, hash_set), ident)));
@ -1531,8 +1532,8 @@ fn make_dot_expr(
Expr::FnCall(mut func, func_pos) => {
// Recalculate hash
func.hash = FnCallHash::from_script_and_native(
calc_fn_hash(empty(), &func.name, func.args.len()),
calc_fn_hash(empty(), &func.name, func.args.len() + 1),
calc_fn_hash(empty(), &func.name, func.num_args()),
calc_fn_hash(empty(), &func.name, func.num_args() + 1),
);
let rhs = Expr::Dot(
@ -1563,7 +1564,7 @@ fn make_dot_expr(
}
// lhs.Fn() or lhs.eval()
(_, Expr::FnCall(x, pos))
if x.args.len() == 0
if x.is_args_empty()
&& [crate::engine::KEYWORD_FN_PTR, crate::engine::KEYWORD_EVAL]
.contains(&x.name.as_ref()) =>
{
@ -1587,8 +1588,8 @@ fn make_dot_expr(
(lhs, Expr::FnCall(mut func, func_pos)) => {
// Recalculate hash
func.hash = FnCallHash::from_script_and_native(
calc_fn_hash(empty(), &func.name, func.args.len()),
calc_fn_hash(empty(), &func.name, func.args.len() + 1),
calc_fn_hash(empty(), &func.name, func.num_args()),
calc_fn_hash(empty(), &func.name, func.num_args() + 1),
);
let rhs = Expr::FnCall(func, func_pos);
Expr::Dot(Box::new(BinaryExpr { lhs, rhs }), op_pos)
@ -1673,7 +1674,7 @@ fn parse_binary_op(
let hash = calc_fn_hash(empty(), &op, 2);
let op_base = FnCallExpr {
name: op,
name: state.get_identifier(op.as_ref()),
hash: FnCallHash::from_native(hash),
capture: false,
..Default::default()
@ -1741,7 +1742,7 @@ fn parse_binary_op(
Box::new(FnCallExpr {
hash: FnCallHash::from_script(hash),
args,
name: OP_CONTAINS.into(),
name: state.get_identifier(OP_CONTAINS),
..op_base
}),
pos,
@ -1829,9 +1830,9 @@ fn parse_custom_syntax(
match required_token.as_str() {
MARKER_IDENT => match input.next().unwrap() {
(Token::Identifier(s), pos) => {
let name = state.get_interned_string(s);
segments.push(name.clone());
tokens.push(state.get_interned_string(MARKER_IDENT));
let name = state.get_identifier(s);
segments.push(name.clone().into());
tokens.push(state.get_identifier(MARKER_IDENT));
let var_name_def = Ident { name, pos };
keywords.push(Expr::Variable(Box::new((None, None, var_name_def))));
}
@ -1842,15 +1843,15 @@ fn parse_custom_syntax(
},
MARKER_EXPR => {
keywords.push(parse_expr(input, state, lib, settings)?);
let keyword = state.get_interned_string(MARKER_EXPR);
segments.push(keyword.clone());
let keyword = state.get_identifier(MARKER_EXPR);
segments.push(keyword.clone().into());
tokens.push(keyword);
}
MARKER_BLOCK => match parse_block(input, state, lib, settings)? {
block @ Stmt::Block(_, _) => {
keywords.push(Expr::Stmt(Box::new(block.into())));
let keyword = state.get_interned_string(MARKER_BLOCK);
segments.push(keyword.clone());
let keyword = state.get_identifier(MARKER_BLOCK);
segments.push(keyword.clone().into());
tokens.push(keyword);
}
stmt => unreachable!("expecting Stmt::Block, but gets {:?}", stmt),
@ -1859,7 +1860,7 @@ fn parse_custom_syntax(
(Token::LexError(err), pos) => return Err(err.into_err(pos)),
(t, _) if t.syntax().as_ref() == s => {
segments.push(required_token.clone());
tokens.push(required_token.clone());
tokens.push(required_token.clone().into());
}
(_, pos) => {
return Err(PERR::MissingToken(
@ -1901,7 +1902,7 @@ fn parse_expr(
match token {
Token::Custom(key) | Token::Reserved(key) | Token::Identifier(key) => {
match state.engine.custom_syntax.get_key_value(key) {
match state.engine.custom_syntax.get_key_value(key.as_str()) {
Some((key, syntax)) => {
input.next().unwrap();
return parse_custom_syntax(
@ -2119,16 +2120,20 @@ fn parse_for(
ensure_not_statement_expr(input, "a boolean")?;
let expr = parse_expr(input, state, lib, settings.level_up())?;
let loop_var = state.get_interned_string(name.clone());
let loop_var = state.get_identifier(name);
let prev_stack_len = state.stack.len();
state.stack.push((loop_var, AccessMode::ReadWrite));
state.stack.push((loop_var.clone(), AccessMode::ReadWrite));
settings.is_breakable = true;
let body = parse_block(input, state, lib, settings.level_up())?;
state.stack.truncate(prev_stack_len);
Ok(Stmt::For(expr, Box::new((name, body.into())), settings.pos))
Ok(Stmt::For(
expr,
Box::new((loop_var, body.into())),
settings.pos,
))
}
/// Parse a variable definition statement.
@ -2156,7 +2161,7 @@ fn parse_let(
(_, pos) => return Err(PERR::VariableExpected.into_err(pos)),
};
let name = state.get_interned_string(name);
let name = state.get_identifier(name);
let var_def = Ident {
name: name.clone(),
pos,
@ -2174,9 +2179,9 @@ fn parse_let(
match var_type {
// let name = expr
AccessMode::ReadWrite => Ok(Stmt::Let(expr, var_def, export, settings.pos)),
AccessMode::ReadWrite => Ok(Stmt::Let(expr, var_def.into(), export, settings.pos)),
// const name = { expr:constant }
AccessMode::ReadOnly => Ok(Stmt::Const(expr, var_def, export, settings.pos)),
AccessMode::ReadOnly => Ok(Stmt::Const(expr, var_def.into(), export, settings.pos)),
}
}
@ -2212,15 +2217,18 @@ fn parse_import(
(_, pos) => return Err(PERR::VariableExpected.into_err(pos)),
};
let name = state.get_interned_string(name);
let name = state.get_identifier(name);
state.modules.push(name.clone());
Ok(Stmt::Import(
expr,
Some(Ident {
name,
pos: name_pos,
}),
Some(
Ident {
name,
pos: name_pos,
}
.into(),
),
settings.pos,
))
}
@ -2269,7 +2277,7 @@ fn parse_export(
let rename = if match_token(input, Token::As).0 {
match input.next().unwrap() {
(Token::Identifier(s), pos) => Some(Ident {
name: state.get_interned_string(s),
name: state.get_identifier(s),
pos,
}),
(Token::Reserved(s), pos) if is_valid_identifier(s.chars()) => {
@ -2284,7 +2292,7 @@ fn parse_export(
exports.push((
Ident {
name: state.get_interned_string(id),
name: state.get_identifier(id),
pos: id_pos,
},
rename,
@ -2511,7 +2519,7 @@ fn parse_stmt(
if lib.contains_key(&hash) {
return Err(PERR::FnDuplicatedDefinition(
func.name.into_owned(),
func.name.to_string(),
func.params.len(),
)
.into_err(pos));
@ -2622,7 +2630,7 @@ fn parse_try_catch(
let var_def = if match_token(input, Token::LeftParen).0 {
let id = match input.next().unwrap() {
(Token::Identifier(s), pos) => Ident {
name: state.get_interned_string(s),
name: state.get_identifier(s),
pos,
},
(_, pos) => return Err(PERR::VariableExpected.into_err(pos)),
@ -2692,7 +2700,7 @@ fn parse_fn(
if params.iter().any(|(p, _)| p == &s) {
return Err(PERR::FnDuplicatedParam(name, s).into_err(pos));
}
let s = state.get_interned_string(s);
let s = state.get_identifier(s);
state.stack.push((s.clone(), AccessMode::ReadWrite));
params.push((s, pos))
}
@ -2739,7 +2747,7 @@ fn parse_fn(
.collect();
Ok(ScriptFnDef {
name: name.into(),
name: state.get_identifier(&name),
access,
params,
#[cfg(not(feature = "no_closure"))]
@ -2755,7 +2763,12 @@ fn parse_fn(
/// Creates a curried expression from a list of external variables
#[cfg(not(feature = "no_function"))]
#[cfg(not(feature = "no_closure"))]
fn make_curry_from_externals(fn_expr: Expr, externals: StaticVec<Ident>, pos: Position) -> Expr {
fn make_curry_from_externals(
state: &mut ParseState,
fn_expr: Expr,
externals: StaticVec<Ident>,
pos: Position,
) -> Expr {
// If there are no captured variables, no need to curry
if externals.is_empty() {
return fn_expr;
@ -2770,12 +2783,14 @@ fn make_curry_from_externals(fn_expr: Expr, externals: StaticVec<Ident>, pos: Po
args.push(Expr::Variable(Box::new((None, None, x.clone()))));
});
let curry_func = crate::engine::KEYWORD_FN_PTR_CURRY;
let expr = Expr::FnCall(
Box::new(FnCallExpr {
name: curry_func.into(),
hash: FnCallHash::from_native(calc_fn_hash(empty(), curry_func, num_externals + 1)),
name: state.get_identifier(crate::engine::KEYWORD_FN_PTR_CURRY),
hash: FnCallHash::from_native(calc_fn_hash(
empty(),
crate::engine::KEYWORD_FN_PTR_CURRY,
num_externals + 1,
)),
args,
..Default::default()
}),
@ -2785,7 +2800,7 @@ fn make_curry_from_externals(fn_expr: Expr, externals: StaticVec<Ident>, pos: Po
// Convert the entire expression into a statement block, then insert the relevant
// [`Share`][Stmt::Share] statements.
let mut statements: StaticVec<_> = Default::default();
statements.extend(externals.into_iter().map(Stmt::Share));
statements.extend(externals.into_iter().map(|v| Stmt::Share(Box::new(v))));
statements.push(Stmt::Expr(expr));
Expr::Stmt(Box::new(StmtBlock { statements, pos }))
}
@ -2812,7 +2827,7 @@ fn parse_anon_fn(
if params.iter().any(|(p, _)| p == &s) {
return Err(PERR::FnDuplicatedParam("".to_string(), s).into_err(pos));
}
let s = state.get_interned_string(s);
let s = state.get_identifier(s);
state.stack.push((s.clone(), AccessMode::ReadWrite));
params.push((s, pos))
}
@ -2880,7 +2895,7 @@ fn parse_anon_fn(
body.hash(hasher);
let hash = hasher.finish();
let fn_name: ImmutableString = format!("{}{:016x}", crate::engine::FN_ANONYMOUS, hash).into();
let fn_name = state.get_identifier(&(format!("{}{:016x}", crate::engine::FN_ANONYMOUS, hash)));
// Define the function
let script = ScriptFnDef {
@ -2896,10 +2911,10 @@ fn parse_anon_fn(
comments: Default::default(),
};
let expr = Expr::FnPointer(fn_name, settings.pos);
let expr = Expr::FnPointer(fn_name.into(), settings.pos);
#[cfg(not(feature = "no_closure"))]
let expr = make_curry_from_externals(expr, externals, settings.pos);
let expr = make_curry_from_externals(state, expr, externals, settings.pos);
Ok((expr, script))
}

8
src/scope.rs
View File

@ -2,7 +2,7 @@
use crate::dynamic::{AccessMode, Variant};
use crate::stdlib::{borrow::Cow, boxed::Box, iter, vec::Vec};
use crate::{Dynamic, ImmutableString, StaticVec};
use crate::{Dynamic, Identifier, StaticVec};
/// Keep a number of entries inline (since [`Dynamic`] is usually small enough).
const SCOPE_SIZE: usize = 16;
@ -54,7 +54,7 @@ pub struct Scope<'a> {
/// Current value of the entry.
values: smallvec::SmallVec<[Dynamic; SCOPE_SIZE]>,
/// (Name, aliases) of the entry.
names: Vec<(Cow<'a, str>, Option<Box<StaticVec<ImmutableString>>>)>,
names: Vec<(Cow<'a, str>, Option<Box<StaticVec<Identifier>>>)>,
}
impl Default for Scope<'_> {
@ -414,7 +414,7 @@ impl<'a> Scope<'a> {
pub(crate) fn add_entry_alias(
&mut self,
index: usize,
alias: impl Into<ImmutableString> + PartialEq<ImmutableString>,
alias: impl Into<Identifier> + PartialEq<Identifier>,
) -> &mut Self {
let entry = self.names.get_mut(index).expect("invalid index in Scope");
if entry.1.is_none() {
@ -449,7 +449,7 @@ impl<'a> Scope<'a> {
#[allow(dead_code)]
pub(crate) fn into_iter(
self,
) -> impl Iterator<Item = (Cow<'a, str>, Dynamic, Vec<ImmutableString>)> {
) -> impl Iterator<Item = (Cow<'a, str>, Dynamic, Vec<Identifier>)> {
self.names
.into_iter()
.zip(self.values.into_iter())

19
src/serde/de.rs
View File

@ -1,6 +1,6 @@
//! Implement deserialization support of [`Dynamic`][crate::Dynamic] for [`serde`].
use super::str::ImmutableStringDeserializer;
use super::str::StringSliceDeserializer;
use crate::dynamic::Union;
use crate::stdlib::{any::type_name, boxed::Box, fmt, string::ToString};
use crate::{Dynamic, EvalAltResult, ImmutableString, LexError, Position};
@ -418,7 +418,12 @@ impl<'de> Deserializer<'de> for &mut DynamicDeserializer<'de> {
#[cfg(not(feature = "no_object"))]
return self.value.downcast_ref::<Map>().map_or_else(
|| self.type_error(),
|map| _visitor.visit_map(IterateMap::new(map.keys(), map.values())),
|map| {
_visitor.visit_map(IterateMap::new(
map.keys().map(|key| key.as_str()),
map.values(),
))
},
);
#[cfg(feature = "no_object")]
@ -512,7 +517,7 @@ impl<'a: 'de, 'de, ITER: Iterator<Item = &'a Dynamic>> SeqAccess<'de> for Iterat
/// `MapAccess` implementation for maps.
struct IterateMap<'a, KEYS, VALUES>
where
KEYS: Iterator<Item = &'a ImmutableString>,
KEYS: Iterator<Item = &'a str>,
VALUES: Iterator<Item = &'a Dynamic>,
{
// Iterator for a stream of [`Dynamic`][crate::Dynamic] keys.
@ -524,7 +529,7 @@ where
#[cfg(not(feature = "no_object"))]
impl<'a, KEYS, VALUES> IterateMap<'a, KEYS, VALUES>
where
KEYS: Iterator<Item = &'a ImmutableString>,
KEYS: Iterator<Item = &'a str>,
VALUES: Iterator<Item = &'a Dynamic>,
{
pub fn new(keys: KEYS, values: VALUES) -> Self {
@ -534,7 +539,7 @@ where
impl<'a: 'de, 'de, KEYS, VALUES> MapAccess<'de> for IterateMap<'a, KEYS, VALUES>
where
KEYS: Iterator<Item = &'a ImmutableString>,
KEYS: Iterator<Item = &'a str>,
VALUES: Iterator<Item = &'a Dynamic>,
{
type Error = Box<EvalAltResult>;
@ -543,11 +548,11 @@ where
&mut self,
seed: K,
) -> Result<Option<K::Value>, Box<EvalAltResult>> {
// Deserialize each `ImmutableString` key coming out of the keys iterator.
// Deserialize each `Identifier` key coming out of the keys iterator.
match self.keys.next() {
None => Ok(None),
Some(item) => seed
.deserialize(&mut ImmutableStringDeserializer::from_str(item))
.deserialize(&mut StringSliceDeserializer::from_str(item))
.map(Some),
}
}

4
src/serde/deserialize.rs
View File

@ -143,8 +143,8 @@ impl<'d> Visitor<'d> for DynamicVisitor {
fn visit_map<M: MapAccess<'d>>(self, mut map: M) -> Result<Self::Value, M::Error> {
let mut m: Map = Default::default();
while let Some((k, v)) = map.next_entry()? {
m.insert(k, v);
while let Some((k, v)) = map.next_entry::<&str, _>()? {
m.insert(k.into(), v);
}
Ok(m.into())

4
src/serde/ser.rs
View File

@ -550,7 +550,7 @@ impl SerializeMap for DynamicSerializer {
})?;
let _value = _value.serialize(&mut *self)?;
let map = self._value.downcast_mut::<Map>().unwrap();
map.insert(key, _value);
map.insert(key.into(), _value);
Ok(())
}
#[cfg(feature = "no_object")]
@ -575,7 +575,7 @@ impl SerializeMap for DynamicSerializer {
})?;
let _value = _value.serialize(&mut *self)?;
let map = self._value.downcast_mut::<Map>().unwrap();
map.insert(_key, _value);
map.insert(_key.into(), _value);
Ok(())
}
#[cfg(feature = "no_object")]

2
src/serde/serialize.rs
View File

@ -54,7 +54,7 @@ impl Serialize for Dynamic {
Union::Map(m, _) => {
let mut map = ser.serialize_map(Some(m.len()))?;
for (k, v) in m.iter() {
map.serialize_entry(k, v)?;
map.serialize_entry(k.as_str(), v)?;
}
map.end()
}

16
src/serde/str.rs
View File

@ -1,17 +1,17 @@
//! Implement deserialization support of [`ImmutableString`][crate::ImmutableString] for [`serde`].
use crate::stdlib::{any::type_name, boxed::Box};
use crate::{EvalAltResult, ImmutableString, Position};
use crate::{EvalAltResult, Position};
use serde::de::{Deserializer, Visitor};
/// Deserializer for `ImmutableString`.
pub struct ImmutableStringDeserializer<'a> {
value: &'a ImmutableString,
pub struct StringSliceDeserializer<'a> {
value: &'a str,
}
impl<'a> ImmutableStringDeserializer<'a> {
/// Create an `ImmutableStringDeserializer` from an `ImmutableString` reference.
pub fn from_str(value: &'a ImmutableString) -> Self {
impl<'a> StringSliceDeserializer<'a> {
/// Create an `ImmutableStringDeserializer` from an `&str` reference.
pub fn from_str(value: &'a str) -> Self {
Self { value }
}
/// Shortcut for a type conversion error.
@ -25,7 +25,7 @@ impl<'a> ImmutableStringDeserializer<'a> {
}
}
impl<'de> Deserializer<'de> for &mut ImmutableStringDeserializer<'de> {
impl<'de> Deserializer<'de> for &mut StringSliceDeserializer<'de> {
type Error = Box<EvalAltResult>;
fn deserialize_any<V: Visitor<'de>>(self, v: V) -> Result<V::Value, Box<EvalAltResult>> {
@ -69,7 +69,7 @@ impl<'de> Deserializer<'de> for &mut ImmutableStringDeserializer<'de> {
}
fn deserialize_str<V: Visitor<'de>>(self, v: V) -> Result<V::Value, Box<EvalAltResult>> {
// Only allow deserialization into a string.
v.visit_borrowed_str(self.value.as_str())
v.visit_borrowed_str(self.value)
}
fn deserialize_string<V: Visitor<'de>>(
self,

7
src/syntax.rs
View File

@ -6,7 +6,8 @@ use crate::fn_native::SendSync;
use crate::stdlib::{boxed::Box, format, string::ToString};
use crate::token::{is_valid_identifier, Token};
use crate::{
Engine, ImmutableString, LexError, ParseError, Position, RhaiResult, Shared, StaticVec,
Engine, Identifier, ImmutableString, LexError, ParseError, Position, RhaiResult, Shared,
StaticVec,
};
pub const MARKER_EXPR: &str = "$expr$";
@ -103,7 +104,7 @@ impl Engine {
///
/// If `new_vars` is negative, then it is expected that only the top `new_vars` variables in the
/// current [`Scope`][crate::Scope] will be _popped_. Do not randomly remove variables.
pub fn register_custom_syntax<S: AsRef<str> + Into<ImmutableString>>(
pub fn register_custom_syntax<S: AsRef<str> + Into<Identifier>>(
&mut self,
keywords: &[S],
new_vars: isize,
@ -221,7 +222,7 @@ impl Engine {
/// Otherwise, custom keywords won't be recognized.
pub fn register_custom_syntax_raw(
&mut self,
key: impl Into<ImmutableString>,
key: impl Into<Identifier>,
parse: impl Fn(&[ImmutableString], &str) -> Result<Option<ImmutableString>, ParseError>
+ SendSync
+ 'static,

95
src/token.rs
View File

@ -842,7 +842,7 @@ pub trait InputStream {
fn peek_next(&mut self) -> Option<char>;
}
/// _(INTERNALS)_ Parse a string literal wrapped by `enclosing_char`.
/// _(INTERNALS)_ Parse a string literal ended by `termination_char`.
/// Exported under the `internals` feature only.
///
/// # Volatile API
@ -852,12 +852,15 @@ pub fn parse_string_literal(
stream: &mut impl InputStream,
state: &mut TokenizeState,
pos: &mut Position,
enclosing_char: char,
termination_char: char,
continuation: bool,
verbatim: bool,
) -> Result<String, (LexError, Position)> {
let mut result: smallvec::SmallVec<[char; 16]> = Default::default();
let mut escape: smallvec::SmallVec<[char; 12]> = Default::default();
let start = *pos;
let mut skip_whitespace_until = 0;
loop {
let next_char = stream.get_next().ok_or((LERR::UnterminatedString, start))?;
@ -871,8 +874,10 @@ pub fn parse_string_literal(
}
match next_char {
// \r - ignore if followed by \n
'\r' if stream.peek_next().unwrap_or('\0') == '\n' => {}
// \...
'\\' if escape.is_empty() => {
'\\' if escape.is_empty() && !verbatim => {
escape.push('\\');
}
// \\
@ -937,18 +942,37 @@ pub fn parse_string_literal(
})?);
}
// \{enclosing_char} - escaped
ch if enclosing_char == ch && !escape.is_empty() => {
// \{termination_char} - escaped
_ if termination_char == next_char && !escape.is_empty() => {
escape.clear();
result.push(ch)
result.push(next_char)
}
// Close wrapper
ch if enclosing_char == ch && escape.is_empty() => break,
_ if termination_char == next_char && escape.is_empty() => break,
// Line continuation
'\n' if continuation && !escape.is_empty() => {
escape.clear();
pos.new_line();
skip_whitespace_until = start.position().unwrap() + 1;
}
// New-line cannot be escaped
// Cannot have new-lines inside non-multi-line string literals
'\n' if !escape.is_empty() || !verbatim => {
pos.rewind();
return Err((LERR::UnterminatedString, start));
}
'\n' => {
pos.new_line();
result.push(next_char);
}
// Unknown escape sequence
ch if !escape.is_empty() => {
escape.push(ch);
_ if !escape.is_empty() => {
escape.push(next_char);
return Err((
LERR::MalformedEscapeSequence(escape.into_iter().collect()),
@ -956,16 +980,14 @@ pub fn parse_string_literal(
));
}
// Cannot have new-lines inside string literals
'\n' => {
pos.rewind();
return Err((LERR::UnterminatedString, start));
}
// Whitespace to skip
_ if next_char.is_whitespace() && pos.position().unwrap() < skip_whitespace_until => {}
// All other characters
ch => {
_ => {
escape.clear();
result.push(ch);
result.push(next_char);
skip_whitespace_until = 0;
}
}
}
@ -1272,12 +1294,15 @@ fn get_next_token_inner(
return get_identifier(stream, pos, start_pos, c);
}
// " - string literal
('"', _) => {
return parse_string_literal(stream, state, pos, '"').map_or_else(
|err| Some((Token::LexError(err.0), err.1)),
|out| Some((Token::StringConstant(out), start_pos)),
)
// " or ` - string literal
('"', _) | ('`', _) => {
let multi_line = c == '`';
return parse_string_literal(stream, state, pos, c, !multi_line, multi_line)
.map_or_else(
|err| Some((Token::LexError(err.0), err.1)),
|out| Some((Token::StringConstant(out), start_pos)),
);
}
// ' - character literal
@ -1288,19 +1313,21 @@ fn get_next_token_inner(
))
}
('\'', _) => {
return Some(parse_string_literal(stream, state, pos, '\'').map_or_else(
|err| (Token::LexError(err.0), err.1),
|result| {
let mut chars = result.chars();
let first = chars.next().unwrap();
return Some(
parse_string_literal(stream, state, pos, c, false, false).map_or_else(
|err| (Token::LexError(err.0), err.1),
|result| {
let mut chars = result.chars();
let first = chars.next().unwrap();
if chars.next().is_some() {
(Token::LexError(LERR::MalformedChar(result)), start_pos)
} else {
(Token::CharConstant(first), start_pos)
}
},
))
if chars.next().is_some() {
(Token::LexError(LERR::MalformedChar(result)), start_pos)
} else {
(Token::CharConstant(first), start_pos)
}
},
),
)
}
// Braces

48
src/utils.rs
View File

@ -6,7 +6,6 @@ use crate::stdlib::{
borrow::Borrow,
boxed::Box,
cmp::Ordering,
collections::BTreeSet,
fmt,
fmt::{Debug, Display},
hash::{BuildHasher, Hash, Hasher},
@ -15,7 +14,7 @@ use crate::stdlib::{
str::FromStr,
string::{String, ToString},
};
use crate::Shared;
use crate::{Identifier, Shared};
/// A hasher that only takes one single [`u64`] and returns it as a hash key.
///
@ -590,6 +589,22 @@ impl PartialOrd<ImmutableString> for String {
}
}
#[cfg(not(feature = "no_smartstring"))]
impl From<ImmutableString> for Identifier {
#[inline(always)]
fn from(value: ImmutableString) -> Self {
value.into_owned().into()
}
}
#[cfg(not(feature = "no_smartstring"))]
impl From<Identifier> for ImmutableString {
#[inline(always)]
fn from(value: Identifier) -> Self {
value.to_string().into()
}
}
impl ImmutableString {
/// Consume the [`ImmutableString`] and convert it into a [`String`].
/// If there are other references to the same string, a cloned copy is returned.
@ -606,18 +621,31 @@ impl ImmutableString {
}
}
/// A collection of interned strings.
/// A factory of identifiers from text strings.
///
/// When [`SmartString`](https://crates.io/crates/smartstring) is used as [`Identifier`],
/// this just returns one because most identifiers in Rhai are short and ASCII-based.
///
/// When [`ImmutableString`] is used as [`Identifier`], this type acts as an interner which keeps a
/// collection of strings and returns shared instances, only creating a new string when it is not
/// yet interned.
#[derive(Debug, Clone, Default, Hash)]
pub struct StringInterner(BTreeSet<ImmutableString>);
pub struct IdentifierBuilder(
#[cfg(feature = "no_smartstring")] crate::stdlib::collections::BTreeSet<Identifier>,
);
impl StringInterner {
/// Get an interned string, creating one if it is not yet interned.
impl IdentifierBuilder {
/// Get an identifier from a text string.
#[inline(always)]
pub fn get(&mut self, text: impl AsRef<str> + Into<ImmutableString>) -> ImmutableString {
self.0.get(text.as_ref()).cloned().unwrap_or_else(|| {
let s = text.into();
pub fn get(&mut self, text: impl AsRef<str> + Into<Identifier>) -> Identifier {
#[cfg(not(feature = "no_smartstring"))]
return text.as_ref().into();
#[cfg(feature = "no_smartstring")]
return self.0.get(text.as_ref()).cloned().unwrap_or_else(|| {
let s: Identifier = text.into();
self.0.insert(s.clone());
s
})
});
}
}

2
tests/operations.rs
View File

@ -72,7 +72,7 @@ fn test_max_operations_functions() -> Result<(), Box<EvalAltResult>> {
fn inc(x) { x + 1 }
let x = 0;
while x < 28 {
while x < 31 {
print(x);
x = inc(x);
}

8
tests/string.rs
View File

@ -8,6 +8,14 @@ fn test_string() -> Result<(), Box<EvalAltResult>> {
engine.eval::<String>(r#""Test string: \u2764""#)?,
"Test string: ❤"
);
assert_eq!(
engine.eval::<String>(" \"Test string: \\u2764\\\n hello, world!\"")?,
"Test string: ❤ hello, world!"
);
assert_eq!(
engine.eval::<String>(" `Test string: \\u2764\nhello,\\nworld!`")?,
"Test string: \\u2764\nhello,\\nworld!"
);
assert_eq!(
engine.eval::<String>(r#""Test string: \x58""#)?,
"Test string: X"