kjuulh/rhai
1
0
Fork 0
Code Issues 1 Pull Requests 1 Packages Projects Releases Wiki Activity

Add Engine::disable_doc_comments and smarter doc-comments treatment.

Browse Source
This commit is contained in:
Stephen Chung 2020-12-20 20:05:23 +08:00
parent 22039b24b3
commit f99703f951
12 changed files with 172 additions and 75 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
RELEASES.md
View File

@ -33,12 +33,13 @@ New features
------------
* `AST::iter_functions` now returns `ScriptFnMetadata` which includes, among others, _doc-comments_ for functions prefixed by `///` or `/**`.
* A functions lookup cache is added to make function call resolution faster.
* _Doc-comments_ can be enabled/disabled with the new `Engine::set_doc_comments` method.
* A new feature `metadata` is added that pulls in `serde_json` and enables `Engine::gen_fn_metadata_to_json` which exports the full list of functions metadata (including those inside an `AST`) in JSON format.
Enhancements
------------
* A functions lookup cache is added to make function call resolution faster.
* Capturing a constant variable in a closure is now supported, with no cloning.
* Provides position info for `debug` statements.
* A _look-ahead_ symbol is provided to custom syntax parsers, which can be used to parse variable-length symbol streams.

21
doc/src/SUMMARY.md
View File

@ -55,6 +55,7 @@ The Rhai Scripting Language
2. [Export a Rust Function](plugins/function.md)
5. [Rhai Language Reference](language/index.md)
1. [Comments](language/comments.md)
1. [Doc-Comments](language/doc-comments.md)
2. [Values and Types](language/values-and-types.md)
1. [Dynamic Values](language/dynamic.md)
2. [Serialization/Deserialization with `serde`](rust/serde.md)
@ -79,14 +80,14 @@ The Rhai Scripting Language
9. [If Statement](language/if.md)
10. [Switch Expression](language/switch.md)
11. [While Loop](language/while.md)
11. [Do Loop](language/do.md)
12. [Loop Statement](language/loop.md)
13. [For Loop](language/for.md)
12. [Do Loop](language/do.md)
13. [Loop Statement](language/loop.md)
14. [For Loop](language/for.md)
1. [Iterators for Custom Types](language/iterator.md)
14. [Return Values](language/return.md)
15. [Throw Exception on Error](language/throw.md)
16. [Catch Exceptions](language/try-catch.md)
17. [Functions](language/functions.md)
15. [Return Values](language/return.md)
16. [Throw Exception on Error](language/throw.md)
17. [Catch Exceptions](language/try-catch.md)
18. [Functions](language/functions.md)
1. [Call Method as Function](language/method.md)
2. [Overloading](language/overload.md)
3. [Namespaces](language/fn-namespaces.md)
@ -94,11 +95,11 @@ The Rhai Scripting Language
5. [Currying](language/fn-curry.md)
6. [Anonymous Functions](language/fn-anon.md)
7. [Closures](language/fn-closure.md)
18. [Print and Debug](language/print-debug.md)
19. [Modules](language/modules/index.md)
19. [Print and Debug](language/print-debug.md)
20. [Modules](language/modules/index.md)
1. [Export Variables, Functions and Sub-Modules](language/modules/export.md)
2. [Import Modules](language/modules/import.md)
20. [Eval Function](language/eval.md)
21. [Eval Function](language/eval.md)
6. [Safety and Protection](safety/index.md)
1. [Checked Arithmetic](safety/checked.md)
2. [Sand-Boxing](safety/sandbox.md)

4
doc/src/engine/metadata/gen_fn_sig.md
View File

@ -48,7 +48,7 @@ In this case, the first parameter should be `&mut T` of the custom type and the
> `set$prop(_, _, _)`
### Script-defined functions
### Script-Defined Functions
Script-defined [function] signatures contain parameter names. Since all parameters, as well as
the return value, are [`Dynamic`] the types are simply not shown.
@ -69,7 +69,7 @@ is the same as:
> `foo(x: Dynamic, y: Dynamic, z: Dynamic) -> Result<Dynamic, Box<EvalAltResult>>`
### Plugin functions
### Plugin Functions
Functions defined in [plugin modules] are the best. They contain all the metadata
describing the functions.

21
doc/src/engine/options.md
View File

@ -6,13 +6,14 @@ Engine Configuration Options
A number of other configuration options are available from the `Engine` to fine-tune behavior and safeguards.
| Method | Not available under | Description |
| ------------------------ | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `set_optimization_level` | [`no_optimize`] | sets the amount of script _optimizations_ performed. See [script optimization]. |
| `set_max_expr_depths` | [`unchecked`] | sets the maximum nesting levels of an expression/statement. See [maximum statement depth]. |
| `set_max_call_levels` | [`unchecked`] | sets the maximum number of function call levels (default 50) to avoid infinite recursion. See [maximum call stack depth]. |
| `set_max_operations` | [`unchecked`] | sets the maximum number of _operations_ that a script is allowed to consume. See [maximum number of operations]. |
| `set_max_modules` | [`unchecked`] | sets the maximum number of [modules] that a script is allowed to load. See [maximum number of modules]. |
| `set_max_string_size` | [`unchecked`] | sets the maximum length (in UTF-8 bytes) for [strings]. See [maximum length of strings]. |
| `set_max_array_size` | [`unchecked`], [`no_index`] | sets the maximum size for [arrays]. See [maximum size of arrays]. |
| `set_max_map_size` | [`unchecked`], [`no_object`] | sets the maximum number of properties for [object maps]. See [maximum size of object maps]. |
| `disable_symbol` | | disables a certain keyword or operator. See [disable keywords and operators]. |
| ------------------------ | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `set_doc_comments` | | enables/disables [doc-comments] |
| `set_optimization_level` | [`no_optimize`] | sets the amount of script _optimizations_ performedSee [script optimization] |
| `set_max_expr_depths` | [`unchecked`] | sets the maximum nesting levels of an expression/statementSee [maximum statement depth] |
| `set_max_call_levels` | [`unchecked`] | sets the maximum number of function call levels (default 50) to avoid infinite recursionSee [maximum call stack depth] |
| `set_max_operations` | [`unchecked`] | sets the maximum number of _operations_ that a script is allowed to consumeSee [maximum number of operations] |
| `set_max_modules` | [`unchecked`] | sets the maximum number of [modules] that a script is allowed to loadSee [maximum number of modules] |
| `set_max_string_size` | [`unchecked`] | sets the maximum length (in UTF-8 bytes) for [strings]See [maximum length of strings] |
| `set_max_array_size` | [`unchecked`], [`no_index`] | sets the maximum size for [arrays]See [maximum size of arrays] |
| `set_max_map_size` | [`unchecked`], [`no_object`] | sets the maximum number of properties for [object maps]See [maximum size of object maps] |
| `disable_symbol` | | disables a certain keyword or operatorSee [disable keywords and operators] |

40
doc/src/language/comments.md
View File

@ -22,43 +22,3 @@ let /* intruder comment */ name = "Bob";
/*/*/*/*/**/*/*/*/*/
*/
```
Doc-Comments
------------
Similar to Rust, comments starting with `///` (three slashes) or `/**` (two asterisks) are
_doc-comments_.
Doc-comments can only appear in front of [function] definitions, not any other elements:
```rust
/// This is a valid one-line doc-comment
fn foo() {}
/** This is a
** valid block
** doc-comment
**/
fn bar(x) {
/// Syntax error - this doc-comment is invalid
x + 1
}
/** Syntax error - this doc-comment is invalid */
let x = 42;
/// Syntax error - this doc-comment is also invalid
{
let x = 42;
}
```
Doc-comments are stored within the script's [`AST`] after compilation.
The `AST::iter_functions` method provides a `ScriptFnMetadata` instance
for each function defined within the script, which includes doc-comments.
Doc-comments never affect the evaluation of a script nor do they incur
significant performance overhead. However, third party tools can take advantage
of this information to auto-generate documentation for Rhai script functions.

76
doc/src/language/doc-comments.md Normal file
View File

@ -0,0 +1,76 @@
Doc-Comments
============
Similar to Rust, comments starting with `///` (three slashes) or `/**` (two asterisks) are
_doc-comments_.
Doc-comments can only appear in front of [function] definitions, not any other elements:
```rust
/// This is a valid one-line doc-comment
fn foo() {}
/** This is a
** valid block
** doc-comment
**/
fn bar(x) {
/// Syntax error - this doc-comment is invalid
x + 1
}
/** Syntax error - this doc-comment is invalid */
let x = 42;
/// Syntax error - this doc-comment is also invalid
{
let x = 42;
}
```
Special Cases
-------------
Long streams of `//////...` and `/*****...` do _NOT_ form doc-comments.
This is consistent with popular comment block styles for C-like languages.
```rust
/////////////////////////////// <- this is not a doc-comment
// This is not a doc-comment // <- this is a normal comment
/////////////////////////////// <- this is not a doc-comment
// However, watch out for comment lines starting with '///'
////////////////////////////////////////// <- this is not a doc-comment
/// This, however, IS a doc-comment!!! /// <- this starts with '///'
////////////////////////////////////////// <- this is not a doc-comment
/****************************************
* *
* This is also not a doc-comment block *
* so we don't have to put this in *
* front of a function. *
* *
****************************************/
```
Using Doc-Comments
------------------
Doc-comments are stored within the script's [`AST`] after compilation.
The `AST::iter_functions` method provides a `ScriptFnMetadata` instance
for each function defined within the script, which includes doc-comments.
Doc-comments never affect the evaluation of a script nor do they incur
significant performance overhead. However, third party tools can take advantage
of this information to auto-generate documentation for Rhai script functions.
Disabling Doc-Comments
----------------------
Doc-comments can be disabled via the `Engine::set_doc_comments` method.

2
doc/src/links.md
View File

@ -91,7 +91,7 @@
[timestamp]: {{rootUrl}}/language/timestamps.md
[timestamps]: {{rootUrl}}/language/timestamps.md
[doc-comments]: {{rootUrl}}/language/comments.md#doc-comments
[doc-comments]: {{rootUrl}}/language/doc-comments.md
[function]: {{rootUrl}}/language/functions.md
[functions]: {{rootUrl}}/language/functions.md
[function overloading]: {{rootUrl}}/rust/functions.md#function-overloading

9
examples/repl.rs
View File

@ -147,6 +147,15 @@ fn main() {
println!();
continue;
}
// "json" => {
// println!(
// "{}",
// engine
// .gen_fn_metadata_to_json(Some(&main_ast), false)
// .unwrap()
// );
// continue;
// }
_ => (),
}

7
src/engine.rs
View File

@ -654,6 +654,9 @@ pub struct Engine {
/// Max limits.
#[cfg(not(feature = "unchecked"))]
pub(crate) limits: Limits,
/// Disable doc-comments?
pub(crate) disable_doc_comments: bool,
}
impl fmt::Debug for Engine {
@ -794,6 +797,8 @@ impl Engine {
#[cfg(not(feature = "no_object"))]
max_map_size: 0,
},
disable_doc_comments: false,
};
engine.load_package(StandardPackage::new().get());
@ -847,6 +852,8 @@ impl Engine {
#[cfg(not(feature = "no_object"))]
max_map_size: 0,
},
disable_doc_comments: false,
}
}

6
src/engine_settings.rs
View File

@ -40,6 +40,12 @@ impl Engine {
pub fn optimization_level(&self) -> crate::OptimizationLevel {
self.optimization_level
}
/// Enable/disable doc-comments.
#[inline(always)]
pub fn set_doc_comments(&mut self, enable: bool) -> &mut Self {
self.disable_doc_comments = !enable;
self
}
/// Set the maximum levels of function calls allowed for a script in order to avoid
/// infinite recursion and stack overflows.
#[cfg(not(feature = "unchecked"))]

32
src/token.rs
View File

@ -354,9 +354,7 @@ impl Token {
Reserved(s) => s.clone().into(),
Custom(s) => s.clone().into(),
LexError(err) => err.to_string().into(),
Comment(s) if is_doc_comment(s) => s[..3].to_string().into(),
Comment(s) => s[..2].to_string().into(),
Comment(s) => s.clone().into(),
token => match token {
LeftBrace => "{",
@ -759,6 +757,8 @@ pub struct TokenizeState {
pub end_with_none: bool,
/// Include comments?
pub include_comments: bool,
/// Disable doc-comments?
pub disable_doc_comments: bool,
}
/// _(INTERNALS)_ Trait that encapsulates a peekable character input stream.
@ -1020,7 +1020,8 @@ fn is_binary_char(c: char) -> bool {
/// Test if the comment block is a doc-comment.
#[inline(always)]
pub fn is_doc_comment(comment: &str) -> bool {
comment.starts_with("///") || comment.starts_with("/**")
(comment.starts_with("///") && !comment.starts_with("////"))
|| (comment.starts_with("/**") && !comment.starts_with("/***"))
}
/// Get the next token.
@ -1040,7 +1041,9 @@ fn get_next_token_inner(
state.comment_level = scan_block_comment(stream, state.comment_level, pos, &mut comment);
if state.include_comments || is_doc_comment(comment.as_ref().unwrap()) {
if state.include_comments
|| (!state.disable_doc_comments && is_doc_comment(comment.as_ref().unwrap()))
{
return Some((Token::Comment(comment.unwrap()), start_pos));
}
}
@ -1288,9 +1291,14 @@ fn get_next_token_inner(
eat_next(stream, pos);
let mut comment = match stream.peek_next() {
Some('/') => {
Some('/') if !state.disable_doc_comments => {
eat_next(stream, pos);
Some("///".to_string())
// Long streams of `///...` are not doc-comments
match stream.peek_next() {
Some('/') => None,
_ => Some("///".to_string()),
}
}
_ if state.include_comments => Some("//".to_string()),
_ => None,
@ -1316,9 +1324,14 @@ fn get_next_token_inner(
eat_next(stream, pos);
let mut comment = match stream.peek_next() {
Some('*') => {
Some('*') if !state.disable_doc_comments => {
eat_next(stream, pos);
Some("/**".to_string())
// Long streams of `/****...` are not doc-comments
match stream.peek_next() {
Some('*') => None,
_ => Some("/**".to_string()),
}
}
_ if state.include_comments => Some("/*".to_string()),
_ => None,
@ -1785,6 +1798,7 @@ impl Engine {
comment_level: 0,
end_with_none: false,
include_comments: false,
disable_doc_comments: self.disable_doc_comments,
},
pos: Position::new(1, 0),
stream: MultiInputsStream {

24
tests/comments.rs
View File

@ -29,7 +29,7 @@ fn test_comments() -> Result<(), Box<EvalAltResult>> {
#[cfg(not(feature = "no_function"))]
#[test]
fn test_comments_doc() -> Result<(), Box<EvalAltResult>> {
let engine = Engine::new();
let mut engine = Engine::new();
let ast = engine.compile(
r"
@ -54,6 +54,16 @@ fn test_comments_doc() -> Result<(), Box<EvalAltResult>> {
)
.is_err());
engine.compile(
r"
///////////////
let x = 42;
/***************/
let x = 42;
",
)?;
let ast = engine.compile(
r"
/** Hello world
@ -78,5 +88,17 @@ fn test_comments_doc() -> Result<(), Box<EvalAltResult>> {
)
.is_err());
engine.set_doc_comments(false);
engine.compile(
r"
/// Hello world!
let x = 42;
/** Hello world! */
let x = 42;
",
)?;
Ok(())
}