Skip to content

Commit 351103e

Browse files
committed
spruce up README
1 parent fb8a89c commit 351103e

2 files changed

Lines changed: 58 additions & 15 deletions

File tree

README.md

Lines changed: 42 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,23 @@
1-
Spindle is a simple and efficient expression and byte sequence generator to aid fuzz testing parsers and de-serializers.
1+
# Spindle
2+
[![Crates.io](https://img.shields.io/crates/v/spindle-lib.svg)](https://crates.io/crates/spindle-lib)
3+
[![docs.rs](https://docs.rs/spindle-lib/badge.svg)](https://docs.rs/spindle-lib)
4+
[![License: APACHE](https://img.shields.io/badge/License-Apache-blue.svg)](https://github.com/awslabs/spindle/blob/main/LICENSE)
5+
![Downloads](https://img.shields.io/crates/d/spindle-lib)
26

3-
# Usage
4-
Spindle offers a syntax similar to [Extended Backus–Naur form](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form) which compiles to a state machine --`Grammar`--that produces structured matching arbitrary sentences from an unstructured feed of bytes.
7+
Spindle is a simple and efficient expression and byte sequence generator to aid fuzz testing parsers and de-serializers. Spindle spins raw, untyped byte buffers into structured data.
58

6-
Spindle integrates with [libfuzzer](https://llvm.org/docs/LibFuzzer.html) and [cargo-fuzz](https://crates.io/crates/cargo-fuzz): Unstructured bytes, from the [arbitrary](https://crates.io/crates/arbitrary) crate, are manipulated by the fuzzer based on code coverage.
9+
## Overview
10+
Spindle's syntax, similar to [Extended Backus–Naur form](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form), lets users define the structure of generated data. This syntax compiles to `Grammar`, a state machine that can be arbitrarily traversed to produce structure-aware, matching expressions.
711

8-
Spindle can be used to generate database expressions, big decimal strings, JSON, and other syntaxes, as well as slightly malformed variants of correct expressions to test interesting edge cases of parser or de-serializer.
12+
Spindle works with fuzzers such as [cargo-fuzz](https://crates.io/crates/cargo-fuzz) or [AFL](https://crates.io/crates/afl) because it is an extension of [arbitrary](https://crates.io/crates/arbitrary); the traversal of the state machine is deterministically dependent on [`Unstructured`](https://docs.rs/arbitrary/latest/arbitrary/struct.Unstructured.html).
13+
14+
Spindle is particularily useful for generating semi-correct and interesting inputs that attack edge cases of parsers and de-serializers, such as mixing familar tokens in incorrect places or sprinkling in Unicode characters.
15+
16+
Spindle is developed and leveraged by AWS to fuzz test the parsers and de-serializers in their backend systems.
17+
18+
## Examples
19+
**For more examples, see the [examples](https://github.com/awslabs/spindle/tree/main/examples) folder.**
920

10-
# Example
1121
```rust
1222
use spindle_lib::Grammar;
1323
use arbitrary::Unstructured;
@@ -18,17 +28,36 @@ let math: Grammar = r#"
1828
symbol : r"-|\+|\*|÷" ;
1929
"#.parse().unwrap();
2030

21-
let mut u = Unstructured::new(b"poiuyt5r4321sdnlknasdbvcxeygrey");
22-
let sentence: String = math.expression(&mut u, None).unwrap(); // (30057+(12594+((((25976+(0*0))*0*0)*0)*0)*0*0))
31+
let mut u = Unstructured::new(b"poiuytasdbvcxeygrey");
32+
let sentence: String = math.expression(&mut u, None).unwrap();
33+
// (21359*39933))+13082-62216
2334
```
2435
The state machine traversal always starts at the first rule. In the example,
2536
- `expr` is the first rule and evaluates to either `u16`, `paren`, or the concatenation of `expr` and `symbol` and `expr`.
2637
- `;` delimits different rules.
27-
- `u16` is a pre-defined data types that directly evaluates to `u16::arbitrary(u)`
38+
- `u16` is a pre-defined data types that directly evaluates to `u16::arbitrary(u)`.
2839
- `paren` evaluates to the concatenation of the literal `"("`, `expr`, `symbol`, `expr` and, `")"`.
2940
- `symbol` evaluates to the an arbitrary string matching the regex `-|\+|\*|÷`.
3041

31-
## Usage in Fuzzer
42+
### Semi-Correct Expression
43+
This grammar is similar to the well formed math expression grammar, but sometimes includes an extra closing parenthesis and/or an arbitrary symbol.
44+
45+
```rust
46+
use spindle_lib::Grammar;
47+
use arbitrary::Unstructured;
48+
49+
let math: Grammar = r#"
50+
expr : u16 | paren | expr symbol expr ;
51+
paren : "(" expr symbol expr ")" ")"? ;
52+
symbol : r"-|\+|\*|÷" | String ;
53+
"#.parse().unwrap();
54+
55+
let mut u = Unstructured::new(b"poiuytasdbvcxeygrey");
56+
let sentence: String = math.expression(&mut u, None).unwrap();
57+
// (44637*32200)Ѱ'x124928390-27338)
58+
```
59+
60+
### Usage in Fuzzer
3261
```rust,ignore
3362
use spindle_lib::Grammar;
3463
use libfuzzer_sys::fuzz_target;
@@ -56,13 +85,11 @@ fuzz_target!(|expr: MathExpression| {
5685
});
5786
```
5887

59-
For more examples, see the `examples` folder.
60-
61-
# Pre-defined Rules
88+
## Pre-defined Rules
6289
- `String` evaluates to `str::arbitrary(u)`
6390
- `u16` evaluates to `u16::arbitrary(u)`
6491

65-
# Visitor
92+
## Visitor
6693
A `Visitor` is some state that is initialized before traversal and mutated as different rules are visited during the traversal, e.g. `visit_or`. Vistors that are already implemented are `String` and `Vec<u8>` for output buffers, and `u64` for classification.
6794

6895
Users can use their own implementation of `Visitor`, for example if they want to
@@ -71,7 +98,7 @@ Users can use their own implementation of `Visitor`, for example if they want to
7198
- gather data
7299
- build an abstract syntax tree
73100

74-
## Example
101+
### Example
75102
```rust
76103
use spindle_lib::{Grammar, Visitor};
77104

examples/math.rs

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,20 @@ fn main() {
1616
let mut u = rand_u(&mut buf);
1717
let sentence: String = grammar.expression(&mut u, None).unwrap();
1818
println!("{}", sentence);
19+
20+
// This grammar is similar to the well formed math expression grammar,
21+
// but sometimes includes an extra closing parenthesis and/or an arbitrary symbol.
22+
let grammar: Grammar = r#"
23+
expr : num | paren | expr symbol expr ;
24+
paren : "(" expr symbol expr ")" ")"? ;
25+
symbol : r"-|\+|\*|÷" | String;
26+
num : r"[0-9]+" ;
27+
"#
28+
.parse()
29+
.unwrap();
30+
31+
let mut buf = [0; 4096];
32+
let mut u = rand_u(&mut buf);
33+
let sentence: String = grammar.expression(&mut u, None).unwrap();
34+
println!("{}", sentence);
1935
}

0 commit comments

Comments
 (0)