Skip to content

Commit 61f0a17

Browse files
authored
feat(annotations): complete DSL surface — composite tests, See selectors, P0 bugfixes (v0.9.6) (#109)
Closes every gap identified in the v0.9.5 audit of flutter_probe_annotation and flutter_probe_gen. Goes from ~70% coverage of the ProbeScript surface to full parity, with comprehensive golden tests cross-validated against the Go parser. P0 correctness bugs (would silently break user tests at runtime): * Mock path is now quoted in the emitted 'when the app calls ...' line. Was unquoted, so the Go lexer split on '/' and the parser only kept the first IDENT segment — Mock(path: '/api/products') silently became /api. Now emits 'when the app calls GET "/api/products"' and the full path round-trips. * See suffixes (state + containing + matching) now compose additively instead of overwriting. See('x', state: SeeState.enabled, containing: 'y') used to silently drop the state; now emits both. P1 missing feature: * @ProbeCompositeTest DSL — Device, OnDevice, Sync classes. Emitter walks devices then body, rendering <alias>: groups and sync "label" barriers. Round-trips through the Go parser's full composite test machinery (parser.go:1433+). P1 incoherent surface: * Press and Pinch are now @deprecated — the Go parser has no case for them, emitted text fell through to parseRecipeCall. Use GoBack() in place of Press('back'). P2 under-modeled assertions: * See.id(key) / See.selector(Selector) factories — assertions can now target by ValueKey or any rich selector (Ordinal, Below/Above/ LeftOf/RightOf, InContainer, TypeSel). Same on DontSee. The Go parser always supported this; the DSL just didn't expose it. * WaitUntil.idAppears(key) / .idDisappears(key) — emits unquoted #key so the Go parser's WaitSelector branch (parser.go:846) matches. P3 robustness: * Emitter no longer indexes hard-coded arrays by enum.index. Reads the enum constant identifier via _name. Reordering Direction, HttpMethod, or SeeState no longer silently corrupts output. Comprehensive tests (the user's strictness ask): * 6 new golden fixture pairs in probe_gen/test/fixtures/: - mock_and_call (catches P0 #1 regression) - see_states (catches P0 #2 regression + new selector forms) - composite_chat (exercises every composite construct) - wait_variants (every wait kind + id-based) - examples_inline (data-driven Examples) - kitchen_sink (one of every step + selector + control flow) * Builder tests: 5 → 11 (one per golden + existing 5). * Cross-language goldens: 4 → 10. Every Dart-emitted .probe round- trips through internal/parser/parser.go via golden_integration_test. Docs: * New website page: website/src/content/docs/probescript/annotations.md — full reference for the DSL with every step class, selector kind, and composite syntax. Added to Starlight sidebar. * docs/wiki/Annotations.md version refs bumped. * README.md highlights composite tests in the annotation section. * Per-package CHANGELOGs detail the changes. Versions bumped to 0.9.6: probe_agent, probe_annotation, probe_gen pubspec.yaml + CHANGELOG.md vscode/package.json docs/wiki/Home.md website/src/content/docs/tools/mcp.md verify snippet probe_gen depends on flutter_probe_annotation: ^0.9.6 Verification (all green before commit): go test ./... 16/16 packages pass staticcheck ./... zero issues dart analyze (annot) no issues found dart test (annot) 3/3 const tests pass dart analyze (gen) no issues found dart test (gen) 11/11 builder tests pass (5 existing + 6 new) go test ...GoldenIntegration 10/10 Dart goldens parse cleanly dart pub publish --dry-run both packages 0 errors Astro build 37 pages generated successfully
1 parent 625226d commit 61f0a17

32 files changed

Lines changed: 1040 additions & 59 deletions

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,25 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
66

77
## [Unreleased]
88

9+
## [0.9.6] - 2026-05-12
10+
11+
### Fixed
12+
- **`flutter_probe_gen`: `Mock` path silently truncated.** The emitter wrote the path unquoted (`when the app calls GET /api/products`), so the Go lexer split on `/` and the parser only recorded the first IDENT segment. Now emits the canonical quoted form. Caught by a new `mock_and_call` golden + the existing cross-language integration test.
13+
- **`flutter_probe_gen`: `See` suffixes silently dropped.** When `state`, `containing`, and `matching` were all set on a single assertion, only the last branch's text reached the output. Now composes all three suffixes additively: `see "x" is enabled contains "y" matching "z"`. Caught by a new `see_states` golden covering the matrix.
14+
15+
### Added
16+
- **`flutter_probe_annotation`: `@ProbeCompositeTest` annotation.** The flagship multi-device composite testing feature finally has a DSL surface. Pair with `Device(alias, target: …)`, `OnDevice(alias, steps: […])` per-device groups, and `Sync(label)` cross-device barriers. Emitter generates standard `composite test` / `devices` / `<alias>:` / `sync` blocks that the existing CLI runner picks up unchanged.
17+
- **`flutter_probe_annotation`: `See.id` / `See.selector` factories** — assertions can now target by `ValueKey` or any rich selector (Ordinal, Below/Above/LeftOf/RightOf, InContainer, TypeSel) rather than only by literal visible text. Same factories on `DontSee`. The Go parser always supported this; the DSL just didn't expose it.
18+
- **`flutter_probe_annotation`: `WaitUntil.idAppears` / `.idDisappears`** — emits unquoted `#key` selector form (Go parser's WaitSelector branch), which is more reliable than text matching for stable `ValueKey`-tagged widgets.
19+
- **`flutter_probe_gen`: 6 new golden fixtures.** `mock_and_call`, `see_states`, `composite_chat`, `wait_variants`, `examples_inline`, `kitchen_sink`. The kitchen sink fixture exercises one of every step, selector kind, and control-flow construct. Every fixture round-trips through `internal/parser/golden_integration_test.go`. Total golden coverage went from 4 → 10 fixtures, builder tests from 5 → 11.
20+
21+
### Changed
22+
- **`flutter_probe_annotation`: `Press` and `Pinch` are now `@Deprecated`.** The Go parser has no `press` or `pinch` case, so emitted text fell through to `parseRecipeCall` and was misinterpreted. Marked deprecated until runtime support lands. Use `GoBack()` in place of `Press('back')`.
23+
- **`flutter_probe_gen`: emitter no longer coupled to enum declaration order.** `_direction`, `_httpMethod`, and the `See` state lookup now read the enum constant identifier (`_name` field) instead of indexing a hard-coded array by `.index`. Reordering `Direction`, `HttpMethod`, or `SeeState` no longer silently corrupts emitted ProbeScript.
24+
25+
### Docs
26+
- New website page: [Annotation-driven Tests](https://flutterprobe.dev/probescript/annotations/) — full reference for the annotation DSL with every step class, selector kind, and the new composite test syntax.
27+
928
## [0.9.5] - 2026-05-12
1029

1130
### Fixed

README.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -369,18 +369,18 @@ run dart:
369369

370370
Co-locate `.probe` tests with the Flutter widgets they exercise. Two Dart packages handle this:
371371

372-
- **`flutter_probe_annotation`**`@ProbeSuite`, `@ProbeTest`, `@ProbeRecipe` decorators plus a fully type-checked step DSL (all 31 ProbeScript verbs, all 6 selector kinds, hooks, loops, conditionals, recipes, examples).
372+
- **`flutter_probe_annotation`**`@ProbeSuite`, `@ProbeTest`, `@ProbeRecipe`, `@ProbeCompositeTest` decorators plus a fully type-checked step DSL (all 31 ProbeScript verbs, all 6 selector kinds, hooks, loops, conditionals, recipes, examples, multi-device composite tests).
373373
- **`flutter_probe_gen`** — a `build_runner` builder that reads the annotations and emits matching `.probe` files into `tests/generated/`.
374374

375375
Add to your Flutter app's `pubspec.yaml`:
376376

377377
```yaml
378378
dependencies:
379-
flutter_probe_annotation: ^0.9.3
380-
flutter_probe_agent: ^0.9.3
379+
flutter_probe_annotation: ^0.9.6
380+
flutter_probe_agent: ^0.9.6
381381

382382
dev_dependencies:
383-
flutter_probe_gen: ^0.9.3
383+
flutter_probe_gen: ^0.9.6
384384
build_runner: ^2.15.0
385385
```
386386
@@ -415,7 +415,9 @@ probe test tests/ # picks up tests/generated/login_screen.probe
415415

416416
Test definitions are now type-checked by `flutter analyze` — a misspelt step name is a compile error rather than a runtime surprise. Selectors stay in sync with widget code because they live in the same file. The generated `.probe` file goes through the same parser, agent, and reporter as a hand-written one.
417417

418-
Full reference: [`docs/wiki/Annotations.md`](docs/wiki/Annotations.md).
418+
**v0.9.6** completes the annotation surface: full composite-test DSL (`@ProbeCompositeTest`, `Device`, `OnDevice`, `Sync`), id/selector-based `See`/`DontSee`, `WaitUntil.idAppears`, and composable `state` + `containing` + `matching` assertions. Plus fixes for two emitter bugs (`Mock` paths and `See` suffix dropping).
419+
420+
Full reference: [flutterprobe.dev/probescript/annotations](https://flutterprobe.dev/probescript/annotations/) (or [`docs/wiki/Annotations.md`](docs/wiki/Annotations.md) on GitHub).
419421

420422
## CLI Commands
421423

docs/wiki/Annotations.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,11 @@ tests. Annotations:
2828
```yaml
2929
# pubspec.yaml
3030
dependencies:
31-
flutter_probe_annotation: ^0.9.3
32-
flutter_probe_agent: ^0.9.3
31+
flutter_probe_annotation: ^0.9.6
32+
flutter_probe_agent: ^0.9.6
3333

3434
dev_dependencies:
35-
flutter_probe_gen: ^0.9.3
35+
flutter_probe_gen: ^0.9.6
3636
build_runner: ^2.15.0
3737
```
3838

docs/wiki/Home.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Welcome to the FlutterProbe wiki. This documentation covers architecture details
1818

1919
## Project Status
2020

21-
FlutterProbe is in active development. Current version: **0.9.5**.
21+
FlutterProbe is in active development. Current version: **0.9.6**.
2222

2323
### Repository Structure
2424

internal/parser/golden_integration_test.go

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,9 +49,11 @@ func TestGoldenIntegration_DartEmittedFiles(t *testing.T) {
4949
t.Fatalf("parser rejected golden %s:\n%v\n--- contents ---\n%s",
5050
filepath.Base(path), err, string(data))
5151
}
52-
// Sanity: every golden defines at least one test, recipe, or hook.
53-
if len(prog.Tests) == 0 && len(prog.Recipes) == 0 && len(prog.Hooks) == 0 {
54-
t.Errorf("golden %s parsed but produced no tests/recipes/hooks",
52+
// Sanity: every golden defines at least one test, composite test,
53+
// recipe, or hook.
54+
if len(prog.Tests) == 0 && len(prog.CompositeTests) == 0 &&
55+
len(prog.Recipes) == 0 && len(prog.Hooks) == 0 {
56+
t.Errorf("golden %s parsed but produced no tests/composite tests/recipes/hooks",
5557
filepath.Base(path))
5658
}
5759
})

probe_agent/CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# Changelog
22

3+
## 0.9.6 - 2026-05-12
4+
5+
- Version bump to match CLI v0.9.6. No agent code changes — annotation DSL
6+
completeness work is in the flutter_probe_annotation &
7+
flutter_probe_gen packages.
8+
39
## 0.9.5 - 2026-05-12
410

511
- **Fix: iOS/Impeller screenshots**`take_screenshot` previously called

probe_agent/pubspec.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ description: >-
33
On-device E2E test agent for FlutterProbe. Embeds in your Flutter app and
44
executes test commands via direct widget-tree access with sub-50ms latency.
55
6-
version: 0.9.5
6+
version: 0.9.6
77
homepage: https://flutterprobe.dev
88
repository: https://github.com/AlphaWaveSystems/flutter-probe
99
issue_tracker: https://github.com/AlphaWaveSystems/flutter-probe/issues

probe_annotation/CHANGELOG.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,36 @@
11
# Changelog
22

3+
## 0.9.6 - 2026-05-12
4+
5+
### Added
6+
7+
- **`@ProbeCompositeTest`** — declare multi-device composite tests as
8+
annotations. Pair with `Device(alias, target: ...)` declarations,
9+
`OnDevice(alias, steps: [...])` per-device step groups, and
10+
`Sync(label)` cross-device barriers. The generated `.probe` block
11+
uses the standard `composite test` / `devices` / `sync` syntax that
12+
the CLI runner already understands.
13+
- **`See.id(key)` / `See.selector(Selector)`** — target assertions by
14+
`ValueKey` or by rich selector (Ordinal, Below/Above/LeftOf/RightOf,
15+
InContainer, TypeSel). Same factories on `DontSee`. Previously,
16+
`See`/`DontSee` only accepted a literal text string — the parser
17+
always supported every selector kind but the DSL didn't expose it.
18+
- **`WaitUntil.idAppears(key)` / `.idDisappears(key)`** — emit
19+
unquoted `wait until #key appears`, exercising the Go parser's
20+
WaitSelector branch. More reliable than text matching for widgets
21+
with stable `ValueKey`s.
22+
- **Composable `See` suffixes** — `See('x', state: SeeState.enabled,
23+
containing: 'y')` now emits `see "x" is enabled contains "y"`
24+
(both suffixes present). Previously the second suffix silently
25+
overwrote the first.
26+
27+
### Changed
28+
29+
- **`Press` and `Pinch` are now `@Deprecated`** with a clear note —
30+
the Go-side parser has no `press` or `pinch` case and emitted text
31+
would be misparsed as a recipe call. They'll be re-enabled when
32+
runtime support lands. Use `GoBack()` in place of `Press('back')`.
33+
334
## 0.9.5 - 2026-05-12
435

536
- Version bump to match CLI v0.9.5. No annotation API changes.

probe_annotation/lib/src/annotations.dart

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,3 +114,70 @@ class ProbeRecipe {
114114
this.steps = const [],
115115
});
116116
}
117+
118+
/// Declares a multi-device composite test. The annotated class becomes a
119+
/// `composite test "name"` block in the generated `.probe` file.
120+
///
121+
/// Devices are declared by alias (`A`, `B`, `Sender`, etc.) and steps are
122+
/// scoped per-device using [OnDevice]. [Sync] barriers between [OnDevice]
123+
/// groups force every device to reach the same checkpoint before any
124+
/// device proceeds.
125+
///
126+
/// Example — chat between two users on two simulators:
127+
///
128+
/// ```dart
129+
/// @ProbeCompositeTest(
130+
/// name: 'alice sends bob a message',
131+
/// tags: ['composite', 'smoke'],
132+
/// devices: [
133+
/// Device('A', target: 'iPhone 15 Simulator'),
134+
/// Device('B', target: 'Pixel 9 Emulator'),
135+
/// ],
136+
/// body: [
137+
/// OnDevice('A', steps: [
138+
/// Open(),
139+
/// Tap(text: 'Sign in as Alice'),
140+
/// ]),
141+
/// OnDevice('B', steps: [
142+
/// Open(),
143+
/// Tap(text: 'Sign in as Bob'),
144+
/// ]),
145+
/// Sync('both signed in'),
146+
/// OnDevice('A', steps: [
147+
/// Tap(text: 'New message'),
148+
/// Type('hello bob'),
149+
/// Tap(text: 'Send'),
150+
/// ]),
151+
/// OnDevice('B', steps: [
152+
/// WaitUntil.appears('hello bob'),
153+
/// See('hello bob'),
154+
/// ]),
155+
/// ],
156+
/// )
157+
/// class ChatComposite {}
158+
/// ```
159+
class ProbeCompositeTest {
160+
final String name;
161+
final List<String> tags;
162+
final List<Device> devices;
163+
164+
/// Composite body — must contain only [OnDevice] and [Sync] elements.
165+
final List<Step> body;
166+
167+
const ProbeCompositeTest({
168+
required this.name,
169+
this.tags = const [],
170+
this.devices = const [],
171+
this.body = const [],
172+
});
173+
}
174+
175+
/// One device entry in a [ProbeCompositeTest.devices] list. The [alias]
176+
/// is referenced by [OnDevice]; [target] is an optional human-readable
177+
/// device name shown in the generated `.probe` header and in failure
178+
/// messages.
179+
class Device {
180+
final String alias;
181+
final String? target;
182+
const Device(this.alias, {this.target});
183+
}

0 commit comments

Comments
 (0)