Skip to content

docs: add chaos testing example script#836

Open
ybdarrenwang wants to merge 2 commits into
strands-agents:mainfrom
ybdarrenwang:docs/chaos-tool
Open

docs: add chaos testing example script#836
ybdarrenwang wants to merge 2 commits into
strands-agents:mainfrom
ybdarrenwang:docs/chaos-tool

Conversation

@ybdarrenwang
Copy link
Copy Markdown
Contributor

@ybdarrenwang ybdarrenwang commented May 13, 2026

Description

Adds documentation and example for the chaos testing module in Strands Evals.

The example demonstrates:

  • Defining named effect maps (dict[str, dict[str, list[ChaosEffect]]]) for readable failure conditions (e.g., "search_timeout", "total_chaos")
  • Using ChaosCase.expand(cases, effect_maps) to generate the Cartesian product of test cases × failure conditions with optional baseline
  • Running a ChaosExperiment with ChaosPlugin for transparent fault injection
  • Evaluating agent resilience with GoalSuccessRateEvaluator

The example uses ToolSimulator for reproducible tool responses and covers pre-hook effects (tool call failures), post-hook effects (response corruption), and compound multi-tool chaos.

Related Issues

#114

Type of Change

  • New content

Checklist

  • I have read the CONTRIBUTING document
  • My changes follow the project's documentation style
  • I have tested the documentation locally using npm run dev
  • Links in the documentation are valid and working

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant