Let the agent run experiments, keep what works, and discard what doesn't.
A fork of pi-autoresearch and inspired by karpathy/autoresearch, ported to the agentplugins CLI.
The vision is simple: give an agent a benchmark, then let it try ideas, measure results, commit winners, revert losers, and keep going until you tell it to stop.
# install via agentplugins
pnpm dlx agentplugins install agentplugins-autoresearch
# start a loop
/skill:autoresearch-createThe agent will ask for:
- Goal — what you want to optimize
- Metric — the number that decides keep vs discard
- Command — the benchmark that produces that number
- Scope — files the agent is allowed to change
Then it writes .auto/prompt.md and .auto/measure.sh, runs a baseline, and starts iterating.
Opens the autoresearch skill. Answer a few questions and the agent begins experimenting.
Puts the agent into a tight loop: propose change → run benchmark → log result → keep or revert → repeat.
/autoresearch optimize test runtime while keeping correctness
/autoresearch reduce bundle size after pnpm build
/autoresearch improve Lighthouse performance score
Launches a browser dashboard with a live chart and results table.
Groups kept experiments into clean, independent branches from the merge-base. Each branch can be reviewed and merged on its own.
| Extension | init_experiment, run_experiment, log_experiment tools + live widget + /autoresearch dashboard |
| Skills | autoresearch-create, autoresearch-finalize, and optional autoresearch-hooks |
| Session files | One .auto/ folder per project: prompt.md, measure.sh, log.jsonl, optional checks.sh and hooks/ |
- Benchmark-driven. The metric is the judge. No hand-waving.
- Persistent.
.auto/log.jsonlsurvives restarts and context resets. - Safe. Every experiment is committed; regressions are reverted.
- Composable. One extension handles any domain — tests, builds, ML, Lighthouse, anything that prints a number.
- Installation and setup
- Usage guide
- Configuration
- Confidence scoring
- Backpressure checks
- Hooks
- CHANGELOG
MIT