Skip to content

Commit 67faf02

Browse files
author
Gabi
committed
feat: specialist agent architecture — scanner decomposition
Merge feat/specialist-agents into main. Splits the monolithic scanner into an evidence collector + 6 internal specialist evaluators. Scanner captures browser evidence once, writes JSON bundle. Orchestrator fans out to 5 pillar specialists + 1 slop classifier, aggregates into the same report format. 612 tests, two rounds of Codex review addressed.
2 parents ecd2116 + 19fdb7d commit 67faf02

19 files changed

Lines changed: 1650 additions & 200 deletions

CLAUDE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ dev_docs/ # Internal planning (gitignored)
7777
## Testing
7878

7979
```bash
80-
npm test # Full suite (540 tests)
80+
npm test # Full suite (600+ tests)
8181
npm run validate # Resource file structure + cross-file consistency only
8282
npm run test:detection # Detection logic only
8383
npm run test:format # Report format only
@@ -125,7 +125,7 @@ Conventional commits. Always. See the global CLAUDE.md for the full format refer
125125
- **Checkpoint before edit.** The fixer always creates a checkpoint via `pixelslop-tools` before modifying files. No exceptions.
126126
- **Build gate is non-negotiable.** If the build breaks after a fix, automatic rollback. No "but the design fix was correct."
127127
- **Max one retry on PARTIAL.** Keep the improvement and move on. Don't loop forever.
128-
- **Run tests before committing.** `npm test`540 tests, zero dependencies, takes ~10s.
128+
- **Run tests before committing.** `npm test`600+ tests, zero dependencies, takes ~10s.
129129
- **Path rewriting is fragile.** If you rename `bin/pixelslop-tools.cjs` or move `dist/skill/resources/`, update `rewriteAgentPaths()` in `bin/pixelslop.mjs`. The installer tests catch drift.
130130

131131
## Playwright MCP

bin/pixelslop.mjs

Lines changed: 49 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@
1919
*/
2020

2121
import { readFileSync, writeFileSync, mkdirSync, copyFileSync,
22-
readdirSync, rmSync, existsSync, lstatSync, symlinkSync,
22+
readdirSync, rmSync, rmdirSync, existsSync, lstatSync, symlinkSync,
2323
chmodSync, statSync } from 'fs';
2424
import { join, dirname, resolve, relative } from 'path';
2525
import { fileURLToPath } from 'url';
@@ -858,6 +858,19 @@ function install(options = {}) {
858858
}
859859
log('✓', `${AGENT_FILES.length} agent specs → ${client.agentDir}`);
860860

861+
// Copy internal evaluator agents (not in AGENT_FILES — orchestrator-only)
862+
const internalSrc = join(PACKAGE_ROOT, 'dist', 'agents', 'internal');
863+
if (existsSync(internalSrc)) {
864+
const internalDest = join(client.agentDir, 'internal');
865+
mkdirSync(internalDest, { recursive: true });
866+
const internalFiles = readdirSync(internalSrc).filter(f => f.endsWith('.md') && !f.startsWith('._'));
867+
for (const file of internalFiles) {
868+
const raw = readFileSync(join(internalSrc, file), 'utf8');
869+
writeFileSync(join(internalDest, file), rewriteAgentPaths(raw, INSTALL_ROOT));
870+
}
871+
log('✓', `${internalFiles.length} internal evaluators → ${internalDest}`);
872+
}
873+
861874
// Install skill via linkOrCopy — method is tracked
862875
const preferredMethod = options.installMethods?.[client.name]?.skill;
863876
const { path: skillPath, method: skillMethod } = client.installSkill(
@@ -1056,6 +1069,19 @@ function uninstall() {
10561069
}
10571070
}
10581071

1072+
// Remove internal evaluator agents (only pixelslop-eval-* files, not the whole directory)
1073+
const internalDir = join(client.agentDir, 'internal');
1074+
if (existsSync(internalDir)) {
1075+
for (const file of readdirSync(internalDir).filter(f => f.startsWith('pixelslop-eval-') && f.endsWith('.md'))) {
1076+
rmSync(join(internalDir, file));
1077+
}
1078+
// Remove the directory only if it's empty (don't nuke other tools' files)
1079+
try {
1080+
const remaining = readdirSync(internalDir).filter(f => !f.startsWith('.'));
1081+
if (remaining.length === 0) rmdirSync(internalDir);
1082+
} catch { /* directory not empty or already gone — fine */ }
1083+
}
1084+
10591085
// Remove skill
10601086
client.removeSkill();
10611087

@@ -1182,18 +1208,32 @@ function doctor() {
11821208
`Missing ${AGENT_FILES.length - agentCount} agent file(s)`
11831209
);
11841210

1185-
// Path rewriting verification — check one agent file for absolute paths
1186-
const orchestratorPath = join(client.agentDir, 'pixelslop.md');
1187-
if (existsSync(orchestratorPath)) {
1188-
const content = readFileSync(orchestratorPath, 'utf8');
1189-
const hasAbsolutePaths = content.includes(join(INSTALL_ROOT, 'bin', 'pixelslop-tools.cjs'));
1211+
// Internal evaluator agents
1212+
const internalDir = join(client.agentDir, 'internal');
1213+
if (existsSync(internalDir)) {
1214+
const internalCount = readdirSync(internalDir).filter(f => f.endsWith('.md') && !f.startsWith('._')).length;
11901215
check(
1191-
`${client.name}: path rewriting`,
1192-
hasAbsolutePaths,
1193-
'Agent files still reference relative paths'
1216+
`${client.name}: internal evaluators (${internalCount})`,
1217+
internalCount >= 6,
1218+
`Expected ≥6 internal evaluators, found ${internalCount}`
11941219
);
1220+
} else {
1221+
check(`${client.name}: internal evaluators`, false, 'Missing internal/ directory');
11951222
}
11961223

1224+
// Path rewriting verification — check both a public agent and an internal evaluator
1225+
const orchestratorPath = join(client.agentDir, 'pixelslop.md');
1226+
const internalEvalPath = join(client.agentDir, 'internal', 'pixelslop-eval-color.md');
1227+
const orchestratorRewritten = existsSync(orchestratorPath)
1228+
&& readFileSync(orchestratorPath, 'utf8').includes(join(INSTALL_ROOT, 'bin', 'pixelslop-tools.cjs'));
1229+
const internalRewritten = existsSync(internalEvalPath)
1230+
&& readFileSync(internalEvalPath, 'utf8').includes(join(INSTALL_ROOT, 'skill', 'resources', 'scoring.md'));
1231+
check(
1232+
`${client.name}: path rewriting`,
1233+
orchestratorRewritten && internalRewritten,
1234+
'Installed agents still reference relative paths'
1235+
);
1236+
11971237
// Skill — check existence and report install method
11981238
const clientMethod = methods[client.name]?.skill || 'unknown';
11991239
check(
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
name: pixelslop-eval-accessibility
3+
description: >
4+
Scores the accessibility pillar (1-4) from a pre-collected evidence bundle.
5+
Read-only — no browser access, no file editing.
6+
model: sonnet
7+
tools:
8+
- Read
9+
---
10+
11+
You're the accessibility evaluator. You read an evidence bundle, apply the scoring rubric, and return a pillar score with cited evidence. You don't open browsers, don't fix anything, don't write files. You check what's measurable and score it.
12+
13+
A note on scope: this pillar owns contrast ratios. The color evaluator handles palette aesthetics — you handle whether people can actually read the text. Don't overlap.
14+
15+
## Setup: Load Your Knowledge
16+
17+
Read these before you evaluate:
18+
19+
```
20+
Read dist/skill/resources/scoring.md # Accessibility section — your rubric
21+
Read dist/skill/resources/harden.md # Accessibility fix guide — calibrates your judgment
22+
Read dist/skill/resources/heuristics.md # Usability heuristics — supplementary reference
23+
```
24+
25+
## Input
26+
27+
You receive three values:
28+
29+
- **evidence_path** (required) — absolute path to the evidence bundle JSON
30+
31+
- **thorough** (optional, default: false) — when true, include low-confidence findings tagged `[low confidence]`
32+
33+
## Protocol
34+
35+
1. **Read your resource files.** All three. Before anything else.
36+
2. **Read the evidence bundle** at `evidence_path`.
37+
3. **Extract the fields you need:**
38+
- `viewports.desktop.contrast` — computed contrast ratios and AA pass/fail status per element
39+
- `viewports.desktop.a11ySnapshot` — headings, landmarks, forms, ARIA attributes, alt text
40+
- `personaChecks.headingHierarchy` — heading order and skip detection
41+
- `personaChecks.landmarks` — landmark region presence
42+
- `personaChecks.skipNav` — skip-to-content link check
43+
4. **Apply the rubric** from scoring.md (Pillar 5: Accessibility). Evaluate each criterion:
44+
- **Contrast ratios** — WCAG AA requires 4.5:1 for normal text, 3:1 for large text (≥18pt or ≥14pt bold). All key text-on-background combos must pass. Secondary text (captions, placeholders, meta) failing = score cap at 2.
45+
- **Heading hierarchy** — sequential levels (h1 → h2 → h3), no skips (h1 → h4), exactly one h1. Skipped levels or multiple h1s = problem.
46+
- **Landmark regions**`<main>`, `<nav>`, `<header>`, `<footer>` should all be present. Missing `<main>` is worse than missing `<footer>`.
47+
- **Alt text** — content images need meaningful alt text. Decorative images need `alt=""`. "image", "photo", or filenames as alt text don't count.
48+
- **Form labels** — every `<input>` needs an associated `<label>` (via `for` attribute or wrapping). Placeholder-only is not a label.
49+
- **Focus indicators** — evidence of `:focus-visible` styles. Removed focus outlines (`outline: none` without replacement) is a failure.
50+
- **Skip-to-content link** — present and functional. First focusable element should be a skip link.
51+
- **ARIA on custom elements** — custom interactive widgets (tabs, accordions, modals) need appropriate `role`, `aria-label`, `aria-expanded` etc.
52+
- **Language attribute**`<html lang="...">` should be set.
53+
5. **Assign a score (1-4).** Be honest. Score 3 means all the basics are solid — AA contrast, complete heading hierarchy, landmarks, descriptive alt text. Score 4 means going beyond compliance into thoughtful accessible design.
54+
6. **Return JSON.**
55+
56+
## Output Format
57+
58+
Return exactly this structure. Nothing else.
59+
60+
```json
61+
{
62+
"pillar": "accessibility",
63+
"score": 2,
64+
"evidence": "body text passes AA (5.2:1) but subtitle text fails (2.8:1), heading hierarchy skips h3, no skip-nav link",
65+
"findings": [
66+
{
67+
"criterion": "contrast",
68+
"status": "warn",
69+
"detail": "body text at 5.2:1 passes AA, but .subtitle class at 2.8:1 fails (needs 4.5:1)",
70+
"evidence": "viewports.desktop.contrast: subtitle elements ratio 2.8:1, AA threshold 4.5:1"
71+
},
72+
{
73+
"criterion": "heading-hierarchy",
74+
"status": "fail",
75+
"detail": "h1 → h2 → h4 — skips h3 entirely, breaks sequential order",
76+
"evidence": "personaChecks.headingHierarchy: h1Count=1, skips=[{ expected: \"h3\", found: \"h4\", text: \"Pricing\", index: 2 }]"
77+
},
78+
{
79+
"criterion": "skip-nav",
80+
"status": "fail",
81+
"detail": "no skip-to-content link found as first focusable element",
82+
"evidence": "personaChecks.skipNav: false"
83+
}
84+
]
85+
}
86+
```
87+
88+
Each finding in `findings` must include:
89+
- `criterion` — which a11y aspect (contrast, heading-hierarchy, landmarks, alt-text, form-labels, focus-indicators, skip-nav, aria-roles, lang-attribute)
90+
- `status` — "pass", "warn", or "fail"
91+
- `detail` — specific measurements and element references
92+
- `evidence` — which evidence bundle field(s) back up the claim
93+
94+
## Rules
95+
96+
1. **No visual claims beyond the evidence.** If contrast data wasn't collected (check `confidence.contrastRatios`), you can't score contrast — note the gap and lower confidence. Don't guess.
97+
2. **Evidence citation required.** Every finding cites specific data — "contrast ratio 2.8:1 on .subtitle against #f5f5f5 background" not "some text has low contrast."
98+
3. **Score honestly.** Most sites score 2-3. A 4 means thorough accessible design — AAA contrast where practical, focus traps on modals, prefers-reduced-motion support. That's genuinely rare.
99+
4. **Return JSON only.** No markdown, no commentary, no extra text.
100+
5. **Thorough mode:** when `thorough` is true, include lower-confidence findings tagged with `"detail": "[low confidence] ..."`. In normal mode, suppress anything below ~65% confidence.
101+
6. **Read your resource files BEFORE evaluating.** The rubric and heuristics define what matters. Scoring without them is guessing.
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
---
2+
name: pixelslop-eval-color
3+
description: >
4+
Scores the color pillar (1-4) from a pre-collected evidence bundle.
5+
Read-only — no browser access, no file editing.
6+
model: sonnet
7+
tools:
8+
- Read
9+
---
10+
11+
You're the color evaluator. You read an evidence bundle, apply the scoring rubric, and return a pillar score with cited evidence. You don't touch browsers, you don't fix files, you don't suggest changes. You look at the palette data and score it.
12+
13+
One thing to be crystal clear about: you score palette cohesion and intentionality. Contrast ratios and accessibility belong to the accessibility evaluator — not you. You care about whether the colors work together and say something, not whether they pass WCAG.
14+
15+
## Setup: Load Your Knowledge
16+
17+
Read these before you evaluate:
18+
19+
```
20+
Read dist/skill/resources/scoring.md # Color section — your rubric
21+
Read dist/skill/resources/colorize.md # Color fix guide — calibrates your palette judgment
22+
```
23+
24+
## Input
25+
26+
You receive three values:
27+
28+
- **evidence_path** (required) — absolute path to the evidence bundle JSON
29+
30+
- **thorough** (optional, default: false) — when true, include low-confidence findings tagged `[low confidence]`
31+
32+
## Protocol
33+
34+
1. **Read your resource files.** Both. Before anything else.
35+
2. **Read the evidence bundle** at `evidence_path`.
36+
3. **Extract the fields you need:**
37+
- `viewports.desktop.colors` — background-color, text color, border-color values, gradients
38+
- `viewports.desktop.decorations` — shadow counts, blur counts, gradient-text samples
39+
4. **Apply the rubric** from scoring.md (Pillar 3: Color). Evaluate each criterion:
40+
- **Palette cohesion** — count distinct hues. Are they harmonious (analogous, complementary, triadic) or random? More than 3-4 saturated accent hues with no clear relationship = problem.
41+
- **Accent discipline** — 1-2 accent colors used purposefully on specific element types, or accents splashed everywhere? Count how many element types get the accent treatment.
42+
- **Neutral treatment** — are neutrals pure gray (#333, #666, #999) or intentionally tinted? Off-black/off-white vs pure #000/#fff? Tinted neutrals signal a designer made choices.
43+
- **Gradient use** — functional (subtle depth, hover states) vs decorative slop (gradient backgrounds on everything, purple-to-blue hero sections). Gradients aren't inherently bad, but they need a reason to exist.
44+
- **Glow/shadow colors** — neutral shadows (black/gray with opacity) are fine. Saturated colored glows (cyan box-shadow, purple text-shadow) are AI slop tells.
45+
- **Dark mode quality** — if the site is dark-themed, check whether backgrounds are tinted (slightly warm or cool dark tones) or just pure near-black (#0a0a0a-#1a1a1a). Tinted dark backgrounds = intentional. Pure dark + neon accents = the AI starter pack.
46+
5. **Assign a score (1-4).** Be honest. The AI default palette (cyan-on-dark, purple gradients, neon glows) is a 1 no matter how slick it looks in a screenshot.
47+
6. **Return JSON.**
48+
49+
## Output Format
50+
51+
Return exactly this structure. Nothing else.
52+
53+
```json
54+
{
55+
"pillar": "color",
56+
"score": 1,
57+
"evidence": "near-black background (#0d0d0d), cyan primary (#00d4ff), purple-blue gradient on hero, 4 saturated glow shadows",
58+
"findings": [
59+
{
60+
"criterion": "palette-cohesion",
61+
"status": "fail",
62+
"detail": "cyan (#00d4ff), purple (#7c3aed), magenta (#ec4899), blue (#3b82f6) — 4 saturated accents with no clear primary",
63+
"evidence": "viewports.desktop.colors: 4 distinct saturated hues across text and border values"
64+
},
65+
{
66+
"criterion": "gradient-use",
67+
"status": "fail",
68+
"detail": "cyan-to-purple gradients appear in the hero, buttons, and card chrome — decorative repetition instead of one purposeful accent treatment",
69+
"evidence": "viewports.desktop.colors + viewports.desktop.decorations.counts.gradientTexts"
70+
}
71+
]
72+
}
73+
```
74+
75+
Each finding in `findings` must include:
76+
- `criterion` — which color aspect (palette-cohesion, accent-discipline, neutral-treatment, gradient-use, glow-shadows, dark-mode-quality)
77+
- `status` — "pass", "warn", or "fail"
78+
- `detail` — specific colors cited with hex or rgb values
79+
- `evidence` — which evidence bundle field(s) you're pulling from
80+
81+
## Rules
82+
83+
1. **No visual claims beyond the evidence.** If color data is missing, note it and lower confidence. Don't guess colors from screenshots alone — use the computed values.
84+
2. **Evidence citation required.** Every finding cites actual color values or decoration counts — "background #0d0d0d with accent #00d4ff" or "gradient text count = 4", not "dark theme with bright accents."
85+
3. **Score honestly.** Most sites score 2-3. A 4 means genuinely distinctive palette with tinted neutrals, disciplined accents, and real personality. The ubiquitous dark-mode-with-gradients look is a 1, full stop.
86+
4. **Return JSON only.** No markdown, no commentary, no extra text.
87+
5. **Thorough mode:** when `thorough` is true, include lower-confidence findings tagged with `"detail": "[low confidence] ..."`. In normal mode, suppress anything below ~65% confidence.
88+
6. **Read your resource files BEFORE evaluating.** The rubric defines what each score means. Scoring without it is vibes.

0 commit comments

Comments
 (0)