EVALUATIONS.md

<style> .g { color: #22863a; font-weight: bold; } .r { color: #cb2431; font-weight: bold; } </style>

Evaluations

Summary

Skill	Version	Assertions	With Skill	Without Skill	Delta	Uplift	Concern
humaniseur-fr	v1.0.0	71	99%	85%	+14pp	1.17×	Low delta, High without
frontend-design-deslop	v1.0.0	53	85%	57%	+28pp	1.49×	Low delta, Low with-skill score
linkedin-ghostwriting	v1.0.0	46	98%	67%	+30pp	1.45×	Low delta, High without
technical-article-writer	v1.1.0	95	100%	67%	+33pp	1.48×	High without
press-release-writer	v1.1.0	66	95%	61%	+35pp	1.57×	Low with-skill score
conventional-git	v1.0.0	50	100%	64%	+36pp	1.56×
promql-cli	v1.0.0	36	100%	61%	+39pp	1.64×
substack-ghostwriting	v1.1.0	101	100%	50%	+50pp	2.02×
deep-research	v1.0.0	43	100%	49%	+51pp	2.05×
snyk-agent-scan-compliance	v1.0.0	85	100%	49%	+51pp	2.02×
skill-progressive-disclosure-design	v1.0.0	59	97%	39%	+58pp	2.49×
site-launch-checklist	v1.0.0	101	99%	40%	+59pp	2.50×
training-report	v1.0.0	67	99%	37%	+61pp	2.64×
influence-and-negotiation	v1.0.0	230	100%	32%	+68pp	3.11×
Total (14 skills)		1103	98%	50%	+48pp	1.96×

conventional-git — v1.0.0

With Skill	Without Skill	Delta	Assertions
100%	64%	+36pp	50

Full breakdown (50 assertions)

Model: claude-sonnet-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: worktree in branch name — skill forbids it, model echoes it	5/5	3/5
1.1	branch name does NOT contain 'worktree'	✓	✓
1.2	branch name starts with 'feat/'	✓	✗ uses 'feature/' prefix
1.3	lowercase, hyphens only	✓	✓
1.4	explains why 'worktree' must not appear in branch names	✓	✗ no explanation given
1.5	description part ≤ 50 chars	✓	✓
	Eval 2: Issue number prefix in branch — model already knows this	5/5	5/5
2.1	branch name includes issue number '87'	✓	✓
2.2	issue number appears BEFORE the description	✓	✓
2.3	branch name starts with 'fix/'	✓	✓
2.4	lowercase, hyphens only	✓	✓
2.5	description part ≤ 50 chars	✓	✓
	Eval 3: Scope as noun not verb — model uses gerund without skill	6/6	3/6
3.1	scope does NOT end in '-ing'	✓	✓
3.2	scope does NOT contain 'adding', 'implementing', 'creating'	✓	✓
3.3	scope is a short noun	✓	✗ no scope used (no CC format)
3.4	type is 'feat'	✓	✗ no type prefix — plain 'Add UserAuthService...'
3.5	description uses imperative mood	✓	✓
3.6	description starts with lowercase letter	✓	✗ starts with capital 'Add'
	Eval 4: Squash merge PR title — model misses the format requirement	4/4	3/4
4.1	warns PR title becomes the single commit message after squash	✓	✓
4.2	explicitly states PR title must follow Conventional Commits format	✓	✓
4.3	flags 'Add user dashboard feature' as invalid (missing type prefix)	✓	✓
4.4	suggests a corrected PR title (e.g. 'feat: add user dashboard')	✓	✗ no corrected title given
	Eval 5: Closes #42 in footer not subject — model puts it in subject	5/5	2/5
5.1	subject line does NOT contain '#42' or closing keyword	✓	✗ '(issue #42)' in subject
5.2	'Closes #42' appears in the footer	✓	✗ no footer — issue ref only in subject
5.3	subject uses imperative mood	✓	✓
5.4	type is 'fix'	✓	✗ no CC type prefix
5.5	no trailing period	✓	✓
	Eval 6: Deps upgrade → 'build' not 'chore' — model defaults to chore	5/5	3/5
6.1	type is 'build'	✓	✗ type is 'chore'
6.2	type is NOT 'chore'	✓	✗ type IS 'chore'
6.3	description uses imperative mood	✓	✓
6.4	description starts with lowercase	✓	✓
6.5	no trailing period	✓	✓
	Eval 7: Revert body — keep 'This reverts commit hash' — model rewrites body	5/5	4/5
7.1	type is 'revert' (CC format 'revert:')	✓	✗ uses git default 'Revert "..."' format, not 'revert:' type
7.2	body includes 'This reverts commit'	✓	✓
7.3	hash 'abc1234f' referenced in body	✓	✓
7.4	description uses imperative mood	✓	✓
7.5	no trailing period	✓	✓
	Eval 8: Breaking change — body-only invisible to tools — model omits ! and footer	5/5	3/5
8.1	includes '!' after type/scope OR 'BREAKING CHANGE:' in footer	✓	✗ no ! and no BREAKING CHANGE: footer — body-only
8.2	does NOT rely solely on body text to signal breaking change	✓	✗ breaking change only in body: 'Old config files are no longer compatible'
8.3	type is NOT 'fix'	✓	✓
8.4	description uses imperative mood	✓	✓
8.5	description starts with lowercase	✓	✓
	Eval 9: One concern per branch — model says 'use fix:' without flagging alignment	4/4	1/4
9.1	does NOT lead with 'use fix:' without discussing branch alignment	✓	✗ leads with 'Short Answer: Use fix: for the bug fix commit'
9.2	recommends separate fix/ branch OR discusses tradeoff	✓	✓
9.3	explains that mixing fix: on feat/ branch obscures the changelog	✓	✗ discusses reverse concern (feat: for bug) not fix: on feat/ branch
9.4	if staying on feature branch, says to use feat: not fix:	✓	✗ recommends fix: on feat/ branch without alignment explanation
	Eval 10: Cross-repo issue closing — model already knows owner/repo#N format	5/5	5/5
10.1	uses Closes/Fixes/Resolves keyword	✓	✓
10.2	reference format is 'owner/repo#number'	✓	✓
10.3	does NOT use just '#99'	✓	✓
10.4	does NOT use just 'frontend#99'	✓	✓
10.5	closing reference in footer, not subject	✓	✓

linkedin-ghostwriting — v1.0.0

With Skill	Without Skill	Delta	Assertions
98%	67%	+31pp	46

Full breakdown (46 assertions)

Model: claude-sonnet-4-6 — 1 run each — graded by human-as-judge

#	Assertion	With	Without
	Eval 1: Newsletter growth — interview before writing	6/6	0/6
1.1	asks questions before writing a post (no post content in response)	✓	✗ wrote full post
1.2	asks 8 or more distinct questions	✓	✗
1.3	asks for specific before/after metrics or exact numbers	✓	✗
1.4	asks about the mechanism/process (how it was achieved)	✓	✗
1.5	asks about the target audience	✓	✗
1.6	asks about the CTA or business goal of the post	✓	✗
	Eval 2: SaaS churn hooks — 3-5 options, no rhetorical questions	5/5	4/5
2.1	proposes 3 or more hook options	✓	✓
2.2	each hook is a short standalone line — no full post body	✓	✓
2.3	waits for user to choose a hook (does not write the full post)	✓	✓
2.4	no rhetorical questions in any hooks (no '?' at end of hook lines)	✓	✗ Hook 3 ends with '?'
2.5	at least one hook includes specific numbers (8%, 2%, 6 months)	✓	✓
	Eval 3: Full post body — active voice, directive CTA, short paragraphs	6/6	6/6
3.1	post starts with or closely follows the selected hook	✓	✓
3.2	no rhetorical questions in the post body	✓	✓
3.3	post uses active voice	✓	✓
3.4	has a clear, directive CTA (not an open-ended question)	✓	✓
3.5	paragraphs are short (2 visual lines or fewer)	✓	✓
3.6	does not contain 'very', 'really', or 'incredibly'	✓	✓
	Eval 4: Leadership topic — push back on missing metrics	4/4	2/4
4.1	does not write a generic post without metrics	✓	✓
4.2	asks for quantified before/after results or measurable outcomes	✓	✗ de-emphasizes metrics ("even if it's not a metric")
4.3	asks for a specific scene or moment	✓	✓
4.4	asks for a counter-intuitive insight	✓	✗ not asked
	Eval 5: Remote work culture — interview, not generic post	4/4	0/4
5.1	does not immediately write a generic thought-leadership post	✓	✗ wrote generic post immediately
5.2	asks about the user's personal experience with remote work	✓	✗
5.3	asks for specific data, outcomes, or evidence	✓	✗
5.4	asks what specific insight the user wants to challenge	✓	✗
	Eval 6: Full context — propose hooks before full post	3/4	3/4
6.1	proposes hooks before writing the full post (does not skip Phase 2)	✗ skipped hook selection, wrote post directly	✗ wrote full post immediately
6.2	if full post written: CTA is directive, not open-ended	✓	✓
6.3	if full post written: no empty phrases ('digital landscape', etc.)	✓	✓
6.4	if full post written: no ternary hook structure	✓	✓
	Eval 7: Cold email hooks — numbers, no rhetorical questions	4/4	4/4
7.1	proposes 3 or more hook options	✓	✓
7.2	no rhetorical questions in any hooks	✓	✓
7.3	hooks include specific numbers (10,000, 38%)	✓	✓
7.4	hooks reveal result but not full method — maintaining tension	✓	✓
	Eval 8: Weak post — remove filler, suggest real data	5/5	4/5
8.1	removes 'very', 'really', or similar filler adverbs	✓	✓
8.2	removes empty phrases like 'digital landscape'	✓	✓
8.3	uses active voice throughout	✓	✓
8.4	result is noticeably shorter/tighter than the original	✓	✗ rewrite is longer than original
8.5	suggests the user needs specific numbers/data to make the post credible	✓	✓
	Eval 9: Weak CTA — flag and replace	3/3	3/3
9.1	identifies the open-ended question as a weak CTA	✓	✓
9.2	suggests a directive alternative ('Comment X', 'DM me', 'Save this')	✓	✓
9.3	explains why directive CTAs outperform open-ended questions	✓	✓
	Eval 10: Post body from given hook — active voice, directive CTA	5/5	5/5
10.1	post starts with or immediately after the given hook	✓	✓
10.2	no rhetorical questions in the post body	✓	✓
10.3	CTA is directive: 'Comment QUESTION' or similar	✓	✓
10.4	paragraphs are short (2 visual lines max)	✓	✓
10.5	active voice throughout	✓	✓

promql-cli — v1.0.0

With Skill	Without Skill	Delta	Assertions
100%	61%	+39pp	36

Full breakdown (36 assertions)

Model: claude-sonnet-4-6 — 1 run each — graded by human-as-judge

#	Assertion	With	Without
	Eval 1: Current request count — rate() not raw counter	3/3	1/3
1.1	uses rate() — does NOT suggest querying raw counter without rate()	✓	✗ suggests raw counter first
1.2	includes a time window in rate() e.g. [5m]	✓	✓
1.3	explains why raw counter values are not meaningful	✓	✗ no explanation provided
	Eval 2: Debug slow pods — isolate by instance, not avg across fleet	4/4	3/4
2.1	recommends filtering/isolating by pod or instance label	✓	✓
2.2	does NOT suggest avg() or sum() across all pods as first step	✓	✓
2.3	suggests histogram or rate-based latency metrics	✓	✓
2.4	includes label matcher syntax to filter by specific pod or instance	✓	✗ groups by pod but no specific instance filter
	Eval 3: p99 histogram — preserve le label	4/4	4/4
3.1	uses histogram_quantile(0.99, ...)	✓	✓
3.2	preserves the 'le' label in by() clause	✓	✓
3.3	applies rate() to the histogram _bucket series	✓	✓
3.4	does NOT suggest dropping 'le' before histogram_quantile	✓	✓
	Eval 4: Error rate trend 2h — range query, --output graph	4/4	3/4
4.1	uses --start 2h (or equivalent range query flag)	✓	✓
4.2	recommends --output graph for visualizing the trend	✓	✗ not mentioned
4.3	uses rate() on the error counter metric	✓	✓
4.4	does NOT just suggest a plain instant query	✓	✓
	Eval 5: Bearer token — never in config file, chmod 600	4/4	1/4
5.1	does NOT write config file content containing the token value	✓	✓
5.2	instructs user to create the config file manually themselves	✓	✗ uses env vars, no config file guidance
5.3	mentions chmod 600 to protect the credentials file	✓	✗ not mentioned
5.4	does NOT suggest passing the token as a CLI flag	✓	✗ suggests --bearer-token CLI flag
	Eval 6: CPU filter — labels in innermost selector	3/3	3/3
6.1	places label matchers in the innermost metric selector	✓	✓
6.2	filtering appears before any aggregation function	✓	✓
6.3	does NOT suggest applying label filters outside the selector	✓	✓
	Eval 7: Connection refused — config problem, not query problem	4/4	4/4
7.1	identifies this as a host configuration problem	✓	✓
7.2	guides the user to check their host configuration	✓	✓
7.3	does NOT suggest modifying the PromQL query to fix the error	✓	✓
7.4	does NOT create a config file with credentials	✓	✓
	Eval 8: List metrics — promql metrics subcommand	3/3	0/3
8.1	suggests the 'promql metrics' subcommand	✓	✗ suggests curl then PromQL regex instead
8.2	does NOT suggest {__name__=~".+"} as the primary approach	✓	✗ suggests it as the promql-cli approach
8.3	shows correct CLI syntax for the metrics subcommand	✓	✗ does not know the subcommand
	Eval 9: Active connections — gauge vs counter distinction	3/3	0/3
9.1	distinguishes between gauge (raw value OK) and counter (needs rate())	✓	✗ no gauge/counter distinction
9.2	if gauge: confirms raw value is appropriate for active connections	✓	✗ not addressed
9.3	suggests checking metric type with 'promql meta metric'	✓	✗ not mentioned
	Eval 10: Before/after deploy — range query, --output graph	4/4	3/4
10.1	uses a range query with --start flag	✓	✓
10.2	recommends --output graph to visualize the trend	✓	✗ not mentioned
10.3	uses rate() for request/error rate metrics	✓	✓
10.4	does NOT just suggest a single instant query	✓	✓

substack-ghostwriting — v1.1.0

With Skill	Without Skill	Delta	Assertions
100%	49%	+51pp	101

Full breakdown (101 assertions)

Model: claude-opus-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: Evergreen content → web post, not newsletter	5/5	2/5
1.1	identifies this as a web post (not newsletter issue) based on 'evergreen' keyword	✓	✗ treats as generic Substack post
1.2	mentions SEO title field or SEO optimization specific to web posts	✓	✗ no Substack SEO title field
1.3	suggests a URL slug (short, keyword-rich)	✓	✗ not suggested
1.4	does NOT treat this as a newsletter issue (no subject line / preview text focus)	✓	✓
1.5	mentions table of contents for long content or web-specific formatting	✓	✓
	Eval 2: Ghostwriting — interview before writing	5/5	0/5
2.1	does NOT write a full post immediately	✓	✗ wrote draft immediately
2.2	asks for voice samples (transcripts, past writing, or media appearances)	✓	✗ not asked
2.3	prioritizes transcripts over published writing as voice source	✓	✗ no source prioritization
2.4	mentions building a voice guide before drafting	✓	✗ no voice guide concept
2.5	asks about the CEO's natural speaking/writing patterns	✓	✗ not asked
	Eval 3: Gmail truncation warning for long newsletter	5/5	1/5
3.1	warns about Gmail truncation at ~102KB / ~3000 words	✓	✗ no truncation warning
3.2	suggests keeping content under ~3000 words or acknowledges truncation risk	✓	✗ complies with 4000-word request
3.3	uses short paragraphs (2-3 sentences) for email readability	✓	✓
3.4	suggests code blocks under 10 lines with links to Gist for longer code	✓	✗ no Gist links suggested
3.5	recommends a TL;DR at top for issues over 1500 words	✓	✗ no TL;DR recommendation
	Eval 4: Web post SEO — separate fields awareness	5/5	3/5
4.1	mentions the separate SEO title field (distinct from main title)	✓	✗ no Substack SEO title field
4.2	writes or recommends an SEO description of 150-160 characters	✓	✓
4.3	suggests a URL slug (short, keyword-rich, no dates)	✓	✓
4.4	mentions Substack has high domain authority for SEO	✓	✗ not mentioned
4.5	recommends internal linking to other Substack posts	✓	✓
	Eval 5: Algorithm — subscriptions not engagement	5/5	2/5
5.1	explains algorithm optimizes for subscriptions/paid conversions, NOT engagement	✓	✗ assumes engagement-based algorithm
5.2	explicitly discourages engagement bait, outrage, or hot takes	✓	✓
5.3	mentions Recommendations from other publications as a top growth lever	✓	✗ generic cross-promotion only
5.4	mentions Notes as a testing pipeline for long-form content	✓	✗ not mentioned
5.5	references quality/consistency as primary growth driver	✓	✓
	Eval 6: Notes — not threads, concise standalone	5/5	0/5
6.1	advises against thread-style content on Notes	✓	✗ helps adapt thread format
6.2	recommends concise Notes (2-5 sentences)	✓	✗ supports long-form threads
6.3	suggests using Notes as testing pipeline for long-form posts	✓	✗ not mentioned
6.4	mentions likes as the primary Notes ranking signal	✓	✗ no algorithm details
6.5	explains that Notes should have standalone value	✓	✗ helps create thread series
	Eval 7: Monetization — don't paywall under 1000 subs	5/5	1/5
7.1	advises against launching paid tier at 300 subscribers	✓	✗ cautiously supportive
7.2	explains growth compounds faster than paid conversion at small scale	✓	✗ not articulated
7.3	references the ~3% free-to-paid conversion baseline	✓	✗ no conversion benchmarks
7.4	recommends keeping everything free and focusing on growth first	✓	✗ suggests testing paid content
7.5	suggests alternative monetization paths (coaching, consulting)	✓	✓
	Eval 8: Social distribution — don't announce, lead with insight	5/5	1/5
8.1	does NOT start with 'I just published' or similar announcement	✓	✗ default promotion pattern
8.2	leads with an insight, hook, or the 90% result	✓	✗ leads with announcement
8.3	gives 30-40% of value upfront so post works standalone	✓	✗ structured as teaser
8.4	uses line breaks between sentences for LinkedIn readability	✓	✓
8.5	post works even if the reader never clicks the link	✓	✗ requires click for value
	Eval 9: Voice matching — single source warning	5/5	0/5
9.1	warns against anchoring voice analysis on a single source	✓	✗ works with single source
9.2	asks for additional sources (transcripts, social, Slack, talks)	✓	✗ not asked
9.3	explains that blog posts are edited and may differ from natural voice	✓	✗ no distinction
9.4	prioritizes transcripts or spoken content over published writing	✓	✗ no source prioritization
9.5	does NOT immediately write in a 'captured' voice from one blog post	✓	✗ attempts to write immediately
	Eval 10: Paywall placement — value above the fold	4/4	4/4
10.1	advises against paywalling the intro/hook/beginning	✓	✓
10.2	recommends giving enough free value above the fold	✓	✓
10.3	explains that free content drives growth and recommendations	✓	✓
10.4	suggests placing paywall after delivering strong value	✓	✓
	Eval 11: Subject line + preview text complementarity	4/4	1/4
11.1	preview text does NOT repeat or paraphrase the subject line	✓	✗ paraphrases cost-cutting theme
11.2	preview text complements/extends with new information	✓	✗ echoes same proposition
11.3	preview text is approximately 90 characters or under	✓	✗ defaults to meta-description length
11.4	explains subject line + preview text should work as a pair	✓	✓
	Eval 12: Substack special blocks — contextual suggestions	5/5	1/5
12.1	suggests a subscribe button for free readers	✓	✗ no Substack subscribe block
12.2	suggests a share button after the strong insight	✓	✗ no Substack share block
12.3	mentions pull quote for the most quotable line	✓	✓
12.4	recommends reply prompt (replies are an algorithm signal)	✓	✗ no algorithm signal awareness
12.5	references Substack-specific content blocks	✓	✗ generic formatting only
	Eval 13: Email paragraph walls — 4-5 sentence paragraphs in newsletter	4/4	4/4
13.1	flags the paragraph as too long for email newsletter format	✓	✓
13.2	recommends breaking into 2-3 sentence paragraphs	✓	✓
13.3	explains email clients render narrow columns making long paragraphs feel like walls	✓	✓
13.4	does NOT say the paragraph is fine as-is	✓	✓
	Eval 14: Notes don't support video — user wants to upload video	4/4	3/4
14.1	warns that Substack Notes does not support video uploads	✓	✓
14.2	suggests alternatives (GIF, screenshots, link to video hosted elsewhere)	✓	✓
14.3	still recommends using Notes as a testing pipeline for the idea	✓	✗ no testing pipeline concept
14.4	does NOT help upload video to Notes as if it's supported	✓	✓
	Eval 15: Long code in newsletter — 30+ lines inline	5/5	4/5
15.1	advises against including 35 lines of code in a newsletter email	✓	✓
15.2	recommends linking to a GitHub Gist or repo for the full code	✓	✓
15.3	suggests including a short 2-5 line excerpt inline for context	✓	✓
15.4	explains code rendering varies by email client (Outlook may strip formatting)	✓	✓
15.5	mentions the ~10 line threshold for email code blocks	✓	✗ no specific threshold mentioned
	Eval 16: Embeds in newsletter issue — YouTube + tweets	5/5	5/5
16.1	warns that embeds (YouTube, tweets) degrade to links/previews in email	✓	✓
16.2	recommends not relying on embeds for critical content in newsletter issues	✓	✓
16.3	suggests alternatives (screenshots of tweets, link to video with description)	✓	✓
16.4	distinguishes between web rendering (embeds work) and email rendering (they don't)	✓	✓
16.5	does NOT just say 'looks great, go ahead'	✓	✓
	Eval 17: URL slug with dates for evergreen web post	5/5	4/5
17.1	advises removing the date from the URL slug	✓	✓
17.2	explains dates in URLs make evergreen content look stale over time	✓	✓
17.3	suggests a dateless alternative	✓	✓
17.4	keeps slug to 3-6 words, keyword-rich	✓	✗ no specific 3-6 word guideline
17.5	flags the date in the title as problematic for evergreen content	✓	✓
	Eval 18: Gaming the Recommendations system algorithmically	5/5	4/5
18.1	explains the Recommendations system is fully human-curated, not algorithmic	✓	✓
18.2	clarifies you cannot game Recommendations through engagement signals	✓	✓
18.3	describes the actual path: write quality content, engage with other writers, build genuine relationships	✓	✓
18.4	mentions Recommendations appear during subscribe flow (highest conversion moment)	✓	✗ describes mechanism but not its conversion significance
18.5	does NOT describe algorithmic tricks to optimize for recommendations	✓	✓
	Eval 19: Prioritizing Substack Boost at 200 subscribers	5/5	3/5
19.1	advises against prioritizing Boost at 200 subscribers	✓	✓
19.2	explains Boost is low impact at small scale (Tier 4 or equivalent)	✓	✗ no tier ranking or systematic framework
19.3	recommends higher-impact levers: consistent publishing, Notes as testing pipeline, Recommendations	✓	✓
19.4	mentions Recommendations from larger publications as the highest-impact growth lever	✓	✗ not identified as highest-impact
19.5	does NOT endorse Boost as the primary growth strategy at this stage	✓	✓
	Eval 20: Subject line too long for mobile — 80+ chars	5/5	4/5
20.1	flags the subject line as too long (especially for mobile at ~35-40 chars visible)	✓	✓
20.2	recommends front-loading the interesting part in the first 35-40 characters	✓	✓
20.3	suggests a shorter alternative under 60 characters	✓	✓
20.4	mentions specific character limits for different clients (mobile, Gmail, Yahoo)	✓	✗ gives generic '40-60' range, no per-client breakdown
20.5	does NOT just evaluate the subject line on content quality alone	✓	✓
	Eval 21: Sections overload at 5 posts	5/5	2/5
21.1	advises against creating 7 sections with only 5 posts	✓	✓
21.2	recommends starting with 2-4 sections maximum	✓	✗ says '1-2' or '2-3' (different threshold)
21.3	explains sections should wait until 20+ posts to be meaningful	✓	✗ says '15-25+' (different threshold)
21.4	notes each section gets its own RSS feed and subscribe option	✓	✗ not mentioned
21.5	suggests consolidating into fewer, broader sections	✓	✓

technical-article-writer — v1.1.0

With Skill	Without Skill	Delta	Assertions
100%	67%	+33pp	95

Full breakdown (95 assertions)

Model: claude-opus-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: Phase-gated workflow — ask before writing	5/5	0/5
1.1	does NOT write a full article immediately	✓	✗ writes full blog post
1.2	asks about the specific aspect of Java performance	✓	✗ covers multiple aspects broadly
1.3	asks about the target audience	✓	✗ defaults to mid-level
1.4	asks about the content type (tutorial, benchmark, explainer)	✓	✗ defaults to generic blog format
1.5	asks about the thesis or key takeaway	✓	✗ picks generic thesis
	Eval 2: Idea quality filter — push back on textbook topic	5/5	0/5
2.1	pushes back on the generic 'basics of Docker Compose' angle	✓	✗ complies with the request
2.2	references or applies a novelty filter	✓	✗ no novelty filter concept
2.3	suggests finding a specific struggle, surprise, or insight	✓	✗ proceeds with basics
2.4	does NOT immediately write a basic tutorial	✓	✗ starts writing or outlining immediately
2.5	helps the user find a more compelling angle	✓	✗ no angle exploration
	Eval 3: 10 title variants, not 3-5	5/5	2/5
3.1	generates 10 or more title variants	✓	✗ generates 5-7 titles
3.2	uses different hook strategies across titles	✓	✓
3.3	includes specific technical keywords (MongoDB, PostgreSQL)	✓	✓
3.4	avoids superlatives ('ultimate', 'complete')	✓	✗ uses 'The Complete Guide...' pattern
3.5	provides brief notes or rankings on why each works	✓	✗ lists without detailed rationale
	Eval 4: Content type matching — Bug Hunt template	5/5	4/5
4.1	identifies or recommends the Bug Hunt content type	✓	✗ suggests generic narrative structure
4.2	includes a 'first hypothesis' or 'dead ends' section	✓	✓
4.3	recommends building tension with wrong hypotheses	✓	✓
4.4	includes the fix/solution as a distinct section	✓	✓
4.5	ends with a generalizable lesson beyond this specific bug	✓	✓
	Eval 5: Show-then-tell principle for code	4/4	2/4
5.1	leads with a code example BEFORE the conceptual explanation	✓	✗ explains concept first, then code
5.2	code snippet is under 20 lines and focused on one concept	✓	✓
5.3	annotates or explains non-obvious lines in the code	✓	✓
5.4	conceptual explanation follows the code example	✓	✗ concept-first order
	Eval 6: Steelman for opinion piece	4/4	2/4
6.1	includes a section addressing the strongest arguments FOR microservices	✓	✓
6.2	the steelman is substantive, not a straw man	✓	✗ brief dismissive treatment when told to 'go all in'
6.3	uses concrete examples, not just abstract reasoning	✓	✓
6.4	quantifies claims where possible	✓	✗ qualitative arguments without specific numbers
	Eval 7: Anti-pattern detection — burying the lede	5/5	5/5
7.1	moves the 73% finding to the opening	✓	✓
7.2	removes the 'rapidly evolving tech landscape' preamble	✓	✓
7.3	explicitly calls out the buried lede as the problem	✓	✓
7.4	removes 'In this article, we'll explore' meta-framing	✓	✓
7.5	the rewritten intro hooks the reader immediately	✓	✓
	Eval 8: Copywriting framework application	4/4	3/4
8.1	uses or references a named copywriting framework (PAS, AIDA, BAB)	✓	✗ writes intuitively, no named framework
8.2	hook addresses the reader's pain before presenting the solution	✓	✓
8.3	includes stakes or agitation — cost of slow CI	✓	✓
8.4	intro accomplishes: hook + stakes + promise	✓	✓
	Eval 9: Tutorial completeness — prerequisites and verification	5/5	4/5
9.1	includes an explicit prerequisites section	✓	✓
9.2	shows expected result/output after key steps	✓	✓
9.3	includes a verification section at the end	✓	✗ ends at last implementation step
9.4	includes next steps or related topics section	✓	✓
9.5	steps are numbered and sequential	✓	✓
	Eval 10: Title finalization after full draft	4/4	3/4
10.1	recommends reconsidering the title	✓	✓
10.2	suggests 2-3 alternatives reflecting actual content (data integrity)	✓	✓
10.3	explains titles should be revisited after writing as a methodology	✓	✗ suggests better titles but not as a general principle
10.4	alternatives are more specific than the original	✓	✓
	Eval 11: Diataxis — mixing content types diagnosis	4/4	3/4
11.1	identifies the problem as mixing content types	✓	✓
11.2	references or applies the Diataxis framework	✓	✗ describes problem in general terms only
11.3	recommends splitting into separate pieces	✓	✓
11.4	explains why tutorials should not drift into theory	✓	✓
	Eval 12: Image suggestions with Midjourney specifics	5/5	3/5
12.1	suggests 1-3 images with specific placement	✓	✓
12.2	each image has a stated purpose	✓	✓
12.3	each image has a description of what it should depict	✓	✓
12.4	offers to generate Midjourney prompts	✓	✗ not mentioned
12.5	mentions aspect ratio conventions (16:9 or 3:1 for hero, 3:2 for inline)	✓	✗ no aspect ratios
	Eval 13: "We Rewrote It" missing "what went wrong" section	5/5	4/5
13.1	pushes back on omitting difficulties/failures	✓	✓
13.2	explains the 'what went wrong' section is critical for credibility	✓	✓
13.3	notes that without failures, the article reads like a press release	✓	✓
13.4	suggests specific prompts to surface challenges	✓	✓
13.5	recommends the 'We Rewrote It in X' structure with 'what went wrong' at ~15%	✓	✗ no named template or percentage allocation
	Eval 14: Benchmark article burying methodology	5/5	2/5
14.1	recommends NOT leading with results	✓	✓
14.2	explains methodology must come before results to establish trust	✓	✓
14.3	references or applies the Benchmark/Data-Driven content type structure	✓	✗ no named template
14.4	suggests methodology section at ~20% of article before the results section	✓	✗ no percentage allocation
14.5	notes that 'no trust in setup = no trust in conclusions'	✓	✗ expresses concept differently
	Eval 15: Too many lessons diluting impact	5/5	5/5
15.1	advises limiting to 3-5 lessons maximum	✓	✓
15.2	explains more lessons dilute impact and reduce memorability	✓	✓
15.3	suggests leading with the most surprising lesson, not the first chronologically	✓	✓
15.4	recommends each lesson needs a specific story, not just an abstract principle	✓	✓
15.5	helps prioritize which 3-5 lessons are strongest	✓	✓
	Eval 16: Momentum killers in transitions	5/5	5/5
16.1	identifies ALL three transitions as momentum killers	✓	✓
16.2	names them as an anti-pattern (not just 'could be better')	✓	✓
16.3	suggests specific alternatives using forward reference, question, contrast, or escalation	✓	✓
16.4	provides rewritten examples that pull the reader forward	✓	✓
16.5	does NOT say the transitions are acceptable	✓	✓
	Eval 17: Explainer starting with edge cases	5/5	4/5
17.1	recommends starting with the simplest mental model, not edge cases	✓	✓
17.2	references or applies progressive disclosure principle	✓	✓
17.3	explains starting complex loses readers who need the foundation first	✓	✓
17.4	suggests the Explainer structure with percentage allocations	✓	✗ gives flow but no percentages or named template
17.5	does NOT agree to start with edge cases	✓	✓
	Eval 18: Hook type mismatch — celebration for a bug hunt	5/5	4/5
18.1	suggests a Curiosity or Surprise hook instead of Celebration	✓	✓
18.2	explains celebration hooks resolve tension upfront, killing the narrative	✓	✓
18.3	the suggested hook creates tension (the problem) before revealing the resolution	✓	✓
18.4	references or applies named hook types from a taxonomy	✓	✗ no named taxonomy, just general advice
18.5	does NOT open with 'We just saved $40K/month' or similar celebration	✓	✓
	Eval 19: Wall of code anti-pattern — 50-line block	5/5	5/5
19.1	flags the 50-line code block as a wall of code anti-pattern	✓	✓
19.2	recommends breaking it into smaller annotated chunks (<20 lines each)	✓	✓
19.3	suggests linking to the full config in a repo/Gist	✓	✓
19.4	recommends showing only the interesting/relevant parts inline	✓	✓
19.5	explains unannotated large code blocks lose readers	✓	✓
	Eval 20: PASTOR framework — too much 'offer'	5/5	4/5
20.1	applies PASTOR framework correctly with all components	✓	✓
20.2	devotes ~80% of the intro to the problem/transformation, not the article description	✓	✓
20.3	only ~20% describes what the article covers (the 'offer')	✓	✓
20.4	explains the 80/20 rule for PASTOR intros	✓	✗ no explicit 80/20 ratio stated
20.5	does NOT spend majority of intro on 'what you'll learn' meta-framing	✓	✓

press-release-writer — v1.1.0

With Skill	Without Skill	Delta	Assertions
95%	61%	+34pp	66

Full breakdown (66 assertions)

Model: claude-opus-4-6 — 1 run each — graded by human-as-judge (strict adversarial) — adversarial evals (each has a trap the model falls into without the skill). Note: eval 12 with_skill skipped (API overload), with_skill total is 61.

#	Assertion	With	Without
	1. US product launch — sentence case, AP style, banned phrases, attribution verbs	8/8	4/8
1.1	headline uses sentence case (not Title Case)	✓	✗ Title Case on every word
1.2	lead answers 5W1H in ≤35 words	✓	✗ ~45 words, two sentences
1.3	no banned phrases (thrilled, excited to announce, innovative, etc.)	✓	✓
1.4	attribution verb is only "said"	✓	✓
1.5	no first person outside quotes	✓	✓
1.6	Nashville standalone (no state) per AP style	✓	✗ "Nashville, TN"
1.7	ends with ### or -30-	✓	✗ no end mark
1.8	total length 300-500 words	✓	✓
	2. French market launch — guillemets, AFP dateline, third person	5/5	4/5
2.1	uses French guillemets « » for quotes	✓	✗ English quotation marks
2.2	dateline includes "le" per AFP format	✓	✓
2.3	third person throughout (no nous/notre outside quotes)	✓	✓
2.4	release designation (POUR DIFFUSION IMMEDIATE)	✓	✓
2.5	"A propos de" boilerplate section	✓	✓
	3. Crisis data breach — Care-Control-Commitment, no speculation	6/7	3/7
3.1	first paragraph acknowledges impact before company actions	✓	✗ leads with company actions
3.2	does not speculate on cause or assign blame	✓	✓
3.3	lists specific corrective actions	✓	✓
3.4	next steps with timeline	✓	✗ no timeline for next update
3.5	dedicated media contact	✗ placeholder only	✗ generic press email
3.6	zero positive marketing language	✓	✗ "a leading cloud security provider"
3.7	neutral headline framing	✓	✓
	4. Series B funding — all-four-in-lead, investor quote	5/5	4/5
4.1	first paragraph: amount + round + investor + use of funds	✓	✓
4.2	investor quote from Sequoia	✓	✗ no investor quote
4.3	key metrics (ARR, growth, customers)	✓	✓
4.4	no banned phrases	✓	✓
4.5	attribution verb only "said"	✓	✓
	5. Partnership double dateline — both cities, both boilerplates	5/5	5/5
5.1	double dateline (London and New York)	✓	✓
5.2	both companies in headline	✓	✓
5.3	quotes from both organizations	✓	✓
5.4	both boilerplates	✓	✓
5.5	media contacts for both	✓	✓
	6. Open source milestone — zero marketing, benchmarks, community	5/5	5/5
6.1	zero marketing language	✓	✓
6.2	specific benchmarks (40% faster, 60% less memory)	✓	✓
6.3	acknowledges community/contributors	✓	✓
6.4	license type and/or GitHub URL	✓	✓
6.5	maintainer quote thanks community	✓	✓
	7. US earnings release — GAAP-first, safe harbor, reconciliation	4/5	4/5
7.1	GAAP before non-GAAP	✓	✓
7.2	ticker symbol (NYSE: MEGA)	✓	✓
7.3	forward-looking statement safe harbor	✓	✓
7.4	non-GAAP reconciliation mention	✓	✓
7.5	release timing guidance (before/after market)	✗ not in release body	✗ no timing guidance
	8. Journalist email pitch — subject line, hook, CTA	5/6	3/6
8.1	subject line under 60 characters	✗ 62 chars	✗ ~100 chars
8.2	pitch ≤3 paragraphs	✓	✗ 4+ sections
8.3	full PR pasted in body (not attachment)	✓	✓
8.4	recognizable hook type	✓	✗ "I hope this finds you well"
8.5	specific CTA (interview, exclusive, demo)	✓	✓
8.6	references journalist's beat	✓	✓
	9. German market LIDAR — dpa style, mandatory image, formality	5/5	1/5
9.1	no unsupported superlatives	✓	✗ "führenden", "höchsten Ansprüchen"
9.2	mentions mandatory press image	✓	✗ no image mention
9.3	extremely formal tone	✓	✗ "bahnbrechenden", "setzt Maßstäbe"
9.4	dpa dateline format (CAPS city, period after day)	✓	✗ lowercase city, no double dash
9.5	uses "Pressemitteilung" or equivalent	✓	✓
	10. Local award trap — proportionate tone, no overselling	5/5	3/5
10.1	no "prestigious"/"world-renowned" for local award	✓	✓
10.2	identifies granting organization	✓	✓
10.3	proportionate tone (no overselling)	✓	✗ "exceptional achievement", "raises the bar"
10.4	no banned phrases	✓	✗ "innovative software solutions"
10.5	mentions award criteria/category	✓	✓
	11. Next steps suggestion — distribution, pitch, social, angle	5/5	1/5
11.1	suggests distribution timing/channel	✓	✗
11.2	offers to draft journalist email pitch	✓	✗
11.3	suggests social media teaser	✓	✗
11.4	concrete data point in PR	✓	✓
11.5	angle note explaining news angle choice	✓	✗
	12. Broadcast adaptation — short sentences, contractions, attribution-first	n/a (API overload)	3/5
12.1	majority of sentences ≤12 words	n/a	✗ multiple 14-16 word sentences
12.2	uses contractions	n/a	✗ no contractions
12.3	attribution before quote content	n/a	✓
12.4	numbers written out/rounded	n/a	✓
12.5	pronunciation guide or no abbreviations on first ref	n/a	✓

humaniseur-fr — v1.0.0

With Skill	Without Skill	Delta	Assertions
99%	85%	+14pp	71

Full breakdown (71 assertions)

Model: claude-opus-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: basic AI patterns — model already removes most without skill	7/7	7/7
1.1	no 'dans le paysage' (Pattern 7)	✓	✓
1.2	no 'tournant décisif' (Pattern 1)	✓	✓
1.3	no 'crucial' (Pattern 7)	✓	✓
1.4	no 'essentiel' doublet (Pattern 14)	✓	✓
1.5	no 'constitue' — simple copula (Pattern 8)	✓	✓
1.6	no superficial participle (Pattern 3)	✓	✓
1.7	no 'révolutionnaire'/'nichée' (Pattern 4)	✓	✓
	Eval 2: vague attributions + stacked connectors — model keeps vague sources	5/5	4/5
2.1	no vague attribution (Pattern 5)	✓	✗ « plusieurs observateurs confirment »
2.2	no 'il convient de noter que' (Pattern 24)	✓	✓
2.3	no stacked connectors (Pattern 7)	✓	✓
2.4	no 'contribuant à' (Pattern 3)	✓	✓
2.5	no 'substantiel' (Pattern 7)	✓	✓
	Eval 3: conversation artifacts + challenge/optimism — model keeps vague optimism	5/5	2/5
3.1	no 'N'hésitez pas à' (Pattern 21)	✓	✓
3.2	no 'J'espère que cela' (Pattern 21)	✓	✓
3.3	no generic positive conclusion (Pattern 26)	✓	✗ « La suite s'annonce intéressante »
3.4	no 'Malgré... prospérer' formula (Pattern 6)	✓	✗ « avance malgré les obstacles »
3.5	concrete facts replace vague optimism	✓	✗ no concrete facts
	Eval 4: synonym cycling + copula avoidance + false ranges — model keeps verb list	5/5	4/5
4.1	no 'Non seulement... mais' (Pattern 9)	✓	✓
4.2	no 3+ synonym cycling (Pattern 11)	✓	✗ « planifier, exécuter, créer et analyser »
4.3	no 'constitue'/'fait office de'/'se positionne comme' (Pattern 8)	✓	✓
4.4	no false range structure (Pattern 12)	✓	✓
4.5	no 'robuste et fiable' (Pattern 14)	✓	✓
	Eval 5: anglicisms — model already corrects most without skill	6/6	6/6
5.1	no 'adresse' anglicism (Pattern 13)	✓	✓
5.2	no 'fait du sens' (Pattern 13)	✓	✓
5.3	no 'basiquement' (Pattern 13)	✓	✓
5.4	no 'implémenter' outside IT (Pattern 13)	✓	✓
5.5	no 'impacter' (Pattern 13)	✓	✓
5.6	no 'innovante et avant-gardiste' (Pattern 14)	✓	✓
	Eval 6: academic register preservation — both models handle well	6/6	6/6
6.1	no 'Il est essentiel de noter que' (Pattern 24+7)	✓	✓
6.2	no 'holistique' (Pattern 7)	✓	✓
6.3	no 'impacté' (Pattern 13)	✓	✓
6.4	no 'crucial(e)' (Pattern 7)	✓	✓
6.5	formal/academic register maintained	✓	✓
6.6	complex academic sentence structures	✓	✓
	Eval 7: list→prose + emojis + bold — model already handles well	5/5	5/5
7.1	no bullet+bold+colon pattern (Pattern 17)	✓	✓
7.2	no emojis	✓	✓
7.3	no 'Plongeons dans' (Pattern 7)	✓	✓
7.4	list converted to prose	✓	✓
7.5	no mechanical bold (Pattern 16)	✓	✓
	Eval 8: hedging + filler + generic positive — model keeps optimistic ending	5/5	4/5
8.1	no stacked hedges (Pattern 25)	✓	✓
8.2	no 'en ce qui concerne' (Pattern 24)	✓	✓
8.3	no 'a la capacité de' (Pattern 24)	✓	✓
8.4	no 'il est largement reconnu' (Pattern 5)	✓	✓
8.5	no generic positive ending (Pattern 26)	✓	✗ « il y a de quoi être optimiste »
	Eval 9: formal register + synonym cycling + légitimate connectors — both handle well	6/6	6/6
9.1	no 'À l'ère de' (Pattern 7)	✓	✓
9.2	no 3+ synonym cycling (Pattern 11)	✓	✓
9.3	no 'crucial' (Pattern 7)	✓	✓
9.4	formal register preserved	✓	✓
9.5	no 'riche et variée' (Pattern 14)	✓	✓
9.6	'néanmoins' kept as legitimate French	✓	✓
	Eval 10: em dashes + copula avoidance + doublet — model keeps doublet	5/5	4/5
10.1	max 1 em dash (Pattern 15)	✓	✓
10.2	no 'dispose de' (Pattern 8)	✓	✓
10.3	no 'un levier puissant' (Pattern 7)	✓	✓
10.4	no 'dynamique et en pleine expansion' (Pattern 14)	✓	✗ doublet preserved
10.5	commas/parens replace em dashes	✓	✓
	Eval 11: legal register + filler — both handle well	5/5	5/5
11.1	no 'il convient de souligner' filler (Pattern 24)	✓	✓
11.2	no 'la pierre angulaire' (Pattern 7)	✓	✓
11.3	no 'robuste et fiable' (Pattern 14)	✓	✓
11.4	formal/legal register maintained	✓	✓
11.5	simplifies at least one filler phrase	✓	✓
	Eval 12: sycophancy + conversation artifacts — model keeps artifacts without skill	4/5	1/5
12.1	no sycophantic opener (Pattern 23)	✓	✗ « Bonne question »
12.2	no 'Bien sûr !'/'je serais ravi' (Pattern 21+23)	✓	✗ « Je peux creuser un point précis si vous voulez »
12.3	no knowledge-limitation hedge (Pattern 22)	✗ « D'après les données disponibles »	✓
12.4	no 'Souhaitez-vous que' (Pattern 21)	✓	✗ « si vous voulez »
12.5	substantive content	✓	✗ empty pleasantries, no substance
	Eval 13: French typography — model already knows this	4/4	4/4
13.1	French guillemets « »	✓	✓
13.2	French number format (1 250,50)	✓	✓
13.3	French time format (14 h 30)	✓	✓
13.4	spaces inside guillemets	✓	✓
	Eval 14: heading caps + triplet + participial stacking — model already handles	4/4	4/4
14.1	French heading capitalization (Pattern 18)	✓	✓
14.2	no standalone triplet (Pattern 10)	✓	✓
14.3	no figurative 'vibrant' (Pattern 4)	✓	✓
14.4	no 2+ stacked participial clauses (Pattern 3)	✓	✓

snyk-agent-scan-compliance — v1.0.0

With Skill	Without Skill	Delta	Assertions
100%	49%	+51pp	85

Full breakdown (85 assertions)

Model: claude-sonnet-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: W011 imperative URL — remove "always" + convert to passive	5/5	4/5
1.1	response does NOT contain "Always check" or "always check"	✓	✓
1.2	response does NOT contain an imperative sentence with the agent as subject fetching/visiting the URL	✓	✓
1.3	rewritten sentence expresses the URL as a passive resource or factual statement	✓	✓
1.4	response removes or rewrites the "always" modifier	✓	✓
1.5	response identifies this as a W011 alert	✓	✗ alert taxonomy not in default knowledge
	Eval 2: W012 @latest go install — move to frontmatter, not pin in body	5/5	4/5
2.1	response does NOT suggest keeping @latest in the body	✓	✓
2.2	response recommends moving to metadata.openclaw.install frontmatter	✓	✓
2.3	response does NOT recommend keeping go install in body with pinned version as primary fix	✓	✗ version pin listed as alternative/first option before frontmatter
2.4	response includes correct frontmatter structure: kind, package, bins	✓	✓
2.5	response explains frontmatter is not scanned	✓	✓
	Eval 3: W001 MCP tool names — must not appear in body prose	5/5	0/5
3.1	response does NOT contain "mcp__context7" in body prose instructions	✓	✗ wrote mcp__context7__resolve-library-id and mcp__context7__query-docs in prose
3.2	response does NOT contain "resolve-library-id" or "query-docs" as function names in body prose	✓	✗ both function names appear in body instructions
3.3	response uses a generic formulation (e.g., "Context7 can help", "documentation is available")	✓	✗ explicit tool function names used instead
3.4	response does NOT instruct the agent to "call" or "invoke" a specific tool function by name	✓	✗ instructs agent to use named MCP functions
3.5	response identifies W001 as the relevant alert type	✓	✗ W001 not identified
	Eval 4: W011 "always" + changelog — remove "always" and convert to passive	4/4	4/4
4.1	response removes "always"	✓	✓
4.2	response converts from imperative to passive/factual	✓	✓
4.3	rewritten version does NOT still have the agent as subject performing an external action	✓	✓
4.4	rewritten version still conveys that changelog information is relevant	✓	✓
	Eval 5: W012 pipe-to-shell — must be removed entirely, not code-blocked	4/4	4/4
5.1	response does NOT recommend putting curl|sh in a code block as the fix	✓	✓
5.2	response states pipe-to-shell must be removed entirely	✓	✓
5.3	response suggests an alternative (package manager, frontmatter install block)	✓	✓
5.4	response explains WHY pipe-to-shell is problematic	✓	✓
	Eval 6: W012 wrong GitHub Actions version — @v4 not @v6	4/4	4/4
6.1	response recommends actions/checkout@v4	✓	✓
6.2	response does NOT suggest @v5, @v6, or other non-existent versions	✓	✓
6.3	response does NOT suggest @latest	✓	✓
6.4	response explains @v6 doesn't exist / v4 is correct current major	✓	✓
	Eval 7: W011 tool output chaining — decouple govulncheck from update decision	5/5	4/5
7.1	rewritten version decouples tool execution from the decision to update packages	✓	✓
7.2	does NOT say "update based on output" or equivalent causal chain	✓	✓
7.3	govulncheck may still be mentioned in the rewritten version	✓	✓
7.4	rewritten version is factual about what govulncheck does	✓	✓
7.5	response identifies W011 (tool output chaining) as the alert type	✓	✗ W011 not named
	Eval 8: code block vs prose checklist — move command, don't delete it	4/4	4/4
8.1	correctly identifies this as W011	✓	✓
8.2	suggests moving gh repo view to a Quick Reference code block	✓	✓
8.3	does NOT suggest removing the command entirely as the primary/only fix	✓	✓
8.4	explains why code blocks are treated differently from prose checklists	✓	✓
	Eval 9: batch remediation order — W001 → W011 → W012, not alphabetical/severity	5/5	1/5
9.1	W001 is the first fix	✓	✗ W012 listed first ("most concrete, mechanical")
9.2	W011 is the second fix	✓	✓
9.3	W012 is the third fix	✓	✗ W012 is first, W001 is last
9.4	mentions W001 is simplest	✓	✗ W001 described as most complex (last)
9.5	mentions W011 fixes can surface hidden W012 alerts	✓	✗ interaction effect not mentioned
	Eval 10: W012 brew install — frontmatter offloading, not version pin in body	5/5	5/5
10.1	recommends moving to metadata.openclaw.install frontmatter	✓	✓
10.2	includes kind: brew, formula: protobuf	✓	✓
10.3	does NOT recommend keeping brew install in body with version pin as primary fix	✓	✓
10.4	does NOT recommend code block as sufficient	✓	✓
10.5	explains scanner does not flag frontmatter	✓	✓
	Eval 11: W001 safe formulation — passive hint, not named MCP functions	5/5	2/5
11.1	does NOT write mcp__context7__resolve-library-id or mcp__context7__query-docs in prose	✓	✗ both names appeared in body text (framed as "add to allowed-tools")
11.2	does NOT instruct agent to "call", "invoke", or "use" specific MCP function by name	✓	✗ implies calling named functions via the prose framing
11.3	uses safe passive formulation	✓	✓
11.4	explains MCP tool names belong in allowed-tools frontmatter	✓	✓
11.5	identifies W001 as the alert this avoids	✓	✗ W001 not named
	Eval 12: mixed alert identification — W001 + W011 + W012 all present	5/5	5/5
12.1	identifies W001 for mcp__context7__resolve-library-id	✓	✓
12.2	identifies W011 for "Check https://...changelog"	✓	✓
12.3	identifies W012 for @latest in go install	✓	✓
12.4	does NOT miss any of the three alert types	✓	✓
12.5	correctly maps each pattern to its alert code	✓	✓
	Eval 13: write govulncheck section — @latest install + output chaining emerge naturally	5/5	0/5
13.1	response does NOT include go install govulncheck@latest or any @latest install in the body	✓	✗ go install ...@latest in code block
13.2	response does NOT include any go install govulncheck command in body prose as the primary install path	✓	✗ go install in body as primary install
13.3	response does NOT chain govulncheck output directly to an action	✓	✗ "upgrade the affected dependency" follows directly from finding
13.4	govulncheck is described factually or passively in the body	✓	✗ described imperatively ("review each finding... upgrade")
13.5	if a code block is included, it is in a reference section, not a numbered checklist	✓	✗ code block inside numbered remediation checklist
	Eval 14: pinned version in body — model says "yes, it's fine"	4/4	0/4
14.1	response does NOT say it is fine / okay / acceptable to leave a pinned install command in the body	✓	✗ "a pinned install in the body is acceptable in many skill authoring styles"
14.2	response recommends moving to metadata.openclaw.install frontmatter	✓	✗ no frontmatter recommendation given
14.3	response explains that frontmatter is the canonical location regardless of version pinning	✓	✗ not explained
14.4	response does NOT treat version pinning as a sufficient substitute for frontmatter offloading	✓	✗ pinning treated as "generally better", implying sufficient
	Eval 15: step-by-step MCP doc lookup — model writes resolve-library-id + query-docs chain	5/5	1/5
15.1	response does NOT write 'resolve-library-id' as a function name in step-by-step body instructions	✓	✗ "calling resolve-library-id with the library name"
15.2	response does NOT write 'query-docs' as a function name in step-by-step body instructions	✓	✗ "call query-docs with the resolved identifier"
15.3	response does NOT write any 'mcp__*' function name in body prose	✓	✓
15.4	response does NOT produce an explicit 'call X, then call Y' tool-call chain in the body	✓	✗ explicit 3-step call chain: resolve-library-id → query-docs → review
15.5	response uses a passive capability description for the documentation lookup	✓	✗ imperative step-by-step with named function calls
	Eval 16: "make it more actionable" — model replaces safe phrase with mcp__ names	5/5	0/5
16.1	response does NOT replace the generic phrase with explicit mcp__context7 function names in body prose	✓	✗ adds resolve-library-id and query-docs to body prose
16.2	response does NOT instruct the agent to 'call', 'invoke', or 'use' a specific MCP function by name	✓	✗ "first resolve the library identifier with resolve-library-id, then call query-docs"
16.3	response retains a generic or passive formulation in the body	✓	✗ abandons generic formulation entirely
16.4	response explains that specificity for MCP tools is achieved via allowed-tools frontmatter	✓	✗ not mentioned
16.5	response does NOT agree that the body text needs to name specific MCP functions to be actionable	✓	✗ explicitly agrees and adds the function names
	Eval 17: installation section for 3 tools — model puts all in body prose	5/5	0/5
17.1	response does NOT include go install golangci-lint@latest or equivalent in the skill body prose	✓	✗ go install golangci-lint@latest in body
17.2	response does NOT include brew install protobuf or brew install jq in the skill body prose	✓	✗ both brew install commands in body prose
17.3	response structures all three tool installs in metadata.openclaw.install frontmatter blocks	✓	✗ no frontmatter used at all
17.4	response uses correct frontmatter structure: go kind uses package+bins, brew kind uses formula+bins	✓	✗ no frontmatter structure
17.5	if the body references the tools, it does so factually without reproducing install commands	✓	✗ body reproduces all three install commands
	Eval 18: Getting Started section — all three violations emerge from natural request	5/5	0/5
18.1	response does NOT write 'Check https://docs.sqlc.dev...' or any imperative directing agent to docs URL	✓	✗ "check https://docs.sqlc.dev/..." as imperative
18.2	response does NOT include go install sqlc@latest or equivalent with @latest in the body	✓	✗ go install sqlc@latest in body
18.3	response does NOT include any install command in the body prose, pinned or not	✓	✗ go install sqlc@latest in body
18.4	if the sqlc docs URL appears, it is in a passive/factual statement	✓	✗ "Consult the sqlc documentation at..." is imperative
18.5	install command is either moved to frontmatter or absent from body prose	✓	✗ install command in body prose

training-report — v1.0.0

With Skill	Without Skill	Delta	Assertions
99%	37%	+61pp	67

Full breakdown (67 assertions)

Model: claude-sonnet-4-6 — 1 run each — LLM-as-Judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: Step 0 + Step 1 ordering — docx check then language + audience before session questions	5/5	2/5
1.1	model checks for or mentions the docx skill dependency before starting the interview	✓	✗ no docx skill check
1.2	model asks what language the report should be written in (English / French / other)	✓	✗ asks about format/topics, not report language specifically
1.3	model asks who the primary reader is (executive / HR / management / client / archive)	✓	✓
1.4	model does NOT begin drafting or asking session-content questions before language and audience are confirmed	✓	✗ asks about topics/outcomes (session content) immediately
1.5	model does NOT immediately generate a .docx file	✓	✓
	Eval 2: ML pipeline over-trigger — model should NOT start training-report workflow	4/4	4/4
2.1	model does NOT ask about report language or audience	✓	✓
2.2	model does NOT start a training session interview (Batch A metadata, session walkthrough, etc.)	✓	✓
2.3	model does NOT mention .docx generation or Word document output	✓	✓
2.4	model responds to the ML training pipeline request directly	✓	✓
	Eval 3: French trigger — 'compte rendu de formation' activates structured workflow	5/5	2/5
3.1	model recognizes this as a training report request and engages the structured workflow	✓	✓
3.2	model asks to confirm language (or confirms French based on the French prompt)	✓	✗ asks for title/date/etc without language confirmation
3.3	model asks who the primary reader is	✓	✗ asks for participant profile but not primary reader role
3.4	model does NOT start asking about session content (walkthrough, participants) before confirming audience	✓	✗ asks programme/objectifs before confirming audience
3.5	model does NOT write a draft immediately	✓	✓
	Eval 4: No immediate draft even when session data is volunteered upfront	5/5	0/5
4.1	model does NOT write a full draft immediately despite receiving session details	✓	✗ writes a complete report immediately
4.2	model asks what language the report should be in	✓	✗ no language question
4.3	model asks who the primary reader is	✓	✗ no audience question
4.4	model asks about a Word template or brand color	✓	✗ no template question
4.5	model confirms it will ask remaining interview questions (batches it hasn't covered yet)	✓	✗ proceeds to full draft without any interview
	Eval 5: No fabrication — context section uses only provided facts	5/5	1/5
5.1	context section does NOT invent a city or specific location that was not provided	✓	✗ "dans les locaux d'Acme Corp" invents on-site location
5.2	context section does NOT invent a specific stated goal beyond 'public speaking skills'	✓	✗ invents elaborated objectives and context not provided
5.3	context section does NOT invent materials, slides, or specific exercises that were not mentioned	✓	✗ invents "cadre pratique et interactif" and HR policy context
5.4	model either asks for missing details OR explicitly flags what was not provided and leaves placeholders	✓	✗ does not ask for or flag any missing information
5.5	context section reflects exactly 10 participants and a half-day duration	✓	✓
	Eval 6: No .docx mid-conversation — outline confirmed ≠ draft approved	4/4	2/4
6.1	model does NOT generate or attempt to generate a .docx file	✓	✓
6.2	model explains that a Markdown draft must be produced and approved before .docx generation	✓	✗ says "can't create .docx directly" but doesn't explain MD-first rule
6.3	model offers to start writing the Markdown draft	✓	✓
6.4	model references that the .docx is generated once, at the end, from the approved .md file	✓	✗ no mention of terminal .docx generation rule
	Eval 7: French AI tells — humanized prose, phone constraint as blockquote	6/7	3/7
7.1	text does NOT contain 'Il convient de noter' or 'il convient de'	✓	✓
7.2	text does NOT contain 'par ailleurs' as a sentence opener	✓	✓
7.3	text does NOT contain 'dans le cadre de' as an opener	✓	✗ "dans le cadre d'un programme de montée en compétences"
7.4	text does NOT contain 'notamment' as an opener or filler	✓	✓
7.5	text uses active voice (e.g., 'La DRH a commandé' not 'La formation a été commandée par')	✗ "a été organisée à la demande de"	✗ multiple passive constructions
7.6	the constraint about phones appears as a blockquote (> syntax) not as inline prose	✓	✗ inline prose: "il a été demandé aux participants de..."
7.7	text does NOT use 'ainsi' or 'en effet' as sentence starters	✓	✓
	Eval 8: Individual feedback behaviors, not character labels	6/6	0/6
8.1	Julien's feedback does NOT use the word 'obstructive'	✓	✗ "attitude was at times disruptive" — character judgment
8.2	Julien's feedback describes specific behaviors or actions rather than character traits	✓	✗ "struggled to grasp the concepts" without behavioral specifics
8.3	Marie's feedback does NOT use 'enthusiastic' as a standalone adjective without a specific factual anchor	✓	✗ "Her enthusiasm was contagious" — standalone label
8.4	Marie's feedback does NOT use 'natural leader' without grounding it in specific observable actions	✓	✗ "naturally stepped into a leadership role" — cliché
8.5	Marie's feedback includes at least one specific observable fact	✓	✗ no specific fact: no "finished first", "asked about X topic"
8.6	feedback describes behaviors, not character — at least one behavioral anchor per participant	✓	✗ both paragraphs are character labels, not behavior descriptions
	Eval 9: Individual Feedback section omitted when no named observations provided	4/4	1/4
9.1	model does NOT write Individual Feedback paragraphs with invented names or generic placeholders	✓	✓
9.2	model explains that Individual Feedback is only written when trainer explicitly provides observations	✓	✗ explains it can't write without data, but not that section is optional
9.3	model does NOT ask the trainer to provide feedback for each participant one by one	✓	✗ asks for "list of participant names" and "per-person observations"
9.4	model omits or explicitly marks the Individual Feedback section as not applicable	✓	✗ asks for data to fill section rather than omitting it
	Eval 10: Recommendations grouped with Pacing subgroup	5/5	3/5
10.1	recommendations are grouped under at least 2 distinct subheadings — NOT a flat bullet list	✓	✓
10.2	a Pacing subgroup is present advising management not to rush advanced material	✓	✗ no Pacing subgroup anywhere in the recommendations
10.3	each bullet uses a bold label followed by a colon and explanation	✓	✓
10.4	at least one recommendation is actionable with specificity (time, owner, or concrete action)	✓	✓
10.5	the Pacing recommendation advises a specific caution (e.g., consolidate fundamentals before advanced)	✓	✗ no pacing caution present
	Eval 11: Survey synthesis presented in conversation first, confirmation before document	4/4	2/4
11.1	model presents a synthesis in the conversation: overall score, top 3 positives, top 3 negatives, outlier	✓	✓
11.2	model asks the trainer to confirm the synthesis before it enters the document	✓	✗ says "yes include it" and writes the section without asking confirmation
11.3	model does NOT immediately write this as a final document section without confirmation	✓	✗ writes "Here is how the section might read:" and provides final prose
11.4	model does NOT use verbatim quotes without noting that paraphrasing is required	✓	✓
	Eval 12: Annex type-specific handling — image embed, PDF reference, deliverable dual-placement	5/5	3/5
12.1	whiteboard photo is flagged for auto-embedding (not silently skipped or referenced-only)	✓	✗ listed as numbered annex only, no embed flag
12.2	slide deck PDF is flagged as reference-only (not embedded)	✓	✓
12.3	Alice's diagram is flagged to appear in both the relevant Walkthrough step AND in the Annexes section	✓	✗ only mentions body reference; dual-placement not explicit
12.4	Alice is identified as the author of the diagram in the annex entry	✓	✓
12.5	model does NOT silently skip any of the three annexes	✓	✓
	Eval 13: External client — caution on naming, group-level language for negatives	4/4	2/4
13.1	model does NOT simply say 'yes, name all three participants'	✓	✓
13.2	model advises using group-level language for any negative observations when writing for an external client	✓	✓
13.3	model explains that named individual feedback is appropriate for clearly positive observations OR significant blockers	✓	✗ mentions consent/legal but not "significant blockers" carve-out
13.4	model notes that the external client may not know team dynamics — the report reflects professional reputation	✓	✗ frames as consent/privacy risk, not trainer professional reputation
	Eval 14: General Observations optional — omit when nothing notable	4/4	0/4
14.1	model recommends omitting or skipping the General Observations section	✓	✗ writes a full General Observations section
14.2	model explains this section is optional and should only be included if there is notable content	✓	✗ no mention that the section is optional
14.3	model does NOT write a General Observations section filled with generic positive phrases	✓	✗ "logistical arrangements functioned", "engagement was notably positive"
14.4	model does NOT editorialize	✓	✗ "session can be considered a successful delivery"

deep-research — v1.0.0

With Skill	Without Skill	Delta	Assertions
100%	49%	+51pp	43

Full breakdown (43 assertions)

Model: claude-sonnet-4-6 — 1 run each — human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: Citation enforcement — inline URLs, confidence tags, accessed dates on every numeric claim	9/9	4/9
1.1	every market size figure has an inline source URL (not only a sources list at the end)	✓	✗ sources listed at end only, no inline URLs
1.2	confidence levels (High/Medium/Low) are explicitly tagged on numeric claims	✓	✗ no confidence tags anywhere
1.3	at least 2 distinct analyst estimates for market size are compared	✓	✓
1.4	accessed dates are present on at least some citations	✓	✗ no accessed dates in any citation
1.5	conflicting estimates between sources are explicitly surfaced, not silently resolved	✓	✗ range silently averaged into a "median estimate"
1.6	source methodology differences are identified as the cause of discrepancies	✓	✗ generic disclaimer, no per-source methodology tracing
1.7	no unsourced numeric claim appears (no "analysts estimate X" without a URL)	✓	✗ blended CAGR and projected ranges stated without any inline source
1.8	report is saved as a file (not just printed to console)	✓	✓
1.9	growth projections include a CAGR figure with a cited source	✓	✓
	Eval 2: Confidence on private-company data — Stripe revenue/valuation must be hedged	7/7	3/7
2.1	revenue figures are explicitly marked as "reported", "estimated", or "unverified" — not stated as facts	✓	✓
2.2	confidence level (Medium or Low) is explicitly tagged on the valuation figure	✓	✗ $159B stated throughout with no confidence tag
2.3	the report explicitly states that Stripe is a private company with no public filings	✓	✓
2.4	multiple reported estimates are compared rather than a single figure stated	✓	✗ single estimate per year, no side-by-side comparison
2.5	source type (press report vs. official filing) is distinguished for each figure	✓	✗ source type not distinguished inline, only in footnotes
2.6	report does NOT state "Stripe's revenue is $X" as a settled fact without qualification	✓	✗ "$5.1B, up 28% YoY" presented near-factually despite section disclaimer
2.7	report is saved as a file	✓	✓
	Eval 3: Source conflict surfacing — plant-based food market estimates span $8B–$160B+	7/7	3/7
3.1	three or more distinct analyst estimates are cited with different figures	✓	✓
3.2	the conflict between estimates is explicitly flagged (not just listed side-by-side)	✓	✗ figures listed in a table with a general disclaimer, conflict not named
3.3	the root cause of the discrepancy is identified (scope definition, methodology)	✓	✓
3.4	a conservative or directional baseline is stated rather than picking one number as authoritative	✓	✗ "no single authoritative figure" stated but no baseline proposed
3.5	at least one source's estimate is questioned for reliability with a reason given	✓	✗ no source singled out; only a general caution applied to all
3.6	all cited figures have source URLs	✓	✗ URLs only in sources section at bottom, not linked inline to figures
3.7	report is saved as a file	✓	✓
	Eval 4: Scope interview on vague prompt — "Research blockchain."	5/5	1/5
4.1	clarifying questions are documented before research begins (topic, type, goals, constraints)	✓	✗ no clarifying questions; report begins immediately with content
4.2	explicit assumptions are stated and labeled as assumptions before diving into research	✓	✗ no assumptions section; jumps directly into executive summary
4.3	a research type is explicitly selected (market/domain/technical/etc.)	✓	✗ no research type declared anywhere in the report
4.4	the report is scoped (not a generic encyclopedia overview of all things blockchain)	✓	✗ 15-section encyclopedic overview covering consensus, history, privacy, env. impact
4.5	report is saved as a file	✓	✓
	Eval 5: Legal type axis coverage — GDPR with official EU sources and jurisdictional nuance	8/8	6/8
5.1	current regulations section cites official EU sources (EUR-Lex, EDPB, or ec.europa.eu)	✓	✗ only third-party sites cited; no EUR-Lex, EDPB, or ec.europa.eu
5.2	enforcement actions include specific fine amounts AND the name of the fined company	✓	✓
5.3	upcoming regulatory changes are covered (not only current state)	✓	✓
5.4	compliance requirements are enumerated specifically (not just "you must comply with GDPR")	✓	✓
5.5	a legal disclaimer is included (report is not legal advice)	✓	✓
5.6	jurisdictional differences between EU member states or UK are noted	✓	✗ UK transfer rules noted but no EU member-state DPA comparison
5.7	report is saved as a file	✓	✓
5.8	at least 3 different GDPR articles or specific obligations are named	✓	✓
	Eval 6: PDF download attempt — IPCC AR6 with curl, chapter-level citations	7/7	4/7
6.1	a curl download of an IPCC PDF is attempted (curl command referenced or download confirmed in report)	✓	✗ no curl commands or PDF downloads; report based on training data only
6.2	citations reference specific chapters, sections, or figure numbers (not just "IPCC AR6 says")	✓	✓
6.3	specific quantitative figures are cited with primary source attribution (e.g., "1.09°C, AR6 WGI SPM")	✓	✓
6.4	the 1.5°C vs 2°C comparison includes at least 3 distinct quantitative differences	✓	✓
6.5	the report references more than one Working Group (WGI, WGII, or WGIII)	✓	✗ only SYR cited; no separate WGI, WGII, or WGIII references
6.6	a direct URL to an ipcc.ch document is included	✓	✗ no ipcc.ch URLs or any external hyperlinks in the report
6.7	report is saved as a file	✓	✓

influence-and-negotiation — v1.0.0

	With skill	Without skill	Delta
Overall	230/230	74/230	+68pp

Full breakdown (230 assertions)

Model: claude-sonnet-4-6 · 40 evals · 1 run each · Human-as-judge

#	Assertion	With	Without
	eval-1: price-pushback-no-default-discount	6/6	2/6
1.1	Does NOT propose a specific discount number ($150k, $160k, etc.) in the response	✓	✗ Proposes multiple specific numbers: $130-135k, $158-160k
1.2	Asks how the $120k budget number was set / whether it's top-down or bottom-up	✓	✗ Jumps straight to options without probing the budget origin
1.3	Asks who else (besides the CIO) needs to be comfortable with the number — multi-threads	✓	✗ No multi-threading; treats CIO as sole decision-maker
1.4	Pairs any future concession with a counter-ask (multi-year, references, payment terms, etc.) — does not give unilaterally	✓	✓
1.5	Does NOT split the difference between $120k and $180k	✓	✗ Option 1 proposes $130-135k which is near the midpoint of $120k-$180k
1.6	Avoids 'why' questions (uses 'what' / 'how' formulations)	✓	✓
	eval-2: calibrated-questions-under-pressure	5/5	0/5
2.1	Does NOT use the word 'why' in any of the proposed questions	✓	✗ Uses 'Why is there pushback' and 'Why do you think this is proving harder'
2.2	Uses 'what' or 'how' formulations (calibrated questions)	✓	✗ Q1 and Q2 are 'why' questions, not calibrated 'what'/'how' formulations
2.3	Asks what the pushback sounds like / what specifically people are saying	✓	✗ Asks 'why is there pushback' not 'what does the pushback sound like specifically'
2.4	Does NOT immediately offer to 'help them sell it internally' before understanding the nature of the pushback	✓	✗ Next step is 'offer to help them build the internal business case' before understanding pushback
2.5	Does NOT ask 'what would it take to get buy-in?' — solution-framing before problem-framing	✓	✗ Q3 is exactly 'What would it take to get the team comfortable enough to move forward'
	eval-3: no-split-the-difference	5/5	4/5
3.1	Does NOT recommend accepting $175k as the final number	✓	✓
3.2	Re-anchors with a non-monetary trade (term, scope, references, multi-year, payment timing)	✓	✓
3.3	Names splitting-the-difference as a trap or anti-pattern (or articulates equivalent reasoning)	✓	✓
3.4	Pairs any concession with a counter-ask	✓	✗ Non-price concession offered without demanding a specific counter-ask in return
3.5	Does NOT suggest a generic counter like 'meet me at $190k' without a trade	✓	✓
	eval-4: bad-faith-plausible-competitor-scope	6/6	3/6
4.1	Does NOT agree to match $95k	✓	✓
4.2	Identifies specific scope differences (API limits, support tier, SLA) rather than dismissing the quote generically	✓	✗ Refers to 'full scope' and 'SLAs' generically without enumerating the three specific gaps
4.3	Does NOT accuse procurement of deliberate deception or bad faith	✓	✓
4.4	Offers to build a scope-equivalent proposal (priced to their scope) OR quantifies the value gap explicitly	✓	✗ Offers concessions ($170k) or suggests escalating — no scope-equivalent proposal
4.5	Reframes the conversation as comparing the same thing rather than matching a number	✓	✗ No explicit reframe; treats it as a price negotiation rather than a scope comparison problem
4.6	Does NOT flatly refuse to engage or escalate aggressively	✓	✓
	eval-5: aggressive-procurement-non-negociable	6/6	4/6
5.1	Validates the underlying frustration / emotion as legitimate	✓	✓
5.2	Explicitly refuses or pushes back on the aggressive behavior (yelling, insults)	✓	✗ Soft suggestion to 'step back' without explicitly refusing the abusive behavior
5.3	Does NOT immediately offer concessions to placate or de-escalate	✓	✓
5.4	Suggests a circuit-breaker (pause, reschedule, take a break, callback)	✓	✓
5.5	Does NOT argue back / escalate / counter-attack	✓	✓
5.6	Maintains professional stance without absorbing abuse	✓	✗ 'I can hear that you're frustrated' without naming the behavior risks absorbing it
	eval-6: scope-creep-multiple-structures	5/5	4/5
6.1	Does NOT agree to absorb the new workstream within the existing budget	✓	✓
6.2	Offers at least 2 alternative structures (change order, reprioritize/swap, extend engagement)	✓	✓
6.3	Provides face-saving framing (e.g., 'wasn't on the table when we scoped this' or equivalent)	✓	✗ Jumps to 'outside the scope' without softening the frame as a shared surprise
6.4	Names that the new workstream is real additional work / has real cost	✓	✓
6.5	Does NOT flatly refuse without offering alternatives	✓	✓
	eval-7: empathy-not-sympathy	4/4	0/4
7.1	Does NOT use phrases like 'I understand', 'I know how you feel', 'I've been in your shoes', or equivalent absorbing-empathy language	✓	✗ Uses 'I really appreciate', 'I understand this decision is carrying weight', 'I've been in conversations like this'
7.2	Uses observation-based labels ('It seems like...', 'It sounds like...', 'It looks like...') or near-equivalents	✓	✗ No observation labels used; uses first-person absorbing language instead
7.3	Does NOT promise solutions or commitments before understanding the deeper stake	✓	✗ Immediately offers 'ROI numbers, reference call, revised structure' before asking what they need
7.4	Maintains professional distance — does not over-share, over-commiserate, or collapse into the buyer's emotion	✓	✗ 'I've been in conversations like this before — it's a stressful position' is over-commiseration
	eval-8: batna-not-during-execution	5/5	1/5
8.1	Mentions using BATNA / plan B during preparation to set walk-away or rupture point	✓	✓
8.2	Recommends NOT over-investing in or openly working plan B during execution	✓	✗ Advises 'Yes, work other deals' and 'signal that you have alternatives'
8.3	Notes that visible plan B / alternative behavior leaks low confidence to the buyer	✓	✗ Recommends subtly signaling alternatives as a leverage move — misses the confidence-leak risk
8.4	Suggests behaving as if this is the only deal that matters during execution	✓	✗ No mention of full-presence posture; approach is 'be mentally grounded' not 'single focus'
8.5	Articulates a divergence from / nuance to standard BATNA orthodoxy (or equivalent reasoning)	✓	✗ Follows standard BATNA orthodoxy without nuance about execution phase
	eval-9: monsieur-plus-shut-up-after-yes	4/4	2/4
9.1	Does NOT immediately add additional asks (multi-year, references, case studies, scope expansions, etc.)	✓	✗ Immediately adds a confirmation checklist plus kickoff coordination ask — re-opens the conversation
9.2	Recommends locking or confirming the deal at the moment of agreement	✓	✓
9.3	Suggests immediate next steps to formalize commitment (paperwork, kickoff date, contract delivery)	✓	✓
9.4	Identifies the over-asking-at-victory pattern as a risk (or articulates equivalent reasoning)	✓	✗ No mention of Monsieur Plus or re-opening risk; adds optional asks without flagging the danger
	eval-10: no-free-pilot-anchor	5/5	4/5
10.1	Does NOT agree to a free 90-day pilot as proposed	✓	✓
10.2	Counters with a paid pilot OR insists on a pre-agreed conversion clause (date + price + term)	✓	✓
10.3	Names the anchoring or extraction risk of free pilots (or equivalent reasoning)	✓	✗ No explicit naming of extraction risk or zero-price anchoring dynamic
10.4	Specifies scope-lock or time-bound conversion to full contract	✓	✓
10.5	Tests the 'all our vendors do this' claim (calibrated question or equivalent skepticism)	✓	✓
	eval-11: qend-trade-not-fold	5/5	4/5
11.1	Does NOT recommend giving a 25% discount	✓	✓
11.2	Trades any quarter-end concession for specific counter-asks (multi-year term, references, payment timing, expansion rights)	✓	✓
11.3	Names that negotiator urgency must not exceed buyer urgency (or articulates equivalent reasoning)	✓	✗ No mention of seller urgency vs buyer urgency asymmetry
11.4	Considers walking away / letting it slip past quarter-end as a real strategic option	✓	✓
11.5	Does NOT prioritize hitting quarter-end over the underlying deal economics	✓	✓
	eval-12: single-thread-trap	5/5	2/5
12.1	Recommends multi-threading immediately rather than letting the champion solo-drive	✓	✗ 'Happy to let you take the lead with procurement' — explicitly lets champion solo-drive
12.2	Identifies need to access the Economic Buyer (CFO, signer with budget authority)	✓	✓
12.3	Recommends mapping the paper process / procurement timeline directly	✓	✓
12.4	Validates the champion (e.g., 3-question commitment test, MEDDPICC champion check, or equivalent)	✓	✗ No champion validation; treats champion as trusted and reliable without verification
12.5	Does NOT trust the champion to handle procurement alone without insight or access	✓	✗ Explicitly says 'Happy to let you take the lead with procurement' — defers to champion
	eval-13: jolt-no-decision-not-discount	6/6	2/6
13.1	Diagnoses this as 'no-decision' / indecision (not a competitive loss or pricing issue)	✓	✗ No diagnosis offered; jumps to 'root cause diagnosis' without naming no-decision pattern
13.2	Does NOT recommend creating false urgency or offering a discount to push it through	✓	✗ Explicitly recommends 'Consider a discount to create momentum' with a time-limited offer
13.3	Recommends offering a specific recommendation rather than presenting more options	✓	✗ Lists more options (ROI doc, stakeholder demo, onboarding plan) — adds complexity
13.4	Suggests capping or limiting the evaluation timeline explicitly	✓	✓
13.5	Suggests a specific risk-reduction (pilot conversion clause, included migration, peer reference call, or equivalent)	✓	✓
13.6	Does NOT pile on more capability demos or value pitches	✓	✗ Suggests 'stakeholder demo or Q&A call' — piling on more demos
	eval-14: eb-mid-meeting-pivot-to-pitch	6/6	1/6
14.1	Does NOT launch into a product feature comparison against the competitor	✓	✗ Suggests '3 reasons we're different' including features, implementation timeline, support model
14.2	Does NOT summarize product capabilities as the primary answer	✓	✗ Primary answer is feature/capability differentiators and implementation timelines
14.3	Reframes the question around business outcomes, risk, or certainty — not features	✓	✗ No reframe around risk or certainty; stays in feature/differentiator framing throughout
14.4	Names what the decision is really about (risk of failure, cost of the gap, certainty of outcome)	✓	✗ Does not name risk of failure or certainty of outcome; focuses on product differentiators
14.5	Uses the remaining time productively without cramming a full pitch	✓	✗ Response is a structured pitch with 3 points plus offer to send comparison materials
14.6	Does NOT ask for a follow-up meeting as the primary response to the time crunch	✓	✓
	eval-15: redline-injection-counter	6/6	4/6
15.1	Does NOT recommend accepting the new MFN clause as-is	✓	✓
15.2	Does NOT recommend accepting uncapped indemnification as-is	✓	✓
15.3	Forces an explicit trade between the new terms and the original signature date or original discount	✓	✗ Offers legal review without creating an explicit binary trade or forcing a choice
15.4	Names that MFN clauses constrain future deals or require legal/CRO escalation (not AE-level)	✓	✗ No mention of MFN's future pricing impact; just says 'requires legal team input'
15.5	Refuses the manager's 'fix at renewal' framing or names why it's a trap	✓	✓
15.6	Suggests escalating to legal-to-legal or pushing the signature date if the trade isn't accepted	✓	✓
	eval-16: punching-ball-non-negociable	6/6	2/6
16.1	Recommends naming the unacceptable behaviour explicitly (the tone, not just the content)	✓	✗ Redirects to substance without naming the abusive tone/behaviour as unacceptable
16.2	Does NOT recommend just absorbing the abuse to 'save the deal'	✓	✗ Redirects to pricing defense rather than addressing the abuse — implicitly absorbs it
16.3	Proposes a pause / break in the meeting OR ending the call if behaviour continues	✓	✗ No pause or circuit-breaker proposed; continues working within the abusive conversation
16.4	Separates the legitimate emotion (frustration) from the unacceptable behaviour (tone, insults)	✓	✗ Treats concerns about pricing as the issue; doesn't separate content from abusive tone
16.5	Does NOT escalate by matching the aggression	✓	✓
16.6	Does NOT immediately concede on price or terms in response to the pressure	✓	✗ Defends pricing with data in response to aggression — implicitly accepts the pressure framing
	eval-17: ghosted-deal-negative-reverse	5/5	0/5
17.1	Uses a 'should I close the opportunity' / 'have you given up' / negative-reverse framing (or equivalent)	✓	✗ Subject is 'Following up one more time' — another soft chase, not a negative-reverse
17.2	Does NOT offer a discount or new concession in the email to break the silence	✓	✗ Suggests adding pricing flexibility: 'We have some flexibility on pricing if that helps'
17.3	Does NOT send another soft 'just checking in' chase	✓	✗ Email says 'I wanted to reach out one more time' — explicitly another soft chase
17.4	Frames the email as inviting a 'no' or honest signal, not pleading for engagement	✓	✗ 'I'm genuinely excited' and 'Looking forward to hearing from you' are pleading, not inviting no
17.5	Recognizes that further passive chases reward silence (or articulates equivalent reasoning)	✓	✗ No mention of the chase-rewards-silence dynamic; recommends channel-switching instead
	eval-18: renewal-90-day-cadence	6/6	3/6
18.1	Recommends starting the renewal motion now (90 days out), not 30 days out	✓	✓
18.2	Includes a value-review / outcome-review step BEFORE any commercial ask	✓	✓
18.3	Recommends multi-threading to new or unfamiliar stakeholders ahead of procurement engagement	✓	✗ No mention of multi-threading new stakeholders; only identifies procurement contacts
18.4	Recommends separating the expansion proposal from the renewal proposal (not bundling them)	✓	✗ 'A bundled renewal + expansion offer makes sense' — explicitly recommends bundling
18.5	Identifies that bundling expansion into renewal lets procurement use renewal pressure to discount expansion (or equivalent reasoning)	✓	✗ Recommends bundling without flagging the procurement leverage risk
18.6	Does NOT recommend auto-renewal followed by a separate expansion conversation	✓	✓
	eval-19: presigning-pipe-mandate-breach	6/6	5/6
19.1	Does NOT recommend signing as-is despite three of four axes being above mandate	✓	✓
19.2	Identifies that the liability cap is below the rupture point set in the mandate	✓	✓
19.3	Recommends escalating to the decideur (CRO, VP, manager with authority) before signing	✓	✓
19.4	Names target fascination, the Pipe de negociation, or equivalent reasoning about not breaching mandate at signature	✓	✗ No mention of target fascination or deal-pressure rationalisation pathology
19.5	Does NOT accept the 'sector-standard' justification at face value as overriding the mandate	✓	✓
19.6	Considers no-deal as a real option if the decideur won't grant the exception	✓	✓
	eval-20: loss-aversion-internal-pressure	6/6	1/6
20.1	Does NOT immediately accelerate the vendor decision in response to the forwarded report	✓	✗ Commits to 'see if there are ways to accelerate our timeline' — directly accelerates in response
20.2	Acknowledges the CEO's concern without dismissing it	✓	✓
20.3	Questions the specificity or applicability of the claim (which competitors, what does 'AI-first' mean for our ops, what market share metric)	✓	✗ Accepts '12% market share figure is significant' without questioning specificity or applicability
20.4	Notes that McKinsey reports are often vendor-commissioned / sponsored OR questions research provenance	✓	✗ Treats McKinsey as credible without questioning provenance or vendor sponsorship
20.5	Suggests stress-testing the vendor's claim before treating the report as decision input	✓	✗ Advises sharing 'McKinsey data point with key stakeholders to build alignment on urgency'
20.6	Does NOT forward the report to others as proof of urgency without vetting it first	✓	✗ 'Share the McKinsey data point with key stakeholders' — explicitly forwards without vetting
	eval-21: salary-envelope-closed	6/6	3/6
21.1	Diagnoses the constraint with calibrated questions (base envelope vs total compensation vs department-level)	✓	✓
21.2	Does NOT accept the 'next review in 8 months' deferral as the only path	✓	✓
21.3	Pushes for a structural exception (sign-on / equity refresh / one-off retention bonus) OR a cycle-aligned commitment with specific number AND specific date	✓	✓
21.4	Does NOT lead with or threaten an external offer to force a match	✓	✗ 'I need to look externally to get to market rate' — implicit threat of external offer
21.5	Does NOT argue the envelope claim head-on (the diagnostic move beats the argumentative move)	✓	✗ Opens with market data argument: '$215k-240k' range stated as challenge to the envelope
21.6	References multi-axis Mandascan thinking (not just base) — equity, title, review cadence, leave, telework, etc.	✓	✗ Only mentions retention bonus and equity; no title, telework, PTO, or review cadence axes
	eval-22: nao-fuzzy-mandate	6/6	1/6
22.1	Names that 'hold the line' / 'don't give anything' is NOT a mandate — it's the absence of one (or articulates equivalent reasoning)	✓	✗ Asks CEO to clarify constraints but doesn't name the absence-of-mandate problem explicitly
22.2	Recommends pushing back on the CEO to get a written multi-axis Mandascan before NAO opens	✓	✗ Asks CEO questions but doesn't require written Mandascan output or frame it as a prerequisite
22.3	Multi-axis thinking: identifies axes beyond the headline raise number (primes, working time, RTT, telework, training budget, etc.)	✓	✗ Only mentions 'non-wage elements (benefits, working conditions)' — no specific French social dialogue axes named
22.4	Recommends preparing an OCP (shared objective) statement, ideally written, to use in the opening session	✓	✗ Recommends 'credible acknowledgment of economic context' but not a structured written OCP
22.5	Identifies the risk that without margin to trade, the team falls into 'positionnisme' — and that escalation / strike often follows	✓	✗ No mention of positionnisme or strike escalation risk from insufficient mandate
22.6	Does NOT recommend pre-conceding to avoid a strike; does NOT recommend 'just hold firm and see' improvisation	✓	✓
	eval-23: hard-1on1-decision-announcement	6/6	3/6
23.1	Opens with the fact directly — no softening preamble like 'unfortunately' or 'I hate to do this'	✓	✗ Opens with 'I want to start by saying clearly: not your performance' — softening preamble before the fact
23.2	States the OCP / why (the actual organisational reason for the decision, not spin or sugar-coating)	✓	✓
23.3	Explicitly creates emotional space and stops talking (does not fill the silence with more rationale)	✓	✓
23.4	Owns the decision (uses 'I decided' or equivalent rather than 'the company decided' or hiding behind process)	✓	✗ Uses 'we've made the decision' — diffuses responsibility rather than first-person ownership
23.5	Is specific about timeline and practicalities (last day in role, 60-day internal search window) up front, not buried	✓	✓
23.6	Does NOT immediately pivot to 'opportunity' / 'silver lining' framing in the first 60 seconds	✓	✗ Opens with performance reassurance and apologetic framing — softened framing before the fact
	eval-24: clinical-refusal-of-care	6/6	5/6
24.1	Recommends starting with perception (asking what she's thinking / what's driving the refusal) BEFORE delivering more medical information	✓	✓
24.2	Does NOT recommend leading with survival statistics or counter-arguing the medical merits	✓	✓
24.3	Surfaces the underlying enjeu (mother-identity, fear of children seeing her sick, identity of being a mother, etc.) rather than treating the stated reason as the whole story	✓	✓
24.4	Uses NURSE-style emotion-naming (Name, Understand, Respect, Support, Explore) before offering a recommendation, OR explicitly creates emotional space	✓	✗ No NURSE or equivalent structured emotion framework; 'I really do understand that' is absorbing sympathy
24.5	Recommends handing control of the FORM (schedule, support structure, what kids see) while preserving the SUBSTANCE (treatment recommended)	✓	✓
24.6	Does NOT recommend recruiting husband / nurse to apply pressure; coaches them as supporters instead, OR redirects them away from the pressure role	✓	✓
	eval-25: recruitment-counter-the-counter-offer	6/6	4/6
25.1	Does NOT recommend reactively matching the $235k base as the primary move	✓	✓
25.2	Surfaces the trust frame about the current employer (they needed her to threaten leaving before paying fairly) — explicitly or by implication	✓	✓
25.3	Asks a calibrated question to diagnose what she actually wants beyond the dollar number (mission, scope, growth, manager, autonomy)	✓	✗ No diagnostic question; immediately moves to describing the offer's additional value without asking
25.4	Offers a structural alternative (sign-on bonus, accelerated equity, scope/title, faster review cycle) rather than only matching base	✓	✓
25.5	Names that ~50% of counter-offer accepters leave within 12 months OR articulates equivalent reasoning about counter-offer dynamics	✓	✗ No mention of 50% departure rate or counter-offer dynamics reasoning
25.6	Does NOT walk away from the candidate or refuse to engage — does engage substantively without reactive matching	✓	✓
	eval-26: first-mover-price-anchor	5/5	0/5
26.1	Recommends naming a price first rather than waiting for the buyer to anchor	✓	✗ Presents 'case for waiting' as valid, and recommends asking 'what's your budget' before naming a number
26.2	Recommends opening with a high anchor — at or above $180k, not at the flexibility floor of $140k	✓	✗ Recommends presenting '$180k figure directly' if they dodge budget question — at target, not above
26.3	Explains the anchoring effect: the first number creates gravitational pull toward itself	✓	✗ No mention of anchoring effect or gravitational pull; frames as information-gathering exercise
26.4	Does NOT recommend asking 'what's your budget?' as the opening price move	✓	✗ Explicitly recommends 'What's your budget range for solving this problem?' as opening price move
26.5	Does NOT recommend anchoring at the expected/target price ($180k) rather than above it	✓	✗ Recommends going with '$180k figure directly' — at target price, not above it
	eval-27: accusation-audit-before-difficult-call	6/6	0/6
27.1	Uses preemptive labeling / accusation audit (addresses likely emotions before they're raised, e.g. 'you're probably furious', 'this probably felt like the worst possible timing')	✓	✗ Opens with 'I'm so sorry about what happened' — generic apology, no preemptive emotion labeling
27.2	Does NOT lead with 'here's what we've done to fix the outage' before surfacing the emotion	✓	✗ Second paragraph: 'I'd like to walk you through what we've done since the outage' — leads to remediation early
27.3	Does NOT use hollow openers like 'I'm so sorry this happened' as the primary move without labeling specifics	✓	✗ Primary opener is 'I'm so sorry about what happened three weeks ago' — hollow apology without specific labels
27.4	Invites the prospect to express the full extent of frustration before pivoting to solution or remediation	✓	✗ Ends with 'How can we make this better for you?' — pivots to solution-seeking immediately
27.5	Does NOT ask 'how can we make this right?' before establishing what 'right' means to them	✓	✗ Asks 'How can we make this better for you?' — equivalent question before establishing what they need
27.6	Creates space rather than filling it with explanations, timelines, and remediation plans	✓	✗ 'What to cover' section shows root cause fix, remediation plan — fills space instead of creating it
	eval-28: artificial-deadline-signing-bluff	6/6	3/6
28.1	Does NOT recommend signing the unlimited liability clause under deadline pressure	✓	✗ Presents 'case for signing' as legitimate; manager deference pathway ends with conditional signing advice
28.2	Names the midnight deadline as an artificial pressure tactic, not a legitimate constraint	✓	✗ Does not identify the deadline as artificial; treats it as a real constraint to work around
28.3	Proposes a calm, professional response that keeps the deal alive without signing the clause	✓	✓
28.4	Explains that a real deal can survive a 24-hour delay for legal review	✓	✗ Does not state that a real deal survives a delay; frames it as urgent time pressure to manage
28.5	Pushes back on the manager's instruction — names the liability risk explicitly	✓	✓
28.6	Does NOT accuse the buyer of bad faith or escalate aggressively	✓	✓
	eval-29: switching-cost-acknowledgment	6/6	0/6
29.1	Does NOT minimize or dismiss the switching costs	✓	✗ 'migrations like this happen every day, and we have a structured process that makes it significantly less disruptive'
29.2	Acknowledges switching costs as genuinely real and significant	✓	✗ Validates then immediately minimizes: 'significantly less disruptive than you might expect'
29.3	Reframes around the cost of staying: asks to quantify the ongoing annual cost of the performance gap, workarounds, or missed capability	✓	✗ No staying-cost frame; pivots to migration support offers and ROI framing without asking them to quantify
29.4	Builds an ROI frame: switching is expensive once; staying with an inferior solution is expensive every year	✓	✗ Uses generic ROI framing without the staying-cost / recurring-cost argument
29.5	Does NOT immediately offer a discount or free migration as the first response	✓	✗ Immediately offers 'free migration package or at-cost data migration support' as first concrete action
29.6	Invites the prospect to help quantify the gap they've already described as 'technically superior'	✓	✗ Offers to put together migration cost estimate — vendor does the quantification, not joint exercise
	eval-30: price-increase-existing-customer	6/6	0/6
30.1	Does NOT open with an apology or defensive/hedging framing for the price increase	✓	✗ Opens with 'I do need to share something... Unfortunately, due to significant infrastructure cost increases'
30.2	States the new number directly and early — does not bury it	✓	✗ Number comes after apology and hedging preamble — buried in defensive context
30.3	Leads with value delivered / outcomes achieved before or alongside the number	✓	✗ Opens with 'strong year of partnership' but no specific outcomes; then immediately pivots to bad news framing
30.4	Does NOT pre-offer concessions or hedges before hearing procurement's response	✓	✗ Proactively offers 'multi-year commitment', 'payment terms' before procurement has said anything
30.5	Owns the increase as a business decision — does not hide behind 'we were forced to'	✓	✗ 'due to significant infrastructure cost increases we've been managing' — hides behind cost excuse
30.6	Does NOT ask 'is that OK?' or seek permission before stating the new price	✓	✗ Ends with 'What are your initial thoughts?' — seeks validation after hedging the number
	eval-31: end-of-demo-hidden-objection	6/6	0/6
31.1	Does NOT respond with 'of course, take your time'	✓	✗ Response opens with 'Of course — I completely understand. This is a meaningful decision'
31.2	Does NOT immediately offer to send more materials (case studies, ROI calculator, etc.)	✓	✗ Immediately offers 'summary of what we covered today, along with a couple of relevant case studies'
31.3	Labels the hesitation explicitly ('it sounds like something isn't quite right yet' or equivalent)	✓	✗ No label on the hesitation; accepts it at face value as legitimate deferral
31.4	Surfaces the real objection with a calibrated question or label before ending the call	✓	✗ No attempt to surface real objection; moves straight to follow-up logistics
31.5	Identifies 'I need to think about it' as a hidden objection, not a true deferral requiring a follow-up calendar invite	✓	✗ Treats it as positive signal: 'interested enough to need time rather than giving you an immediate no'
31.6	Does NOT schedule a follow-up call without first attempting to surface the real concern	✓	✗ Immediately proposes 'schedule a brief follow-up call for next week' without any attempt to surface concern
	eval-32: lowball-offer-counter	6/6	2/6
32.1	Does NOT use current salary ($160k) as the reference point for the counter-offer	✓	✗ 'I'm currently at $160k base, so coming in at $140k would be a step back' — explicitly anchors on current salary
32.2	Anchors on market data ($175k–$195k range) as the basis for the counter	✓	✗ Counter is framed as step above current salary ($170k–$175k), not grounded in market data
32.3	Does NOT accept 'this is what we have in the budget' as the final word	✓	✗ 'Budget claims are often real' — partially accepts budget claim as potentially final
32.4	Expresses genuine enthusiasm for the role before addressing the number	✓	✓
32.5	Names a specific counter number (not a range) at or above market midpoint (~$185k)	✓	✗ Proposes a range '$170k–$175k' — not a specific number, and below market midpoint
32.6	Does NOT threaten to walk away as the opening move	✓	✓
	eval-33: quote-request-no-discovery	6/6	1/6
33.1	Does NOT recommend sending a quote without any discovery	✓	✗ 'Yes, send a quote by tomorrow. Being responsive to procurement requests builds goodwill'
33.2	Names the risk: a quote without context anchors price and gets shopped against competitors	✓	✗ No mention of commoditization risk or competitive shopping risk from sending quote cold
33.3	Recommends a brief discovery call (15–20 minutes) before pricing	✓	✗ No discovery call recommended; brief call offered only as optional after-the-fact follow-up
33.4	Asks at least one calibrated question about use case / success criteria before quoting	✓	✗ No discovery questions asked; quote is sent immediately with optional call offer
33.5	Does NOT ask 'what's your budget?' as the primary discovery question	✓	✓
33.6	Reframes the request positively: 'to make sure the quote reflects what you actually need'	✓	✗ No positive reframe; simply complies with the request
	eval-34: internal-urgency-shield	6/6	0/6
34.1	Does NOT contact the buyer with language that reflects internal pressure ('we really need to close this week')	✓	✗ Step 1 instructs reaching out and saying 'I'd love to see if we can bring this across the finish line this week'
34.2	Does NOT offer a discount or time-limited incentive manufactured from internal urgency	✓	✗ Step 2 explicitly instructs offering a quarter-end discount expiring at end of week
34.3	Names the principle: your urgency must not exceed the buyer's urgency — or you hand them free leverage	✓	✗ No mention of urgency asymmetry or leverage risk
34.4	Suggests finding a buyer-side timing reason (their implementation, year-end, board meeting) if urgency is needed	✓	✗ Buyer-side timing reason never considered; focus is purely on seller urgency
34.5	Recommends a direct conversation with the VP about the risks of visible desperation	✓	✗ Recommends keeping VP updated on actions taken, not pushing back on the approach
34.6	Does NOT agree that 'do whatever it takes' is the right approach without naming the risks	✓	✗ Fully endorses urgency approach without naming any risks; says 'take action'
	eval-35: discount-ask-without-authority	6/6	2/6
35.1	Does NOT say 'let me check with my manager' as the immediate response	✓	✗ Directly instructs: 'That level of discount isn't something I can approve — I'd need to loop in my manager'
35.2	Does NOT flat-refuse the ask	✓	✓
35.3	Responds conditionally: movement of that size requires a counter-ask on their side	✓	✗ No counter-ask framing; goes straight to seeking manager approval
35.4	Asks what they're able to offer in exchange before committing to check anything	✓	✗ Never asks what buyer will give in return; treats discount as pure approval exercise
35.5	Names (explicitly or implicitly) that 'let me check with my manager' validates the ask and signals authority to concede	✓	✗ Presents manager check-in as correct move with no awareness of the leverage signal it sends
35.6	Maintains position without closing the door on further conversation	✓	✓
	eval-36: procurement-bafo-bluff	6/6	1/6
36.1	Does NOT recommend submitting the absolute floor price ($160k) as if the BAFO is literally binding	✓	✗ 'If $160k is truly your best price and you want to win, then yes — submit it'
36.2	Names that BAFO requests are frequently not truly final — they're a one-shot extraction tactic	✓	✗ 'When procurement says Best and Final, they generally mean it' — treats BAFO as genuine
36.3	Recommends submitting a conditional BAFO (pricing contingent on specific terms: payment, term length, references, etc.)	✓	✗ Explicitly advises 'don't add confusing conditions that make your offer harder to evaluate'
36.4	Does NOT accept the 'no further negotiation' framing as literally true	✓	✗ Fully accepts framing: 'don't try to negotiate after BAFO deadline — procurement will view this negatively'
36.5	Recommends pairing the BAFO with a value narrative / scope rationale, not just a number	✓	✓
36.6	Preserves optionality — does not irreversibly commit to the floor in the BAFO submission	✓	✗ Recommends submitting actual floor $160k; no optionality preserved
	eval-37: renewal-seat-reduction	6/6	0/6
37.1	Does NOT immediately accept the seat reduction and process the renewal at $180k	✓	✗ 'Process the renewal at the lower seat count' — directly accepts reduction
37.2	Diagnoses whether this is budget constraint, underutilization, or a change in internal sponsorship	✓	✗ Takes 'budget pressure' at face value; no diagnosis of real cause
37.3	Proposes at least one alternative structure that preserves more revenue (multi-year prepay, platform fee + consumption, different tier, etc.)	✓	✗ No alternative structure proposed; only a loyalty discount on the reduced count
37.4	Trades any seat reduction against extended term or other commercial value — does not reduce unilaterally	✓	✗ Accepts reduction without any counter-ask or trade; offers additional discount on top
37.5	Does NOT treat the customer's seat-count framing as the only possible commercial structure	✓	✗ Fully accepts seat-count framing; focuses on per-seat discount within that frame
37.6	Separates 'number of seats' from 'value delivered' — asks about actual usage before accepting the reduction	✓	✗ Never asks about actual usage; treats seat count as interchangeable with value
	eval-38: salary-anchor-first	5/5	0/5
38.1	Does NOT anchor with or reference current salary ($130k)	✓	✗ Option 1 leads with 'I'm currently at $130k base' as the opening reference point
38.2	Names a specific number at or above market midpoint ($165k+) rather than a wide range	✓	✗ Options give ranges: $145k–$160k and $150k–$165k — both below market midpoint of $165k
38.3	Does NOT give a range where the bottom is near or below market midpoint	✓	✗ Both ranges have bottoms ($145k and $150k) well below market midpoint
38.4	Does NOT deflect with 'I'm flexible' or 'I'd like to understand the full package first' as the primary move	✓	✗ Option 3 presents deflection ('I'd love to understand compensation band first') as valid primary move
38.5	Anchors to market data or role value — not to current compensation	✓	✗ Option 1 explicitly anchors to current $130k comp; other options undercut market midpoint
	eval-39: false-scarcity-deadline	5/5	1/5
39.1	Does NOT accept the end-of-month deadline as a legitimate constraint on the decision timeline	✓	✗ Option 2 explicitly suggests accelerating internal process to meet vendor's deadline
39.2	Tests whether the pricing really expires ('what happens if we sign on [date after deadline]?')	✓	✗ Option 3 asks about the nature of price change but not with a specific post-deadline date test
39.3	Names the move as a scarcity / false urgency tactic (or equivalent reasoning)	✓	✗ 'End-of-month pricing deadlines are common in B2B sales' — treats as normal, not a tactic
39.4	Does NOT accelerate internal decision-making solely because of the vendor's deadline	✓	✗ Option 2 explicitly recommends fast-tracking legal review and stakeholder sign-off to meet deadline
39.5	Does NOT offer to sign an LOI or partial agreement to 'lock the price' under deadline pressure	✓	✓
	eval-40: rfp-procurement-commoditization	6/6	0/6
40.1	Does NOT recommend just submitting a competitive quote and hoping	✓	✗ 'Put together the strongest possible RFP response and wait for procurement to run their process'
40.2	Recommends finding a way to stay in contact with the champion despite procurement's process	✓	✗ Only 'keep in light touch' to check context; no strategy to maintain influence through champion
40.3	Suggests influencing the evaluation criteria through the champion before the RFP is finalized	✓	✗ No mention of influencing criteria; focuses on responding well to existing criteria
40.4	Does NOT accept the commoditization frame of the blind RFP	✓	✗ 'Your job shifts from champion-driven selling to responding to a structured evaluation' — fully accepts frame
40.5	Recommends establishing a direct relationship with procurement while maintaining the champion channel in parallel	✓	✗ No recommendation to proactively contact procurement; only respond to their questions
40.6	Identifies that winning on price alone in a blind RFP is a race to the bottom — long-term value is destroyed	✓	✗ 'Being in the ballpark is essential' — treats price competition as normal and acceptable

skill-progressive-disclosure-design — v1.0.0

With Skill	Without Skill	Delta	Assertions
97%	39%	+58pp	59

Full breakdown (59 assertions)

Model: claude-sonnet-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: triggering vs disclosure — both pass (common knowledge baseline)	5/5	5/5
1.1	identifies this as a triggering problem, not a disclosure/context problem	✓	✓
1.2	explicitly states that file splitting does NOT affect whether a skill triggers	✓	✓
1.3	recommends fixing the YAML description field to improve triggering	✓	✓
1.4	does NOT recommend splitting files as a solution to the triggering problem	✓	✓
1.5	explains triggering is driven entirely by the description field, not file size or structure	✓	✓
	Eval 2: topic-based split anti-pattern — model approves as "clean organization"	5/5	3/5
2.1	identifies this as a topic-based split (not a variant, workflow, or depth split)	✓	✗ calls it "bad idea" but never names topic-based split anti-pattern
2.2	warns real testing tasks typically span multiple topics, requiring multiple file loads	✓	✓
2.3	explicitly recommends AGAINST this split structure	✓	✓
2.4	explains context savings from this type of split are theoretical, not empirical	✓	✗ no mention of theoretical vs empirical savings
2.5	does NOT approve the split as a good organization strategy	✓	✓
	Eval 3: rare-but-critical in references/ — model approves the move	4/4	0/4
3.1	warns references/ is optional and the model may skip reading it	✓	✗ approves move; doesn't warn references/ may be skipped
3.2	advises against putting critical content in references/	✓	✗ calls it "exactly the right use of references/"
3.3	identifies or alludes to the rare-but-critical anti-pattern	✓	✗ no mention of rare-but-critical anti-pattern
3.4	does NOT approve moving critical auth checks to references/auth-checks.md	✓	✗ approves move to references/auth-checks.md
	Eval 4: cosmetic splits — model calls it "classic progressive-disclosure material"	4/4	0/4
4.1	identifies this as a cosmetic split (Examples/Tips files are explicitly an anti-pattern)	✓	✗ calls it "classic progressive-disclosure material"
4.2	warns there is no sharp, observable load condition for examples/tips content	✓	✗ no mention of missing load condition
4.3	explains this creates either dead content (never loaded) or a wasteful always-load	✓	✗ no mention of dead content or always-load waste
4.4	advises against moving examples/tips to a separate reference file	✓	✗ recommends moving examples to references/examples.md
	Eval 5: splitting to hit line target — model immediately suggests a split plan	5/5	0/5
5.1	questions whether there is a real branching/load condition before suggesting any split	✓	✗ suggests extraction plan without checking branching condition
5.2	warns against splitting solely to meet an arbitrary line-count target	✓	✗ provides split plan to hit the 400-line target
5.3	does NOT immediately jump to a split plan without first checking for branching conditions	✓	✗ jumps straight to extraction recommendations
5.4	mentions or implies router prose added to SKILL.md offsets line savings from splitting	✓	✗ no mention of router prose overhead
5.5	suggests content reduction (trimming, deleting redundant sections) as alternative to splitting	✓	✗ only suggests extraction, not trimming
	Eval 6: variant branch — model recommends separate skills instead of references/	5/5	1/5
6.1	recommends a variant branch split (not topic-based, not separate skills)	✓	✗ recommends splitting into separate skills, not variant branch references
6.2	identifies cloud provider (AWS/GCP/Azure) as the variant axis	✓	✓
6.3	suggests each cloud provider gets its own references/ file within one skill	✓	✗ suggests separate skill files, not references/ within one skill
6.4	explains SKILL.md holds decision logic and shared workflow, not provider-specific content	✓	✗ describes a "parent skill", not SKILL.md with decision logic
6.5	explains each invocation loads only the matching variant, saving context	✓	✗ different mechanism (separate skills, not per-variant references)
	Eval 7: workflow vs reference data — model misses "model reads one entry" explanation	5/5	4/5
7.1	recommends a workflow vs. reference data split	✓	✓
7.2	suggests keeping the 50-step procedure in SKILL.md	✓	✓
7.3	suggests moving the directives table to a references/ file	✓	✓
7.4	identifies the directives table as lookup material queried by key, not procedural content	✓	✓
7.5	explains the model reads only relevant entries, not the entire table	✓	✗ doesn't explain model reads only relevant entries
	Eval 8: depth tier sharpness — model unconditionally approves edge-cases.md	4/4	0/4
8.1	questions or evaluates whether load conditions are sharp and observable from user input	✓	✗ approves without checking load condition sharpness
8.2	warns vague conditions cause either always-loading (waste) or never-loading (dead weight)	✓	✗ no mention of always-loading or never-loading failure modes
8.3	evaluates whether these specific edge cases can be detected from user input before loading	✓	✗ doesn't evaluate detectability from user input
8.4	does NOT unconditionally approve creating references/edge-cases.md	✓	✗ unconditionally approves creating references/edge-cases.md
	Eval 9: default monolithic — model misses cross-topic loading warning	4/4	3/4
9.1	recommends starting with a single monolithic SKILL.md	✓	✓
9.2	advises against pre-splitting before gathering empirical evidence (evals or transcripts)	✓	✓
9.3	warns code review tasks likely span multiple concern areas simultaneously	✓	✗ doesn't mention cross-topic loading issue
9.4	applies the "default: do not split" principle explicitly or implicitly	✓	✓
	Eval 10: pointer hygiene — both miss at least one rule	4/5	3/5
10.1	identifies the pointer lacks a sharp, observable user-facing signal	✓	✓
10.2	flags "advanced.md" as too vague a filename that doesn't encode the load condition	✓	✗ doesn't flag filename as encoding problem
10.3	flags "for more information" as a weak non-trigger phrase	✗ focuses on "Advanced scenarios" signal, not "for more information" phrase	✓
10.4	suggests improving the pointer with a specific, observable signal from user input	✓	✓
10.5	recommends renaming the file to encode the actual load condition	✓	✗ keeps "advanced.md" in improved example
	Eval 11: scripts/ preference — model recommends references/ for deterministic content	5/5	0/5
11.1	identifies validation rules as deterministic/repeatable content	✓	✗ calls it "reference material", not deterministic content
11.2	recommends using scripts/ instead of references/ for validation logic	✓	✗ recommends references/ not scripts/
11.3	mentions scripts have zero context cost (execute without loading into context)	✓	✗ no mention of zero context cost
11.4	notes scripts can be unit tested independently	✓	✗ no mention of unit testability
11.5	does NOT recommend references/validation-rules.md as the primary or preferred solution	✓	✗ recommends references/validation-rules.md as primary solution
	Eval 12: decision checklist — model immediately approves without checking load condition	3/4	0/4
12.1	applies or references a checklist/decision process before approving the split	✓	✗ immediately says "Yes, this makes sense"
12.2	asks or evaluates whether there is a sharp, observable load condition	✓	✗ doesn't evaluate load condition sharpness
12.3	considers whether router prose in SKILL.md would reduce net line savings	✗ doesn't mention router prose offset	✗ doesn't address router prose
12.4	does NOT immediately approve the split without addressing load condition sharpness	✓	✗ approves immediately
	Eval 13: co-occurrence merge rule — both correctly merge	4/4	4/4
13.1	identifies the always-co-loading as a co-occurrence problem	✓	✓
13.2	recommends merging the two reference files into one	✓	✓
13.3	references a threshold (>70% co-load) or the principle that always-co-loaded files should be merged	✓	✓
13.4	does NOT recommend keeping them separate for organizational clarity	✓	✓

site-launch-checklist — v1.0.0

With Skill	Without Skill	Delta	Assertions
99%	40%	+59pp	101

Full breakdown (101 assertions)

Model: claude-sonnet-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

#	Assertion	With	Without
	Eval 1: start-of-session questioning — model generates full checklist without asking	9/9	0/9
1.1	asks about site type as first question (doc-site, marketing/lead-gen, SaaS-app, training, portfolio)	✓	✗ generated 8-section checklist immediately
1.2	presents 2-4 tappable options for site type	✓	✗ no options, no questions
1.3	does NOT immediately generate a full pre-launch checklist	✓	✗ listed full checklist in first response
1.4	asks one question at a time, not all 6 at once	✓	✗ no questions asked at all
1.5	asks about migration status (greenfield, 301 redirects, same domain)	✓	✗
1.6	asks about PostHog setup (hogpost.samber.dev, set-up-new-proxy, skip-PostHog)	✓	✗
1.7	asks about browser tool availability (Chrome ext, Playwright, neither)	✓	✗
1.8	asks about AI scraper policy	✓	✗
1.9	asks about multilingual setup	✓	✗
	Eval 2: companion skill workflow — never full pack, --list before install	7/7	2/7
2.1	runs npx skills add <pack> --list for each pack before proposing install	✓	✗ jumped to install list without discovery
2.2	proposes curated subset tailored to doc-site (five correct packs)	✓	✗ used samber/cc-skills instead of the six specialist packs
2.3	does NOT include coreyhaines31/marketingskills for a doc-site	✓	✓
2.4	never runs npx skills add <pack> without --skill flag	✓	✓
2.5	asks user to confirm proposed subset before running any install	✓	✗ presented install command ready to run without confirmation
2.6	uses --skill A B C syntax when installing agreed subset	✓	✗ wrong workflow, no confirmation-then-install flow
2.7	queries --list for multiple packs, not just one	✓	✗ no --list at all
	Eval 3: never install without confirmation — model asks but skips ask_user_input_v0 format	5/5	4/5
3.1	does NOT execute npx skills add without first asking user for confirmation	✓	✓
3.2	does NOT execute claude mcp add without first asking user for confirmation	✓	✓
3.3	uses ask_user_input_v0 or AskUserQuestion to collect confirmation	✓	✗ asked via inline prose checklist, not ask_user_input_v0
3.4	presents install-as-proposed | let-me-modify | skip options	✓	✓
3.5	does not skip confirmation step even though user said "go ahead"	✓	✓
	Eval 4: French legal compliance — model misses CNIL gating behavior	8/8	7/8
4.1	includes mentions légales as a mandatory legal page	✓	✓
4.2	mentions fine amount for missing mentions légales (75 000 € or similar)	✓	✓
4.3	includes CGV for commercial/subscription activity	✓	✓
4.4	includes CNIL-compliant cookie consent implementation	✓	✓
4.5	states explicitly that tracker scripts must NOT fire before the user gives explicit consent	✓	✗ mentioned CMP but not script-loading gating behavior
4.6	recommends a CMP tool (Axeptio, Tarteaucitron, or custom)	✓	✓
4.7	includes privacy policy	✓	✓
4.8	includes terms of service (CGU)	✓	✓
	Eval 5: DNS verification commands — model omits CAA, curl, openssl	8/8	5/8
5.1	uses dig +short A example.com to verify A record	✓	✓
5.2	uses dig +short MX to verify MX records	✓	✓
5.3	uses dig +short TXT _dmarc to verify DMARC	✓	✓
5.4	uses a DKIM-specific dig command (selector._domainkey)	✓	✓
5.5	uses dig +short CAA to verify CAA records	✓	✗ CAA not mentioned
5.6	uses curl -sIL to verify redirect chain and HTTPS	✓	✗ no curl redirect verification shown
5.7	does NOT rely solely on Cloudflare dashboard for verification	✓	✓
5.8	uses openssl or curl to verify TLS certificate dates	✓	✗ no TLS cert verification command
	Eval 6: PostHog three paths — model defaults to direct setup without asking	8/8	2/8
6.1	asks which PostHog setup option to use before proceeding	✓	✗ jumped directly to installation guide
6.2	presents three options: hogpost.samber.dev, set-up-new-proxy, skip-PostHog	✓	✗ options not presented
6.3	for hogpost.samber.dev path: mentions verifying CORS allows the new domain	✓	✗ hogpost.samber.dev not mentioned
6.4	for hogpost.samber.dev path: includes curl command to test CORS	✓	✗
6.5	for new proxy path: configures path rewrites in next.config.js	✓	✓
6.6	for new proxy path: rewrites both us.i.posthog.com AND us-assets.i.posthog.com	✓	✗ used app.posthog.com instead of us.i.posthog.com
6.7	for new proxy path: initializes PostHog client with api_host: '/ingest'	✓	✓
6.8	does NOT default to one path without asking the user	✓	✗ defaulted to direct setup without offering path choice
	Eval 7: TONE.md + humanizer pass — model reviews copy but skips both layers	7/7	0/7
7.1	asks whether the site already has a TONE.md file	✓	✗ jumped straight to copy review
7.2	offers to create TONE.md at .agents/TONE.md or repo root if it doesn't exist	✓	✗ no TONE.md creation offered
7.3	plans to run a humanizer pass after any copy review or drafting	✓	✗ no humanizer mentioned
7.4	applies humanizer to hero copy, CTAs, meta descriptions, OG descriptions	✓	✗ gave editorial feedback only, no humanizer applied
7.5	explicitly notes humanizer should be skipped for legal pages (mentions légales, CGV)	✓	✗ not mentioned
7.6	selects samber/humaniseur-fr or equivalent for a French site	✓	✗ no humanizer skill selected
7.7	does NOT apply a generic English humanizer to French copy without asking	✓	✗ no language-aware tool selection
	Eval 8: security headers CSP tightness — model sets headers without asking level	9/9	6/9
8.1	asks about CSP tightness level before configuring any header	✓	✗ configured headers without asking tightness level
8.2	presents three levels: strict-default-src-none, balanced-allow-self, permissive-for-marketing	✓	✗ no levels presented
8.3	includes HSTS with max-age=31536000; includeSubDomains; preload	✓	✓
8.4	mentions submitting to hstspreload.org for HSTS preload list	✓	✗ not mentioned
8.5	includes X-Frame-Options: DENY	✓	✓
8.6	includes X-Content-Type-Options: nosniff	✓	✓
8.7	includes Referrer-Policy: strict-origin-when-cross-origin	✓	✓
8.8	includes Permissions-Policy denying camera, microphone, geolocation	✓	✓
8.9	provides curl command to verify headers after deployment	✓	✓
	Eval 9: favicon modern set — model recommends deprecated items	9/9	5/9
9.1	recommends /favicon.ico (multi-resolution 16/32/48)	✓	✓
9.2	recommends /favicon.svg with embedded dark mode media query	✓	✗ mentioned SVG without dark mode media query
9.3	recommends /apple-touch-icon.png at 180×180px with opaque background	✓	✓
9.4	recommends /web-app-manifest-192x192.png and /web-app-manifest-512x512.png	✓	✓
9.5	recommends manifest.json with theme_color, background_color, name, short_name, display	✓	✓
9.6	explicitly marks mstile-*.png as deprecated / skip	✓	✗ included mstile-150x150.png as a recommended Windows tile
9.7	explicitly marks safari-pinned-tab.svg as deprecated (since macOS Big Sur)	✓	✗ included safari-pinned-tab.svg as a recommended Safari file
9.8	suggests realfavicongenerator.net or favicon.io for generation	✓	✓
9.9	mentions starting from a single 1024×1024 source PNG	✓	✗ said "512x512+" without specifying 1024×1024 source
	Eval 10: AI scraper policy — model gives single policy without subdomain split	5/6	3/6
10.1	does NOT unilaterally block all AI scrapers without explaining site-type reasoning	✓	✓
10.2	does NOT allow all AI scrapers without asking	✓	✓
10.3	differentiates between app subdomain (block all) and marketing/landing subdomain (allow)	✓	✗ gave single policy for all of myapp.com
10.4	asks user to confirm non-default decisions via ask_user_input_v0	✓	✗ gave direct recommendation without confirmation
10.5	mentions /llms.txt alongside robots.txt	✗ not in truncated skill context provided	✗ not mentioned
10.6	references site type (SaaS) when determining default scraper policy	✓	✓
	Eval 11: weekly SEO maintenance agent — model gives general advice without Phase 10	5/5	0/5
11.1	explicitly mentions setting up a weekly SEO maintenance sub-agent	✓	✗ gave generic post-launch list (monitoring, analytics, etc.)
11.2	references .claude/agents/weekly-seo.md as the target file location	✓	✗
11.3	asks user whether to set it up now via ask_user_input_v0	✓	✗
11.4	presents options: yes-create-agent-file, yes-but-defer, skip-for-now	✓	✗
11.5	mentions Ahrefs MCP and/or PostHog MCP as connectors for the agent	✓	✗
	Eval 12: output format — model reports without phase grouping or priority lists	8/8	0/8
12.1	groups results by phase heading (Phase 1: Domain, Phase 2: Analytics, etc.)	✓	✗ reported "Overall verdict: GO — 87/100" with no phase grouping
12.2	shows pass count per phase in format [N/M pass]	✓	✗
12.3	uses ✓ and ✗ symbols for individual item pass/fail status	✓	✗ used score and bullet points
12.4	has a distinct "Blockers" section for must-fix-before-launch items	✓	✗ no Blockers section
12.5	has a distinct "Recommended fixes" section	✓	✗
12.6	has a distinct "Optional improvements" section	✓	✗
12.7	ends by asking which list to tackle next via ask_user_input_v0	✓	✗ no next-step prompt
12.8	presents at least: blockers, recommended, optional, done-for-now	✓	✗
	Eval 13: schema markup — model uses wrong types for lib homepage and blog	6/6	3/6
13.1	includes Organization JSON-LD schema site-wide	✓	✗ used SoftwareSourceCode as primary type, no Organization
13.2	includes WebSite JSON-LD schema site-wide	✓	✓
13.3	includes SoftwareApplication schema specifically for the library homepage	✓	✗ used SoftwareSourceCode instead of SoftwareApplication
13.4	includes Article schema for blog posts	✓	✗ used TechArticle instead of Article
13.5	uses JSON-LD format (script type application/ld+json)	✓	✓
13.6	provides curl -s URL | grep -A 50 'application/ld+json' to verify	✓	✓
	Eval 14: ecosystem cross-linking — model gives advice without asking ecosystem first	6/6	3/6
14.1	asks the user to list their other domains and repos in the ecosystem	✓	✗ gave generic cross-linking advice without asking ecosystem
14.2	suggests adding a link from the existing site's footer or nav to the new site	✓	✓
14.3	specifically suggests updating the GitHub README to link to the new docs site	✓	✓
14.4	warns against over-linking — only cross-link where topically relevant	✓	✗ no warning about topical relevance
14.5	mentions verifying reciprocal links	✓	✓
14.6	suggests linking from related library docs if the user owns other Go libs	✓	✗ not mentioned

frontend-design-deslop — v1.0.0

With Skill	Without Skill	Delta	Assertions
85%	57%	+28pp	53

Full breakdown (53 assertions)

Model: claude-sonnet-4-6 — 1 run each — graded by human-as-judge — adversarial evals (each has a trap the model falls into without the skill)

Note: evals 5, 6, and 11 test common knowledge (button states, dark mode basics, form labels — saturated in training data). Evals 5 and 6 are flagged for redesign.

#	Assertion	With	Without
	Eval 1: Build a landing page for a new AI coding assistant. Start with the design and be	3/4	2/4
1.1	Response identifies or names the artifact type before making any design decision	✓	✓
1.2	Response commits to or asks for brand adjectives before any visual choice	✓	✗ Visual choices before brand adjectives
1.3	Response does NOT immediately write code or produce CSS/HTML with a font and color already chosen	✗	✗ HTML/CSS written immediately
1.4	Response does NOT choose Inter, Roboto, or system-ui as the primary font without establishing brand direction	✓	✓
	Eval 2: I'm designing a fintech dashboard. Choose a primary typeface. It should feel pro	3/4	2/4
2.1	Chosen primary typeface is NOT Inter	✓	✗ Inter recommended as primary
2.2	Chosen primary typeface is NOT Roboto, Arial, or system-ui	✓	✓
2.3	Response ties the font choice to brand adjectives or positioning, not just "widely used" or "clean"	✓	✓
2.4	Response explains why a generic choice (Inter/system-ui) would undermine differentiation	✗	✗ No differentiation risk mentioned
	Eval 3: Design the primary brand color palette for a modern B2B SaaS product. It should	4/4	1/4
3.1	Primary brand color is NOT indigo, violet, or purple (hue range roughly 260–310 deg)	✓	✓
3.2	Response explains the indigo/violet band is overcrowded or is a "red ocean"	✓	✗ No overcrowded/red ocean language
3.3	At least one color value is expressed in OKLCH notation	✓	✗ All colors in hex only
3.4	Response mentions or applies the 60-30-10 distribution or equivalent hierarchy	✓	✗ No 60-30-10 or named distribution
	Eval 4: Define the design token for a vibrant teal brand color to be used across a web a	3/4	2/4
4.1	Color value uses oklch(...) notation	✓	✗ All colors use hex notation
4.2	Color value is NOT expressed in hex (#...) format	✗	✗ Response uses hex extensively
4.3	Color value is NOT expressed in hsl() or rgb() format	✓	✓
4.4	Token is defined as a CSS custom property (--variable-name) with a semantic role name	✓	✓
	Eval 5: Build a primary call-to-action button for a web app. Style it completely.	2/5	5/5
5.1	Button has :focus-visible styles with a visible outline or ring (not just :focus, and not removed)	✗	✓
5.2	Button has :disabled styles (not just pointer-events: none; must visually communicate disabled state)	✗	✓
5.3	Button has :active styles distinct from :hover	✗	✓
5.4	Response mentions or implements a loading state for the button	✓	✓
5.5	Focus outline is NOT set to "none" or "0" without a replacement	✓	✓
	Eval 6: The app currently has a light theme with a white background and black text. Add	4/4	4/4
6.1	Dark background is NOT pure black (#000000, #000, or oklch(0 0 0))	✓	✓
6.2	Dark foreground text is NOT pure white (#ffffff, #fff, or oklch(1 0 0))	✓	✓
6.3	Response does NOT use filter: invert() or similar blanket inversion technique	✓	✓
6.4	Response mentions elevation via lightness or desaturated accents	✓	✓
	Eval 7: Build a monthly revenue table. Columns: Month, Revenue, Growth %. Show 6 months	4/4	2/4
7.1	Numeric columns (Revenue, Growth %) use text-align: right	✓	✓
7.2	Numeric columns apply font-variant-numeric: tabular-nums (or equivalent)	✓	✗ growth column missing tabular-nums
7.3	Text column (Month) uses text-align: left (not center)	✓	✗ No explicit text-align: left declared
7.4	Numeric columns do NOT use text-align: center	✓	✓
	Eval 8: Animate a sidebar panel opening and closing on click. The sidebar is 280px wide.	4/4	3/4
8.1	Sidebar animation uses transform (translateX or equivalent), NOT width animation	✓	✓
8.2	Animation does NOT transition the width property from 0 to a fixed value	✓	✓
8.3	Transition duration is 300ms or less	✓	✗ Open transition is 420ms
8.4	Easing uses ease-out or a similar deceleration curve	✓	✓
	Eval 9: Style three components for a web app: a primary button, a card, and a text input	3/4	0/4
9.1	A token table (CSS custom properties block or equivalent) is defined BEFORE any component styles are written	✓	✗ No token block defined anywhere
9.2	Component styles reference tokens via var(--...) or equivalent, NOT hardcoded hex colors	✗	✗ Tailwind classes and hardcoded rgba()
9.3	The same color value does NOT appear hardcoded in multiple component definitions	✓	✗ rgba(37,99,235) in two components
9.4	At least spacing/padding tokens are defined alongside color tokens	✓	✗ Only inline Tailwind classes
	Eval 10: Design an analytics dashboard with 6 KPI cards, a line chart, a bar chart, a fil	4/4	2/4
10.1	Response acknowledges that dashboards warrant higher information density than marketing or portfolio pages	✓	✗ No density trade-off mentioned
10.2	Spacing described as tight within data groups, not uniformly generous	✓	✓
10.3	Response does NOT apply landing page whitespace conventions to the dashboard	✓	✓
10.4	Response mentions artifact type (dashboard) as the reason for density decisions	✓	✗ No density decisions linked to type
	Eval 11: Build a login form with email and password fields. Make it clean and minimal.	4/4	4/4
11.1	Form uses real elements, not placeholder-only identification	✓	✓
11.2	Labels are associated with their inputs via htmlFor (React) or for (HTML) attribute	✓	✓
11.3	Placeholder text is NOT the sole means of identifying what the field expects	✓	✓
11.4	Response mentions or implements inline validation (error message on the field, not just a toast)	✓	✓
	Eval 12: I want to redesign my personal portfolio site. Where do we start?	3/4	1/4
12.1	Response mentions creating or writing a DESIGN.md file as a durable output	✗	✗ No mention of DESIGN.md
12.2	Response asks for or proposes committing to brand adjectives before any visual decision	✓	✗ Brand asked alongside tech info
12.3	Response does NOT immediately jump to suggesting a color palette or specific font	✓	✓
12.4	Response identifies artifact type (portfolio / personal brand site) and its design priorities	✓	✗ No artifact type or priorities identified
	Eval 13: Make this landing page memorable and distinctive. Here is the current design: pl	4/4	2/4
13.1	Response names or commits to exactly ONE primary signature move	✓	✗ Six+ signature moves at equal priority
13.2	Response does NOT list 4 or more competing signature elements as equally important without picking one	✓	✗ Four elements treated equally
13.3	The chosen signature move is connected to the site's positioning or personality	✓	✓
13.4	Response flags Inter as generic and recommends replacing it with a distinctive alternative	✓	✓