feat: improve mixpanelyst skill score 74% → 84%#186
Conversation
Hey @jaredmixpanel 👋 I ran your skills through `tessl skill review` at work and found some targeted improvements for the `mixpanelyst` skill. Here's the full before/after: | Skill | Before | After | Change | |-------|--------|-------|--------| | mixpanelyst | 74% | 84% | +10% | | dashboard-expert | 94% | — | (unchanged) | | setup | 17%* | — | (unchanged) | * `setup` scores 17% because the name `mixpanel-headless:setup` contains colons, which fails the kebab-case validation — the LLM judge is skipped entirely when validation fails. The skill content itself looks solid; renaming to e.g. `mixpanel-headless-setup` would unlock a real score. Happy to send a follow-up for that if you're interested. Changes made to `mixpanelyst`: Description (82% → 100%): - Replaced generic verbs with specific actions (build segmentation queries, generate retention curves, create funnel reports, etc.) - Added explicit exclusion clause to reduce conflict risk with generic analytics skills Content — progressive disclosure (55% → 63%): - Created references/entity-crud-reference.md — moved Entity CRUD method listings and Legacy Query signatures out of main SKILL.md - Created references/custom-property-formulas.md — moved formula function reference to dedicated file, kept Mixpanel-specific regex quirks inline - Condensed Statistical Analysis and Visualization sections to 1-line summaries Net effect: SKILL.md reduced from 1058 → 907 lines while preserving all Mixpanel-specific domain expertise. I also stress-tested your `mixpanelyst` skill against a few real-world task evals and it held up really well on multi-step funnel analysis with custom conversion windows and inline cohort definitions for behavioral segmentation. Kudos for that. Honest disclosure — I work at @tesslio where we build tooling around skills like these. Not a pitch — just saw room for improvement and wanted to contribute. Want to self-improve your skills? Just point your agent (Claude Code, Codex, etc.) at this Tessl guide (https://docs.tessl.io/evaluate/optimize-a-skill-using-best-practices) and ask it to optimize your skill. Ping me — @yogesh-tessl — if you hit any snags. Thanks in advance 🙏
Confidence Score: 4/5Safe to merge; only skill documentation is changed, no functional code. The structural reorganization is sound and entity-crud-reference.md is a faithful lift. Three small omissions in custom-property-formulas.md could trip up agents writing complex formulas but do not break anything today. custom-property-formulas.md — missing escape-backslash warning, $0 backreference, and concrete Braze cleanup code.
|
| Filename | Overview |
|---|---|
| mixpanel-plugin/skills/mixpanelyst/SKILL.md | Description updated with action-verb lead and exclusion clause; bulk content condensed or delegated to reference files. Minor omissions: $0 whole-match and escape-backslash warnings are gone from both this file and the reference files. |
| mixpanel-plugin/skills/mixpanelyst/references/custom-property-formulas.md | New reference file capturing formula function reference. Missing: escape-backslash warning and the concrete Braze multi-step cleanup code example. |
| mixpanel-plugin/skills/mixpanelyst/references/entity-crud-reference.md | New reference file faithfully preserving all entity CRUD method listings and legacy query signatures from the original SKILL.md; content matches the original accurately. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Agent receives Mixpanel task] --> B{Skill routing}
B -->|matches description| C[SKILL.md loaded]
C --> D{Content needed?}
D -->|formula functions| E[references/custom-property-formulas.md]
D -->|entity CRUD / legacy queries| F[references/entity-crud-reference.md]
D -->|core query API| G[Inline examples in SKILL.md]
E --> H[Agent writes formula]
F --> I[Agent calls entity method]
G --> J[Agent runs typed query]
Comments Outside Diff (1)
-
mixpanel-plugin/skills/mixpanelyst/SKILL.md, line 213 (link)$0whole-match backreference droppedThe original noted that
$0(whole-match) works inREGEX_REPLACEreplacements alongside$1/$2. The condensed line now only mentions$1,$2, andcustom-property-formulas.mdlikewise omits$0. An agent trying to echo the full match in a replacement will either guess or assume it isn't supported.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Reviews (1): Last reviewed commit: "feat: improve mixpanelyst skill score 74..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| **CamelCase splitting:** `REGEX_REPLACE(text, "(?-i)([a-z])([A-Z])", "$1 $2")` → "ChickenSundaysApril" becomes "Chicken Sundays April" | ||
|
|
||
| **Multi-step cleanup** (campaign names from Braze): Chain `LET` + `REGEX_REPLACE` to strip date prefixes, targeting codes, channel suffixes, and underscores in a single formula. |
There was a problem hiding this comment.
The original SKILL.md had an explicit warning about needing \\\\ for a literal \ depending on how the formula is constructed (Python string → JSON → regex engine). This is a common stumbling block for agents writing formulas with literal backslashes, and it appears in neither the condensed SKILL.md regex-quirks block nor this reference file. Agents hitting this will get silent, hard-to-debug mismatch errors.
|
|
||
| **CamelCase splitting:** `REGEX_REPLACE(text, "(?-i)([a-z])([A-Z])", "$1 $2")` → "ChickenSundaysApril" becomes "Chicken Sundays April" | ||
|
|
||
| **Multi-step cleanup** (campaign names from Braze): Chain `LET` + `REGEX_REPLACE` to strip date prefixes, targeting codes, channel suffixes, and underscores in a single formula. |
There was a problem hiding this comment.
The original file had the full LET + REGEX_REPLACE chain for stripping date prefixes, targeting codes, and channel suffixes from Braze campaign names. The replacement here describes what it does but omits the actual formula. Agents prompted to clean up campaign names will have no runnable pattern to work from, defeating the purpose of this reference file.
1 participant
Hey @jaredmixpanel 👋
a full programmable interface to Mixpanel with typed queries for insights, funnels, retention, and flows is a great idea. Especially the part about being designed with coding agents in mind. That's forward-thinking.
also ran your skills through
tessl skill reviewat work and found some targeted improvements for themixpanelystskill. Here's the full before/after:Changes made to `mixpanelyst`
Description (82% → 100%):
Content — progressive disclosure (55% → 63%):
quick honest disclosure. I work at https://github.com/tesslio where we build tooling around skills like these. Not a pitch, just saw room for improvement and wanted to contribute.
If you want to self-improve your skills, or define your own scenarios to pressure test, just ask your agent (Claude Code, Codex, etc.) to evaluate and optimize your skill with Tessl. Ping me @yogesh-tessl, if you hit any snags.