feat(build): run unit tests in build via pytest

[scottschreckengaust]

Wires unit tests into the build so they run locally and in CI alike, using each language's standard test runner.

Related

• Closes Establish a Python unit-test framework wired into mise run build #226 — establishes the test plumbing (a first test to cover the parser from feat(security): vendor Semgrep r/all ruleset with tracked exclusions #220 is a follow-up).

Problem

mise run build ran lint / fmt / validate / security but no tests. The one existing test file (tools/evals/databases-on-aws/dsql/scripts/test_safe_query.py) was never invoked, so Python logic under tools/ could regress silently.

Approach

Use the ecosystem's standard runners rather than a bespoke script:

• test:py → uv run --with pytest pytest tools plugins. pytest already does discovery, per-file execution, and result aggregation. --with installs pytest into an ephemeral cached env for the run only, so no dependency is added to the repo — which was the whole reason the earlier custom shim existed. Exit code 5 (no tests collected) is treated as success.
• test:js → npm test only when a package.json defines a test script; otherwise skips. Leaves room for jest/other runners with no custom code.
• test aggregates the per-language tasks and stays in the build chain: lint → fmt:check → validate → test → security.

Why not a custom runner?

An earlier revision added a 91-line tools/run-tests.py that reimplemented discovery/aggregation/exclusion. pytest does all of that natively — and better: the custom runner's no-pytest fallback shim silently skipped parametrized and fixture-based tests. Switching to pytest raises the passing count from 63 → 65 while deleting ~91 lines.

Verification

• mise run test → test:py 65 passed; test:js skips cleanly (no package.json).
• mise run build runs the test step; lint / fmt:check pass.
• Net change: +18 / −94 (custom runner deleted).

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

Generated with Claude Code

[theagenticguy]

✅ Approve

Small, correct build-plumbing change (+27/−2 in mise.toml + AGENTS.md). Reviewed the full diff and verified the PR's claims against the committed head branch and CI config rather than the description alone.

Why this is a solid approve:

• Correct exit-code handling. uv run --with pytest pytest tools plugins || [ $? -eq 5 ] masks only pytest's exit 5 (no tests collected) as success. Real failures (exit 1) and collection/usage errors (2–4) still propagate and break the build.
• No new repo dependency. --with pytest installs into an ephemeral cached env for the run only — the whole reason the earlier custom shim existed, now unnecessary.
• Standard runner over bespoke code. pytest handles discovery/parametrize/fixtures natively; the deleted 91-line shim's fallback silently skipped parametrized/fixture tests (why the passing count rises 63→65).
• CI is genuinely exercised. The build job runs mise run build, so the new test step actually executes in CI (passed at 2m32s), and the mutation guard confirms the step writes nothing. All checks green.
• Docs consistent. Edits land in AGENTS.md (CLAUDE.md is a symlink to it). The updated line lint + fmt:check + validate + test + security also corrects a pre-existing omission — validate was already in the chain but undocumented.

Parser-covering tests are a documented fast-follow (#220), so their absence here is by design.

Non-blocking nits (fine to merge as-is):

1. The PR body names the tasks test:py / test:js, but the committed names are test:python / test:node — description drift only.
2. test:node's grep -q '"test"' is a loose match against package.json; jq -e '.scripts.test' would be exact. Zero impact today (no package.json, so it always skips cleanly) — worth tightening only if a plugin ever ships one.