|
| 1 | +# CLAUDE.md — aidlc-evaluator |
| 2 | + |
| 3 | +Agent guidance for working in `scripts/aidlc-evaluator`. |
| 4 | + |
| 5 | +## Security scanning |
| 6 | + |
| 7 | +Run all local scans in one pass before pushing: |
| 8 | + |
| 9 | +```bash |
| 10 | +cd scripts/aidlc-evaluator |
| 11 | + |
| 12 | +# Semgrep (matches CI ruleset exactly) |
| 13 | +uv run semgrep scan --oss-only --metrics=off --config=r/all \ |
| 14 | + --exclude .venv --exclude runs --exclude test_cases |
| 15 | + |
| 16 | +# Bandit (source only, exclude .venv) |
| 17 | +uv run bandit -r packages/ scripts/ run.py -ll |
| 18 | + |
| 19 | +# Checkov (IaC/config) |
| 20 | +uv run checkov -d . --output cli |
| 21 | + |
| 22 | +# gitleaks (secrets) |
| 23 | +gitleaks detect --source . --no-git --config .gitleaks.toml |
| 24 | + |
| 25 | +# Grype (dependency vulnerabilities) |
| 26 | +/usr/local/bin/grype dir:. --exclude './.venv' --exclude './runs' |
| 27 | + |
| 28 | +# Markdownlint (source docs only) |
| 29 | +npx markdownlint-cli2 "*.md" "docs/*.md" "packages/**/*.md" \ |
| 30 | + "!**/.venv/**" "!**/node_modules/**" |
| 31 | +``` |
| 32 | + |
| 33 | +## Semgrep suppression — CRITICAL |
| 34 | + |
| 35 | +**Always put `# nosemgrep` on the same line as the flagged code.** Preceding-line comments are not reliably associated with new findings in PRs. |
| 36 | + |
| 37 | +Use the **full dotted rule ID**. CI uses `--config=r/all` (full Semgrep registry). |
| 38 | + |
| 39 | +```python |
| 40 | +# CORRECT — inline on the flagged line |
| 41 | +result = subprocess.run(cmd) # nosec B603 nosemgrep: python.lang.security.audit.dangerous-subprocess-use-audit.dangerous-subprocess-use-audit |
| 42 | + |
| 43 | +# WRONG — preceding line not reliable for new findings in new files |
| 44 | +# nosemgrep: dangerous-subprocess-use-audit |
| 45 | +result = subprocess.run(cmd) |
| 46 | +``` |
| 47 | + |
| 48 | +## Interpreting CI results |
| 49 | + |
| 50 | +The PR check list has TWO semgrep entries: |
| 51 | + |
| 52 | +| Check name | What it is | How to read it | |
| 53 | +| ---------- | ---------- | -------------- | |
| 54 | +| `semgrep` (security-scanners workflow) | Live scan of the PR diff | **This is the authoritative result** | |
| 55 | +| `Semgrep OSS` (Code Scanning) | GitHub's annotation of the SARIF from the previous push | Can show stale findings; ignore if `semgrep` is green | |
| 56 | + |
| 57 | +If `semgrep` passes but `Semgrep OSS` shows findings, the findings are stale from a prior commit. They will clear on the next push that re-uploads a clean SARIF. |
| 58 | + |
| 59 | +## Working location |
| 60 | + |
| 61 | +All changes must be inside `scripts/aidlc-evaluator/`. Verify before committing: |
| 62 | + |
| 63 | +```bash |
| 64 | +git diff --name-only main..HEAD | grep -v "^scripts/aidlc-evaluator/" |
| 65 | +# Should produce no output |
| 66 | +``` |
| 67 | + |
| 68 | +## Branch |
| 69 | + |
| 70 | +Active development branch: `fix/evaluator-update` on `harmjeff/aidlc-workflows` fork. |
| 71 | +Target: `awslabs/aidlc-workflows` PR #235. |
0 commit comments