Show stats panel in occurrence list sidebar

[mihow]

Summary

Adds a Stats panel at the top of the occurrence list sidebar. It's the frontend for the model-agreement endpoint from #1307 (now merged). The panel shows, for the current filtered result set, how much has been human-verified and how closely the model's predictions agree with those human verifications. Because it reuses the list view's active filters, the numbers always match what's on screen — change a filter and the stats re-query.

The goal is to give reviewers a quick, honest read on model quality for whatever slice they're looking at, with the uncertainty made visible rather than hidden behind a single percentage.

Screenshots

List of Changes

#	What the user sees	How
1	A Stats panel above the filter sections in the occurrence list sidebar	New OccurrenceStats component (ui/src/pages/occurrences/occurrence-stats.tsx), wired into occurrences.tsx
2	Numbers always match the visible list (taxon, deployment, date, verification status, default filters)	Threads the same active filters array the list sends to useOccurrences, converted to query params with the same active/error rules as getFetchUrl
3	Verified occurrences — share of the filtered set that's human-verified (shows <1% instead of 0% when the count rounds down but is non-zero)	verified_pct; exact counts moved into the tooltip
4	Agreement (exact taxon) shown above the fold; any rank, coarser rank, and Cohen's κ tucked under a More toggle (collapsed by default) to reduce clutter	Collapsible section
5	An (i) tooltip on every metric with the exact counts and a plain-language explanation — including why the agreement denominator (verified occurrences with a model prediction) can be smaller than the verified count	Dynamic tooltip text built from the response
6	The 95% confidence interval shown as the agreement headline (e.g. 83–94%) plus a fuzzy diagonal-hatch band on the bar marking the uncertain zone	Solid fill to the lower CI bound, hatch across the CI range over the gray track
7	Cohen's κ as a signed, zero-centred bar	[-1, 1] bar
8	Panel shows a loading skeleton, and renders nothing on error so it never blocks the list	—

How the confidence interval is drawn

Each agreement bar is one 0–100% track. A solid fill runs up to the lower 95% CI bound (the "confident floor"), and a diagonal hatch covers the CI range (low→high) — the uncertain zone where the true value sits. The hatch is drawn over the gray track rather than over the solid fill, so it stays visible no matter where the point estimate lands (an earlier version layered the hatch on the solid fill, which hid it blue-on-blue when the estimate sat near 100%). A wide hatch reads immediately as "shaky number", a narrow one as "confident".

Design notes (why these particular metrics)

The agreement rate is the share of human-verified occurrences where the human's pick matched the model's pick. Three calibration ideas are baked in:

• Confidence interval instead of a hard cutoff. Rather than a yes/no "enough data" line, the Wilson 95% CI shows how shaky the number is — wide when few occurrences are verified, tightening as more get verified. A Wilson score interval behaves well at small samples. This is more honest than a magic threshold like "30 verifications", which is a rule of thumb that only holds if verifications are a random sample — they aren't; people verify the unusual or eye-catching ones first.
• Agreement beyond chance (Cohen's κ). Plain agreement % has a blind spot: if most occurrences in a project are one common species, human and model "agree" most of the time just by both guessing the common one — luck, not skill. κ subtracts the expected-by-chance agreement (1.0 = perfect, 0 = no better than guessing, negative = worse).
• Counts in the tooltip. The exact K of N lives in the (i) tooltip so the headline stays uncluttered, but the reader can still see how many verifications the rate is built on.

Same caveat applies to all three: they describe only the occurrences people chose to verify, not the whole project.

Known follow-up (backend, not this PR)

While testing against real data, the agreement rates looked high even on lightly-verified projects. On inspection, roughly half the verifications on a sample project are users accepting the model's suggestion (Identification.agreed_with_prediction), which sets the human taxon equal to the model's by construction and counts as an exact match automatically. Measured on a sample project, excluding accept-the-suggestion IDs drops exact agreement from 90% (n=100, CI 83–94%) to 38% (n=16, CI 18–61%). This is a definition issue in the merged endpoint (#1307), not in this UI. Details and a suggested fix are in a comment on this PR; a backend follow-up can exclude those IDs from the agreement denominator.

Test plan

• tsc --noEmit, eslint, and prettier --check clean on touched files (run in a pinned Node 22.12 container; the worktree's host node_modules is unavailable).
• Live browser render verified against a well-populated project via the worktree dev server, proxied to the staging backend. All bars render; the More section expands; tooltips carry the live counts.
• CI hatch verified across point estimates — including near-100% cases (e.g. 21–100% renders as mostly hatch) and mid-range cases.
• Filter reactivity verified: toggling Default Filters off sets ?apply_defaults=false and the panel re-queries with the same param, so the same filter array drives both list and stats.
• Staging backend deployed to current main so the Netlify deploy preview serves the live endpoint.

🤖 Generated with Claude Code

[netlify]

✅ Deploy Preview for antenna-preview ready!

Name	Link
🔨 Latest commit	d3c6acf
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-preview/deploys/6a3359e0c1c3240008ae4821
😎 Deploy Preview	https://deploy-preview-1308--antenna-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
Lighthouse	1 paths audited Performance: 62 (🔴 down 3 from production) Accessibility: 81 (🔴 down 8 from production) Best Practices: 92 (🔴 down 8 from production) SEO: 92 (no change from production) PWA: 80 (no change from production) View the detailed breakdown and full score reports
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b8b08405-d33c-470e-8a17-f51294883c42

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/occurrence-stats-ui

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[netlify]

✅ Deploy Preview for antenna-ssec ready!

Name	Link
🔨 Latest commit	d3c6acf
🔍 Latest deploy log	https://app.netlify.com/projects/antenna-ssec/deploys/6a3359e0b9dad100080e1521
😎 Deploy Preview	https://deploy-preview-1308--antenna-ssec.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[mihow]

Claude says: While iterating on the panel UI, Anna noticed the agreement numbers look high and the confidence intervals look narrow even when only a small fraction of occurrences are verified. I looked into why. There are two separate things going on, and one of them is a measurement issue worth fixing in the endpoint (merged in #1307), not in this UI PR.

1. The narrow CI is correct, and it is not about the sampling fraction

CI width is driven by the absolute size of the comparable set — verified occurrences that have both a human taxon and a model prediction — not by what percentage of all occurrences are verified. On a sample project with roughly 24k occurrences and about 100 comparable verifications, n ≈ 100 gives roughly ±6% at 95%, which is genuinely narrow. The fact that only ~0.4% of the population is verified does not widen it: the Wilson interval assumes an effectively infinite population, and a finite-population correction would make it narrower, not wider.

A higher confidence level does widen the band — that is the z constant (WILSON_Z_95 = 1.96 in ami/utils/stats.py). Using 99% (z ≈ 2.576) would turn 83–94% into roughly 80–96%. But that only widens the band; it does not explain why the point estimate is high.

2. The high agreement appears inflated by accept-the-suggestion verifications

About half of the verifications in the sample project are users accepting the model's prediction (Identification.agreed_with_prediction is set). For those, the human taxon equals the model's predicted taxon by construction, so they count as an exact match automatically. model_agreement_for_project currently takes the best non-withdrawn identification's taxon without distinguishing independent identifications from accept-the-suggestion ones, so this circularity inflates exact agreement, any-rank agreement, and Cohen's κ.

Measured on the sample project, excluding accept-the-suggestion identifications and keeping only independent human IDs:

Cohort	Exact agreement	n	95% CI
As shipped (all identifications)	90%	100	83–94%
Independent identifications only	38%	16	18–61%

So genuine independent human-vs-model agreement is substantially lower, with a wide CI — which matches the intuition that a small verified set should be uncertain. The narrow 83–94% came from the inflated n. Selection bias (verifiers tending to confirm easy detections) pushes in the same direction, but the accept-the-suggestion circularity is the larger and more fixable effect. These are measured numbers from staging data, so the interpretation (circularity is the cause) is well supported, though the exact split will vary by project.

Suggested implementation

In model_agreement_for_project (ami/main/models_future/occurrence.py), exclude identifications that merely agreed with the model prediction from the human side of the comparison:

best_user_ident = Identification.objects.filter(
    occurrence=OuterRef("pk"),
    withdrawn=False,
    agreed_with_prediction__isnull=True,   # exclude accept-the-suggestion IDs
).order_by(*BEST_IDENTIFICATION_ORDER)

This makes agreement measure independent confirmation. It affects agreed_exact, agreed_any_rank, the coarser-rank variant, and cohens_kappa, since they all derive from best_user_taxon_id.

A few things worth deciding before implementing:

• Do we want a single independent-only number, or do we report both (independent vs. inclusive) so the accept rate stays visible? Reporting both is more transparent but adds fields to the response.
• Should "agreed with another human identification" (agreed_with_identification) be treated the same way? In the sample it was 0, but it raises the same independence question.
• Worth exposing the confidence level as a parameter if anyone wants 99%.

Happy to open a follow-up PR against the endpoint with the filter plus a test, since #1307 is already merged. Flagging it here for visibility on the UI work.