-
Notifications
You must be signed in to change notification settings - Fork 14
Add processed/unprocessed filter to captures list view #1326
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
7b14b8d
9a41c7d
d18392f
98e259b
741daa8
0d85574
8e71fd5
dcb98cc
658d58c
ca47bb0
097e5d2
dae73c5
a3fab55
7c4d1da
b42bfcd
bb01ef7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,7 +38,13 @@ | |
| from ami.ml.models.pipeline import Pipeline | ||
| from ami.ml.models.processing_service import ProcessingService | ||
| from ami.ml.models.project_pipeline_config import ProjectPipelineConfig | ||
| from ami.tests.fixtures.main import create_captures, create_occurrences, create_taxa, setup_test_project | ||
| from ami.tests.fixtures.main import ( | ||
| create_captures, | ||
| create_detections, | ||
| create_occurrences, | ||
| create_taxa, | ||
| setup_test_project, | ||
| ) | ||
| from ami.tests.fixtures.storage import populate_bucket | ||
| from ami.users.models import User | ||
| from ami.users.roles import BasicMember, Identifier, MLDataManager, ProjectManager, create_roles_for_project | ||
|
|
@@ -1390,6 +1396,40 @@ def test_unrelated_list_endpoints_still_work_without_project_id(self): | |
| self.assertEqual(response.status_code, status.HTTP_200_OK, path) | ||
|
|
||
|
|
||
| class TestCapturesProcessedFilter(APITestCase): | ||
| """ | ||
| The captures list supports ?has_detections=true|false, which the UI surfaces | ||
| as the "Processing status" filter. A capture is "processed" when it has any | ||
| Detection row (including null markers for "processed, found nothing"). | ||
| """ | ||
|
|
||
| def setUp(self) -> None: | ||
| self.project, self.deployment = setup_test_project(reuse=False) | ||
| self.captures = create_captures(self.deployment, num_nights=1, images_per_night=4) | ||
| # Mark the first two captures as processed by giving them a detection. | ||
| for capture in self.captures[:2]: | ||
| create_detections(capture, bboxes=[(0.1, 0.1, 0.2, 0.2)]) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Use pixel-space bbox fixtures in these new detection rows. These new tests use normalized-looking bbox values, but this repo’s detection bbox convention is pixel coordinates. Please switch to integer pixel boxes to match canonical behavior and avoid brittle assumptions in future validation paths. 💡 Proposed test fixture update- create_detections(capture, bboxes=[(0.1, 0.1, 0.2, 0.2)])
+ create_detections(capture, bboxes=[(10, 10, 20, 20)])
...
- create_detections(self.captures[0], bboxes=[(0.1, 0.1, 0.2, 0.2)])
+ create_detections(self.captures[0], bboxes=[(10, 10, 20, 20)])Based on learnings: Also applies to: 1460-1460 🤖 Prompt for AI Agents |
||
| self.user = User.objects.create_user(email="proc-filter@insectai.org", is_staff=True) # type: ignore | ||
| self.client.force_authenticate(user=self.user) | ||
| self.list_url = f"/api/v2/captures/?project_id={self.project.pk}" | ||
| return super().setUp() | ||
|
|
||
| def test_no_filter_returns_all_captures(self): | ||
| response = self.client.get(self.list_url) | ||
| self.assertEqual(response.status_code, status.HTTP_200_OK) | ||
| self.assertEqual(response.json()["count"], 4) | ||
|
|
||
| def test_has_detections_true_returns_only_processed(self): | ||
| response = self.client.get(f"{self.list_url}&has_detections=true") | ||
| self.assertEqual(response.status_code, status.HTTP_200_OK) | ||
| self.assertEqual(response.json()["count"], 2) | ||
|
|
||
| def test_has_detections_false_returns_only_unprocessed(self): | ||
| response = self.client.get(f"{self.list_url}&has_detections=false") | ||
| self.assertEqual(response.status_code, status.HTTP_200_OK) | ||
| self.assertEqual(response.json()["count"], 2) | ||
|
|
||
|
|
||
| class TestProjectOwnerAutoAssignment(APITestCase): | ||
| def setUp(self) -> None: | ||
| self.user_1 = User.objects.create_user(email="testuser@insectai.org", is_staff=True, is_superuser=True) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| # Captures list — "Processed / Not processed" filter | ||
|
|
||
| Date: 2026-05-28 | ||
| Status: design approved, pending spec review | ||
| Scope: first of several planned captures-list filters; this PR ships the processed filter only. | ||
|
|
||
| ## Goal | ||
|
|
||
| Add a "Processing status" filter to the Captures (SourceImage) list view, letting users | ||
| narrow to captures that have been processed, not processed, or all (no filter). Lay the | ||
| groundwork (a planned filter set) for additional filters in later PRs. | ||
|
|
||
| "Processed" = the image has been run through detection. Because PR #1093 writes a null | ||
| Detection marker for the "processed, found nothing" case, the presence of *any* Detection | ||
| row is an accurate signal of "was processed." | ||
|
|
||
| ## Backend — no change required | ||
|
|
||
| The filter already exists and is exercised by the list endpoint: | ||
|
|
||
| - `ami/main/api/views.py:630-636` — `SourceImageViewSet.filter_by_has_detections` | ||
| handles `?has_detections=true|false` by annotating | ||
| `Exists(Detection.objects.filter(source_image=OuterRef("pk")))` and filtering on it. | ||
| (`SourceImageViewSet` at `views.py:528`.) | ||
| - Called from `get_queryset` only for the `list` action (`views.py:600`), which is what | ||
| the captures list uses. | ||
|
|
||
| Decision: reuse the existing `has_detections` query param. Zero backend change, already | ||
| tested behavior. The param name (`has_detections`) means "was processed" because of the | ||
| null-marker convention; we surface it to users with the label "Processing status" and keep | ||
| `has_detections` as the internal query key. This name/meaning gap is the one known wart and | ||
| is documented here rather than fixed (a `was_processed` alias was considered and rejected to | ||
| avoid extra surface area). | ||
|
|
||
| ## Frontend — four wiring changes | ||
|
|
||
| 1. **New component** `ui/src/components/filtering/filters/processing-status-filter.tsx`. | ||
| Model on `verification-status-filter.tsx`. Two options: "Processed" (true) / | ||
| "Not processed" (false). Wire `onValueChange={onAdd}` directly so both true and false | ||
| are settable. (The generic `BooleanFilter` is unusable here: its "No" branch calls | ||
| `onClear()` instead of filtering to false — see `boolean-filter.tsx:21-27`.) | ||
| Use a translated label string for the two options (add to `utils/language` if needed). | ||
|
|
||
| 2. **Register the component** in `ui/src/components/filtering/filter-control.tsx` | ||
| `ComponentMap`: `has_detections: ProcessingStatusFilter`. | ||
|
|
||
| 3. **Register the filter** in `ui/src/utils/useFilters.ts` `AVAILABLE_FILTERS`: | ||
| `{ label: 'Processing status', field: 'has_detections', tooltip: { text: ... } }`. | ||
|
|
||
| 4. **Render it** on the captures page `ui/src/pages/captures/captures.tsx` (inside the | ||
| existing `FilterSection`, alongside `deployment` and `collections`): | ||
| `<FilterControl field="has_detections" />`. | ||
|
|
||
| State, URL params, page reset, and the clear-X ("All") behavior all come from the existing | ||
| `useFilters` machinery — no changes there. | ||
|
|
||
| ## Data flow | ||
|
|
||
| UI select -> `addFilter('has_detections', 'true'|'false')` -> URL search param -> | ||
| `useFilters` -> `useCaptures` builds `?has_detections=...` via `getFetchUrl` | ||
| (`ui/src/data-services/utils.ts`) -> DRF `filter_by_has_detections` -> filtered queryset. | ||
| Clear-X removes the param -> "All". | ||
|
|
||
| ## Testing | ||
|
|
||
| - Backend: verify existing coverage for `?has_detections=true|false` on the captures list | ||
| endpoint; add a test if missing (both branches + absent param). | ||
| - Frontend: manual verification against the running stack — select Processed, Not processed, | ||
| and clear; confirm result counts change and the URL param round-trips. | ||
|
|
||
| ## Out of scope (planned follow-up PRs) | ||
|
|
||
| To live in a collapsible "Advanced" `FilterSection` on the captures page later: | ||
|
|
||
| - **Date range** — `date_start`/`date_end` already in the FE registry with a `DateFilter` | ||
| component, but the SourceImage viewset needs backend support mapping them to a `timestamp` | ||
| range (new work). | ||
| - **Station** — already available via the existing `deployment` filter. | ||
| - **Site** — add `deployment__research_site` to `filterset_fields` + a Site filter component. | ||
| - **Device** — add `deployment__device` to `filterset_fields` + a Device filter component. |
Uh oh!
There was an error while loading. Please reload this page.