fix(recall): widen context tag candidate fetch#215
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
context_profile.priority_tags, socontext_tagsboosts can promote candidates outside a small requested limitcontext_tagsCloses #201.
Public/API Notes
/recallresponse shape is unchanged.context_tagsremains a soft boost; this only widens the pre-scoring vector candidate pool when a semantic query or embedding is present.context_tags.Validation
tests/test_recall_overfetch.py::test_context_tags_widen_vector_fetch_before_scoringfailed with requested vector limit20instead of50tests/lab/test_lab_corpus.py::test_recall_serializes_paramsfailed with unexpectedcontext_tagstests/lab/test_run_recall_test.py::test_run_single_query_forwards_row_context_tagsfailed because row-level tags were ignoredPYTEST_DISABLE_PLUGIN_AUTOLOAD=1 .venv/bin/pytest tests/test_recall_overfetch.py tests/test_context_tag_separator.py -q-> 17 passedPYTEST_DISABLE_PLUGIN_AUTOLOAD=1 .venv/bin/pytest tests/lab/test_lab_corpus.py tests/lab/test_run_recall_test.py -q-> 11 passedPYTEST_DISABLE_PLUGIN_AUTOLOAD=1 .venv/bin/pytest tests/test_api_endpoints.py -q -k recall-> 69 passed, 124 deselectedmake test-> 631 passed, 1 skipped, 65 deselectedmake lint-> passedgit diff --check-> passedpython -m json.tool lab/test_sets/queries_context_tags_20260704.json-> passedmake bench-eval BENCH=locomo-mini-> 84.26% overall accuracy (198/235), 367.5s, 69 cat-5 questions skipped by judge-off run; matches latest comparable local LoCoMo-mini artifactmake bench-eval BENCH=locomo-> 81.91% overall accuracy (1263/1542), 2072.6s, 444 cat-5 questions skipped by judge-off run; slightly above latest comparable local LoCoMo-full artifact (81.84%, 1262/1542)Live Lab A/B
Ran two isolated local stacks from the same restored production snapshot,
prod-api-20260611-post-repair-001241:origin/main(7636021) athttp://localhost:8011a29e9fb) athttp://localhost:8012memory_count=9900,vector_count=9900,sync_status=syncedContext-tag probes (
lab/test_sets/queries_context_tags_20260704.json,limit=5,current_only=false):0.000, MRR0.000, NDCG@100.000, 5 misses1.000, MRR1.000, NDCG@101.000, 0 missesPlain production set (
lab/test_sets/queries_prod_20260611.json,limit=20,current_only=true):0.045, MRR0.024, NDCG@100.029, latency avg765ms0.275, MRR0.220, NDCG@100.231, latency avg972ms208msavg on this local restored stackThe context-tag probe is the direct #201 validation signal. The plain-query production lift is observational and likely reflects existing derived
context_profile.priority_tagsnow widening the candidate pool as intended. Ignored result files are underlab/results/issue201/, includingissue201_live_ab_summary.json.