Heal LLM-simplified citations to canonical reference - #3069
Open
John CSA (jluocsa) wants to merge 3 commits into
Open
Heal LLM-simplified citations to canonical reference#3069John CSA (jluocsa) wants to merge 3 commits into
John CSA (jluocsa) wants to merge 3 commits into
Conversation
…ples#2950) Adds backend post-processing of the LLM answer so simplified citation tokens (e.g. [PyCon US 2025.pdf#page=1]) are mapped back to the canonical indexed reference via exact match, else a unique endswith match. Complements the frontend fix so corrected citations are present in the raw response for streaming, API consumers, and exports - matching the maintainer's suggested Solution Azure-Samples#1 (post-process the LLM response).
John CSA (jluocsa)
force-pushed
the
fix/heal-simplified-citations
branch
from
May 31, 2026 01:54
3c3acda to
006841c
Compare
Contributor
Author
|
Updated this PR to implement the maintainer's suggested Solution #1 (post-process the LLM response) server-side, in addition to the original frontend safety net. What changed
This means corrected citations are now present in the raw response itself (streaming, API consumers, exports), not just the rendered UI. Rebased onto current |
…-citations # Conflicts: # tests/test_chatapproach.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Fixes #2950.
When a document filename contains leading special characters (e.g. a leading
hyphen, as produced for some user-uploaded files:
- PyCon US 2025.pdf), the LLMtends to "simplify" the citation it emits, dropping the leading characters. For
example, given the source
- PyCon US 2025.pdf#page=1, the model writes[PyCon US 2025.pdf#page=1]in its answer.The frontend citation parser validated the cited text with
endsWith, whichcorrectly recognized it as a valid citation, but then used the simplified text
as the citation reference. The resulting
/content/<path>request usedPyCon US 2025.pdfinstead of the indexed- PyCon US 2025.pdf, socheck_path_auth()filtered by the wrongsourcefilevalue and the documentreturned 403/404.
This change resolves the cited text back to the canonical citation from
data_points.citations(exact match preferred, otherwise theendsWithmatch)and uses that canonical value for the reference and for the
citation_activity_detailslookup. Links and lookups now use the indexed value,so the content route resolves correctly.
Does this introduce a breaking change?
Citations that already match exactly continue to resolve exactly (the exact match
is preferred). Only previously-broken simplified citations change behavior.
Does this require changes to learn.microsoft.com docs?
Type of change
Code quality checklist
tsctype-checks cleanly (npx tsc --noEmit).prettierapplied to the changed file.python -m pytest).