Studio: extract text from PDF web results#7154
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for fetching and extracting text from PDF files during web searches, handling mislabeled PDFs via magic byte sniffing, and enforcing download and page limits. The review feedback suggests defensively converting the pages range object to a list before passing it to pymupdf4llm.to_markdown to prevent potential compatibility issues.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
oobabooga
force-pushed
the
studio/web-fetch-pdf-extract
branch
from
668691e to
e68dbc6
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
1 participant
web_searchcurrently treats PDF URLs as non-text content, so Studio cannot read papers, manuals, or other search results served as PDFs even though the RAG pipeline already supports PDF text extraction.Fix
Direct PDF fetches now reuse the RAG PyMuPDF/PyMuPDF4LLM parser and return page-delimited text to the model. This works for correctly labeled PDFs and PDFs served as
text/plain,text/html,application/octet-stream, or without a Content-Type.PDF downloads are capped at 10 MiB and read through EOF so the parser receives the cross-reference data. Extraction is limited to 50 pages and 16,000 characters. Oversized, malformed, encrypted, and textless PDFs return short placeholders instead of raw bytes or tracebacks. Other binary formats remain rejected.
Verification
Added regression coverage for declared and mislabeled PDFs, multi-page extraction, the RAG byte-stream path, download and processing limits, preserved page formatting, deadlines, encrypted and malformed files, and PDFs without a text layer.
Focused and related suites: 158 passed, 1 optional dependency test skipped. Repository formatting, Ruff, and whitespace checks also pass.