Skip to content

feat(cli): add gzip compression to v2 docs publish requests#16035

Draft
broady wants to merge 1 commit into
mainfrom
devin/1779338596-v2-gzip-compression
Draft

feat(cli): add gzip compression to v2 docs publish requests#16035
broady wants to merge 1 commit into
mainfrom
devin/1779338596-v2-gzip-compression

Conversation

@broady
Copy link
Copy Markdown
Contributor

@broady broady commented May 21, 2026

Description

Refs FER-10484

Adds transparent gzip compression to the v2 (current default) docs publish request bodies. This is the companion to #16033 which does the same for the docs-ledger write path.

Changes Made

  • New compressedFetch.ts: createGzipFetch() returns a fetch wrapper that pipes request bodies through CompressionStream("gzip") and sets Content-Encoding: gzip. Also exports gzipJsonBody() for raw fetch calls.
  • publishDocs.ts: The FdrClient is constructed while globalThis.fetch is temporarily replaced with the gzip fetch, so oRPC's LinkFetchClient captures it for all subsequent calls (register, finish, API registration).
  • publishDocs.ts: The raw fetch translation registration call now uses gzipJsonBody() to compress its JSON body.
  • Changelog entry added.

How it works

  1. oRPC's LinkFetchClient snapshots globalThis.fetch at construction time
  2. We temporarily swap in a gzip-compressing fetch before building the FdrClient, then restore immediately after
  3. All subsequent requests through the oRPC client use the captured gzip fetch — bodies are piped through CompressionStream("gzip") with Content-Encoding: gzip header
  4. FDR's @fastify/compress transparently decompresses on the server side
  5. File uploads to S3 (via axios) and dynamic IR uploads are unaffected

Testing

  • Biome lint, format, and check all pass
  • Manual testing completed

Link to Devin session: https://app.devin.ai/sessions/08fa84ed296b46629c0316c86ee1aca1
Requested by: @broady

Co-Authored-By: cbro <cbro@buildwithfern.com>
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@devin-ai-integration
Copy link
Copy Markdown
Contributor

Test Results: Gzip Compression for V2 Docs Publish

Tested by building the dev CLI with these changes and running end-to-end docs publish against registry-dev2.buildwithfern.com using DEV_SMOKE_TEST_FERN_TOKEN.

Content-Length Investigation (Key Concern)

Finding: Content-Length was never set in the first place — the "loss" concern is moot.

  • Node.js Request objects do NOT auto-set Content-Length for string bodies
  • oRPC's toFetchBody only sets Content-Length for Blob bodies, not JSON
  • Our headers.delete("Content-Length") is purely defensive (deletes a header that was never there)
  • Node.js fetch automatically uses transfer-encoding: chunked for streaming bodies
Test 1: Unit-Level Transport Verification

Local HTTP test server receiving gzip-compressed requests via the exact createGzipFetch() pattern:

  • Content-Encoding: gzip — present ✓
  • Content-Length — NOT PRESENT (never was) ✓
  • transfer-encoding: chunked — present (auto-set by Node.js) ✓
  • Decompressed body matches original payload exactly (1027 bytes) ✓
Test 2: E2E Docs Publish — Minimal Fixture (1 page)

2 FDR POST requests, all gzip'd, all 200 OK:

POST /v2/registry/docs/preview/init  | Content-Encoding: gzip | Content-Length: NOT SET | 200 OK
POST /v2/registry/docs/register/{id} | Content-Encoding: gzip | Content-Length: NOT SET | 200 OK

Published to: https://smoke-test-preview-2f0ca998-0a51-4059-8080-afec8491713e.docs.dev.buildwithfern.com

Test 3: E2E Docs Publish — Full Smoke-Test Fixture (multiple APIs, products, pages)

7 FDR POST requests, all gzip'd, all 200 OK:

POST /v2/registry/docs/preview/init   | Content-Encoding: gzip | Content-Length: NOT SET | 200 OK
POST /registry/api/register (x5)      | Content-Encoding: gzip | Content-Length: NOT SET | 200 OK
POST /v2/registry/docs/register/{id}  | Content-Encoding: gzip | Content-Length: NOT SET | 200 OK

14/14 assertions passed. Non-FDR requests (health, slugs, versions) correctly NOT compressed.

Devin session

@broady broady enabled auto-merge (squash) May 21, 2026 05:11
@broady broady requested a review from thesandlord May 21, 2026 05:11
@broady broady marked this pull request as draft May 21, 2026 06:52
auto-merge was automatically disabled May 21, 2026 06:52

Pull request was converted to draft

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant