Skip to content

perf(core): cache parsed JSON request body to avoid redundant decoding#13377

Open
AlinsRan wants to merge 7 commits into
masterfrom
perf/cache-json-request-body
Open

perf(core): cache parsed JSON request body to avoid redundant decoding#13377
AlinsRan wants to merge 7 commits into
masterfrom
perf/cache-json-request-body

Conversation

@AlinsRan
Copy link
Copy Markdown
Contributor

@AlinsRan AlinsRan commented May 15, 2026

Summary

get_json_request_body_table() previously called json.decode() on every invocation with no caching. In the ai-proxy plugin request path, this function is called 3 times per request across different phases:

Request
  │
  ├─ Route matching (access phase, ctx.lua)
  │    └─ post_arg.* variable resolution
  │         └─ get_json_request_body_table()   ← decode #1
  │
  ├─ ai-proxy access phase (ai-proxy/base.lua: detect_request_type)
  │    └─ detect protocol (chat / responses / anthropic), extract model
  │         └─ get_json_request_body_table()   ← decode #2
  │
  └─ ai-proxy before_proxy phase (ai-proxy/base.lua: before_proxy)
       └─ build upstream HTTP request, apply model options, auth
            └─ get_json_request_body_table()   ← decode #3

(+ decode #4 if ai-rag or ai-aliyun-content-moderation plugin is also active)

Each decode is redundant: the body bytes have not changed between phases.

Changes

apisix/core/request.lua

  • Add get_request_body_table(ctx): a unified, content-type-aware body parser that handles application/json, application/x-www-form-urlencoded, and multipart/form-data. Caches the parsed result in ctx._request_body_tab for the lifetime of the request.
  • Rewrite get_json_request_body_table() as a thin wrapper: validates Content-Type is application/json (returning an error for non-JSON requests to prevent type confusion), then delegates to get_request_body_table().

apisix/core/ctx.lua

Remove the local _get_parsed_request_body / get_parsed_request_body functions and the multipart dependency. The post_arg.* variable resolution now calls request.get_request_body_table(ctx) directly, sharing the same cache.

apisix/patch.lua

Simplify the set_body_data patch: use _request_body_tab as the single guard. If it is non-nil, clear it and scan var._cache for stale post_arg.* entries. If it is nil, body was never parsed — skip both cleanups entirely.

Performance

Benchmarked on a single APISIX worker with a mock upstream (100 ms fixed delay, ~10 KB response):

Body size Before (3 decodes/req) After (1 decode/req) Gain
1 MB ~70 RPS ~180 RPS +157%
10 MB ~7 RPS ~16 RPS +129%

CPU was the bottleneck in all cases. Gain scales with body size because json.decode cost grows linearly with input.

Test

Added TEST 17 in t/core/request.t:

  • Patches json.decode with a call counter
  • Verifies 3 successive calls produce only 1 decode and return the same table object
  • Calls ngx.req.set_body_data() to replace the body, then verifies the next call re-decodes (counter becomes 2) and returns the new content

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. performance generate flamegraph for the current PR labels May 15, 2026
Comment thread apisix/patch.lua Outdated
membphis
membphis previously approved these changes May 15, 2026
@AlinsRan AlinsRan dismissed stale reviews from shreemaan-abhishek and membphis via 3ae1190 May 15, 2026 06:41
@AlinsRan AlinsRan force-pushed the perf/cache-json-request-body branch from 3105a55 to 3ae1190 Compare May 15, 2026 06:41
nic-6443
nic-6443 previously approved these changes May 15, 2026
@AlinsRan AlinsRan force-pushed the perf/cache-json-request-body branch from 3ae1190 to 9bf8392 Compare May 15, 2026 08:19
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels May 15, 2026
@AlinsRan AlinsRan force-pushed the perf/cache-json-request-body branch 3 times, most recently from f8a25a7 to 3cff6a7 Compare May 15, 2026 08:36
`get_json_request_body_table()` is called on every request by plugins that
need to inspect or modify the JSON request body (e.g. AI proxy, body
transformations). Each call previously triggered a full `json.decode()`,
even when the body had not changed.

This commit adds a per-request cache in `ctx._request_body_tab`. On the
first call the body is decoded and stored; subsequent calls within the same
request return the cached table directly, skipping redundant decoding.

The existing `set_body_data` patch in `patch.lua` is extended to also
clear `_request_body_tab`, so any plugin that rewrites the body will
cause the next call to re-decode from the new content.

Performance impact (single APISIX worker, 1 MB JSON body):
- Before: ~70 RPS  (4 redundant decodes per request)
- After:  ~180 RPS (1 decode per request)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@AlinsRan AlinsRan force-pushed the perf/cache-json-request-body branch from 3cff6a7 to 3c55f9f Compare May 15, 2026 09:06
AlinsRan and others added 6 commits May 15, 2026 17:56
Replace abstract 'json' string with CONTENT_TYPE_JSON constant as the
content_type parameter to get_request_body_table(), so callers and
_request_body_type use the same content-type string constants throughout.

Also update patch.lua to clear _request_body_type (replacing the old
_json_request_body_tab approach) and fix error message pattern in test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…lback

Use 'local ct = content_type or _M.header(ctx, "Content-Type") or ""'
so the caller-supplied content_type takes precedence over the header,
removing the separate content_type == CONTENT_TYPE_JSON branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This block was accidentally removed in a previous commit. Restore it
to match upstream master behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Restore get_post_args signature to (ctx) with req_get_post_args(0) to
match upstream master. Remove extra blank line in get_body.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance generate flamegraph for the current PR size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants