Skip to content

feat(http): add int64_as_string parameter to query endpoints#6699

Open
waynercheung wants to merge 9 commits intotronprotocol:developfrom
waynercheung:feat/support_param_int64
Open

feat(http): add int64_as_string parameter to query endpoints#6699
waynercheung wants to merge 9 commits intotronprotocol:developfrom
waynercheung:feat/support_param_int64

Conversation

@waynercheung
Copy link
Copy Markdown
Collaborator

@waynercheung waynercheung commented Apr 23, 2026

What does this PR do?

Closes #6568. Adds an optional int64_as_string request parameter on whitelisted query servlets. When set to true, int64/uint64 protobuf fields in the response are serialized as quoted JSON strings, avoiding precision loss in clients whose native number type cannot safely represent integers above 2^53 - 1 (most notably JavaScript).

Design:

  • Core mechanism: JsonFormat.pushInt64AsString(boolean) returns an AutoCloseable controlling a scope-bound ThreadLocal. Each whitelisted servlet wraps the final response write in try (AutoCloseable ignored = JsonFormat.pushInt64AsString(flag)) { ... }.
  • Parameter reading mirrors each servlet's existing parameter-reading shape where applicable (GET query / POST body). One special case: GetRewardServlet.doPost parses both address and int64_as_string from the same JSON body in a single read.
  • Opt-in whitelist: write paths, endpoints bypassing JsonFormat, and endpoints whose response has no int64 fields are left untouched.
  • JsonFormat.merge is deliberately not modified — input contract stays strict (see "Known limitation" below).
  • /walletsolidity/* routes inherit the feature via Spring bean reuse in SolidityNodeHttpApiService (two solidity-specific servlets — GetTransactionByIdSolidityServlet and GetTransactionInfoByIdSolidityServlet — needed explicit edits).
  • /walletpbft/* routes inherit via Java subclassing: the PBFT query servlets covered here extend their corresponding wallet servlet and only wrap it in walletOnPBFT.futureGet(...), so the superclass's int64_as_string handling applies without further changes.

Why are these changes required?

Full rationale and scope discussion in #6568. TL;DR: JS clients lose precision parsing int64 values like balance, total_supply, and asset/market balances once the value exceeds 2^53 - 1. This PR gives clients an opt-in way to receive those fields as strings.

Supported /wallet/* endpoints (this list enumerates /wallet/* endpoints only; equivalent /walletsolidity/* and /walletpbft/* routes are supported where those routes already exist and reuse the same servlet logic per the mechanisms described above):

  • Account / asset / resource queries — getaccount, getaccountresource, getaccountnet, getaccountbyid, getaccountbalance, getassetissuebyid, getassetissuebyname, getassetissuelist, getassetissuelistbyname, getassetissuebyaccount, getpaginatedassetissuelist, getdelegatedresource(v2), getdelegatedresourceaccountindex(v2), getcanwithdrawunfreezeamount, getavailableunfreezecount, getcandelegatedmaxsize
  • Block / transaction queries — getnowblock, getblock, getblockbynum, getblockbyid, getblockbylimitnext, getblockbylatestnum, getblockbalance, gettransactionbyid, gettransactionfrompending, gettransactioninfobyid, gettransactioninfobyblocknum, gettransactionreceiptbyid, gettransactioncountbyblocknum, totaltransaction
  • Governance / chain queries — listwitnesses, listproposals, listexchanges, getpaginatednowwitnesslist, getpaginatedproposallist, getpaginatedexchangelist, getchainparameters, getnextmaintenancetime, getproposalbyid, getexchangebyid
  • Contract / market queries — getcontract, getcontractinfo, getmarketorderbyaccount, getmarketorderbyid, getmarketorderlistbypair, getmarketpricebypair
  • Other int64-bearing queries — getReward, getburntrx, getpendingsize
    Explicitly not supported (code unchanged; the flag is silently ignored):
Endpoint(s) Reason
getnodeinfo Response is serialized via JSON.toJSONString(nodeInfo) on a POJO, bypassing JsonFormat; the pushInt64AsString helper does not apply. NodeInfo has long fields that do fall within this issue's domain; supporting them needs a separate fastjson-based mechanism. Deferred to follow-up as a deliberate scope cut, not a claim of "no precision risk".
getsignweight, getapprovedlist Response embeds a TransactionExtention that clients round-trip back for signing.
getBrokerage Response field is a 0-100 int, not long.
getbandwidthprices, getenergyprices, getmemofee Response (PricesResponseMessage.prices) has no int64 fields.
gettransactionlistfrompending TransactionIdList has no int64 fields (repeated string txId only).
getmarketpairlist MarketOrderPair has only bytes token ids.
listnodes NodeList/Node/Address contain only string ip and int32 port.
All create* / trigger* / broadcast* / deploy* / transfer* / freeze* / unfreeze* / vote* / proposal* / exchange* (create/inject/transaction/withdraw) / write-path market endpoints (e.g. marketsellasset, marketcancelorder) / participate* / withdraw* Write paths; their responses enter JsonFormat.merge on the client side for signing and re-broadcasting.

This PR has been tested by:

  • Unit Tests:
    • JsonFormatInt64AsStringTest (20 cases): default behavior, int64/uint64 quoting, non-int64 fields unaffected, nested / map / boundary values (2^53 ± 1, Long.MAX/MIN, -1), scope lifecycle (clean close, exception, explicit close, nested scopes), thread isolation, thread-reuse anti-pollution, PostParams body parsing.
    • UtilMockTest (+3 cases): Util.decodeAddress blank-input and mainnet-hex branches, Util.processAddressError error-JSON shape.
  • Integration Tests (one representative per template shape):
    • GetRewardServletTest — hand-rolled JSON servlet + POST JSON body fix.
    • GetNowBlockServletTest — JsonFormat path with doPost → doGet delegation.
    • GetBlockServletTest (new) — shared private handle() with custom parseParams.
    • GetPaginatedProposalListServletTest (new) — doPost that manually reads the body via Util.getInt64AsStringPost.
    • ListExchangesServletTest (new) — URL-query-only POST contract. testPostBodyFlagIsIgnored pins the counter-intuitive "body-only flag must not take effect" semantic against future refactors.
  • All existing tests in org.tron.core.services.http.* continue to pass.
  • ./gradlew :framework:checkstyleMain :framework:checkstyleTest passes.

Follow up

  • developers.tron.network API reference needs an int64_as_string parameter entry for each supported endpoint plus a top-level section documenting GET vs POST read paths and the round-trip limitation.
  • GetNodeInfoServlet: add int64_as_string support via a fastjson local SerializeConfig binding Long/long to ToStringSerializer, since this endpoint bypasses JsonFormat. Deferred as a deliberate scope cut.

Extra details

  • Known limitation — round-trip on Transaction / Block responses

JsonFormat.merge() only accepts unquoted 64-bit integer tokens. As a result, responses from block and transaction endpoints whose payload embeds a Transaction or Block protobuf (the getnowblock / getblock* family and gettransactionbyid / gettransactionfrompending, plus their /walletsolidity/* and /walletpbft/* counterparts where exposed) cannot be locally parsed back into a protobuf message via JsonFormat.merge when int64_as_string=true is set. This is a deliberate design choice, not a bug — keeping merge strict preserves backward-compatible input semantics and avoids widening the signing/broadcast input contract. Clients needing round-trip simply don't set the flag.

  • Thread safety & wire compatibility

  • Preserve-restore ThreadLocal with try-with-resources guarantees cleanup on normal return, exception, and nested scopes; explicitly validated against Jetty's QueuedThreadPool reuse by noPollutionOnThreadReuse and threadIsolation unit tests.

  • Zero protobuf schema changes. Default behavior (flag absent) is preserved: existing clients see no change in response encoding.

  • GetRewardServlet.doPost was restructured so application/json POST parses both address and int64_as_string from a single body read — fixes a body-internal inconsistency surfaced during review (address from body, flag from URL query). Form-encoded POSTs still delegate to doGet.

…#6568

Introduce JsonFormat.pushInt64AsString(boolean) returning an AutoCloseable,
using a strictly scoped ThreadLocal to serialize int64/uint64 protobuf fields
as quoted JSON strings within try-with-resources blocks. This avoids precision
loss in clients whose native number type cannot safely represent integers
above 2^53 - 1 (e.g. JavaScript).

- JsonFormat: add INT64_AS_STRING ThreadLocal + pushInt64AsString helper;
  split printFieldValue INT64/SINT64/SFIXED64 and UINT64/FIXED64 branches so
  they emit quoted strings only when the helper is active.
- Util: add INT64_AS_STRING constant + getInt64AsString (URL query, mirrors
  getVisible) + getInt64AsStringPost (JSON body, mirrors getVisiblePost).
- PostParams: add int64AsString field; 3-arg primary constructor; keep legacy
  2-arg constructor as adapter defaulting to false; getPostParams parses the
  new field from the body.
- Test: JsonFormatInt64AsStringTest covers default behavior, int64/uint64
  quoting, non-int64 fields unaffected, nested/map/boundary values (2^53 +/-
  1, Long.MAX/MIN, -1), scope lifecycle (clean after normal close, after
  exception, after explicit close, across nested scopes), thread isolation,
  thread-reuse anti-pollution, PostParams body parsing, and the adapter
  constructor defaults.
tronprotocol#6568

Thread the new JsonFormat.pushInt64AsString helper through every query
servlet whose response contains int64/uint64 fields, preserving existing
`visible` semantics on each route. Parameter reading mirrors visible:
GET reads URL query via Util.getInt64AsString; POST that uses PostParams
reads the JSON body via PostParams.isInt64AsString; the single POST path
that manually reads the request body (GetPaginatedProposalListServlet)
calls Util.getInt64AsStringPost directly.
…ve servlets

Extend GetRewardServletTest and GetNowBlockServletTest with int64_as_string matrix cases:
- default (no param) preserves pre-existing number output
- int64_as_string=true stringifies the int64 field(s) in the response
- int64_as_string=false behaves identically to the default
…ckServlet.parseParams

The GET branch of GetBlockServlet.parseParams previously constructed a 3-arg
PostParams inline with Boolean.parseBoolean(request.getParameter(Util.VISIBLE))
for visible and Util.getInt64AsString(request) for int64_as_string. Extract
both into named local booleans so the call site is symmetric and readable
without needing a comment, and switch the visible read to Util.getVisible(request)
to align with how every other servlet in this package reads the flag. Behavior
is unchanged: Util.getVisible is the standard Boolean.parseBoolean + null-safety
wrapper already defined in Util.java.
…aginatedProposalListServlet

Two whitelisted servlets have non-standard integration shapes that deserve
their own end-to-end tests beyond what JsonFormatInt64AsStringTest covers:

- GetBlockServlet delegates both doGet and doPost to a private handle()
  method whose parseParams() assembles a PostParams from either URL query
  (GET) or JSON body (POST). GetBlockServletTest exercises the GET query
  path, the POST body path (int64_as_string=true / =false), and verifies
  that URL query is ignored on POST (mirroring visible semantics).

- GetPaginatedProposalListServlet.doPost manually reads the request body
  and bypasses PostParams.getPostParams, so it takes the direct
  Util.getInt64AsStringPost(input) branch instead of params.isInt64AsString().
  GetPaginatedProposalListServletTest exercises that branch plus the regular
  doGet path.
A later review of FullNodeHttpApiService surfaced these query servlets as
also returning int64 fields:

getproposalbyid, getexchangebyid, getcontract, getcontractinfo,
getmarketorderbyaccount, getmarketorderbyid, getmarketorderlistbypair,
getmarketpricebypair, getblockbalance, getnextmaintenancetime.
Cover Util.decodeAddress (blank-input and mainnet-hex branches) and
Util.processAddressError in UtilMockTest; both were previously exercised
only indirectly.
ListExchangesServlet.doPost reads visible and int64_as_string from the
URL query, not from the JSON body -- a counter-intuitive shape shared
only by GetChainParametersServlet. Without a test, a future refactor
that "normalizes" this POST path to read from the body would silently
break clients that pass the flag via the URL.
public static void processAddressError(Exception e, HttpServletResponse response) {
try {
response.getWriter()
.println("{\"Error\": " + "\"INVALID address, " + e.getMessage() + "\"}");
@waynercheung waynercheung reopened this Apr 23, 2026
@waynercheung waynercheung changed the title feat(http): add int64_as_string parameter to query endpoints feat(http): add int64_as_string parameter to query endpoints Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regarding the potential numeric overflow issue when querying asset values

2 participants