Skip to content

Fix ContextExtension serialization order#843

Open
Luivatra wants to merge 1 commit into
ergoplatform:developfrom
Luivatra:fix/context-extension-hamt-order
Open

Fix ContextExtension serialization order#843
Luivatra wants to merge 1 commit into
ergoplatform:developfrom
Luivatra:fix/context-extension-hamt-order

Conversation

@Luivatra
Copy link
Copy Markdown

@Luivatra Luivatra commented Feb 6, 2026

Root Cause: Issue #763

When a ContextExtension has 5+ entries, the Ergo node (Scala 2.12) serializes them in HAMT hash-based order, not sorted/insertion order. sigma-rust used insertion order for all sizes, causing bytes_to_sign to differ, which makes the TxId different,
which invalidates the proof signature.

Threshold behavior:

  • 1-4 entries: Scala uses Map1-Map4 (insertion order) — matches sigma-rust
  • 5+ entries: Scala uses HashMap (HAMT iteration order) — diverges from sigma-rust

The Fix

Single file modified: ergotree-ir/src/chain/context_extension.rs

What was added:

  1. scala_212_improve(hc: i32) -> i32 — Replicates Scala 2.12's hash improvement function used by its HAMT
  2. scala_212_hamt_sort_key(key: u8) -> u64 — Computes a sort key that matches HAMT iteration order (encodes 5-bit slots at each trie level)
  3. Modified sigma_serialize — For 5+ entries, sorts by HAMT slot order before writing; for <5, preserves insertion order (unchanged behavior)

Tests added (4):

  • test_scala_212_improve — Verifies level-0 HAMT slots match empirically observed node behavior
  • test_hamt_sort_order_6_entries — Keys {0,1,2,3,4,5} sort to [0,5,1,2,3,4] matching the node
  • test_serialize_order_5plus_entries — Verifies serialized bytes have keys in HAMT order
  • test_serialize_order_4_entries_unchanged — Verifies <5 entries still use insertion order

Match the Scala 2.12 HashMap iteration order for ContextExtension
serialization when there are 5 or more entries. This is necessary to
maintain compatibility with the Ergo node, which uses Scala 2.12's
HashMap. For fewer than 5 entries, the existing insertion order
preservation is maintained.
@a-shannon
Copy link
Copy Markdown

I added a small test-only patch PR against this branch: Luivatra#1

It adds Scala 2.12 ContextExtension golden vectors for 8, 32, and 128 ids, plus an exact serialized-byte vector for the 8-id case. This is meant to make the JVM consensus traversal order explicit in sigma-rust CI for this fix.

Local checks on the patch:

  • cargo fmt --check -p ergotree-ir: PASS
  • git diff --check: PASS

I could not run cargo test in a fresh clone because the workspace has no Cargo.lock and resolves core2 = 0.4.0, which crates.io reports as yanked. I did not change dependency metadata in the test-only patch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants