From 969456352ab67cf8400a33351b034c154a0ec96d Mon Sep 17 00:00:00 2001 From: Muad'Dib Date: Wed, 8 Apr 2026 12:41:26 +0200 Subject: [PATCH] ergo-nipopow: add prove_with_reader for db-backed proof construction MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Direct port of JVM `org.ergoplatform.modifiers.history.popow.NipopowProverWithDbAlgs.prove`, the production-side prover used by Ergo nodes to serve NiPoPoW proofs to peers. Unlike the existing `NipopowAlgos::prove(chain: &[PoPowHeader])` (which is paper-style and requires the caller to materialize every block as a `PoPowHeader` up front), `prove_with_reader` walks the interlink hierarchy via a new `PopowHeaderReader` callback and only requests `PoPowHeader`s for blocks the walk actually visits. Asymptotic win: ~`m + k + m * log2(N)` `popow_header_by_id` calls per proof, vs `O(N)` for the in-memory variant. At the JVM P2P defaults `m = 6, k = 10` on a `N ~= 270k` testnet chain that's roughly 120 fetches versus 270k — three orders of magnitude fewer. The in-memory `prove` makes serving NiPoPoW proofs over P2P unviable on long chains (it stalls sync, mining, and mempool for the duration of a single peer's `GetNipopowProof`); `prove_with_reader` is the foundation for making the Rust ergo node a viable NiPoPoW serving peer. Changes (all in `ergo-nipopow`, plus a test in `ergo-chain-generation`): * New `PopowHeaderReader` trait (`ergo-nipopow/src/popow_header_reader.rs`) with five methods: `headers_height`, `popow_header_by_id`, `popow_header_at_height`, `last_headers`, `best_headers_after`. Re-exported from `lib.rs`. The trait is the minimal Rust analogue of the subset of `ErgoHistoryReader` that the JVM `prove` requires. * New `NipopowAlgos::prove_with_reader`, with private helpers `links_with_indexes`, `previous_header_id_at_level`, `collect_level`, and `prove_prefix` ported line-by-line from the JVM source. The Scala `@tailrec` recursion in `collectLevel` is translated to an explicit loop. The `mutable.TreeMap[ModifierId, PoPowHeader]` accumulator becomes a `BTreeMap`. * New `NipopowProofError::MissingPopowHeader` variant. The JVM source uses `Try` and `.get` on every `popowHeader` lookup; the Rust port returns `Err(MissingPopowHeader)` instead, signalling a reader that is inconsistent with the chain it claims to expose. The existing `NipopowAlgos::prove(&[PoPowHeader])` is left untouched — it remains in use by `ergo-chain-generation`'s tests and is still the canonical paper-style implementation. Known divergence from the JVM source: the JVM `prove` accepts a `params.continuous` flag that, when set, embeds extra epoch-boundary popow headers into the prefix so peers can validate difficulty for blocks past the suffix without further sync. This Rust port does NOT implement that mode: sigma-rust's `NipopowProof` currently has no `continuous` field, so adding it requires coordinated changes to the struct, the serializer, and the on-wire format — out of scope for this PR. `prove_with_reader` always produces non-continuous proofs. JVM peers applying non-continuous proofs still succeed (`applyPopowProof` does not strictly require the flag), they just cannot self-validate difficulty for blocks beyond the suffix until they sync more headers — fine for the P2P serve use case. Continuous mode is tracked as a follow-up. Testing: * `test_nipopow_prove_with_reader_matches_in_memory` (`ergo-chain-generation/src/fake_pow_scheme.rs`) asserts byte-for-byte equivalence between `prove(&chain)` and `prove_with_reader(&reader, None, k, m)` on a 100-block synthetic chain at the JVM P2P defaults `m = 6, k = 10`. This is the Rust counterpart to the JVM `PoPowAlgosWithDBSpec`-"proof(chain) is equivalent to proof(histReader)" test. The test is in `fake_pow_scheme.rs` (rather than `chain_generation.rs`) because byte-for-byte equivalence only holds when every non-genesis block has `max_level_of >= 1`: the in-memory `prove` adds blocks at its level-0 iteration that the db-backed interlink walk never visits, so on a chain with level-0-only blocks (which the real-Autolykos `chain_generation.rs` generator produces) the prefixes legitimately diverge. The fake pow scheme forces `d = order / (height + 1)`, guaranteeing positive level for every block — exactly mirroring `DefaultFakePowScheme` in the JVM spec, which uses `d = q / (height + 10)` for the same reason. * `test_nipopow_prove_with_reader_valid_on_real_autolykos_chain` (`ergo-chain-generation/src/chain_generation.rs`) is a sanity check on the real-Autolykos generator: the resulting proof has valid connections, the suffix has length `k`, and the db-backed prefix is a subset of the in-memory prefix (since the db-backed walk only visits level-1+ blocks via interlinks). * `test_nipopow_prove_with_reader_explicit_header_id` (`ergo-chain-generation/src/chain_generation.rs`) covers the `header_id_opt = Some(..)` branch: the resulting proof's suffix head matches the requested header, the suffix tail contains the next `k - 1` headers in ascending-height order, and the proof has valid connections. Verification: cargo test --workspace -p ergo-nipopow -p ergo-chain-generation 883 passed; 0 failed cargo clippy --workspace -p ergo-nipopow --all-targets No issues cargo clippy --workspace -p ergo-chain-generation --all-targets No issues Co-Authored-By: Claude Opus 4.6 (1M context) --- ergo-chain-generation/src/chain_generation.rs | 149 +++++++++++- ergo-chain-generation/src/fake_pow_scheme.rs | 112 ++++++++- ergo-nipopow/src/lib.rs | 2 + ergo-nipopow/src/nipopow_algos.rs | 213 ++++++++++++++++++ ergo-nipopow/src/nipopow_proof.rs | 7 + ergo-nipopow/src/popow_header_reader.rs | 73 ++++++ 6 files changed, 554 insertions(+), 2 deletions(-) create mode 100644 ergo-nipopow/src/popow_header_reader.rs diff --git a/ergo-chain-generation/src/chain_generation.rs b/ergo-chain-generation/src/chain_generation.rs index 40e543507..d0ef0440f 100644 --- a/ergo-chain-generation/src/chain_generation.rs +++ b/ergo-chain-generation/src/chain_generation.rs @@ -367,7 +367,9 @@ fn transactions_root(txs: &[Transaction], block_version: u8) -> Digest32 { #[cfg(test)] mod tests { use super::*; - use ergo_nipopow::{NipopowAlgos, NipopowProof, PoPowHeader}; + use ergo_lib::ergo_chain_types::Header; + use ergo_nipopow::{NipopowAlgos, NipopowProof, PoPowHeader, PopowHeaderReader}; + use std::collections::HashMap; fn generate_popowheader_chain(len: usize, start: Option) -> Vec { block_stream(start.map(|p| ErgoFullBlock { @@ -505,4 +507,149 @@ mod tests { assert_eq!(serde_json::from_str::(&json).unwrap(), header); } } + + /// In-memory `PopowHeaderReader` backed by a synthetic chain. Lives in + /// the test module because `ergo-nipopow` cannot depend on + /// `ergo-chain-generation` (circular dep). + struct MockReader { + by_id: HashMap, + by_height: Vec, // index 0 = height 1 (genesis) + } + + impl MockReader { + fn from_chain(chain: &[PoPowHeader]) -> Self { + let mut by_id = HashMap::with_capacity(chain.len()); + let mut by_height: Vec = Vec::with_capacity(chain.len()); + for ph in chain { + by_id.insert(ph.header.id, ph.clone()); + by_height.push(ph.clone()); + } + // Sanity: heights must be 1..=len contiguous starting at 1. + for (i, ph) in by_height.iter().enumerate() { + assert_eq!(ph.header.height as usize, i + 1); + } + Self { by_id, by_height } + } + } + + impl PopowHeaderReader for MockReader { + fn headers_height(&self) -> u32 { + self.by_height.len() as u32 + } + + fn popow_header_by_id(&self, id: &BlockId) -> Option { + self.by_id.get(id).cloned() + } + + fn popow_header_at_height(&self, height: u32) -> Option { + // Genesis is height 1, so subtract 1 to index into by_height. + if height == 0 { + return None; + } + self.by_height.get((height - 1) as usize).cloned() + } + + fn last_headers(&self, k: usize) -> Vec
{ + let len = self.by_height.len(); + let start = len.saturating_sub(k); + self.by_height[start..] + .iter() + .map(|ph| ph.header.clone()) + .collect() + } + + fn best_headers_after(&self, header: &Header, n: usize) -> Vec
{ + // Heights are 1-indexed; the slot immediately after `header` is + // at vec index `header.height` (because by_height[0] is height 1). + let start = header.height as usize; + let end = (start + n).min(self.by_height.len()); + if start >= end { + return Vec::new(); + } + self.by_height[start..end] + .iter() + .map(|ph| ph.header.clone()) + .collect() + } + } + + /// Sanity check: on a chain produced by the real Autolykos chain + /// generator, `prove_with_reader(None)` produces a proof whose + /// connections validate, the suffix has length `k`, and the prefix is + /// no larger than the in-memory `prove`'s prefix (the db-backed + /// algorithm walks the interlink hierarchy and never visits level-0 + /// blocks, so its prefix is always a subset of the in-memory one when + /// some blocks have `max_level_of == 0`). + /// + /// Strict byte-for-byte equivalence between `prove` and + /// `prove_with_reader` only holds when *every* block in the chain has + /// `max_level_of >= 1`, which is not the case here — the real Autolykos + /// generator produces level-0 blocks. The byte-for-byte assertion lives + /// in `fake_pow_scheme.rs`, which uses a fake pow scheme that forces + /// every block to a positive level (matching what + /// `org.ergoplatform.modifiers.history.PoPowAlgosWithDBSpec` does in + /// the JVM via `DefaultFakePowScheme`). + #[test] + fn test_nipopow_prove_with_reader_valid_on_real_autolykos_chain() { + let m = 6; + let k = 10; + let popow_algos = NipopowAlgos::default(); + let chain = generate_popowheader_chain(100, None); + + let in_memory_proof = popow_algos.prove(&chain, k, m).unwrap(); + let reader = MockReader::from_chain(&chain); + let db_backed_proof = popow_algos + .prove_with_reader(&reader, None, k, m) + .unwrap(); + + assert!(db_backed_proof.has_valid_connections()); + assert_eq!(db_backed_proof.suffix_tail.len(), (k - 1) as usize); + // suffix head must be the (len - k + 1)-th block (1-indexed) of the + // chain — i.e. chain[len - k]. + assert_eq!( + db_backed_proof.suffix_head.header.id, + chain[chain.len() - k as usize].header.id + ); + // Db-backed prefix is a subset of the in-memory prefix on chains + // with level-0 blocks (see doc comment above). + let in_memory_ids: std::collections::HashSet = in_memory_proof + .prefix + .iter() + .map(|p| p.header.id) + .collect(); + for ph in &db_backed_proof.prefix { + assert!( + in_memory_ids.contains(&ph.header.id), + "db-backed prefix contained {:?} which is absent from in-memory prefix", + ph.header.id + ); + } + } + + /// When `prove_with_reader` is given an explicit `header_id`, the + /// resulting suffix head must match the requested header and the proof + /// must have valid connections. + #[test] + fn test_nipopow_prove_with_reader_explicit_header_id() { + let m = 6; + let k = 10; + let popow_algos = NipopowAlgos::default(); + let chain = generate_popowheader_chain(100, None); + let reader = MockReader::from_chain(&chain); + + let target = chain[80].clone(); + let proof = popow_algos + .prove_with_reader(&reader, Some(&target.header.id), k, m) + .unwrap(); + + assert_eq!(proof.suffix_head.header.id, target.header.id); + assert_eq!(proof.suffix_head.header.height, target.header.height); + // suffix_tail must be the next k-1 headers immediately after + // target, in ascending-height order. + assert_eq!(proof.suffix_tail.len(), (k - 1) as usize); + for (i, h) in proof.suffix_tail.iter().enumerate() { + assert_eq!(h.height, target.header.height + 1 + i as u32); + } + assert!(proof.has_valid_connections()); + } } diff --git a/ergo-chain-generation/src/fake_pow_scheme.rs b/ergo-chain-generation/src/fake_pow_scheme.rs index 7baa98fb2..b82411f88 100644 --- a/ergo-chain-generation/src/fake_pow_scheme.rs +++ b/ergo-chain-generation/src/fake_pow_scheme.rs @@ -8,7 +8,7 @@ #[cfg(test)] mod tests { use ergo_lib::ergo_chain_types::{blake2b256_hash, ADDigest, BlockId, Digest32}; - use ergo_nipopow::{NipopowAlgos, NipopowProof}; + use ergo_nipopow::{NipopowAlgos, NipopowProof, PopowHeaderReader}; use ergo_chain_types::{autolykos_pow_scheme::order_bigint, AutolykosSolution, Header, Votes}; use ergo_lib::ergotree_interpreter::sigma_protocol::private_input::DlogProverInput; @@ -16,6 +16,7 @@ mod tests { use ergo_nipopow::PoPowHeader; use num_bigint::BigUint; use rand::{thread_rng, Rng}; + use std::collections::HashMap; use crate::{default_miner_secret, ErgoFullBlock, ExtensionCandidate}; use ergo_merkle_tree::{MerkleNode, MerkleTree}; @@ -355,4 +356,113 @@ mod tests { }; assert!(!proof.has_valid_connections()); } + + /// In-memory `PopowHeaderReader` backed by a synthetic chain. Lives in + /// the test module because `ergo-nipopow` cannot depend on + /// `ergo-chain-generation` (circular dep). + struct MockReader { + by_id: HashMap, + by_height: Vec, // index 0 = height 1 (genesis) + } + + impl MockReader { + fn from_chain(chain: &[PoPowHeader]) -> Self { + let mut by_id = HashMap::with_capacity(chain.len()); + let mut by_height: Vec = Vec::with_capacity(chain.len()); + for ph in chain { + by_id.insert(ph.header.id, ph.clone()); + by_height.push(ph.clone()); + } + for (i, ph) in by_height.iter().enumerate() { + assert_eq!(ph.header.height as usize, i + 1); + } + Self { by_id, by_height } + } + } + + impl PopowHeaderReader for MockReader { + fn headers_height(&self) -> u32 { + self.by_height.len() as u32 + } + + fn popow_header_by_id(&self, id: &BlockId) -> Option { + self.by_id.get(id).cloned() + } + + fn popow_header_at_height(&self, height: u32) -> Option { + if height == 0 { + return None; + } + self.by_height.get((height - 1) as usize).cloned() + } + + fn last_headers(&self, k: usize) -> Vec
{ + let len = self.by_height.len(); + let start = len.saturating_sub(k); + self.by_height[start..] + .iter() + .map(|ph| ph.header.clone()) + .collect() + } + + fn best_headers_after(&self, header: &Header, n: usize) -> Vec
{ + // Heights are 1-indexed; the slot immediately after `header` is + // at vec index `header.height` (because by_height[0] is height 1). + let start = header.height as usize; + let end = (start + n).min(self.by_height.len()); + if start >= end { + return Vec::new(); + } + self.by_height[start..end] + .iter() + .map(|ph| ph.header.clone()) + .collect() + } + } + + /// Asserts byte-for-byte equivalence between `prove(&chain)` and + /// `prove_with_reader(&reader, None, k, m)` on a 100-block synthetic + /// chain at the JVM P2P defaults `m=6, k=10`. This is the Rust + /// counterpart to `PoPowAlgosWithDBSpec` "proof(chain) is equivalent to + /// proof(histReader)" in the JVM ergo node. + /// + /// The test must use the fake-pow-scheme chain in this module (where + /// `d = order / (height + 1)` forces `max_level_of >= 1` for every + /// non-genesis block). On real-Autolykos chains the in-memory algorithm + /// adds level-0-only blocks at its level-0 iteration that the db-backed + /// interlink walk never visits, so byte-for-byte equivalence does not + /// hold there. The JVM equivalent test (`PoPowAlgosWithDBSpec`) + /// likewise relies on `DefaultFakePowScheme`, which mints blocks with + /// `d = q / (height + 10)` for the same reason. + #[test] + fn test_nipopow_prove_with_reader_matches_in_memory() { + use sigma_ser::ScorexSerializable; + let m = 6; + let k = 10; + let popow_algos = NipopowAlgos::default(); + let chain = generate_popowheader_chain(100, None); + + // Sanity: every block must have positive level for the equivalence + // to hold (see test doc). + for ph in &chain { + assert!(popow_algos.max_level_of(&ph.header).unwrap() >= 1); + } + + let in_memory_proof = popow_algos.prove(&chain, k, m).unwrap(); + let reader = MockReader::from_chain(&chain); + let db_backed_proof = popow_algos + .prove_with_reader(&reader, None, k, m) + .unwrap(); + + let in_memory_bytes = in_memory_proof.scorex_serialize_bytes().unwrap(); + let db_backed_bytes = db_backed_proof.scorex_serialize_bytes().unwrap(); + + assert_eq!( + in_memory_bytes, db_backed_bytes, + "prove and prove_with_reader produced different proofs:\n\ + in-memory prefix len: {}, db-backed prefix len: {}", + in_memory_proof.prefix.len(), + db_backed_proof.prefix.len() + ); + } } diff --git a/ergo-nipopow/src/lib.rs b/ergo-nipopow/src/lib.rs index caa76063a..2ba743c1d 100644 --- a/ergo-nipopow/src/lib.rs +++ b/ergo-nipopow/src/lib.rs @@ -21,7 +21,9 @@ mod nipopow_algos; mod nipopow_proof; mod nipopow_verifier; +mod popow_header_reader; pub use nipopow_algos::{NipopowAlgos, INTERLINK_VECTOR_PREFIX}; pub use nipopow_proof::{NipopowProof, NipopowProofError, PoPowHeader}; pub use nipopow_verifier::NipopowVerifier; +pub use popow_header_reader::PopowHeaderReader; diff --git a/ergo-nipopow/src/nipopow_algos.rs b/ergo-nipopow/src/nipopow_algos.rs index f9686abd8..5c829b9d0 100644 --- a/ergo-nipopow/src/nipopow_algos.rs +++ b/ergo-nipopow/src/nipopow_algos.rs @@ -5,8 +5,10 @@ use ergo_chain_types::{ Header, }; use num_traits::ToPrimitive; +use std::collections::{BTreeMap, BTreeSet}; use std::convert::TryInto; +use crate::popow_header_reader::PopowHeaderReader; use crate::{nipopow_proof::PoPowHeader, NipopowProof, NipopowProofError}; use ergo_chain_types::{BlockId, Digest32, ExtensionCandidate}; @@ -195,6 +197,101 @@ impl NipopowAlgos { prefix.sort_by(|a, b| a.header.height.cmp(&b.header.height)); NipopowProof::new(m, k, prefix, suffix_head, suffix_tail) } + + /// Computes a NiPoPow proof for the chain exposed by `reader`, optionally + /// rooted at the prefix that contains a specific header (when + /// `header_id_opt` is `Some`, that header becomes the suffix head). + /// + /// This is a direct port of the JVM + /// `org.ergoplatform.modifiers.history.popow.NipopowProverWithDbAlgs.prove` + /// method (the db-backed prover used in production for serving NiPoPoW + /// proofs to peers). Unlike [`NipopowAlgos::prove`], which requires the + /// caller to materialize the entire chain as `PoPowHeader`s up front, + /// `prove_with_reader` walks the interlink hierarchy via the + /// [`PopowHeaderReader`] callback and only materializes the headers it + /// actually needs — see the trait docs for the asymptotic motivation. + /// + /// # Known divergence from the JVM source + /// + /// The JVM `prove` accepts a `params.continuous` flag that, when set, + /// embeds extra epoch-boundary popow headers into the prefix so peers can + /// validate difficulty for blocks past the suffix without further sync. + /// This Rust port does NOT implement that mode: sigma-rust's + /// [`NipopowProof`] currently has no `continuous` field, so adding it + /// would require coordinated changes to the struct, the serializer, and + /// the on-wire format — out of scope for this patch. `prove_with_reader` + /// always produces non-continuous proofs (equivalent to + /// `params.continuous = false`). Continuous-mode support is tracked as a + /// follow-up. JVM peers applying non-continuous proofs still succeed: + /// `applyPopowProof` does not strictly require the flag, the proof + /// recipient just cannot self-validate difficulty for blocks beyond the + /// suffix until they sync more headers. + pub fn prove_with_reader( + &self, + reader: &R, + header_id_opt: Option<&BlockId>, + k: u32, + m: u32, + ) -> Result { + if k == 0 { + return Err(NipopowProofError::ZeroKParameter); + } + if reader.headers_height() < k + m { + return Err(NipopowProofError::ChainTooShort); + } + + // Build the suffix: either rooted at an explicit header_id, or the + // last `k` headers at the chain tip. + let (suffix_head, suffix_tail): (PoPowHeader, Vec
) = match header_id_opt { + Some(header_id) => { + let suffix_head = reader + .popow_header_by_id(header_id) + .ok_or(NipopowProofError::MissingPopowHeader)?; + let suffix_tail = reader.best_headers_after(&suffix_head.header, (k - 1) as usize); + (suffix_head, suffix_tail) + } + None => { + let suffix = reader.last_headers(k as usize); + let head = suffix + .first() + .ok_or(NipopowProofError::MissingPopowHeader)?; + let suffix_head = reader + .popow_header_by_id(&head.id) + .ok_or(NipopowProofError::MissingPopowHeader)?; + let suffix_tail: Vec
= suffix.iter().skip(1).cloned().collect(); + (suffix_head, suffix_tail) + } + }; + + // Mirror the JVM `prefixBuilder` / `storedHeights` accumulators. + // The genesis popow header is always in the prefix; height 1 is + // pre-recorded so the dedup loop below skips it. + const GENESIS_HEIGHT: u32 = 1; + let mut stored_heights: BTreeSet = BTreeSet::new(); + let mut prefix_builder: Vec = Vec::new(); + + let genesis = reader + .popow_header_at_height(GENESIS_HEIGHT) + .ok_or(NipopowProofError::MissingPopowHeader)?; + prefix_builder.push(genesis); + stored_heights.insert(GENESIS_HEIGHT); + + // (Continuous mode would inject additional epoch-boundary headers + // here. See the doc comment above for why this is omitted.) + + let prefix_collected = prove_prefix(reader, GENESIS_HEIGHT, &suffix_head, m)?; + for ph in prefix_collected { + if !stored_heights.contains(&ph.header.height) { + stored_heights.insert(ph.header.height); + prefix_builder.push(ph); + } + } + + prefix_builder.sort_by_key(|p| p.header.height); + + NipopowProof::new(m, k, prefix_builder, suffix_head, suffix_tail) + } + /// Packs interlinks into key-value format of the block extension. pub fn pack_interlinks(interlinks: Vec) -> Vec<([u8; 2], Vec)> { let mut res = vec![]; @@ -348,3 +445,119 @@ fn extension_merkletree(kv: &[([u8; 2], Vec)]) -> ergo_merkle_tree::MerkleTr .collect::>(); ergo_merkle_tree::MerkleTree::new(leafs) } + +// Helpers for `NipopowAlgos::prove_with_reader`. Direct ports of the +// nested helpers in JVM `NipopowProverWithDbAlgs.prove`. + +/// Port of JVM `linksWithIndexes(header) = header.interlinks.tail.reverse.zipWithIndex`. +/// +/// Given `interlinks = [genesis, X_max, ..., X_2, X_1]` (genesis at index 0, +/// highest superlevel pointer at index 1, lowest at the last index), this +/// returns `[(X_1, 0), (X_2, 1), ..., (X_max, max-1)]`. Index 0 maps to the +/// LOWEST superlevel pointer (i.e. `interlinks[len-1]`), and the highest +/// index maps to the HIGHEST superlevel pointer (i.e. `interlinks[1]`). +fn links_with_indexes(header: &PoPowHeader) -> Vec<(BlockId, usize)> { + if header.interlinks.len() < 2 { + return Vec::new(); + } + header + .interlinks + .iter() + .skip(1) + .rev() + .copied() + .enumerate() + .map(|(i, id)| (id, i)) + .collect() +} + +/// Port of JVM `previousHeaderIdAtLevel(level, currentHeader)` — +/// `linksWithIndexes(currentHeader).find(_._2 == level).map(_._1)`. +/// +/// Looks up the interlink pointer for the given linksWithIndexes index. +/// Returns `None` if `level` is out of range for this header's interlinks. +fn previous_header_id_at_level(level: usize, header: &PoPowHeader) -> Option { + let n = header.interlinks.len(); + if n < 2 { + return None; + } + // tail length = n - 1, so valid indices are 0..=n-2. + if level > n - 2 { + return None; + } + Some(header.interlinks[n - 1 - level]) +} + +/// Port of JVM `collectLevel(prevHeaderId, level, anchoringHeight, acc)`, +/// translated from a `@tailrec` recursion to an explicit loop. +/// +/// Walks backward through the interlink at `level` starting at +/// `start_id`, accumulating `PoPowHeader`s until either the walk passes +/// below `anchoring_height` or a header with no interlink at this level is +/// reached. The returned `Vec` is in ascending-height order, matching the +/// JVM `prevHeader +: acc` prepend semantics. +fn collect_level( + reader: &R, + start_id: BlockId, + level: usize, + anchoring_height: u32, +) -> Result, NipopowProofError> { + // We push to the back during the walk (descending-height order) and + // reverse once at the end, to avoid the O(n^2) cost of `insert(0, ..)`. + let mut walked: Vec = Vec::new(); + let mut current_id = start_id; + loop { + let prev_header = reader + .popow_header_by_id(¤t_id) + .ok_or(NipopowProofError::MissingPopowHeader)?; + if prev_header.header.height < anchoring_height { + walked.reverse(); + return Ok(walked); + } + let next_link = previous_header_id_at_level(level, &prev_header); + walked.push(prev_header); + match next_link { + Some(next_id) => current_id = next_id, + None => { + walked.reverse(); + return Ok(walked); + } + } + } +} + +/// Port of JVM `provePrefix(initAnchoringHeight, lastHeader)`. +/// +/// Iterates over `linksWithIndexes(last_header)` from the highest level +/// down to the lowest (matching Scala `foldRight`), running `collect_level` +/// at each level and updating the running anchoring height as the JVM +/// version does. Returns the deduplicated set of collected headers (sorted +/// by `BlockId` because we use a `BTreeMap`, mirroring Scala +/// `mutable.TreeMap[ModifierId, PoPowHeader]`). +fn prove_prefix( + reader: &R, + init_anchoring_height: u32, + last_header: &PoPowHeader, + m: u32, +) -> Result, NipopowProofError> { + let mut collected: BTreeMap = BTreeMap::new(); + let levels = links_with_indexes(last_header); + + // `levels.foldRight(initAnchoringHeight)` in Scala visits elements from + // last to first, i.e. highest superlevel first. + let mut anchoring_height = init_anchoring_height; + for (prev_header_id, level_idx) in levels.into_iter().rev() { + let level_headers = collect_level(reader, prev_header_id, level_idx, anchoring_height)?; + for ph in &level_headers { + collected.insert(ph.header.id, ph.clone()); + } + if (m as usize) < level_headers.len() { + anchoring_height = level_headers[level_headers.len() - (m as usize)] + .header + .height; + } + // else: anchoring_height unchanged, matching the JVM else branch. + } + + Ok(collected.into_values().collect()) +} diff --git a/ergo-nipopow/src/nipopow_proof.rs b/ergo-nipopow/src/nipopow_proof.rs index d10fbac11..48f13e052 100644 --- a/ergo-nipopow/src/nipopow_proof.rs +++ b/ergo-nipopow/src/nipopow_proof.rs @@ -199,6 +199,13 @@ pub enum NipopowProofError { /// Chain must be of length `>= k + m` #[error("Chain must be of length `>= k + m`")] ChainTooShort, + /// A `PopowHeaderReader` lookup returned `None` for a header that the + /// proof construction algorithm expected to be present (genesis, an + /// interlink target, the suffix head, or a header in the suffix tail). + /// Indicates the reader is inconsistent with the chain it claims to + /// expose. + #[error("Popow header reader returned None for an expected header")] + MissingPopowHeader, } #[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)] diff --git a/ergo-nipopow/src/popow_header_reader.rs b/ergo-nipopow/src/popow_header_reader.rs new file mode 100644 index 000000000..55fc62d67 --- /dev/null +++ b/ergo-nipopow/src/popow_header_reader.rs @@ -0,0 +1,73 @@ +//! Trait for reading [`PoPowHeader`]s and [`Header`]s from an external store. +//! +//! Used by [`crate::NipopowAlgos::prove_with_reader`] to construct a NiPoPoW +//! proof without first materializing the entire chain as `PoPowHeader`s. +//! +//! # Relationship to the JVM `ErgoHistoryReader` +//! +//! This trait is the minimal Rust analogue of the subset of methods that +//! `org.ergoplatform.modifiers.history.popow.NipopowProverWithDbAlgs.prove` +//! requires from `ErgoHistoryReader`. Implementations are expected to be +//! cheaply clonable readers backed by some external storage. +//! +//! # Asymptotic motivation +//! +//! [`crate::NipopowAlgos::prove`] requires the caller to materialize the +//! whole chain as `PoPowHeader`s before it runs — `O(N)` `popow_header` +//! constructions per proof, where `N` is the chain length. The internal +//! algorithm only inspects most of those headers' `n_bits` / `id` fields, +//! and reads `interlinks` / `interlinks_proof` only for blocks that land in +//! the final prefix. +//! +//! [`crate::NipopowAlgos::prove_with_reader`] inverts that: it walks the +//! interlink hierarchy starting from the suffix head and only requests +//! `PoPowHeader`s for blocks the walk actually visits — roughly +//! `m + k + m * log2(N)` `popow_header_by_id` calls per proof. For the P2P +//! defaults `m = 6, k = 10` on a `N ~= 270k` chain that's ~120 fetches +//! versus 270k for the in-memory variant: three orders of magnitude fewer. +//! +//! # Consistency expectation +//! +//! Any header `id` returned by one method (for example, the `id` of a +//! `Header` returned by [`PopowHeaderReader::last_headers`] or appearing in +//! the `interlinks` of a `PoPowHeader`) MUST be resolvable via +//! [`PopowHeaderReader::popow_header_by_id`]. Likewise, the genesis block +//! at height `1` MUST be resolvable via +//! [`PopowHeaderReader::popow_header_at_height`]. Inconsistent readers will +//! cause [`crate::NipopowAlgos::prove_with_reader`] to fail with +//! [`crate::NipopowProofError::MissingPopowHeader`]. + +use ergo_chain_types::{BlockId, Header}; + +use crate::nipopow_proof::PoPowHeader; + +/// Read-only access to a chain's [`PoPowHeader`]s and [`Header`]s for +/// db-backed NiPoPoW proof construction. See the module docs for semantics +/// and the asymptotic motivation. +pub trait PopowHeaderReader { + /// Returns the current chain height (number of blocks). Used to enforce + /// the `headers_height >= k + m` precondition before proving. + fn headers_height(&self) -> u32; + + /// Looks up a [`PoPowHeader`] by its block id. Hot path during the + /// interlink walk in [`crate::NipopowAlgos::prove_with_reader`]. + /// Returns `None` if the reader has no record of `id`. + fn popow_header_by_id(&self, id: &BlockId) -> Option; + + /// Looks up a [`PoPowHeader`] by absolute block height (1 = genesis). + /// Used to fetch the genesis popow header and, when proving without an + /// explicit `header_id`, to resolve the suffix head. + fn popow_header_at_height(&self, height: u32) -> Option; + + /// Returns up to `k` headers at the chain tip in ascending-height + /// order. Called only when the caller does not specify an explicit + /// `header_id_opt` to [`crate::NipopowAlgos::prove_with_reader`]; the + /// first element becomes the suffix head and the rest the suffix tail. + fn last_headers(&self, k: usize) -> Vec
; + + /// Returns up to `n` headers immediately following `header` in + /// ascending-height order. Used to construct the suffix tail when an + /// explicit `header_id_opt` is supplied to + /// [`crate::NipopowAlgos::prove_with_reader`]. + fn best_headers_after(&self, header: &Header, n: usize) -> Vec
; +}