feat(state): authenticate VCT-fast blocks from the successor header

[p0mvn]

Motivation

The Zakura finalized-block committer deadlocks at a checkpoint / verified-commitment-trees (VCT) boundary. A VCT-fast block's peer-supplied note-commitment roots are authenticated against the next block's header commitment, but the committer waited for the successor block body to be buffered. Since header sync runs far ahead of the body frontier, this coupled the committer to the body-download pipeline: the writer holds block N awaiting N+1's body, while N+1's body cannot be applied until N commits — a circular wait (observed frozen for hours at height 1,709,217 on a benchmark node).

Solution

Authenticate against the already-committed successor header instead of its body.

At NU5+ the successor's hashBlockCommitments binds the chain-history (MMR) root together with the successor's own ZIP-244 auth_data_root, so that root is needed as a co-input. It is now:

• Propagated: carried per-height in BlockCommitmentRoots over header sync, and stored in the commitment_roots_by_height serving-index value (64→96 bytes, backward compatible — old rows decode with a zero root and fall back to the body-wait path). Consolidated under state DB format 27.3.0 (no version bump). Carrying it bumps the Zakura header-sync stream format to v5 — a breaking wire change (all peers must run the new format).
• Consumed: a header-driven header_commitment_is_valid_for_chain_history core is extracted (the block-based check delegates to it, behavior preserved); CommitmentRootVerification::header_only is now body-free; the writer sources the successor from the buffered look-ahead or the committed header chain (successor_header_auth_for), and only waits at the genuine header tip.

Test evidence

• cargo check --workspace --tests clean; clippy + fmt clean on touched crates.
• 22 VCT property tests pass (deferral, fast-path-matches-legacy-and-rejects-wrong-roots, byte-identical storage round-trip).
• New header_only_successor_confirms_and_rejects_wrong_root proves the body-free path confirms real roots and rejects wrong ones from the successor header alone.
• zebra-chain serialization round-trip (non-zero auth-data root) + 85 header_sync wire tests + rollback/serve-index tests pass.
• The ignored verifies_real_nu5_range_over_synced_forks can be run against 1.7M snapshot forks (VCT_SEED_DB/VCT_ARCHIVE_DB) for real-data confidence before deploy.

Merge plan

This is the full feature branch. The network-breaking auth-root propagation (the data-model/wire/storage/serve half, without the consensus-bearing committer/verifier changes) is split into a separate PR to merge first; this PR should then be rebased down to just the consensus consumption.

AI disclosure

Implemented with Claude Code (Opus). Scope: the deadlock investigation, the design, and the full implementation + tests in this PR. The contributor is the responsible author.

[v12-auditor]

Note

Complete: Audit complete. V12 found one issue worth reviewing.

Open the full results here.

Finding	Severity	Details
F-94756	🟠 High	Poisoned successor roots stall sync VCT fast-sync verifies a current block’s supplied roots against a header-only successor when the successor body is unavailable. If a peer-supplied Zakura header range stores an invalid auth_data_root for that successor, verify_commitment_roots() fails at the successor height, but the caller discards the returned failure height and calls vct_reject_supplied_root(height, error) with the current height. The rejection path deletes only the current height’s root row from the peer-backed root source and leaves the poisoned successor row in the Zakura header-root database. The write worker treats the resulting VctSuppliedRootUnavailable as retryable and retries the same checkpoint block in place, while the frozen-frontier guard refuses to recompute without a valid supplied root. Since successful header-range commits mark ranges covered, normal header-sync scheduling does not refetch the already-covered successor root from honest peers.

And one more auto-invalidated finding.

Analyzed 21 files, diff 3d307b0...9c7e198.