feat(state): verified commitment trees fast checkpoint sync (state layer)

[p0mvn]

Motivation

Below the last checkpoint, the dominant checkpoint-sync CPU cost is recomputing the per-block Sapling/Orchard note-commitment frontier for every block. This PR adds the zebra-state fast-path logic for verified commitment trees (VCT) checkpoint sync: instead of recomputing the frontier, the node folds peer-supplied, header-verified per-block commitment roots into the anchor set and history tree, and only materializes the real frontier once, at the checkpoint handoff. See docs/design/verified-commitment-trees.md (merged in #402) for the full design.

This is the follow-up slice to #402 (VCT docs, config, and DB storage) and replaces the closed #391: it carries the remaining state-layer logic, without the note-commitment precompute dependency (#387, closed) — the committer uses the existing tree-update path.

Solution

• Verify-before-commit: each peer-supplied root is checked against the node's own checkpoint-committed block headers (the ZIP-221 ChainHistory MMR one-block-lag check, plus direct below-Heartwood / below-NU5 checks) before it can influence consensus state (commitment_aux_verify.rs). A root that cannot be obtained or verified is rejected — never recomputed against the stale frozen frontier — so no untrusted data can affect consensus state (fail-closed / frozen-frontier policy).
• Embedded final frontier at handoff: at the checkpoint handoff an embedded final frontier (vct/mainnet-frontier.bin), verified against that block's proven root, is written as the tip treestate, and normal per-block recompute resumes above the checkpoint. The resulting consensus state is byte-identical to the legacy recompute.
• Root serving across the upgrade height: serve_block_roots stitches the CommitmentRootsByHeight serving index (at/above the upgrade height U) with per-height trees (below U) into one gap-free tree_aux BlockRoots response, so fast-synced nodes keep the root-serving fleet alive.
• Fail-closed reopen policy: a database interrupted below the checkpoint handoff refuses to reopen without a VCT root source instead of silently corrupting state from the stale frontier (updates the feat(state): add VCT docs config and DB storage #402-era reopen tests to #[should_panic]).
• History-tree open-time repair: repair_tip_history_tree_if_incompatible rebuilds an undecodable stored tip history tree from finalized blocks plus commitment roots (index first, per-height trees as fallback) before the background format-validity check reads it.
• VctData adaptation: write_block keeps the VctData bundle introduced by feat(state): add VCT docs config and DB storage #402, with both fields now optional to express the full feature's cases: the checkpoint-handoff block writes the real frontier through the legacy path while still setting the sync marker (anchor_roots: None), and a non-persistent fast sync folds roots without marking the database (sync_below: None).

Dormant feature: the checkpoint_sync / vct_fast_sync config fields are mirrors that zebrad sets at startup. This PR does not include the zebra-consensus vct_fast_sync knob or the zebrad startup wiring, so the fast path is not yet reachable in a running node — a follow-up PR activates it. The state tests exercise the path directly.

Tests

Observed locally on this branch (base origin/ironwood-main):

• cargo fmt --all -- --check — clean.
• cargo clippy --workspace --all-targets -- -D warnings — clean.
• cargo build --workspace — builds (all consumer crates compile).
• cargo test -p zebra-state — 270 passed; 0 failed; 2 ignored (lib, incl. the large VCT finalized_state/tests/prop.rs proptests), 2 passed (tests/basic.rs).

Specifications & References

• docs/design/verified-commitment-trees.md (merged in feat(state): add VCT docs config and DB storage #402)
• ZIP-221 (ChainHistory MMR), ZIP-244 (auth data commitment)
• Follows feat(state): add VCT docs config and DB storage #402; extracted from feat: pre-release main (sync perf, vct, block sync fixes) #290 / the bundled pre-release branch; replaces feat(state): verified commitment trees fast checkpoint sync (state layer) #391.

Follow-up Work

• A follow-up PR adds the zebra-consensus vct_fast_sync knob and the zebrad startup wiring that set the mirror config fields and make the fast path reachable at runtime.
• Not included by design: the note-commitment precompute (perf(chain): precompute note-commitment tree hashing off the committer #387, closed), the zebra-utils frontier generator, .github/workflows/checkpoint-update.yml, and the committer-thread write_block change (perf(state): run write_block on the committer thread instead of the commit-compute pool #247).

AI Disclosure

• AI tools were used: Claude Code — to re-cut this slice from the feat/vct-state branch onto ironwood-main, adapt the VctData plumbing from feat(state): add VCT docs config and DB storage #402, and verify (fmt/clippy/build/tests).

PR Checklist

• The PR title follows conventional commits format: type(scope): description
• The PR follows the contribution guidelines.
• This change was discussed with the team beforehand (split series from feat(state): verified commitment trees fast checkpoint sync (state layer) #391/feat: pre-release main (sync perf, vct, block sync fixes) #290).
• The solution is tested.
• The documentation and changelogs are up to date.

[v12-auditor]

Note

Complete: Audit complete. V12 found one issue worth reviewing.

Open the full results here.

Finding	Severity	Details
F-95113	🟡 Medium	Fallible state validation paths can still panic on bad bytes Multiple state-validation and compatibility paths are exposed as fallible APIs, but they still invoke panic-on-error internals once the outer byte framing or initial decode succeeds. In validate_final_frontiers_bytes, FinalFrontiers::from_bytes validates the serialized container and then constructs Sapling, Orchard, and Sprout trees through FromDisk::from_bytes, whose implementations call bincode::deserialize(...).expect(...). That means malformed inner tree blobs can abort the process instead of producing FinalFrontiersValidationError::InvalidBytes. Similarly, check_tip_history_tree_decodes is documented as a non-panicking startup probe, yet it delegates to HistoryTreeParts::with_network, which begins with assert_eq! on the stored network kind before returning its Result. Both paths therefore violate their fallible API contract and need the same fix pattern: convert invariant checks and deserialization failures into ordinary error returns all the way up the validation stack.

And one more auto-invalidated finding.

Analyzed 20 files, diff e9436d3...4a7a629.

[p0mvn]

Note: have a task tracking Ironwood