fix(state): verify zakura header-store invariants in consensus reads#493
Merged
Conversation
Close the three write-path corruption bugs proven by the Phase-1 header-store coherence suite (PR #490): 1. prepare_header_range_batch_with_roots now rejects ranges whose first header does not link to the anchor or whose headers are not internally contiguous (new CommitHeaderRangeError::UnlinkedRange, classified as a local non-scoring failure in zebrad). 2. The range insert loop skips heights that already have a committed block, so re-deliveries can no longer strand provisional zakura rows below the body tip. 3. prepare_zakura_header_from_committed_block refuses seeds that do not link to the stored row below them as silent no-ops; header-range sync converges the store instead (scenario s11). Flip the four *_upholds_invariants twins to permanent regression gates, delete the corruption_repro_* tests, remove the discovery masking, and un-ignore the random invariant sweep (clean at PROPTEST_CASES=4096).
Pillar 2 of the REORG_PLAN: a corrupted header store can no longer poison difficulty validation — readers surface the storage fault explicitly. - recent_header_context verifies, at every step of its walk, that the stored header is the block its hash row names (HeaderHashMismatch), that it links to the row below (BrokenLinkage), and that no row is missing below a stored one (Gap), returning the new StoreIncoherentError instead of a poisoned or silently shortened difficulty window. - The range writer's anchor round-trip distinguishes a bijection violation in our own indexes (StoreIncoherent::BijectionMismatch) from a genuinely unknown anchor (UnknownAnchor). - CommitHeaderRangeError::StoreIncoherent is classified as a local, non-peer-scoring commit failure in zebrad: the range was rejected because our store cannot supply trustworthy context, not because the peer's range was shown invalid. New reads.rs suite in header_store_coherence hand-corrupts the column families and pins the reader/writer behavior for all four fault shapes, including side-effect freedom of the rejections.
|
Note Complete: Audit complete. No review-worthy issues remain after automatic triage. Three findings were auto-invalidated. Open the full results here. Analyzed five files, diff |
p0mvn
marked this pull request as draft
July 6, 2026 23:43
This was referenced Jul 7, 2026
Author
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 4459656. Configure here.
p0mvn
marked this pull request as ready for review
July 7, 2026 03:29
evan-forbes
changed the base branch from
zakura-header-store-write-guards
to
ironwood-main
July 7, 2026 10:56
evan-forbes
approved these changes
Jul 7, 2026
evan-forbes
left a comment
There was a problem hiding this comment.
for our future selves, I think we can avoid keeping the zebra mixing of logic for state and reactor/services at least slightly
easy approve tho
|
there are some minor compile issues and conflicts I'll clean up quickly then merge |
# Conflicts: # CHANGELOG.md # zebra-state/src/service/finalized_state/zebra_db/block/tests/header_store_coherence/README.md # zebrad/src/commands/start.rs # zebrad/src/commands/start/zakura/header_sync_driver.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This PR helps to identify poisoned header-store, and, instead of downscoring a peer for this, silently failing, which prompts a fallback to legacy.
#491 prevents writing/poisoning the store in the first place, and #494 and #496 add auto-repair mechanisms.
Specifically, today,
recent_header_contextwalks the height index blindly, so one stale row inside the 28-height difficulty window makes the node reject every honest header withInvalidDifficultyThreshold.This has been reproduced and identified via testnet incidents.
This PR catches the poison: corruption found during a read becomes an explicit local error instead of a wrong validation verdict blamed on peers.
Solution
New
StoreIncoherentError(exported fromzebra-state), returned by readers instead of consuming rows that fail an invariant check:recent_header_contextverifies as it walks. At every step: the stored header must be the block its hash row names (HeaderHashMismatch— one header hash per consumed row, negligible next to the validation the window feeds), it must link to the row below (BrokenLinkage), and a missing row below a stored one is aGap, not the end of history (rows are contiguous from genesis: full-block rows below the body tip — retained under pruning — and zakura rows above). A short window is returned only when the walk genuinely reaches genesis; a silently shortened mid-chain window would shift the difficulty adjustment instead of failing.UnknownAnchor; a hash→height entry whose height→hash row disagrees is a bijection violation in our own indexes and now reportsStoreIncoherent(BijectionMismatch).CommitHeaderRangeError::StoreIncoherentis classifiedLocal(non-scoring) in zebrad'sheader_range_commit_failure_kind: the range was rejected because our store cannot supply trustworthy context, not because the peer's range was shown invalid. This hard-codes the rule that local store incoherence must never score or disconnect peers.StoreIncoherentrejections are side-effect free (validation happens before any batch write), so the store is untouched and the fault can only trigger recovery — on the fleet branch the ContextMismatch/walk-back machinery picks it up; Pillar 3's audit-and-repair is the end state.