fix(state): verify zakura header-store invariants in consensus reads

CHANGELOG.md

@@ -35,6 +35,14 @@ and this project adheres to [Semantic Versioning](https://semver.org).
 
 ### Fixed
 
+- Zakura header-store consensus reads now verify store invariants as they
+  walk: the difficulty-context walk checks that every stored header is the
+  block its hash row names and links to the row below it, and the anchor
+  lookup distinguishes a corrupted index round-trip from a genuinely unknown
+  anchor. A corrupted store now surfaces as an explicit local
+  `StoreIncoherent` error (never scored against peers) instead of poisoning
+  difficulty validation and rejecting honest headers with
+  `InvalidDifficultyThreshold`.
 - Fixed three Zakura header-store write paths that could leave the on-disk
   header store internally incoherent after chain forks, causing nodes to
   reject valid headers from honest peers (`InvalidDifficultyThreshold` /

zebra-state/src/error.rs

@@ -195,6 +195,68 @@ impl From<CommitHeaderRangeError> for CommitCheckpointVerifiedError {
     }
 }
 
+/// An internal invariant of the zakura header store was found violated while
+/// reading it.
+///
+/// This is a **local storage fault**, never evidence about a peer: readers
+/// return it instead of feeding rows from more than one branch (or from beside
+/// a gap) into consensus validation, where the corruption would otherwise
+/// surface as a misleading validation failure (`InvalidDifficultyThreshold`,
+/// `UnknownAnchor`) attributed to whoever supplied the input being validated.
+#[derive(Debug, Error, Clone, Copy, PartialEq, Eq)]
+#[non_exhaustive]
+pub enum StoreIncoherentError {
+    /// The header row at `height` does not link to the stored row below it.
+    #[error(
+        "header store incoherent: header at {height:?} links to {expected_parent} but the stored row below is {actual_below}"
+    )]
+    BrokenLinkage {
+        /// Height of the header whose parent link failed to resolve.
+        height: block::Height,
+        /// The parent hash the header claims (`previous_block_hash`).
+        expected_parent: block::Hash,
+        /// The hash actually stored at `height - 1`.
+        actual_below: block::Hash,
+    },
+
+    /// A header row exists at `height` but the row below it is missing.
+    #[error(
+        "header store incoherent: no stored row at {missing:?} below the header at {height:?}"
+    )]
+    Gap {
+        /// Height of the stored header above the gap.
+        height: block::Height,
+        /// The missing height (`height - 1`).
+        missing: block::Height,
+    },
+
+    /// The header row at `height` is not the block its hash row names.
+    #[error(
+        "header store incoherent: header stored at {height:?} hashes to {computed} but the hash row names {indexed}"
+    )]
+    HeaderHashMismatch {
+        /// Height of the divergent rows.
+        height: block::Height,
+        /// The hash the height→hash index names.
+        indexed: block::Hash,
+        /// The stored header's actual hash.
+        computed: block::Hash,
+    },
+
+    /// The hash→height and height→hash indexes disagree about a hash.
+    #[error(
+        "header store incoherent: hash {hash} is indexed at {height:?} but that height stores {stored:?}"
+    )]
+    BijectionMismatch {
+        /// The hash whose round-trip failed.
+        hash: block::Hash,
+        /// The height the hash→height index reports for it.
+        height: block::Height,
+        /// What the height→hash index stores there instead.
+        stored: Option<block::Hash>,
+    },
+}
+
 /// An error describing why a header-only range could not be committed.
 #[derive(Debug, Error, Clone, PartialEq, Eq)]
 #[non_exhaustive]
@@ -323,6 +385,15 @@ pub enum CommitHeaderRangeError {
         height: block::Height,
     },
 
+    /// The local header store was found internally incoherent while reading
+    /// the context needed to validate the range.
+    ///
+    /// This is a local storage fault, not a peer validation failure: the range
+    /// was rejected because the store cannot supply trustworthy context, not
+    /// because the range itself was shown invalid.
+    #[error("header store incoherent while validating range: {0}")]
+    StoreIncoherent(#[from] StoreIncoherentError),
+
     /// Contextual header validation failed.
     #[error("could not contextually validate header")]
     ValidateContextError(#[from] Box<ValidateContextError>),

zebra-state/src/lib.rs

@@ -43,7 +43,8 @@ pub use config::{
 pub use constants::{state_database_format_version_in_code, MAX_BLOCK_REORG_HEIGHT};
 pub use error::{
     BoxError, CloneError, CommitBlockError, CommitCheckpointVerifiedError, CommitHeaderRangeError,
-    CommitSemanticallyVerifiedError, DuplicateNullifierError, ValidateContextError,
+    CommitSemanticallyVerifiedError, DuplicateNullifierError, StoreIncoherentError,
+    ValidateContextError,
 };
 pub use request::{
     CheckpointVerifiedBlock, CommitSemanticallyVerifiedBlockRequest, HashOrHeight, MappedRequest,

zebra-state/src/service/finalized_state/zebra_db/block.rs

@@ -36,7 +36,7 @@ use crate::{
     constants::{
         MAX_BLOCK_REORG_HEIGHT, MAX_HEADER_SYNC_HEIGHT_RANGE, MAX_PRUNE_HEIGHTS_PER_COMMIT,
     },
-    error::{CommitCheckpointVerifiedError, CommitHeaderRangeError},
+    error::{CommitCheckpointVerifiedError, CommitHeaderRangeError, StoreIncoherentError},
     request::FinalizedBlock,
     service::check,
     service::finalized_state::{
@@ -546,31 +546,85 @@ impl ZebraDb {
     }
 
     /// Returns recent header difficulty/time context in reverse height order,
-    /// starting at `height`.
+    /// starting at `height`, verifying `previous_block_hash` linkage at every
+    /// step of the walk.
+    ///
+    /// Returns an empty context when there is no stored row at `height` (the
+    /// caller decides whether that anchor is unknown), and a shorter-than-span
+    /// context when the walk reaches genesis.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`StoreIncoherentError`] when the walk finds a header row that
+    /// is not the block its hash row names, a row that does not link to the
+    /// row below it, or a gap below a stored row. Feeding such a window into
+    /// difficulty validation would mix rows from more than one branch (or
+    /// shift the adjustment window), producing `InvalidDifficultyThreshold`
+    /// rejections of honest input — the reader surfaces the storage fault
+    /// explicitly instead. The per-row hash check costs one header hash per
+    /// consumed row, negligible next to the validation the window feeds.
     pub fn recent_header_context(
         &self,
         height: block::Height,
-    ) -> Vec<(
-        zebra_chain::work::difficulty::CompactDifficulty,
-        DateTime<Utc>,
-    )> {
+    ) -> Result<
+        Vec<(
+            zebra_chain::work::difficulty::CompactDifficulty,
+            DateTime<Utc>,
+        )>,
+        StoreIncoherentError,
+    > {
         let mut context = Vec::with_capacity(check::difficulty::POW_ADJUSTMENT_BLOCK_SPAN);
-        let mut current_height = Some(height);
 
-        while let Some(height) = current_height {
-            let Some((_hash, header)) = self.header_by_height(height) else {
-                break;
-            };
+        let Some((mut hash, mut header)) = self.header_by_height(height) else {
+            return Ok(context);
+        };
+        let mut height = height;
+
+        loop {
+            let computed = block::Hash::from(&*header);
+            if computed != hash {
+                return Err(StoreIncoherentError::HeaderHashMismatch {
+                    height,
+                    indexed: hash,
+                    computed,
+                });
+            }
 
             context.push((header.difficulty_threshold, header.time));
+
             if context.len() == check::difficulty::POW_ADJUSTMENT_BLOCK_SPAN {
-                break;
+                return Ok(context);
             }
+            let Ok(below) = height.previous() else {
+                // The walk reached genesis: a short context is legitimate, and
+                // the difficulty functions handle it (MedianTime clamps
+                // negative heights to zero).
+                return Ok(context);
+            };
 
-            current_height = height.previous().ok();
-        }
+            let Some((below_hash, below_header)) = self.header_by_height(below) else {
+                // Rows must be contiguous from genesis up to the header tip
+                // (full-block rows below the body tip — retained even under
+                // pruning — and zakura rows above it), so a missing row below
+                // a stored one is a gap, not the end of history.
+                return Err(StoreIncoherentError::Gap {
+                    height,
+                    missing: below,
+                });
+            };
 
-        context
+            if header.previous_block_hash != below_hash {
+                return Err(StoreIncoherentError::BrokenLinkage {
+                    height,
+                    expected_parent: header.previous_block_hash,
+                    actual_below: below_hash,
+                });
+            }
+
+            height = below;
+            hash = below_hash;
+            header = below_header;
+        }
     }
 
     /// Returns header-known, body-missing heights.
@@ -2037,17 +2091,26 @@ impl DiskWriteBatch {
             .or_else(|| (anchor == zebra_db.network().genesis_hash()).then_some(block::Height(0)))
             .ok_or(CommitHeaderRangeError::UnknownAnchor { anchor })?;
 
-        if anchor != zebra_db.network().genesis_hash()
-            && zebra_db.header_hash(anchor_height) != Some(anchor)
-        {
-            return Err(CommitHeaderRangeError::UnknownAnchor { anchor });
+        // The hash→height index knows the anchor, so a failed height→hash
+        // round-trip is a bijection violation in our own store — a local
+        // storage fault, not an unknown anchor supplied by the caller.
+        if anchor != zebra_db.network().genesis_hash() {
+            let stored = zebra_db.header_hash(anchor_height);
+            if stored != Some(anchor) {
+                return Err(StoreIncoherentError::BijectionMismatch {
+                    hash: anchor,
+                    height: anchor_height,
+                    stored,
+                }
+                .into());
+            }
         }
 
         let finalized_height = zebra_db.finalized_tip_height();
         let best_header_tip = zebra_db.best_header_tip().map(|(height, _)| height);
         let checkpoints = zebra_db.network().checkpoint_list();
 
-        let mut recent_headers = zebra_db.recent_header_context(anchor_height);
+        let mut recent_headers = zebra_db.recent_header_context(anchor_height)?;
         if recent_headers.is_empty() {
             if anchor == zebra_db.network().genesis_hash() && anchor_height == block::Height(0) {
                 return Err(CommitHeaderRangeError::MissingGenesisAnchor { anchor });

zebra-state/src/service/finalized_state/zebra_db/block/tests/header_store_coherence.rs

@@ -32,5 +32,6 @@ mod fabricate;
 mod ops;
 mod oracle;
 mod prop;
+mod reads;
 mod scenarios;
 mod startup_audit;

zebra-state/src/service/finalized_state/zebra_db/block/tests/header_store_coherence/README.md

@@ -36,6 +36,7 @@ write instead.
 | `ops.rs` | The op alphabet, each op mapped to one real production write-batch shape: `CommitHeaderRange` → `prepare_header_range_batch_with_roots`; `CommitBody` / `Finalize` → `prepare_block_header_and_transaction_data_batch` plus the finalization roots delete (which runs the release path internally); `Seed` → `seed_zakura_header_from_committed_block` (the non-finalized best-chain commit hook); `Reopen` → shutdown and reopen of the persistent store. The `Harness` executes ops, cross-checks the oracle's prediction against the store's response, and audits after every mutation; failures come back as a transcribable `FailureReport` (the executed op prefix plus every violation found). |
 | `scenarios.rs` | Scripted production event shapes (s01–s11): simple reorgs, lower-work rejections and their later reversal, split-range and walk-back deliveries, body commits racing header reorgs, reorgs to a lower height, double reorgs at one fork point, activity across the difficulty-adjustment window edge, restarts at every boundary, seed/range interplay, and refused-seed convergence. Also holds the `*_upholds_invariants` regression gates below. |
 | `prop.rs` | The permanent random sweep: random op sequences over the fixed universe, shrunk to minimal counterexamples on any audit failure. |
+| `reads.rs` | Read-path coherence: hand-corrupts the column families and asserts `recent_header_context` / the anchor round-trip report `StoreIncoherentError` (`HeaderHashMismatch`, `BrokenLinkage`, `Gap`, `BijectionMismatch`) instead of feeding stale rows into difficulty validation, and that the range writer's `StoreIncoherent` rejection is side-effect free. |
 
 ## Fixed corruption bugs gated by this suite