Summary
Zakura header-sync anchors its forward range on ReadRequest::BestHeaderTip. On an equal-height non-finalized reorg, that read returns a stale block hash, so the next header range fails FirstHeaderDoesNotLink against an honest peer that reorged; header-sync transiently bans the peer and lags a block before recovering.
The node recovers, so this is not failing CI — it was surfaced while investigating the nightly Zakura e2e traces, where FirstHeaderDoesNotLink @ tip+1 appears in both passing and failing runs (benign today). Filing as correctness hardening.
Evidence
zebra-state/src/service.rs:1710-1723 — BestHeaderTip handler:
ReadRequest::BestHeaderTip => {
let best_disk_header_tip = state.db.best_header_tip(); // finalized DB + Zakura header store
let verified_block_tip = read::tip(state.latest_best_chain(), &state.db); // non-finalized best chain
Ok(ReadResponse::BestHeaderTip(match (best_disk_header_tip, verified_block_tip) {
(Some(header_tip), Some(block_tip)) if block_tip.0 > header_tip.0 => Some(block_tip),
(Some(header_tip), _) => Some(header_tip), // EQUAL height: returns the stale disk hash
(None, block_tip) => block_tip,
}))
}
At equal height the non-finalized chain's (post-reorg) hash block_tip is discarded in favour of the disk hash.
zebra-state/src/service/finalized_state/zebra_db/block.rs:391-402 — best_header_tip() reads only the finalized body tip and the Zakura header store, never the non-finalized chain.
The Zakura header store is reconciled with the chain only on finalized transitions (finalized commit via prepare_zakura_header_release_from_committed_block; finalized rollback via delete_zakura_headers_above, whose only non-test caller is the offline zebrad rollback CLI). There is no non-finalized-reorg hook. The serving side (headers_by_height_range) is reorg-correct, so this is purely requester-anchor staleness.
Proposed fix
- Read-layer (core): in
BestHeaderTip, when the non-finalized block tip is at the same height as the disk header tip, prefer the non-finalized chain hash — it reflects reorgs.
- Deeper (header-ahead case): when the Zakura header store has rows above the non-finalized block tip and a non-finalized reorg passes through those heights, reconcile/invalidate the affected rows (a non-finalized analogue of
delete_zakura_headers_above). Consensus-adjacent — scope carefully.
Also revisit the header-sync re-anchor recovery in zebra-network/src/zakura/header_sync/reactor.rs (handle_possible_stale_anchor_link_failure), which bails when best_header_tip <= verified_block_tip (the tip-reorg case).
TDD plan
zebra-state test: commit to height N (non-finalized) + seed a Zakura header at N; reorg the best non-finalized chain at N to a different block; assert BestHeaderTip returns the new hash. Fails today, passes after fix (1). A second test covers the header-ahead case for (2).
Notes
Not failing CI; node recovers. The non-blocking block-sync reactor (PR 199) did not introduce this — the read handler and header store predate it; the faster reactor only widens the window. AI-assisted analysis (Claude Code); confirm with the failing test above before fixing.
Summary
Zakura header-sync anchors its forward range on
ReadRequest::BestHeaderTip. On an equal-height non-finalized reorg, that read returns a stale block hash, so the next header range failsFirstHeaderDoesNotLinkagainst an honest peer that reorged; header-sync transiently bans the peer and lags a block before recovering.The node recovers, so this is not failing CI — it was surfaced while investigating the nightly Zakura e2e traces, where
FirstHeaderDoesNotLink @ tip+1appears in both passing and failing runs (benign today). Filing as correctness hardening.Evidence
zebra-state/src/service.rs:1710-1723—BestHeaderTiphandler:At equal height the non-finalized chain's (post-reorg) hash
block_tipis discarded in favour of the disk hash.zebra-state/src/service/finalized_state/zebra_db/block.rs:391-402—best_header_tip()reads only the finalized body tip and the Zakura header store, never the non-finalized chain.The Zakura header store is reconciled with the chain only on finalized transitions (finalized commit via
prepare_zakura_header_release_from_committed_block; finalized rollback viadelete_zakura_headers_above, whose only non-test caller is the offlinezebrad rollbackCLI). There is no non-finalized-reorg hook. The serving side (headers_by_height_range) is reorg-correct, so this is purely requester-anchor staleness.Proposed fix
BestHeaderTip, when the non-finalized block tip is at the same height as the disk header tip, prefer the non-finalized chain hash — it reflects reorgs.delete_zakura_headers_above). Consensus-adjacent — scope carefully.Also revisit the header-sync re-anchor recovery in
zebra-network/src/zakura/header_sync/reactor.rs(handle_possible_stale_anchor_link_failure), which bails whenbest_header_tip <= verified_block_tip(the tip-reorg case).TDD plan
zebra-statetest: commit to height N (non-finalized) + seed a Zakura header at N; reorg the best non-finalized chain at N to a different block; assertBestHeaderTipreturns the new hash. Fails today, passes after fix (1). A second test covers the header-ahead case for (2).Notes
Not failing CI; node recovers. The non-blocking block-sync reactor (PR 199) did not introduce this — the read handler and header store predate it; the faster reactor only widens the window. AI-assisted analysis (Claude Code); confirm with the failing test above before fixing.