Skip to content

refactor(network): make the resident gate the single Zakura block-sync memory authority#415

Draft
p0mvn wants to merge 3 commits into
ironwood-mainfrom
roman/zakura-single-admission-authority
Draft

refactor(network): make the resident gate the single Zakura block-sync memory authority#415
p0mvn wants to merge 3 commits into
ironwood-mainfrom
roman/zakura-single-admission-authority

Conversation

@p0mvn

@p0mvn p0mvn commented Jul 2, 2026

Copy link
Copy Markdown

Tier 2 of the block-sync memory-management simplification (tier 1 was #413).

Motivation

Block-sync had two independent memory ceilings in different units: the wire-byte ByteBudget (in-flight reservations + held reorder/applying/channel bytes, no commit-window exemption) and the resident look-ahead gate (all pools at wire×4, window-exempt). The held-byte tracking in the wire budget was double accounting — the resident gate already bounds retention to budget + one commit window independent of it — and the entire shed-and-fund apparatus (FundFloorReservation oneshot round-trip through the sequencer task, shed_top_until_available, the per-body inline shed, reorder_max_height) existed solely to reconcile the floor against that second ceiling. The round-trip was also a latent liveness hazard: a wedged sequencer task deadlocked floor funding.

Solution

The resident gate (admit()) is now the single retention authority; the ByteBudget is a pure outstanding-request wire limiter.

  • Floor funding: reserve_request_budget is synchronous — try_reserve, else (floor priority only) a bounded overdraft via the existing charge. The WorkQueue single-owner invariant caps cumulative overshoot at one response; the floor can never block, even transiently, and has no cross-task dependency. The whole shed-and-fund mechanism is deleted.
  • Release-at-receipt: the request-estimate reservation releases when the body arrives (ordered after the body is counted in sequencer_input_bytes, so it is never invisible to both counters). Held bodies charge nothing; BlockBudgetLedger shrinks to Reserved → Released; settle/mark_held_direct/apply_budget_delta are gone; the sequencer audit becomes budget.audit(work.reserved_bytes()) — the same paired-call-site drift detector over the one thing the budget now tracks.
  • Config: effective_max_reorder_lookahead_bytes() drops the max_inflight_block_bytes × 4 coupling (the request cap no longer implies retention), clamp_inflight_block_bytes_to_floor is deleted (a sub-range request budget can no longer deadlock checkpoint sync), and max_inflight_block_bytes is re-documented as the outstanding-request/DoS cap. Net: one memory knob (max_reorder_lookahead_bytes, resident-denominated), one concurrency knob.

Trace field and metric names are unchanged; budget_reserved now means outstanding request bytes and still drains at quiescence (the regtest trace-oracle check holds).

Testing

  • cargo clippy --all-targets --features "zakura-testkit,internal-bench" -D warnings clean; zebra-network lib suite 837 green (193 block-sync, 27 config, 14 fuzz scenarios).
  • New pins: floor_overdraft_is_bounded_and_immediate (budget exactly full → floor GetBlocks sent synchronously, overdraft = one request estimate), receipt_releases_wire_reservation_while_body_retained, late-body no-resurrection, floor-watchdog received-height skip. The fuzz memory invariant gained floor-overdraft slack + a quiescence-drain assertion; fuzz_commit_stall was re-premised on the resident plateau (its wire-ceiling premise is void under the demotion); the two resident-plateau anchors are byte-for-byte untouched and green.
  • Live 100K mainnet A/B, 3 runs per binary, alternated against producer drift (same harness as the fix(network): cap Zakura block-sync applying buffer by resident memory to prevent OOM #386 validation):
binary mid-run rate (10K→90K), blk/min mean peak RSS peak retained wire
#386 head 7,464 / 9,986 / 9,344 8,931 4.41–4.64 GiB 807.7–807.8 MB
this PR 10,125 / 9,127 / 7,040 8,764 4.20–4.80 GiB 806.0–807.8 MB

Throughput: no regression, no measurable change — the per-binary means differ by −2%, far inside the ±15% run-to-run producer variance (both distributions span ~7.0K–10.1K). Memory: identical — the retained-wire plateau is pinned at the budget on every run of both binaries, and RSS ranges overlap. The structural change is directly visible in the traces: peak budget_reserved drops from ~802 MB (held bodies) to ~11 MB (outstanding requests only), and it still drains at quiescence.

  • Deep-chain soak: a resync from height ~791K runs under RSS/tip monitoring with this binary; through ~980K it sustained ~9.5K blk/min with the retained plateau flat and RSS in the 3.3–4.8 GiB band (not directly comparable across sessions — different producer conditions — but no anomaly).

AI disclosure

AI (Claude Code) was used for the design exploration, implementation, tests, live validation, and this PR description. The author is responsible for the change.

Refs ZCA-742.

@v12-auditor

v12-auditor Bot commented Jul 2, 2026

Copy link
Copy Markdown

Note

Complete: Audit complete. No review-worthy issues remain after automatic triage. Three findings were auto-invalidated.

Open the full results here.

Analyzed 16 files, diff c67d10d...6f1ab60.

@p0mvn
p0mvn marked this pull request as draft July 2, 2026 20:47
@p0mvn
p0mvn force-pushed the roman/zakura-single-admission-authority branch from 6f1ab60 to da9c4fb Compare July 2, 2026 21:30
@p0mvn
p0mvn changed the base branch from roman/zakura-memmgmt-cleanup to ironwood-main July 2, 2026 21:31
@p0mvn
p0mvn force-pushed the roman/zakura-single-admission-authority branch from da9c4fb to 571f201 Compare July 2, 2026 23:01
@p0mvn
p0mvn force-pushed the roman/zakura-single-admission-authority branch from 571f201 to bc5c72b Compare July 3, 2026 01:12
p0mvn added 2 commits July 3, 2026 01:22
… request budget

Post-rebase integration of #361 (probe-first no-progress policy,
reliability-weighted cwnd, first-completion-wins for unmatched active bodies)
with the single-admission-authority accounting:

- The unmatched-body path gates on the new admit_received_body() check (a
  received body consumes no request budget, so the resident gate is the sole
  admission decision), claims pending heights via claim_received, and ends a
  competing peer's active reservation via release_active_reserved_height —
  the ended estimate is released after the forward, mirroring the matched
  receipt path.
- release_reserved_heights loses its Held-skip special case (a received
  height is Released, which release_reserved() already skips);
  release_and_return_items becomes test-only (production paths must use the
  reserved-only variant so received bodies stay claimed for the commit
  pipeline).
- The Drop-safety and GC-skip tests are re-premised on ended-reservation
  semantics instead of Held.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant