refactor(tracing): add lazy typed trace events by evan-forbes · Pull Request #228 · zakura-core/zakura

evan-forbes · 2026-07-17T00:20:17Z

Motivation

Zakura's production JSONL trace events were assembled through mutable raw JSON maps at call sites. This scattered schema ownership across reactors and drivers, made required fields implicit, and performed some peer hashing and diagnostic projection work before knowing whether the trace channel could accept an event.

Solution

add typed JSONL tables, events, and a reserve-first JsonlEventEmitter while preserving the public raw-map compatibility API
move block reactor, peer-routine, sequencer, header/block driver, chain-tip mirror, handler, discovery, exchange, and legacy trace construction into owner-local trace modules
reduce production control flow to semantic trace_* calls while leaving metrics and ordinary tracing logs beside the state changes they observe
replace the zakurad commit-state property bag and key-dispatch helpers with typed owner projections
absorb newly added exact header-operation identities and legacy peer-attribution traces into typed, reserve-first owner projections
construct peer labels, display hashes, error strings, elapsed durations, collection counts, and event projections only after reserving channel capacity
preserve existing table names, file names, event names, field representations, ordering, and null-versus-omitted behavior
add literal JSON-order tests, parsed-shape tests, laziness tests, and inventory tests that reject serializer construction in core owner files

The block-sync sparse row projection remains internal to block-sync trace modules; it is no longer imported or constructed by reactor, peer-routine, or sequencer core logic.

Size

The initial PR state was GitHub +3,798/-2,309 (net +1,489) and +754 production Rust LoC by the counter below. The current PR is GitHub +6,344/-3,960 (net +2,384) and +1,289 production Rust LoC.

Production LoC is defined as non-blank, non-comment physical Rust lines excluding tests, testkit, benches, bench.rs, tests.rs, and brace-delimited items immediately preceded by #[cfg(test)]; strings and comments are stripped before brace counting. The semantic owner split was kept despite the increase because it removes serialization policy and mutable field assembly from production control flow.

Compatibility results

exact key-order and parsed-shape tests pass for the typed owner projections and optional branches
fixed handler null fields, sparse omissions, legacy-request null peers, numeric widths, hashed peers, display hashes, and error representations remain covered
disabled, full, and closed emitters do not invoke event builders
the smoke trace oracle passed after a 203-block from-scratch kind-6 catch-up and reorg recovery
the plotting parser produced non-empty SVGs from node2: 1,452 trace samples over heights 1-204, with no HoL stall samples or reorder growth
GitHub Actions passed all 44 executed checks after rebasing onto current main

Testing

cargo fmt --all -- --check
cargo test -p zakura-jsonl-trace
cargo test -p zakura-network --lib trace::tests
CXXFLAGS='-include cstdint' cargo test -p zakura --lib trace::tests
cargo clippy -p zakura-jsonl-trace --all-targets -- -D warnings
cargo clippy -p zakura-network --lib -- -D warnings
CXXFLAGS='-include cstdint' cargo clippy -p zakura --lib -- -D warnings
python3 docker/zakura-regtest-e2e/trace_oracle.py --self-test
CXXFLAGS='-include cstdint' ZAKURA_E2E_MODE=smoke docker/zakura-regtest-e2e/run.sh
python3 .agents/skills/zakura-trace-plots/scripts/plot_zakura_traces.py /tmp/zakura-regtest-e2e-traces-zakura-smoke/node2 --out-dir /tmp/zakura-regtest-e2e-trace-plots
git diff --check

Specifications & References

Existing JSONL trace schemas are preserved for downstream trace-oracle and plotting consumers.

v12-auditor · 2026-07-17T00:20:23Z

Note

Complete: Audit complete. V12 did not find any issues that need review.

Open the full results here.

Analyzed 23 files, diff cb83db6...56f9421.

evan-forbes

net gain LoC, mainly due to typing, isolation, and testing

however imo we're getting a massive readability gain

if we wanted to pursue macros we could likely remove most of the tracing files as well

evan-forbes · 2026-07-18T17:36:24Z

-    fn trace_body_submitted(&self, height: block::Height, token: BlockApplyToken) {
-        self.trace.emit_with(BLOCK_SYNC_TABLE, |row| {
-            row.insert(
-                bs_trace::EVENT.to_string(),
-                serde_json::Value::String(bs_trace::BLOCK_BODY_SUBMITTED.to_string()),
-            );
-            bs_insert_height(row, bs_trace::HEIGHT, height);
-            bs_insert_u64(row, bs_trace::APPLY_TOKEN, token);
-        });
-    }


followed the refactor we did in headersync where we move the tracing code to it's own file

evan-forbes · 2026-07-18T17:37:37Z

 ) {
-    trace.emit_with(DISCOVERY_TABLE, |row| {
-        row.insert(
-            d_trace::EVENT.to_string(),
-            serde_json::Value::String(d_trace::DISCOVERY_DIAL_RESULT.to_string()),
-        );
-        row.insert(
-            d_trace::RESULT.to_string(),
-            serde_json::Value::String(result.label().to_string()),
-        );
-        let peer = ZakuraPeerId::new(node_id.as_bytes().to_vec())
-            .map(|peer_id| peer_label(&peer_id))
-            .ok();
-        row.insert(
-            d_trace::PEER.to_string(),
-            peer.map_or(serde_json::Value::Null, serde_json::Value::String),
-        );
-    });
+    trace.emit_event(|| DiscoveryDialResultEvent::new(node_id.as_bytes(), result));


while net gain in LoC

we're getting imo massive readability gain here using a single line to do the same work as before

evan-forbes · 2026-07-18T17:38:29Z

-                        self.trace_get_headers_sent(
-                            &identity.peer,
-                            range,
-                            range.count,
-                            peer_cap,
-                            GetHeadersTraceMeta {
-                                request_id,
-                                session_id: identity.session_id,
-                                stream_version: ZAKURA_HEADER_SYNC_STREAM_VERSION,
-                            },
-                        );
+                        self.trace_get_headers_sent(range, peer_cap, &identity, request_id);


another example

our goal here is just be sane with readability

evan-forbes

net gain LoC, mainly due to typing, isolation, and testing

however imo we're getting a massive readability gain

if we wanted to pursue macros we could likely remove most of the tracing files as well

ebfull · 2026-07-18T20:43:23Z

main's history was rewritten to remove the 68 MB Sprout-history artifact from git (old tip 089a0baa0 → b9d0ebd0c). This branch is based on the old history — merging before rebasing would re-add stale base commits including the blob. Converted to draft.

To fix:

git fetch origin
git rebase --onto origin/main 089a0baa087cbf19cef6e7023014d91a0212f4dc
git push --force-with-lease

This branch contains a merge commit; add --rebase-merges if you want to keep it. Mark ready for review after.

evan-forbes commented Jul 18, 2026

View reviewed changes

evan-forbes marked this pull request as ready for review July 18, 2026 17:40

ebfull force-pushed the main branch from 089a0ba to b9d0ebd Compare July 18, 2026 20:39

ebfull marked this pull request as draft July 18, 2026 20:42

evan-forbes mentioned this pull request Jul 19, 2026

fix(network): do not advertise pruned block bodies #275

Merged

evan-forbes force-pushed the reduce-tracing branch from f9ddb4b to d1a42d9 Compare July 19, 2026 19:22

evan-forbes added 4 commits July 20, 2026 16:40

refactor(tracing): add lazy typed trace events

e31bb69

refactor(tracing): localize semantic trace events

a58ec9f

refactor(tracing): inline typed trace emission

e2b5aa0

refactor(tracing): absorb new semantic trace events

4cdb79e

evan-forbes force-pushed the reduce-tracing branch from d1a42d9 to 4cdb79e Compare July 20, 2026 21:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(tracing): add lazy typed trace events#228

refactor(tracing): add lazy typed trace events#228
evan-forbes wants to merge 4 commits into
mainfrom
reduce-tracing

evan-forbes commented Jul 17, 2026 •

edited

Loading

Uh oh!

v12-auditor Bot commented Jul 17, 2026 •

edited

Loading

Uh oh!

evan-forbes left a comment

Uh oh!

evan-forbes Jul 18, 2026

Uh oh!

evan-forbes Jul 18, 2026 •

edited

Loading

Uh oh!

evan-forbes Jul 18, 2026

Uh oh!

evan-forbes left a comment

Uh oh!

ebfull commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

evan-forbes commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Solution

Size

Compatibility results

Testing

Specifications & References

Uh oh!

v12-auditor Bot commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

evan-forbes left a comment

Choose a reason for hiding this comment

Uh oh!

evan-forbes Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

evan-forbes Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evan-forbes Jul 18, 2026

Choose a reason for hiding this comment

Uh oh!

evan-forbes left a comment

Choose a reason for hiding this comment

Uh oh!

ebfull commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

evan-forbes commented Jul 17, 2026 •

edited

Loading

v12-auditor Bot commented Jul 17, 2026 •

edited

Loading

evan-forbes Jul 18, 2026 •

edited

Loading