Serialize bulk payloads as bytes, not byte-by-byte

[phil-opp]

Stacked on #3153 (migrate-bincode-to-postcard) — review that first. This PR is based on that branch, so its diff shows only the change described below; GitHub will retarget it to main once #3153 merges.

DataMessage::Vec and InterDaemonEvent::Output.data reach the encoder through serde's default slice impl, which emits serialize_seq plus one serialize_u8 per element; postcard turns that into a try_push per byte. This routes them through serialize_bytes / deserialize_bytes so the whole slice moves at once.

No wire change. postcard writes a varint length followed by the raw bytes either way, so this needs no version bump. encoding_is_unchanged_from_the_seq_form pins it against the default sequence encoding at the varint length-prefix boundary (127/128/129) and either side of it, and the golden vectors in libraries/message/tests/uhlc_wire_format.rs are untouched and still pass. JSON is unaffected too — serde_json renders both forms as an array of numbers, and the visitor takes that shape back via visit_seq, which survives_a_json_round_trip asserts byte-for-byte against the sequence form.

DaemonRequest::SendMessage, pre-sized encode:

payload	before	after
64 B	111 ns	56 ns	2.0x
4 KB	3.71 µs	141 ns	26x
64 KB	59.2 µs	1.24 µs	48x
1 MB	932 µs	21.1 µs	44x

Decode is the same shape: 4 KB 3.90 µs → 151 ns, 1 MB 1.15 ms → 22.2 µs.

4 KB is the case that matters most. It's the largest message that stays on the daemon↔node TCP path — anything above ZERO_COPY_THRESHOLD goes via shared memory — so that hop drops from ~7.6 µs of serde overhead to ~0.29 µs.

This isn't a regression the postcard migration introduced; bincode was equally slow on the same path (~979 µs for 1 MB). It only became visible while benchmarking the migration, and I kept it out of that PR so a bisect couldn't confuse "postcard" with "custom serde impl".

One thing worth recording

Now that the payload is a bulk copy, encode_presized only clearly wins on small messages — 2.3x at 64 B, 1.4x at 4 KB. At 64 KB and above, growing from empty is in fact marginally faster, because Vec::extend reserves the payload exactly once and pre-sizing then only adds the envelope overshoot. The small sizes are exactly the ones on the TCP path, so the helper stays as-is; I've corrected the bench comment, which previously claimed a win at every size.

Deserialization still produces a 128-byte-aligned AVec, which the Arrow zero-copy decode path depends on — asserted in round_trips_and_preserves_alignment as well as the pre-existing daemon_path_ipc_roundtrip_preserves_payload_and_alignment.

Verification

• cargo fmt --all -- --check, cargo clippy --all -- -D warnings — clean
• cargo test --all — 125 test binaries, zero failures
• Benchmarks above from cargo bench -p dora-message --bench message_serde, before/after on the same machine

[trunk-io]

😎 Merged successfully - details.

[phil-opp]

🤖 Automated review by Claude — fully automated review; no human has verified these findings.

No issues found.

What I checked:

• Wire-format equivalence (the load-bearing claim): encoding_is_unchanged_from_the_seq_form compares the new serialize_bytes output against the pre-PR default (AVec's serialize_seq) byte-for-byte, straddling postcard's varint length-prefix boundary at 127/128. postcard writes varint(len) + raw bytes for both serialize_seq of u8 and serialize_bytes, so the assertion holds and this is genuinely an unversioned change. The option::serialize path adds only the standard Option discriminant, also covered.
• Deserialization: postcard's deserialize_bytes yields borrowed bytes → visit_borrowed_bytes → AVec::from_slice(128, ..), preserving the 128-byte alignment the Arrow zero-copy path needs (asserted in round_trips_and_preserves_alignment). The self-describing (JSON) path lands in visit_seq and is exercised by survives_a_json_round_trip.
• Cross-version / record-replay compat: because the bytes on the wire are identical, mixed old/new daemons over Zenoh and replay of previously recorded logs remain compatible.
• The tests are not vacuous — they assert on real byte outputs and round-tripped values, not tautologies.

---

Generated by Claude Code