Skip to content

Replace unmaintained bincode with postcard - #3153

Merged
trunk-io[bot] merged 1 commit into
mainfrom
migrate-bincode-to-postcard
Aug 13, 2026
Merged

Replace unmaintained bincode with postcard#3153
trunk-io[bot] merged 1 commit into
mainfrom
migrate-bincode-to-postcard

Conversation

@phil-opp

Copy link
Copy Markdown
Collaborator

bincode is unmaintained (RUSTSEC-2025-0141) — development stopped at 1.3.3 and every version is flagged — so it isn't something to carry into 1.0. This moves every binary plane to postcard.

postcard is serde-based, so this is an encoding change only: no message types change. Unlike bincode it has a documented, stable wire spec, which is the property that actually matters for a format 1.0 commits to. Messages shrink 25–27 bytes each (Metadata alone: 34 → 27 B) from varint integers and length prefixes.

I looked at rkyv too and rejected it: it isn't serde, so every type in dora-message would need a second derive set plus hand-written ArchiveWith wrappers for uhlc::Timestamp, AVec, chrono and semver — and the serde derives couldn't go away, because the WS and descriptor planes still need them. Its zero-copy win also doesn't land here: the bulk path is already zero-copy shared memory carrying Arrow IPC, and the one large thing this touches (DataMessage::Vec) is opaque bytes that any format memcpy's. bitcode is disqualified outright — it lists "stable format across major versions" as a non-goal, which is fatal for a protocol spoken between independently-versioned binaries.

Encode goes through dora_message::encode / encode_presized and decode through dora_message::decode. postcard is now named in exactly one Cargo.toml, so the codec is swappable from one file and a call site can't accidentally reach for the lax variant. Two behaviours there are load-bearing:

Pre-sizing. Plain postcard::to_stdvec benchmarked 12–74% slower than bincode at serialize — entirely StdVec growing from an empty Vec, not the encoder. Encoding into a pre-sized buffer erases it:

payload bincode postcard (grow) postcard (pre-sized)
64 B 107 ns 194 ns 111 ns
4 KB 3.84 µs 4.64 µs 3.71 µs
64 KB 59.4 µs 69.1 µs 59.2 µs
1 MB 960 µs 1083 µs 932 µs

Decode was already at parity or better. The bench keeps a grow_from_empty arm so the win stays measurable in-tree — if that arm stops being slower, encode_presized has stopped earning its complexity.

Trailing-byte strictness. postcard::from_bytes ignores trailing bytes where bincode errored. The zenoh attachment filters read a decode failure as "foreign publisher, ignore it", so without strictness an attachment that merely starts with something Metadata-shaped would be accepted as genuine. decode restores the strict behaviour and is the only decode entry point.

The version gates

All fail loudly rather than misparse:

  • Metadata::CURRENT_VERSION 1 → 2 — a mixed-version node/daemon pair is rejected at register with a clear error.
  • Coordinator store SCHEMA_VERSION 4 → 5 — v4 rows can't be read; RedbStore::open() rejects via the existing schema version mismatch path, so dora up offers --recreate-store as usual.
  • .drec FORMAT_VERSION 1 → 2, plus a new minimum-supported floor. This one matters: the container framing is unchanged, so without the floor a v1 recording passes the header check and then fails per-entry, surfacing as "corrupt or format-drifted recording" instead of a version error.

Two things to know

  1. A latent .drec bug fell out of this. Both writers (binaries/cli/src/command/record.rs, binaries/record-node/src/main.rs) hardcoded version: 1, so any future bump would have produced files their own reader rejects. FORMAT_VERSION is now pub and stamped from one place, with a const _: () = assert!(...) tying it to the reader's floor.
  2. The WS topic-data channel has no version handshake, and both encodings are positional — so a bincode-era third-party subscriber misparses rather than errors. There's no way to make that fail cleanly without adding a handshake to the channel, which felt out of scope here. Documented in docs/websocket-topic-data-channel.md and the changelog; happy to add the handshake separately if you'd rather.

bincode is gone from dora's own dependencies. It remains in the tree transitively via zenoh-ext 1.9, so the deny.toml waiver stays but is re-scoped to that.

Verification

  • cargo fmt --all -- --check, cargo clippy --all -- -D warnings — clean
  • cargo test --all — 125 test binaries, zero failures
  • cargo check --examples, cargo check --all --all-targets — clean
  • Golden wire vectors in libraries/message/tests/uhlc_wire_format.rs re-pinned; the new TIMESTAMP_HEX was hand-computed as LEB128 first, then confirmed against the implementation

All of the above ran with a private CARGO_TARGET_DIR — the shared .shared-target/ was leaking another worktree's artifacts and producing phantom errors in unrelated crates.

Two follow-ups deliberately left out and filed separately: a ~50–65x payload-serialization win that is wire-compatible but shouldn't share a commit with a wire-format change, and a per-connection encode buffer.

bincode is unmaintained (RUSTSEC-2025-0141): development stopped at
1.3.3 and every version is flagged, so it is not something to carry
into 1.0.

postcard is serde-based, so this is an encoding change only -- no
message type changes. Unlike bincode it has a documented, stable wire
spec, which is the property that matters for a format 1.0 commits to.
Messages shrink 25-27 bytes each (Metadata: 34 -> 27 B) from varint
integers and length prefixes.

Encode goes through `dora_message::encode`/`encode_presized` and decode
through `dora_message::decode`; postcard is named in one crate. Two
behaviours are load-bearing there:

- `encode_presized` pre-sizes the buffer from the message's bulk
  payload. postcard's StdVec flavor grows from empty, which cost 12-74%
  over bincode; pre-sized it matches or beats bincode at every size
  (64 B: 107 -> 111 ns, 4 KB: 3.84 -> 3.71 us, 1 MB: 960 -> 932 us).
  Decode was already at parity or better.
- `decode` rejects trailing bytes. postcard's own from_bytes ignores
  them where bincode errored, and the zenoh attachment filters use a
  decode failure to mean "not a dora message, ignore it".

Four format versions move, all failing loudly rather than misparsing:
Metadata::CURRENT_VERSION 1 -> 2 (rejects a mixed-version peer at
register), coordinator store SCHEMA_VERSION 4 -> 5, and .drec
FORMAT_VERSION 1 -> 2 with a new minimum-supported floor -- the
container framing is unchanged, so without the floor a v1 recording
would pass the header check and fail per entry. Both .drec writers
hardcoded `version: 1`, so FORMAT_VERSION is now public and stamped
from one place.

The WebSocket topic-data channel carries these bytes to third-party
subscribers and has no version handshake; both encodings are positional,
so a bincode-era subscriber misparses rather than errors. Documented in
the channel spec and the changelog.

bincode remains in the tree only transitively via zenoh-ext, so the
deny.toml waiver stays but is no longer a dora dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@trunk-io

trunk-io Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

😎 Merged successfully - details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant