Replace unmaintained bincode with postcard - #3153
Merged
Merged
Conversation
bincode is unmaintained (RUSTSEC-2025-0141): development stopped at 1.3.3 and every version is flagged, so it is not something to carry into 1.0. postcard is serde-based, so this is an encoding change only -- no message type changes. Unlike bincode it has a documented, stable wire spec, which is the property that matters for a format 1.0 commits to. Messages shrink 25-27 bytes each (Metadata: 34 -> 27 B) from varint integers and length prefixes. Encode goes through `dora_message::encode`/`encode_presized` and decode through `dora_message::decode`; postcard is named in one crate. Two behaviours are load-bearing there: - `encode_presized` pre-sizes the buffer from the message's bulk payload. postcard's StdVec flavor grows from empty, which cost 12-74% over bincode; pre-sized it matches or beats bincode at every size (64 B: 107 -> 111 ns, 4 KB: 3.84 -> 3.71 us, 1 MB: 960 -> 932 us). Decode was already at parity or better. - `decode` rejects trailing bytes. postcard's own from_bytes ignores them where bincode errored, and the zenoh attachment filters use a decode failure to mean "not a dora message, ignore it". Four format versions move, all failing loudly rather than misparsing: Metadata::CURRENT_VERSION 1 -> 2 (rejects a mixed-version peer at register), coordinator store SCHEMA_VERSION 4 -> 5, and .drec FORMAT_VERSION 1 -> 2 with a new minimum-supported floor -- the container framing is unchanged, so without the floor a v1 recording would pass the header check and fail per entry. Both .drec writers hardcoded `version: 1`, so FORMAT_VERSION is now public and stamped from one place. The WebSocket topic-data channel carries these bytes to third-party subscribers and has no version handshake; both encodings are positional, so a bincode-era subscriber misparses rather than errors. Documented in the channel spec and the changelog. bincode remains in the tree only transitively via zenoh-ext, so the deny.toml waiver stays but is no longer a dora dependency. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
😎 Merged successfully - details. |
This was referenced Aug 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
bincode is unmaintained (RUSTSEC-2025-0141) — development stopped at 1.3.3 and every version is flagged — so it isn't something to carry into 1.0. This moves every binary plane to postcard.
postcard is serde-based, so this is an encoding change only: no message types change. Unlike bincode it has a documented, stable wire spec, which is the property that actually matters for a format 1.0 commits to. Messages shrink 25–27 bytes each (
Metadataalone: 34 → 27 B) from varint integers and length prefixes.I looked at rkyv too and rejected it: it isn't serde, so every type in
dora-messagewould need a second derive set plus hand-writtenArchiveWithwrappers foruhlc::Timestamp,AVec,chronoandsemver— and the serde derives couldn't go away, because the WS and descriptor planes still need them. Its zero-copy win also doesn't land here: the bulk path is already zero-copy shared memory carrying Arrow IPC, and the one large thing this touches (DataMessage::Vec) is opaque bytes that any format memcpy's. bitcode is disqualified outright — it lists "stable format across major versions" as a non-goal, which is fatal for a protocol spoken between independently-versioned binaries.Encode goes through
dora_message::encode/encode_presizedand decode throughdora_message::decode.postcardis now named in exactly oneCargo.toml, so the codec is swappable from one file and a call site can't accidentally reach for the lax variant. Two behaviours there are load-bearing:Pre-sizing. Plain
postcard::to_stdvecbenchmarked 12–74% slower than bincode at serialize — entirelyStdVecgrowing from an emptyVec, not the encoder. Encoding into a pre-sized buffer erases it:Decode was already at parity or better. The bench keeps a
grow_from_emptyarm so the win stays measurable in-tree — if that arm stops being slower,encode_presizedhas stopped earning its complexity.Trailing-byte strictness.
postcard::from_bytesignores trailing bytes where bincode errored. The zenoh attachment filters read a decode failure as "foreign publisher, ignore it", so without strictness an attachment that merely starts with somethingMetadata-shaped would be accepted as genuine.decoderestores the strict behaviour and is the only decode entry point.The version gates
All fail loudly rather than misparse:
Metadata::CURRENT_VERSION1 → 2 — a mixed-version node/daemon pair is rejected at register with a clear error.SCHEMA_VERSION4 → 5 — v4 rows can't be read;RedbStore::open()rejects via the existingschema version mismatchpath, sodora upoffers--recreate-storeas usual..drecFORMAT_VERSION1 → 2, plus a new minimum-supported floor. This one matters: the container framing is unchanged, so without the floor a v1 recording passes the header check and then fails per-entry, surfacing as "corrupt or format-drifted recording" instead of a version error.Two things to know
.drecbug fell out of this. Both writers (binaries/cli/src/command/record.rs,binaries/record-node/src/main.rs) hardcodedversion: 1, so any future bump would have produced files their own reader rejects.FORMAT_VERSIONis nowpuband stamped from one place, with aconst _: () = assert!(...)tying it to the reader's floor.docs/websocket-topic-data-channel.mdand the changelog; happy to add the handshake separately if you'd rather.bincodeis gone from dora's own dependencies. It remains in the tree transitively viazenoh-ext1.9, so thedeny.tomlwaiver stays but is re-scoped to that.Verification
cargo fmt --all -- --check,cargo clippy --all -- -D warnings— cleancargo test --all— 125 test binaries, zero failurescargo check --examples,cargo check --all --all-targets— cleanlibraries/message/tests/uhlc_wire_format.rsre-pinned; the newTIMESTAMP_HEXwas hand-computed as LEB128 first, then confirmed against the implementationAll of the above ran with a private
CARGO_TARGET_DIR— the shared.shared-target/was leaking another worktree's artifacts and producing phantom errors in unrelated crates.Two follow-ups deliberately left out and filed separately: a ~50–65x payload-serialization win that is wire-compatible but shouldn't share a commit with a wire-format change, and a per-connection encode buffer.