Skip to content

Reuse a per-connection encode buffer on the daemon↔node reply path instead of allocating per message #3155

Description

@phil-opp

Summary

TcpConnection::send_reply (binaries/daemon/src/node_communication/tcp.rs)
allocates a fresh Vec for every reply, writes the encoded bytes into it, hands
the slice to the socket, and drops it. The same shape applies to
send_message in apis/rust/node/src/daemon_connection/tcp.rs.

dora_message::encode_presized already removed the realloc churn there by
sizing the buffer up front, but the malloc/free pair per message remains. Since
send_reply takes &mut self, the buffer can simply live on the connection.

Proposed fix

postcard::to_extend returns the container it was given, so this is a small
change:

// TcpConnection gains: buf: Vec<u8>
self.buf.clear();
let buf = dora_message::encode_presized_into(
    &message,
    message.encode_size_hint(),
    std::mem::take(&mut self.buf),
)?;
socket_stream_send(&mut self.0, &buf).await?;
self.buf = buf;

which needs one new dora-message entry point taking the buffer to encode into,
alongside the existing encode / encode_presized.

The node side (daemon_connection::tcp::send_message) is the same idea, but the
buffer has to move onto the connection struct rather than being created per call,
so it's slightly more invasive.

Expected size of the win

Estimated, not measured: roughly 30–100 ns per message, plus the allocator
pressure of one malloc/free pair per reply. For calibration, the migration's own
benchmark puts a whole pre-sized 64-byte SendMessage encode at 111 ns, so the
allocation is a meaningful fraction of that.

This should be measured before it's implemented — it's the weakest-evidence item
of the three that came out of the postcard work, and it's the one most likely to
turn out not to be worth the extra state on the connection type.

Ordering note

This composes with the serialize_bytes follow-up and is worth doing after it.
Today a 4 KB encode is ~3.7 µs, so ~30 ns of allocation is noise. Once the
payload loop drops to ~63 ns, the allocation becomes a large fraction of the
remaining cost — so the same change is worth substantially more then, and is
easier to measure honestly.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions