Summary
TcpConnection::send_reply (binaries/daemon/src/node_communication/tcp.rs)
allocates a fresh Vec for every reply, writes the encoded bytes into it, hands
the slice to the socket, and drops it. The same shape applies to
send_message in apis/rust/node/src/daemon_connection/tcp.rs.
dora_message::encode_presized already removed the realloc churn there by
sizing the buffer up front, but the malloc/free pair per message remains. Since
send_reply takes &mut self, the buffer can simply live on the connection.
Proposed fix
postcard::to_extend returns the container it was given, so this is a small
change:
// TcpConnection gains: buf: Vec<u8>
self.buf.clear();
let buf = dora_message::encode_presized_into(
&message,
message.encode_size_hint(),
std::mem::take(&mut self.buf),
)?;
socket_stream_send(&mut self.0, &buf).await?;
self.buf = buf;
which needs one new dora-message entry point taking the buffer to encode into,
alongside the existing encode / encode_presized.
The node side (daemon_connection::tcp::send_message) is the same idea, but the
buffer has to move onto the connection struct rather than being created per call,
so it's slightly more invasive.
Expected size of the win
Estimated, not measured: roughly 30–100 ns per message, plus the allocator
pressure of one malloc/free pair per reply. For calibration, the migration's own
benchmark puts a whole pre-sized 64-byte SendMessage encode at 111 ns, so the
allocation is a meaningful fraction of that.
This should be measured before it's implemented — it's the weakest-evidence item
of the three that came out of the postcard work, and it's the one most likely to
turn out not to be worth the extra state on the connection type.
Ordering note
This composes with the serialize_bytes follow-up and is worth doing after it.
Today a 4 KB encode is ~3.7 µs, so ~30 ns of allocation is noise. Once the
payload loop drops to ~63 ns, the allocation becomes a large fraction of the
remaining cost — so the same change is worth substantially more then, and is
easier to measure honestly.
Summary
TcpConnection::send_reply(binaries/daemon/src/node_communication/tcp.rs)allocates a fresh
Vecfor every reply, writes the encoded bytes into it, handsthe slice to the socket, and drops it. The same shape applies to
send_messageinapis/rust/node/src/daemon_connection/tcp.rs.dora_message::encode_presizedalready removed the realloc churn there bysizing the buffer up front, but the malloc/free pair per message remains. Since
send_replytakes&mut self, the buffer can simply live on the connection.Proposed fix
postcard::to_extendreturns the container it was given, so this is a smallchange:
which needs one new
dora-messageentry point taking the buffer to encode into,alongside the existing
encode/encode_presized.The node side (
daemon_connection::tcp::send_message) is the same idea, but thebuffer has to move onto the connection struct rather than being created per call,
so it's slightly more invasive.
Expected size of the win
Estimated, not measured: roughly 30–100 ns per message, plus the allocator
pressure of one malloc/free pair per reply. For calibration, the migration's own
benchmark puts a whole pre-sized 64-byte
SendMessageencode at 111 ns, so theallocation is a meaningful fraction of that.
This should be measured before it's implemented — it's the weakest-evidence item
of the three that came out of the postcard work, and it's the one most likely to
turn out not to be worth the extra state on the connection type.
Ordering note
This composes with the
serialize_bytesfollow-up and is worth doing after it.Today a 4 KB encode is ~3.7 µs, so ~30 ns of allocation is noise. Once the
payload loop drops to ~63 ns, the allocation becomes a large fraction of the
remaining cost — so the same change is worth substantially more then, and is
easier to measure honestly.