Skip to content

perf: dispatch pipelined commands concurrently - #48

Merged
kacy merged 1 commit into
mainfrom
perf/connection-affinity
Feb 7, 2026
Merged

perf: dispatch pipelined commands concurrently#48
kacy merged 1 commit into
mainfrom
perf/connection-affinity

Conversation

@kacy

@kacy kacy commented Feb 7, 2026

Copy link
Copy Markdown
Owner

summary

changes pipelined command processing from serial to concurrent dispatch. instead of awaiting each command before starting the next, all commands are dispatched to shards simultaneously using join_all.

rationale

for a pipeline of 16 commands, the old code did 16 sequential channel round-trips:

for frame in frames {
    let response = process(frame).await;  // serial
    responses.push(response);
}

the new code dispatches all 16 concurrently:

let futures: Vec<_> = frames.map(|f| process(f)).collect();
let responses = join_all(futures).await;  // parallel

this allows shards to process commands in parallel rather than one at a time.

benchmarks (GCP c2-standard-8)

test before after change
GET P=16 (8 shards) ~1,069k ~1,126k +5%
SET P=16 (8 shards) ~998k ~1,003k ~same

the improvement is modest because the channel overhead (mpsc send + oneshot reply) is still present for every command. multi-core scaling remains broken.

test plan

  • verified functional correctness with redis-cli (SET, GET, MGET, MSET)
  • benchmarked on GCP VM with redis-benchmark
  • tested with 50 and 200 concurrent clients

next steps

to achieve true multi-core scaling, ember needs architectural changes:

  1. sharded acceptors - one tokio runtime + TcpListener per core via SO_REUSEPORT
  2. connection-local shards - connections pinned to shards, single-key ops bypass channels

the channel-per-request model is the fundamental bottleneck.

instead of processing pipelined commands serially (awaiting each one
before starting the next), dispatch all commands to shards at once
and await them together using join_all.

for a pipeline of 16 commands, the old code did 16 sequential channel
round-trips. the new code dispatches all 16 concurrently, allowing
shards to process them in parallel.

this should significantly improve pipelined throughput by utilizing
all shards simultaneously rather than one at a time.
@kacy
kacy merged commit 1140d95 into main Feb 7, 2026
5 checks passed
@kacy
kacy deleted the perf/connection-affinity branch February 7, 2026 20:06
kacy added a commit that referenced this pull request Feb 11, 2026
perf: dispatch pipelined commands concurrently
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant