Skip to content

perf: add SO_REUSEPORT multiple accept loops (--workers flag) - #49

Closed
kacy wants to merge 1 commit into
mainfrom
perf/sharded-acceptors
Closed

perf: add SO_REUSEPORT multiple accept loops (--workers flag)#49
kacy wants to merge 1 commit into
mainfrom
perf/sharded-acceptors

Conversation

@kacy

@kacy kacy commented Feb 7, 2026

Copy link
Copy Markdown
Owner

summary

adds --workers flag that spawns multiple accept loops on the same port using SO_REUSEPORT. the kernel distributes incoming connections across accept loops, potentially reducing contention under high connection rates.

benchmark results (GCP c2-standard-8)

mode SET P=16 GET P=16
--workers (8 accept loops) ~997k ~1,129k
default (single accept loop) ~999k ~1,109k

no significant difference — the accept loop is not the bottleneck.

investigation findings

this PR is part of a larger investigation into why ember's multi-core scaling is broken (8 shards ≈ 1 shard performance).

what we tried

  1. jemalloc (+13% for SET) — helps with allocation contention
  2. concurrent command dispatch (+5% for GET) — helps with pipelining
  3. SO_REUSEPORT multiple accept loops (~same) — accept loop not the bottleneck

root cause

profiling shows ~17% CPU time in futex operations and memory allocation:

  • futex_wake / futex_wait from mpsc/oneshot channels
  • malloc / cfree for oneshot channel allocation per request

every request crosses thread boundaries twice:

  1. connection → mpsc → shard (futex_wake)
  2. shard → oneshot → connection (futex_wake)

what would actually fix it

to achieve true multi-core scaling, ember needs to bypass channels for local operations:

  1. true sharded acceptors — each core owns its own keyspace partition. connections accepted on that core execute locally without channels. only cross-shard operations use channels.

  2. connection affinity — pin connections to specific shards based on a hash. all operations from that connection execute on that shard without routing.

these are significant architectural changes that require restructuring how shards and connections interact.

test plan

  • verified server starts with --workers flag
  • verified functional correctness (SET, GET, MGET, MSET work correctly)
  • benchmarked on GCP VM with redis-benchmark

changes

  • adds --workers flag to enable multiple accept loops
  • uses socket2 crate for SO_REUSEPORT support
  • each accept loop runs as a tokio task on the same runtime as shards

adds --workers flag that spawns multiple accept loops on the same port:
- each accept loop binds via SO_REUSEPORT
- kernel distributes connections across accept loops
- all loops run on the same tokio multi-threaded runtime as shards
- reduces contention on single accept loop

this approach keeps everything on the same runtime (unlike the previous
worker-per-thread approach which had cross-runtime overhead).
@kacy

kacy commented Feb 7, 2026

Copy link
Copy Markdown
Owner Author

closing in favor of #50 (concurrent keyspace mode).

investigation showed that the accept loop was not the bottleneck - the channel overhead was. PR #50 addresses the actual problem by eliminating channels for GET/SET, achieving 2x better throughput.

SO_REUSEPORT may still be useful at very high connection counts, but it's not needed for the current performance goals.

@kacy kacy closed this Feb 7, 2026
@kacy
kacy deleted the perf/sharded-acceptors branch February 7, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant