perf: add SO_REUSEPORT multiple accept loops (--workers flag) - #49
Closed
kacy wants to merge 1 commit into
Closed
Conversation
adds --workers flag that spawns multiple accept loops on the same port: - each accept loop binds via SO_REUSEPORT - kernel distributes connections across accept loops - all loops run on the same tokio multi-threaded runtime as shards - reduces contention on single accept loop this approach keeps everything on the same runtime (unlike the previous worker-per-thread approach which had cross-runtime overhead).
Owner
Author
|
closing in favor of #50 (concurrent keyspace mode). investigation showed that the accept loop was not the bottleneck - the channel overhead was. PR #50 addresses the actual problem by eliminating channels for GET/SET, achieving 2x better throughput. SO_REUSEPORT may still be useful at very high connection counts, but it's not needed for the current performance goals. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
adds
--workersflag that spawns multiple accept loops on the same port usingSO_REUSEPORT. the kernel distributes incoming connections across accept loops, potentially reducing contention under high connection rates.benchmark results (GCP c2-standard-8)
no significant difference — the accept loop is not the bottleneck.
investigation findings
this PR is part of a larger investigation into why ember's multi-core scaling is broken (8 shards ≈ 1 shard performance).
what we tried
root cause
profiling shows ~17% CPU time in futex operations and memory allocation:
futex_wake/futex_waitfrom mpsc/oneshot channelsmalloc/cfreefor oneshot channel allocation per requestevery request crosses thread boundaries twice:
what would actually fix it
to achieve true multi-core scaling, ember needs to bypass channels for local operations:
true sharded acceptors — each core owns its own keyspace partition. connections accepted on that core execute locally without channels. only cross-shard operations use channels.
connection affinity — pin connections to specific shards based on a hash. all operations from that connection execute on that shard without routing.
these are significant architectural changes that require restructuring how shards and connections interact.
test plan
changes
--workersflag to enable multiple accept loopssocket2crate for SO_REUSEPORT support