epoll: parallelize the central acceptor so workers don't starve under connection churn (keep the central-acceptor model)

[enghitalo]

Context

On HttpArena the epoll backend wins keep-alive profiles — baseline-4096 vanilla-epoll 4.03M @ 6307% CPU beats vanilla-io_uring 3.62M, so the central-acceptor + round-robin model is the right default and should be kept (it has consistently tested better than per-worker SO_REUSEPORT accept).

The one workload where it loses is connection churn: limited-conn (gcannon -r 10 → close every 10 reqs).

• limited-conn-4096: epoll 1.01M @ 3045% CPU vs io_uring 2.25M @ 5164%, zix 2.75M.
• epoll CPU collapses from 6307% (baseline) to 3045% the instant accept churn appears → the workers are idle, starved behind the acceptor.

Root cause

A single acceptor runs handle_accept_loop on the main thread (http_server/backend_epoll/worker_linux.c.v:339). Per accepted connection it does, serially on one core: accept4 (:75), an active_conns atomic load (:93), set_tcp_nodelay = a setsockopt syscall (:98), a cross-thread epoll_ctl(ADD) into a worker epoll (:105), an atomic add (:111). Workers never accept. Under churn this one thread is the ceiling.

Proposed fix — keep the central-acceptor model, remove the single-thread ceiling

Do not move to per-worker SO_REUSEPORT accept. Instead:

1. Small pool of acceptor threads (N≈2–4) sharing the one listener. Multiple threads can accept4() the same listening fd; the kernel hands each connection to one thread. The accept + setsockopt + epoll_ctl(ADD) work then runs in parallel across the pool, while the architecture stays "dedicated acceptors → round-robin into worker epolls" (workers still never accept). Use blocking accept4 loops, or EPOLLEXCLUSIVE on the listener across the acceptors to avoid a thundering herd.
2. Take set_tcp_nodelay off the acceptor hot path. TCP_NODELAY is one setsockopt syscall per accept on the bottlenecked thread. Defer it to the worker (set on the fd's first readable event) so the acceptor only does accept4 + epoll_ctl(ADD).
3. (optional) Make max_connections accounting a relaxed/per-acceptor counter to drop the per-accept atomic contention once there are N acceptors.

This keeps the model that wins everywhere else and should roughly double limited-conn (≈1.0M → ≈2.0M) by un-starving the workers — baseline proves the per-request engine is already as fast as io_uring's, so the entire deficit is the serial accept.

Acceptance

• limited-conn-{512,4096} epoll rps approaches its own io_uring (~2.2M) with worker CPU back near saturation.
• No regression on baseline-* (the keep-alive case the single acceptor already wins).

Fix location: http_server/backend_epoll/worker_linux.c.v (+ listener/worker wiring). Found in the HttpArena profile audit.