io_uring: blocking libpq in the single-threaded ring worker collapses DB profiles (defer to the stateful-async migration)

[enghitalo]

Summary

The io_uring backend has no async runtime (README: request_handler only). The HttpArena DB endpoints call the blocking db.pg client (PQexecParams) synchronously inside the single-threaded ring worker → head-of-line blocking, the ring stalls on every query.

Impact (HttpArena, vanilla-io_uring vs vanilla-epoll)

• async-db ~10.7k rps (epoll ~280k) — ~26× behind its own epoll
• fortunes ~41 rps (epoll ~165k)
• api-16 ~28k, api-4 ~29k, crud ~225k (epoll 307k)

epoll handles all of these fine via the per-worker async pool + async_handler; only io_uring collapses.

Resolution (deferred — per the roadmap)

Tracked for the planned "everything stateful-async in vanilla" direction: give the io_uring backend the same async runtime (await DB readiness on the ring instead of blocking), or route DB calls to a worker thread pool as an interim. Not a quick fix — XL. Filed so the io_uring DB collapse is documented and tied to that future work.

Fix location: io_uring backend (http_server/http_server_io_uring_linux.c.v + the async runtime). Found in the HttpArena profile audit.

[enghitalo]

Design study: how to give the io_uring backend a non-blocking DB path

Deep dive into the two viable solution families, grounded in the actual code. Bottom line up front: build the native IORING_OP_POLL_ADD async runtime (mirror the epoll reactor), phased. Do not build the thread-pool interim — it is more work, not less. Reasoning and a concrete implementation plan below.

---

1. Root cause (confirmed in code)

The io_uring backend has no async path today — new_server rejects it outright (http_server/http_server.c.v:319-323: async_handler requires the epoll backend). The ring worker only ever gets a request_handler / stateful_handler (io_uring_worker_main, http_server_io_uring_linux.c.v:531), and the handler is invoked synchronously at:

• drain_iou_requests → handler(req, conn.fd, mut conn.response_buffer) — http_server_io_uring_linux.c.v:340
• start_iou_body_drain (large-body head path) — :406

The worker is single-threaded: it owns the only io_uring_submit_and_wait (:614/:616) that reaps every connection's completions. A blocking db.pg/libpq call at :340 therefore stalls the whole batch-drain loop (:629-645) — no other connection's recv/send makes progress. That is the 26×–4500× collapse on async-db / fortunes / crud / api.

2. Two key findings that shape the answer

1. The async runtime is already backend-pluggable. core.AsyncCtx / WakeFn / AsyncStep (core/core.v:56-114) are backend-neutral, and the watch primitive is installed by the backend via a fn pointer: RegisterFn = fn (mut ac AsyncCtx, ext_fd int, interest WatchInterest, cont WakeFn, udata voidptr) (core.v:88). AsyncCtx.reactor/loop_fd are opaque voidptr — core never dereferences them. So a backend supports async_handler by (a) installing its own register, (b) resuming continuations on readiness, (c) honoring .done/.suspend/.close. Nothing in core is epoll-specific.
2. pg_async is genuinely non-blocking and backend-neutral. It imports nothing from http_server; async_on_readable drains the socket to EAGAIN (pg_async/conn_async.v:240-282) and only needs a socket fd + a readiness signal + synchronous submit/flush/parse calls. The connect + SCRAM handshake blocks only once at pool bring-up (off the hot path). Given a watch(fd) primitive, the epoll entry's async_handler + pg_async code runs on io_uring verbatim.

Consequence: the DB driver is not the obstacle. The obstacle is purely giving the io_uring worker a watch(fd)+continuation reactor.

3. The two options

Option A — native IORING_OP_POLL_ADD reactor (mirror epoll). Add an op_poll op type; AsyncCtx.register queues a oneshot POLL_ADD on the DB socket fd; the CQE resumes the continuation on the ring thread. pg_async pumps the socket non-blockingly, exactly as on epoll.

Option B — thread-pool + eventfd offload (interim). Keep a blocking query; run it on a pthread pool; wake the ring via an eventfd it polls; resume the continuation when a result lands.

Both options need the same async spine: lift the new_server gate, plumb async_handler into io_uring_worker_main, make the drain loop park-aware, add a parked-connection state to Connection, and add a resume→flush→drain-pipelined-leftovers path. B adds a cross-thread subsystem on top of that. So B is strictly additive complexity over A, and its only real advantage — working with a blocking-only driver — is moot here because pg_async is already non-blocking. B also introduces: a per-worker PG-backend blow-up (one whole connection per in-flight query, no pipelining → workers × threads PG backends), a DB-connection-leak landmine on discarded completions, the codebase's first lock-free cross-thread hot path, and a helgrind blind spot (helgrind OOMs io_uring at 16 workers, so the 16-worker race can't be covered). Recommendation: A.

4. Design A — concrete plan

Reuse pg_async/* and the handler body verbatim; the only handler-side change is io_multiplexing: .io_uring. Re-implement the reactor as a parallel file (the epoll reactor is tightly coupled to PlainState/ConnState; io_uring uses Connection.response_buffer/bytes_sent + iou_flush_response), mirroring how epoll and io_uring already have two separate sync-path implementations.

Favorable fact: SINGLE_ISSUER is satisfied for free — every poll SQE is queued from a continuation that runs inside dispatch_io_uring_cqe on the ring's owning thread. No cross-thread submission, no lock, no new shared state.

4.1 New op + encoding. Add op_poll = u8(4) alongside op_accept/read/write (io_uring/io_uring_linux.c.v:63-65). Encode the ext_fd, not a pointer, in user_data's low 48 bits — the reactor's watch table grows by doubling and reallocates, so a packed &WatchEntry would UAF; the CQE handler instead does watches[ext_fd], fd-indexed, exactly like epoll. Add fn C.io_uring_prep_poll_add (+ poll_remove/cancel_fd for teardown) and pollin/out/err/hup consts, plus a prepare_poll helper (~25 LOC; liburing preps are header-inline, callable via fn C. since <liburing.h> + -luring are already wired).

4.2 Reactor (new file http_server_io_uring_async_linux.c.v, ~330 LOC). IouWatchEntry{active, conn &Connection, cont, udata, queue []IouParkSlot, persistent} + IouReactor{watches, worker, cur_conn}. reactor_watch/reactor_clear/reactor_mark_dead/reactor_orphan_single port near-verbatim from backend_epoll/async_linux.c.v:97-223 — pure array logic — with one substitution: store the stable &Connection slab pointer (the conns slab is allocated once and never reallocated, io_uring_linux.c.v:273-284 — the invariant that makes this sound) instead of epoll's client_fd. cur_conn bridges RegisterFn (which only gets mut ac, no conn) to the currently-running connection.

4.3 iou_async_register (RegisterFn, ~35 LOC). Twin of epoll's async_register (async_linux.c.v:229-260); identical except step 5 swaps epoll_ctl for a queued oneshot POLL_ADD. Mask = (writable ? pollout : pollin) | pollerr | pollhup. Oneshot, re-armed per park — maps 1:1 onto epoll's consume-then-re-arm (on_db_ready calls ac.watch(...) again and returns .suspend); since async_on_readable drains to EAGAIN, one edge suffices, and POLL_ADD reports current readiness at submit time (no lost wakeup). Multishot is deferred — on a pooled/persistent fd it outlives the request and needs explicit cancel-on-release (an ABA hazard).

4.4 Parking — where io_uring is actually simpler than epoll. Add Connection.awaiting_fd int = -1. When has_async, the read path calls a park-aware async_iou_drain (twin of async_drain): answer each complete request via the AsyncHandler, append to response_buffer, stop at the first .suspend, set conn.awaiting_fd. Then: if parked → return doing nothing else (no flush, no recv re-arm) — the connection now has zero client-side ops in flight. It stays alive because nothing calls iou_release (so owner != nil, slab pointer valid) and the reactor holds entry.conn; the timeout sweep skips it (both deadlines are 0, and iou_sweep_timeouts:504-522 only shuts down slots with a >0 deadline).

The elegant part: because a parked conn has no armed op on the client fd, a client hangup produces no CQE — so io_uring doesn't need epoll's entire tombstone machinery (ParkSlot.dead/reactor_mark_dead/reactor_orphan_single) for v1 correctness. The DB reply (query already submitted) resumes the continuation, renders, iou_flush_response → the send CQE fails on the dead peer → handle_io_uring_write:437-444 releases the slot normally. And the slot-reuse UAF that the sync path guards against (why iou_sweep_timeouts uses shutdown() not close()) cannot arise on the parked path — there's no pending CQE whose slot a fresh accept could reuse. This is why v1 arms zero ops on the client while parked.

4.5 Resume. Add an op_poll arm to dispatch_io_uring_cqe (:486-494; the current else {} silently drops unknown ops). handle_io_uring_poll (twin of async_on_ready, :529-621, ~70 LOC): decode ext_fd, guard watches[ext_fd].active (stale→ignore), read cqe.res as the poll mask (not a byte count) for ready_err = res & (pollerr|pollhup), unpark (awaiting_fd = -1), build the AsyncCtx, run entry.cont. On .done → iou_flush_response; handle_io_uring_write's keep-alive re-arm must then re-drain any HTTP request pipelined behind the parked one still in read_buf (io_uring analogue of epoll's .done→async_serve re-read). On .suspend → register already re-armed a fresh oneshot poll; flush if the continuation streamed bytes. On .close → iou_release (never the pooled DB fd).

4.6 Cancellation / UAF safety. Oneshot poll = nothing to cancel on completion (just don't re-arm). Pooled DB fd = never closed on client release (the reactor never owns ext_fd, satisfying watch_persistent). Reactor-growth-safe (fd in user_data, not a pointer). Stale CQE guarded by the active flag.

4.7 Wiring (~90 LOC across existing files): lift the new_server gate (http_server.c.v:321, ~3 LOC — stateful_handler/make_state already allow io_uring since #93, so per-worker pool bring-up timing already matches epoll); pass async_handler through run_io_uring_backend→io_uring_worker_main→io_uring_worker_loop→dispatch_io_uring_cqe; branch handle_io_uring_read/_write to the park-aware drain when has_async.

Total ≈ 420–450 LOC, ~75% of it the reactor port, which has a line-by-line reference in async_linux.c.v. The HttpArena entry becomes byte-identical to the epoll entry except io_multiplexing: .io_uring.

5. Phased plan

• Phase 0 — go/no-go probe (½ day). The one gate that could flip the decision. A ~15-line standalone test: prepare_poll on a pipe/eventfd then a non-loopback PG socket; assert one op_poll CQE carrying the poll mask. Retires the three biggest unknowns at once: (a) does header-inline io_uring_prep_poll_add link via fn C. under -prod (else a 3-line #insert shim); (b) cqe.res-as-mask; (c) io_uring loopback-muting — if POLL_ADD is muted even off-loopback on the target box, Family A's mechanism is undermined and Option B becomes the fallback only then.
• Phase 1 — bindings (½ day): op_poll, poll consts, prepare_poll, Connection.awaiting_fd. Compile clean.
• Phase 2 — single-watch async, no pipelining (~2 days): IouReactor + iou_async_register + async_iou_drain + handle_io_uring_poll (single-watch branch) + wiring + the gate. Run with a pool_size large enough that acquire() is one-client-per-conn. This alone eliminates io_uring: blocking libpq in the single-threaded ring worker collapses DB profiles (defer to the stateful-async migration) #83's HOL collapse (the ring keeps serving plaintext/json/single-query while DB queries are outstanding).
• Phase 3 — pipelining (~1–2 days): port drain_pipelined + the reactor_watch queue-promotion + ParkSlot.dead, and acquire_pipelined. Required for parity on multiplexed profiles (fortunes/crud/api-16); without it Phase 2 recovers from collapse but caps at pool_size concurrency.
• Phase 4 — hardening (~2 days): byte-diff vs epoll; helgrind at low worker count; -gc none leak watch; parked-conn timeout follow-up.

Effort ≈ 6–8 working days for someone fluent in the codebase.

6. Ship-blockers to fold in (not defer)

1. SQ-full at park (A3). RegisterFn returns void (core.v:88) — the handler already async_submit'd and returns .suspend unconditionally. If the poll SQE can't be queued (SQ full), a naive last_watched = -1 makes the drain treat the conn as not parked → arms recv, DB reply never drained, pooled conn stuck, client hangs. Must queue the poll on a per-worker pending list and retry at loop top (don't clear), or hard-close the client and release the DB conn.
2. .done pipelined-behind re-drain (A4) — the sharpest byte-divergence risk vs epoll; gate with a /db /db /db single-connection pipeline byte-diff.
3. Parked-conn timeout (A1). A never-returning query on a vanished client pins the slot + pooled DB conn indefinitely (same known gap as epoll, async_linux.c.v:29). Add a parked-deadline sweep (shutdown the client fd past a deadline, reap on the resume send-fail) plus a PG-side statement_timeout.

7. Verification (immune to io_uring's benchmark noise)

• Pure-array unit tests for the reactor functions (no kernel).
• Byte-diff vs the epoll twin (dual-container harness, bridge net): same handler on .epoll and .io_uring, identical /db + pipelined /db /db /db on one connection → assert byte-identical bodies. This proves park/resume/pipelined-drain without trusting wrk.
• HOL-blocking acceptance test (io_uring: blocking libpq in the single-threaded ring worker collapses DB profiles (defer to the stateful-async migration) #83's criterion): N connections on a slow /db (select pg_sleep(0.2)) + one on a synchronous /health; assert /health p99 stays flat (single-digit ms) vs the collapsed pre-change baseline.
• Perf: report io_uring relative to the epoll twin on the same non-loopback harness (loopback mutes io_uring), success = within run-to-run noise of epoll (closing the gap), not beating it.
• helgrind at 1–2 workers (OOMs at 16): Family A adds no cross-thread state, so a finding = a real bug.

8. Payoff

All four DB profiles park on the pooled pg fd, so all four recover. Phase 2 alone closes the HOL collapse (→ ~epoll parity for single-query async-db with a modest pool). fortunes/crud/api are multiplexed and need Phase 3 for full throughput parity; at Phase 2 they recover from collapse but cap at pool_size concurrency.

9. Risks (ranked)

1. io_uring POLL_ADD reliability / loopback muting on this box (highest — Phase 0 go/no-go).
2. Completion-model read/write glue (draining requests pipelined behind a parked one after resume) — mitigated by the pipelined byte-diff.
3. Parked-conn slot leak on hung query + vanished client (v1 gap; follow-up sweep + statement_timeout).
4. IOURINGINLINE linkage / cqe.res semantics — retired in Phase 0.
5. Pool sizing / shed parity if Phase 3 is skipped (non-fatal).
6. SINGLE_ISSUER drift if a future refactor registers a watch off-thread — add a defensive assert.

---

Summary: the DB driver isn't the problem — pg_async is already non-blocking and the async runtime is already pluggable — so the fix is a per-worker POLL_ADD reactor that mirrors the epoll one and lets the io_uring entry reuse the epoll entry's async_handler verbatim. It stays single-threaded, lock-free, SINGLE_ISSUER-compliant, and is simpler than epoll on the disconnect path (no tombstone in v1). Gate it on a Phase-0 POLL_ADD probe; Phase 2 fixes the collapse, Phase 3 reaches parity. The thread-pool interim is more code and more risk for a moot advantage — skip it.

_{Study produced with a multi-agent code analysis (5 subsystem deep-dives + 2 design proposals + adversarial critique) over the io_uring backend, the core async contract, the epoll async reactor, and pg_async. File:line references are against vanilla main @6fb4244.}