Skip to content

Research: bounded pipe-pool IORING_OP_SPLICE for disk-backed large static on io_uring #77

Description

@enghitalo

Context. io_uring has no native sendfile opcode (through 6.16); the zero-copy file→socket idiom is IORING_OP_SPLICE file→pipe→socket. A C PoC in this campaign showed it works but cliffs at high connection counts: a per-connection 64 KiB pipe hits pipe-user-pages-soft (default 16384 pages = 64 MiB/user) at ~1024 conns → the kernel clamps new pipes to 1 page → p99 explodes (144→220 ms) and throughput drops below epoll's flat line.

Note on priority. vanilla-io_uring currently serves static from RAM (preloaded full responses), so it does NOT need splice today — see the queue_buf borrowed-send work (#75). This issue is only relevant if a disk-backed large static path (files too big to preload) is ever wanted on io_uring.

What to research. A bounded pool of a few large pipes cycled across transfers (instead of one pipe per connection) to dodge the per-user pipe-page cap, plus whether raising pipe-user-pages-soft is acceptable ops-wise. Compare against keeping disk-backed static on an epoll worker.

PoC harness lives in the campaign scratchpad (zerocopy-poc/: epoll-sendfile vs io_uring-splice vs send/send_zc + bench.sh).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions