Skip to content

Add ioxide.Kestrel: ASP.NET Core Kestrel transport on the io_uring reactor#84

Merged
MDA2AV merged 5 commits into
mainfrom
feat/kestrel-transport
Jun 21, 2026
Merged

Add ioxide.Kestrel: ASP.NET Core Kestrel transport on the io_uring reactor#84
MDA2AV merged 5 commits into
mainfrom
feat/kestrel-transport

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Jun 21, 2026

Copy link
Copy Markdown
Owner

Summary

Adds ioxide.Kestrel — an ASP.NET Core Kestrel transport backed by ioxide. Each connection runs on an ioxide reactor, and Kestrel's whole request loop (recv → parse → handler → send) is pinned to the reactor thread — no ThreadPool hop on the hot path. Drop-in via builder.WebHost.UseIoxide().

Engine impact: minimal and opt-in

Only 3 additive touch points in the engine, with no behavioral change to the native inline path:

  • Reactor/Reactor.Post.cs (new): ScheduleOnReactor(Action<object?>, object?) + OnReactorThread, built on the existing _remoteOps/WakeFdWrite machinery — a queue drained once per loop iteration.
  • Reactor.cs and Reactor.Incremental.cs: one DrainPostQ(); per loop.

When nothing uses the scheduler (the native runtime, GenHTTP engine, pg/redis/file clients), DrainPostQ is a single TryDequeue on an always-empty queue — effectively free, and inline execution is byte-for-byte unchanged. If you don't reference ioxide.Kestrel, the engine behaves exactly as before.

How the transport works

HopDuplexPipe hands Kestrel two System.IO.Pipelines.Pipes whose reader schedulers are a ReactorPipeScheduler (over Reactor.ScheduleOnReactor). A recv pump copies received bytes into the inbound pipe and a send pump drains the response into the connection's send slab — both on the reactor. Routing the pipe reader continuations to the reactor is what keeps Kestrel's loop there (it's the one Kestrel-sanctioned hook for loop placement).

Usage

var builder = WebApplication.CreateBuilder(args);
builder.WebHost.UseIoxide();                 // replaces the default sockets transport

var app = builder.Build();
app.MapGet("/", () => "Hello, World!");
app.Run();

Options: UseIoxide(o => { o.ReactorCount = …; o.ConfigureServer = cfg => cfg with { … }; }).

Benchmarks (single box, plaintext)

i9-14900K, net11, server pinned to 4 P-cores / 8 reactors, wrk on 16 E-cores, median of 3:

conns sockets transport ioxide.Kestrel
64 914K rps (p99 2.65 ms) 1.36M rps (p99 0.52 ms)
256 915K rps 1.45M rps
1024 916K rps 1.37M rps

~1.5× the stock sockets transport, with ~100% of request processing on the reactor thread (verified by thread sampling), 0 errors. Caveats: single-box (the load generator shares the machine), small messages, Linux/io_uring only.

Notes

  • New package targets net11.0, references Microsoft.AspNetCore.App, added to ioxide.slnx.
  • Public surface is just UseIoxide() + IoxideTransportOptions; everything else is internal.
  • Thread-per-core caveat: middleware runs on the reactor thread, so blocking work in a handler stalls every connection on that reactor — keep handlers async.

MDA2AV added 5 commits June 21, 2026 17:12
…actor

Adds a Kestrel transport (UseIoxide()) that runs each connection on an ioxide reactor, with the whole HTTP request loop (recv -> parse -> handler -> send) pinned to the reactor thread.

Core engine (additive, opt-in — 3 touch points):
- New Reactor.ScheduleOnReactor(): run a continuation on the reactor loop thread,
  reusing the existing eventfd wake-queue (drained once per loop iteration via
  DrainPostQ). Native inline execution is unchanged: when nothing schedules, the
  drain is a no-op on an empty queue.

New ioxide.Kestrel package:
- HopDuplexPipe: two System.IO.Pipelines pipes whose reader schedulers route to
  the reactor (ReactorPipeScheduler), plus recv->inbound and outbound->send pumps
  that run on the reactor.
- IConnectionListenerFactory/Listener/Context bridging ioxide's push Handle model
  to Kestrel's pull AcceptAsync model. Public surface: UseIoxide() + options.

Plaintext (i9-14900K, 8 reactors on 4 P-cores, net11): ~1.4M rps, ~1.5x the stock
sockets transport, with ~100% of request processing on the reactor thread.
Minimal ASP.NET Core app with GET / and /plaintext endpoints. Selects the transport via the TRANSPORT env var: 'ioxide' (default, via UseIoxide()) or 'sockets' (stock Kestrel sockets). Added to ioxide.slnx.
@MDA2AV MDA2AV merged commit c900769 into main Jun 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant