diff --git a/.docs/adr-002-af-xdp-implementation-strategy.md b/.docs/adr-002-af-xdp-implementation-strategy.md index 90cd8fa..225bb08 100644 --- a/.docs/adr-002-af-xdp-implementation-strategy.md +++ b/.docs/adr-002-af-xdp-implementation-strategy.md @@ -36,6 +36,31 @@ the implementation approach for the remaining work. 5. **Portability**: ARM64 must remain a first-class target alongside x86_64 — the release workflow ships both. Anything pinned to Intel intrinsics or x86-only XDP features is out. +6. **Commodity hardware** (load-bearing for the rest of the project, + not just AF_XDP): `lb-node` must run on any x86_64 or ARM64 Linux + server with a stock kernel ≥ 5.10 and a stock NIC driver. No + SmartNIC requirement, no specific vendor, no kernel patches, no + custom firmware. This rules out: + * **HW-mode XDP offload** (`XDP_FLAGS_HW_MODE`): only Netronome / + Corigine and a few others. Out — implementation can opt in + manually if a deployment happens to have the right NIC, but the + default path must not assume it. + * **Driver-private fast paths** (e.g. Mellanox-only `XDP_FLAGS_DRV_MODE` + extensions): out — anything that needs `ETHTOOL_GFEATURES` flags + not present on every commodity NIC. + * **Userspace-driver bypass** (DPDK): explicitly rejected in + ADR-pre-existing context; needs huge pages, IOMMU bind, vendor + PMDs. + + What this *does* mean operationally: AF_XDP runs in **native mode** + on drivers that support it (Intel ixgbe/i40e/ice, Mellanox CX-4+, + AWS ENA, virtio-net 5.13+, Broadcom bnxt 5.9+, ...) and falls back + to **generic / SKB mode** on everything else. SKB mode loses ~30–40% + of throughput but works on any Linux NIC since it runs in the + kernel's generic XDP hook instead of the driver. `deploy/xdp/load.sh` + defaults to `auto` mode (try native, fall back to skb), so an + operator running on hardware that doesn't support native XDP gets + a working deployment without having to know which mode to pick. ## Options considered diff --git a/.docs/operations.md b/.docs/operations.md index eb70523..bd6ad59 100644 --- a/.docs/operations.md +++ b/.docs/operations.md @@ -370,15 +370,25 @@ sudo apt install clang libbpf-dev bpftool # Build the redirect program (deploy/xdp/redirect.bpf.o is gitignored) make -C deploy/xdp -# Attach to the data interface in native XDP mode -sudo deploy/xdp/load.sh eth0 native +# Attach to the data interface. Default mode is `auto`: try native +# (driver-attached, fastest), fall back to skb (generic kernel hook, +# works on every NIC since 5.10). Pass `native` or `skb` explicitly +# to force a mode. +sudo deploy/xdp/load.sh eth0 ``` The script attaches the program, pins the XSKMAP at -`/sys/fs/bpf/lb/xsks_map`, and prints next steps. `lb-node` opens the -pinned map at startup and inserts its own XSK fd at index `queue_id` -(per `[node].cpu_affinity.rewriters`); no manual `bpftool map update` -is needed. +`/sys/fs/bpf/lb/xsks_map`, and prints which mode it ended up using. +`lb-node` opens the pinned map at startup and inserts its own XSK fd +at index `queue_id` (per `[node].cpu_affinity.rewriters`); no manual +`bpftool map update` is needed. + +The commodity-hardware path is **skb mode**: it works on any Linux +kernel ≥ 5.10 regardless of NIC vendor or driver vintage, at the +cost of ~30–40% throughput compared to native. Native mode is +preferred when available (Intel ixgbe/i40e/ice, Mellanox CX-4+, AWS +ENA, virtio-net 5.13+, Broadcom bnxt 5.9+); the auto path picks it +when the driver supports it and silently falls back when not. To detach (e.g. before swapping to a different program): diff --git a/README.md b/README.md index 7f27c82..efe8e75 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ A high-performance L4 packet-forwarding load balancer written in Rust, inspired - **GRE encapsulation** -- RFC 2784 tunneling to backends, supports IPv4/IPv6 inner packets, Direct Server Return (DSR) - **MTU-aware tunneling** -- automatic MSS clamping on TCP SYN/SYN-ACK (incremental checksum, RFC 1624), ICMP Fragmentation Needed generation for oversized non-TCP packets with DF set, rate-limited. Operator sets `network_mtu`; all tunnel parameters derived automatically - **IP fragment reassembly pinning** -- first fragment flows through the full Maglev pipeline and records its 3-tuple→backend choice in a per-thread Robin Hood table; non-first fragments (which have no L4 header) look up the 3-tuple and GRE-encap to the same backend. Bounded table size + TTL so a fragment flood can't grow memory unbounded -- **Kernel bypass I/O** -- AF_XDP and DPDK backends for line-rate packet processing on commodity hardware +- **Kernel bypass I/O** -- AF_XDP backend for line-rate packet processing on commodity hardware. Auto-fallback from native to generic/SKB XDP mode means it runs on any x86_64 / ARM64 Linux server with kernel ≥ 5.10, no SmartNIC or DPDK toolchain required (see [ADR-002](.docs/adr-002-af-xdp-implementation-strategy.md)) - **Multi-threaded data plane** -- steering thread distributes to per-rewriter SPSC queues; muxer thread collects TX output. Zero-copy hot path: packets live in a shared `PacketPool` arena, only 4-byte frame indices flow through queues (AF_XDP UMEM model) - **Health checking** -- TCP, HTTP, and HTTPS probes with configurable thresholds and state machine (UNKNOWN/HEALTHY/UNHEALTHY) - **BGP speaker** -- minimal custom implementation (OPEN/UPDATE/KEEPALIVE/NOTIFICATION) with active-active announcement to multiple router peers. Each peer holds an independent session with exponential-backoff reconnect (1s→60s); VIPs fan out to every Established peer so loss of any single router does not withdraw the VIP from the others. Driven by the controller: a VIP is announced while at least one backend in its pool is healthy and withdrawn otherwise @@ -220,6 +220,7 @@ cargo build --release - **[Design specification](.docs/lb-spec-v2.md)** -- full architecture and protocol details - **[MTU handling spec](.docs/mtu-handling-spec.md)** -- MSS clamping, ICMP Frag Needed, MTU configuration - **[ADR-001: Configuration model](.docs/adr-001-configuration-model.md)** -- rationale for file-based configuration +- **[ADR-002: AF_XDP implementation strategy](.docs/adr-002-af-xdp-implementation-strategy.md)** -- xdpilone, frame-pool design, **commodity-hardware constraint** (no SmartNICs / DPDK / kernel patches required) ## Project structure diff --git a/deploy/xdp/load.sh b/deploy/xdp/load.sh index 8647d23..79d5e7f 100755 --- a/deploy/xdp/load.sh +++ b/deploy/xdp/load.sh @@ -10,11 +10,24 @@ # sudo ./load.sh [mode] # # Data interface (matches `[node].data_iface` in lb-node.toml). -# [mode] xdp attach mode: `native` (default), `skb`, or `hw`. -# Use `native` on supported NICs (Intel ixgbe/i40e/ice, -# Mellanox CX-4+, AWS ENA). `skb` is the kernel fallback — -# slower but works everywhere. `hw` requires NIC offload -# support (most don't). +# [mode] xdp attach mode: +# `auto` (default) — try `native` first, fall back to `skb` +# if the driver doesn't support it. This honours the +# project's commodity-hardware constraint: no NIC is +# too old to run lb-node. Throughput on `skb` is +# ~30–40% lower than `native`, but every kernel +# ≥ 5.10 supports it. +# `native` — driver-attached XDP (Intel ixgbe/i40e/ice, +# Mellanox CX-4+, AWS ENA, virtio-net 5.13+, …). +# Higher throughput; fails fast if the driver +# doesn't support it. +# `skb` — generic kernel-side XDP. Works everywhere, +# including veth/tun/loopback for tests. Slower. +# `hw` — full NIC offload. Requires SmartNIC support +# (Netronome/Corigine and a few others). Out of +# scope for the commodity-hardware deployment +# target — pass explicitly only if you know your +# NIC supports it. # # Effect: # * Compiles redirect.bpf.o if missing (`make`). @@ -28,16 +41,16 @@ set -euo pipefail IFACE="${1:-}" -MODE="${2:-native}" +MODE="${2:-auto}" if [[ -z "$IFACE" ]]; then - echo "usage: $0 [native|skb|hw]" >&2 + echo "usage: $0 [auto|native|skb|hw]" >&2 exit 2 fi case "$MODE" in - native|skb|hw) ;; - *) echo "mode must be one of: native, skb, hw" >&2; exit 2 ;; + auto|native|skb|hw) ;; + *) echo "mode must be one of: auto, native, skb, hw" >&2; exit 2 ;; esac cd "$(dirname "$0")" @@ -59,19 +72,51 @@ fi PIN_DIR="/sys/fs/bpf/lb" mkdir -p "$PIN_DIR" -# Detach any prior program — re-running load.sh shouldn't fail because -# something is already attached. -ip link set dev "$IFACE" xdp${MODE} off 2>/dev/null || true +# Detach any prior program in any mode — re-running load.sh shouldn't +# fail because something is already attached. +for m in native skb hw; do + ip link set dev "$IFACE" xdp${m} off 2>/dev/null || true +done # Load the program with bpftool. `pinmaps` parks every map (here just # `xsks_map`) under PIN_DIR. bpftool prog loadall redirect.bpf.o "$PIN_DIR/prog" \ type xdp pinmaps "$PIN_DIR" -# Attach the program to the interface. -ip link set dev "$IFACE" xdp${MODE} pinned "$PIN_DIR/prog/lb_redirect" +# Attach. In `auto` mode, try native first and fall back to skb when the +# driver doesn't support it — this is the commodity-hardware path: every +# kernel ≥ 5.10 supports skb mode regardless of NIC vendor or vintage. +attach() { + local m="$1" + ip link set dev "$IFACE" xdp${m} pinned "$PIN_DIR/prog/lb_redirect" 2>&1 +} -echo "✓ XDP redirect program attached to $IFACE ($MODE mode)" +ATTACHED="" +case "$MODE" in + auto) + if attach native >/dev/null 2>&1; then + ATTACHED="native" + else + echo " driver does not support native XDP on $IFACE, falling back to skb" + if attach skb >/dev/null 2>&1; then + ATTACHED="skb" + else + echo "FAIL: could not attach in either native or skb mode" >&2 + attach skb >&2 + exit 1 + fi + fi + ;; + *) + if ! attach "$MODE" >&2; then + echo "FAIL: could not attach in $MODE mode" >&2 + exit 1 + fi + ATTACHED="$MODE" + ;; +esac + +echo "✓ XDP redirect program attached to $IFACE ($ATTACHED mode)" echo " XSKMAP pinned at $PIN_DIR/xsks_map" echo echo "Next: start lb-node. To populate xsks_map manually for queue 0:" diff --git a/deploy/xdp/smoke-test.sh b/deploy/xdp/smoke-test.sh index d656477..89f9836 100755 --- a/deploy/xdp/smoke-test.sh +++ b/deploy/xdp/smoke-test.sh @@ -77,7 +77,10 @@ ip netns exec "$NS" ip link set "$VETH_NS" up sleep 0.5 echo "== Attaching XDP program to $VETH_HOST ==" -"$(dirname "$0")"/load.sh "$VETH_HOST" skb # veth supports SKB mode +# `auto` would also work — veth doesn't support native mode, so it'd +# fall back to skb. Pass `skb` explicitly to make the mode the test +# exercises obvious in the output. +"$(dirname "$0")"/load.sh "$VETH_HOST" skb echo "== Starting lb-node ==" cat > /tmp/lb-smoke-config.toml <