Codify commodity-hardware constraint: auto-fallback XDP mode + ADR amendment

.docs/adr-002-af-xdp-implementation-strategy.md

@@ -36,6 +36,31 @@ the implementation approach for the remaining work.
 5. **Portability**: ARM64 must remain a first-class target alongside
    x86_64 — the release workflow ships both. Anything pinned to Intel
    intrinsics or x86-only XDP features is out.
+6. **Commodity hardware** (load-bearing for the rest of the project,
+   not just AF_XDP): `lb-node` must run on any x86_64 or ARM64 Linux
+   server with a stock kernel ≥ 5.10 and a stock NIC driver. No
+   SmartNIC requirement, no specific vendor, no kernel patches, no
+   custom firmware. This rules out:
+   * **HW-mode XDP offload** (`XDP_FLAGS_HW_MODE`): only Netronome /
+     Corigine and a few others. Out — implementation can opt in
+     manually if a deployment happens to have the right NIC, but the
+     default path must not assume it.
+   * **Driver-private fast paths** (e.g. Mellanox-only `XDP_FLAGS_DRV_MODE`
+     extensions): out — anything that needs `ETHTOOL_GFEATURES` flags
+     not present on every commodity NIC.
+   * **Userspace-driver bypass** (DPDK): explicitly rejected in
+     ADR-pre-existing context; needs huge pages, IOMMU bind, vendor
+     PMDs.
+
+   What this *does* mean operationally: AF_XDP runs in **native mode**
+   on drivers that support it (Intel ixgbe/i40e/ice, Mellanox CX-4+,
+   AWS ENA, virtio-net 5.13+, Broadcom bnxt 5.9+, ...) and falls back
+   to **generic / SKB mode** on everything else. SKB mode loses ~30–40%
+   of throughput but works on any Linux NIC since it runs in the
+   kernel's generic XDP hook instead of the driver. `deploy/xdp/load.sh`
+   defaults to `auto` mode (try native, fall back to skb), so an
+   operator running on hardware that doesn't support native XDP gets
+   a working deployment without having to know which mode to pick.
 
 ## Options considered
 

.docs/operations.md

@@ -370,15 +370,25 @@ sudo apt install clang libbpf-dev bpftool
 # Build the redirect program (deploy/xdp/redirect.bpf.o is gitignored)
 make -C deploy/xdp
 
-# Attach to the data interface in native XDP mode
-sudo deploy/xdp/load.sh eth0 native
+# Attach to the data interface. Default mode is `auto`: try native
+# (driver-attached, fastest), fall back to skb (generic kernel hook,
+# works on every NIC since 5.10). Pass `native` or `skb` explicitly
+# to force a mode.
+sudo deploy/xdp/load.sh eth0
 ```
 
 The script attaches the program, pins the XSKMAP at
-`/sys/fs/bpf/lb/xsks_map`, and prints next steps. `lb-node` opens the
-pinned map at startup and inserts its own XSK fd at index `queue_id`
-(per `[node].cpu_affinity.rewriters`); no manual `bpftool map update`
-is needed.
+`/sys/fs/bpf/lb/xsks_map`, and prints which mode it ended up using.
+`lb-node` opens the pinned map at startup and inserts its own XSK fd
+at index `queue_id` (per `[node].cpu_affinity.rewriters`); no manual
+`bpftool map update` is needed.
+
+The commodity-hardware path is **skb mode**: it works on any Linux
+kernel ≥ 5.10 regardless of NIC vendor or driver vintage, at the
+cost of ~30–40% throughput compared to native. Native mode is
+preferred when available (Intel ixgbe/i40e/ice, Mellanox CX-4+, AWS
+ENA, virtio-net 5.13+, Broadcom bnxt 5.9+); the auto path picks it
+when the driver supports it and silently falls back when not.
 
 To detach (e.g. before swapping to a different program):
 

README.md

@@ -12,7 +12,7 @@ A high-performance L4 packet-forwarding load balancer written in Rust, inspired
 - **GRE encapsulation** -- RFC 2784 tunneling to backends, supports IPv4/IPv6 inner packets, Direct Server Return (DSR)
 - **MTU-aware tunneling** -- automatic MSS clamping on TCP SYN/SYN-ACK (incremental checksum, RFC 1624), ICMP Fragmentation Needed generation for oversized non-TCP packets with DF set, rate-limited. Operator sets `network_mtu`; all tunnel parameters derived automatically
 - **IP fragment reassembly pinning** -- first fragment flows through the full Maglev pipeline and records its 3-tuple→backend choice in a per-thread Robin Hood table; non-first fragments (which have no L4 header) look up the 3-tuple and GRE-encap to the same backend. Bounded table size + TTL so a fragment flood can't grow memory unbounded
-- **Kernel bypass I/O** -- AF_XDP and DPDK backends for line-rate packet processing on commodity hardware
+- **Kernel bypass I/O** -- AF_XDP backend for line-rate packet processing on commodity hardware. Auto-fallback from native to generic/SKB XDP mode means it runs on any x86_64 / ARM64 Linux server with kernel ≥ 5.10, no SmartNIC or DPDK toolchain required (see [ADR-002](.docs/adr-002-af-xdp-implementation-strategy.md))
 - **Multi-threaded data plane** -- steering thread distributes to per-rewriter SPSC queues; muxer thread collects TX output. Zero-copy hot path: packets live in a shared `PacketPool` arena, only 4-byte frame indices flow through queues (AF_XDP UMEM model)
 - **Health checking** -- TCP, HTTP, and HTTPS probes with configurable thresholds and state machine (UNKNOWN/HEALTHY/UNHEALTHY)
 - **BGP speaker** -- minimal custom implementation (OPEN/UPDATE/KEEPALIVE/NOTIFICATION) with active-active announcement to multiple router peers. Each peer holds an independent session with exponential-backoff reconnect (1s→60s); VIPs fan out to every Established peer so loss of any single router does not withdraw the VIP from the others. Driven by the controller: a VIP is announced while at least one backend in its pool is healthy and withdrawn otherwise
@@ -220,6 +220,7 @@ cargo build --release
 - **[Design specification](.docs/lb-spec-v2.md)** -- full architecture and protocol details
 - **[MTU handling spec](.docs/mtu-handling-spec.md)** -- MSS clamping, ICMP Frag Needed, MTU configuration
 - **[ADR-001: Configuration model](.docs/adr-001-configuration-model.md)** -- rationale for file-based configuration
+- **[ADR-002: AF_XDP implementation strategy](.docs/adr-002-af-xdp-implementation-strategy.md)** -- xdpilone, frame-pool design, **commodity-hardware constraint** (no SmartNICs / DPDK / kernel patches required)
 
 ## Project structure
 

deploy/xdp/load.sh

@@ -10,11 +10,24 @@
 #   sudo ./load.sh <iface> [mode]
 #
 #   <iface>  Data interface (matches `[node].data_iface` in lb-node.toml).
-#   [mode]   xdp attach mode: `native` (default), `skb`, or `hw`.
-#            Use `native` on supported NICs (Intel ixgbe/i40e/ice,
-#            Mellanox CX-4+, AWS ENA). `skb` is the kernel fallback —
-#            slower but works everywhere. `hw` requires NIC offload
-#            support (most don't).
+#   [mode]   xdp attach mode:
+#            `auto`   (default) — try `native` first, fall back to `skb`
+#                     if the driver doesn't support it. This honours the
+#                     project's commodity-hardware constraint: no NIC is
+#                     too old to run lb-node. Throughput on `skb` is
+#                     ~30–40% lower than `native`, but every kernel
+#                     ≥ 5.10 supports it.
+#            `native` — driver-attached XDP (Intel ixgbe/i40e/ice,
+#                     Mellanox CX-4+, AWS ENA, virtio-net 5.13+, …).
+#                     Higher throughput; fails fast if the driver
+#                     doesn't support it.
+#            `skb`    — generic kernel-side XDP. Works everywhere,
+#                     including veth/tun/loopback for tests. Slower.
+#            `hw`     — full NIC offload. Requires SmartNIC support
+#                     (Netronome/Corigine and a few others). Out of
+#                     scope for the commodity-hardware deployment
+#                     target — pass explicitly only if you know your
+#                     NIC supports it.
 #
 # Effect:
 #   * Compiles redirect.bpf.o if missing (`make`).
@@ -28,16 +41,16 @@
 set -euo pipefail
 
 IFACE="${1:-}"
-MODE="${2:-native}"
+MODE="${2:-auto}"
 
 if [[ -z "$IFACE" ]]; then
-    echo "usage: $0 <iface> [native|skb|hw]" >&2
+    echo "usage: $0 <iface> [auto|native|skb|hw]" >&2
     exit 2
 fi
 
 case "$MODE" in
-    native|skb|hw) ;;
-    *) echo "mode must be one of: native, skb, hw" >&2; exit 2 ;;
+    auto|native|skb|hw) ;;
+    *) echo "mode must be one of: auto, native, skb, hw" >&2; exit 2 ;;
 esac
 
 cd "$(dirname "$0")"
@@ -59,19 +72,51 @@ fi
 PIN_DIR="/sys/fs/bpf/lb"
 mkdir -p "$PIN_DIR"
 
-# Detach any prior program — re-running load.sh shouldn't fail because
-# something is already attached.
-ip link set dev "$IFACE" xdp${MODE} off 2>/dev/null || true
+# Detach any prior program in any mode — re-running load.sh shouldn't
+# fail because something is already attached.
+for m in native skb hw; do
+    ip link set dev "$IFACE" xdp${m} off 2>/dev/null || true
+done
 
 # Load the program with bpftool. `pinmaps` parks every map (here just
 # `xsks_map`) under PIN_DIR.
 bpftool prog loadall redirect.bpf.o "$PIN_DIR/prog" \
     type xdp pinmaps "$PIN_DIR"
 
-# Attach the program to the interface.
-ip link set dev "$IFACE" xdp${MODE} pinned "$PIN_DIR/prog/lb_redirect"
+# Attach. In `auto` mode, try native first and fall back to skb when the
+# driver doesn't support it — this is the commodity-hardware path: every
+# kernel ≥ 5.10 supports skb mode regardless of NIC vendor or vintage.
+attach() {
+    local m="$1"
+    ip link set dev "$IFACE" xdp${m} pinned "$PIN_DIR/prog/lb_redirect" 2>&1
+}
 
-echo "✓ XDP redirect program attached to $IFACE ($MODE mode)"
+ATTACHED=""
+case "$MODE" in
+    auto)
+        if attach native >/dev/null 2>&1; then
+            ATTACHED="native"
+        else
+            echo "  driver does not support native XDP on $IFACE, falling back to skb"
+            if attach skb >/dev/null 2>&1; then
+                ATTACHED="skb"
+            else
+                echo "FAIL: could not attach in either native or skb mode" >&2
+                attach skb >&2
+                exit 1
+            fi
+        fi
+        ;;
+    *)
+        if ! attach "$MODE" >&2; then
+            echo "FAIL: could not attach in $MODE mode" >&2
+            exit 1
+        fi
+        ATTACHED="$MODE"
+        ;;
+esac
+
+echo "✓ XDP redirect program attached to $IFACE ($ATTACHED mode)"
 echo "  XSKMAP pinned at $PIN_DIR/xsks_map"
 echo
 echo "Next: start lb-node. To populate xsks_map manually for queue 0:"

deploy/xdp/smoke-test.sh

@@ -77,7 +77,10 @@ ip netns exec "$NS" ip link set "$VETH_NS" up
 sleep 0.5
 
 echo "== Attaching XDP program to $VETH_HOST =="
-"$(dirname "$0")"/load.sh "$VETH_HOST" skb   # veth supports SKB mode
+# `auto` would also work — veth doesn't support native mode, so it'd
+# fall back to skb. Pass `skb` explicitly to make the mode the test
+# exercises obvious in the output.
+"$(dirname "$0")"/load.sh "$VETH_HOST" skb
 
 echo "== Starting lb-node =="
 cat > /tmp/lb-smoke-config.toml <<EOF