Add SGLang Qwen3-8B baseline benchmark harness

.github/workflows/ut.yml

@@ -51,7 +51,7 @@ jobs:
 
       - name: Build
         run: |
-          apt-get update && apt-get install -y lcov
+          apt-get update && apt-get install -y --no-install-recommends lcov
           mkdir build && cd build
           CMAKE_ARGS="-DCMAKE_BUILD_TYPE=Debug"
           if [ "${{ matrix.platform }}" = "rocm" ]; then

examples/qwen3/bench/BASELINE.md

@@ -0,0 +1,85 @@
+# SGLang Qwen3-8B Baseline Latency
+
+> **Note:** Values marked "TBD" will be filled after running benchmarks on the target hardware.
+
+Model: `Qwen/Qwen3-8B` (~16 GB fp16)
+
+## Image
+
+| Field          | Value |
+|----------------|-------|
+| Image tag      | `lmsysorg/sglang:v0.4.6.post1-cu124` |
+| Image digest   | TBD (run on 8×A100-80GB) |
+| CUDA version   | 12.4 |
+| SGLang version | 0.4.6.post1 |
+
+## Hardware
+
+| Field             | Value |
+|-------------------|-------|
+| Node              | 8×A100-80GB |
+| GPUs              | 8× NVIDIA A100-SXM4-80GB |
+| GPU clocks (gr)   | TBD (run on 8×A100-80GB) |
+| GPU clocks (mem)  | TBD (run on 8×A100-80GB) |
+| Interconnect      | NVLink + NVSwitch |
+
+## TP=1 — SGLang natural best-latency config
+
+Single-GPU inference. For an 8B model (~16 GB fp16), TP=1 is the
+lowest-overhead configuration and represents SGLang's best achievable
+latency for this model size.
+
+### Server flags
+
+```
+python -m sglang.launch_server \
+    --model Qwen/Qwen3-8B \
+    --tp 1 \
+    --port 30000 \
+    --mem-fraction-static 0.85 \
+    --trust-remote-code
+```
+
+### Results
+
+| Metric                          | Value |
+|---------------------------------|-------|
+| Prefill TTFT (prompt=2048, gen=1) | TBD (run on 8×A100-80GB) |
+| Decode per-token (prompt=2048, gen=128) | TBD (run on 8×A100-80GB) |
+| Trials                          | 5 |
+
+## TP=8 — Matched-regime comparison
+
+TP=8 for an 8B model is the ARK-favorable regime: it minimizes per-GPU
+compute and amplifies kernel-launch and communication overhead
+(hypothesis 1). This matches the parallelism used by the ARK target
+but is **not** SGLang's natural best config for this model size.
+
+### Server flags
+
+```
+python -m sglang.launch_server \
+    --model Qwen/Qwen3-8B \
+    --tp 8 \
+    --port 30000 \
+    --mem-fraction-static 0.85 \
+    --trust-remote-code
+```
+
+### Results
+
+| Metric                          | Value |
+|---------------------------------|-------|
+| Prefill TTFT (prompt=2048, gen=1) | TBD (run on 8×A100-80GB) |
+| Decode per-token (prompt=2048, gen=128) | TBD (run on 8×A100-80GB) |
+| Trials                          | 5 |
+
+## Methodology
+
+- Endpoint: `/generate` with `ignore_eos: true` (not `/v1/chat/completions`).
+- Prompt: ~2048 tokens (repeating pattern).
+- Warmup: 3 requests discarded before measurement.
+- TTFT: time for prompt=2048, max_new_tokens=1 (single request, not streaming).
+- Decode per-token: total_time / max(output_tokens, 1) for prompt=2048, max_new_tokens=128 (end-to-end; includes prefill, which is negligible for 128 output tokens).
+- Reported value: median over 5 trials.
+- Temperature: 0.0 (greedy).

examples/qwen3/bench/launch_sglang.sh

@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# Launch SGLang server with Qwen3-8B on A100 GPUs.
+#
+# Usage:
+#   ./launch_sglang.sh          # TP=8 (default)
+#   ./launch_sglang.sh 1        # TP=1
+#   ./launch_sglang.sh 8        # TP=8
+#
+# Prerequisites:
+#   - Docker with NVIDIA runtime
+#   - 8×A100-80GB (for TP=8) or 1×A100-80GB (for TP=1)
+#   - HuggingFace cache at $HF_HOME (default: ~/.cache/huggingface)
+
+set -euo pipefail
+
+# --- Configuration -----------------------------------------------------------
+
+TP="${1:-8}"
+MODEL="Qwen/Qwen3-8B"
+PORT=30000
+CONTAINER_NAME="sglang-qwen3-bench"
+
+# Pinned SGLang image — CUDA 12.x, A100-compatible.
+# To pin by digest: docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE_TAG"
+IMAGE_TAG="lmsysorg/sglang:v0.4.6.post1-cu124"
+
+HF_HOME="${HF_HOME:-${HOME}/.cache/huggingface}"
+
+# --- Validate -----------------------------------------------------------------
+
+if [[ "${TP}" != "1" && "${TP}" != "8" ]]; then
+    echo "Error: TP must be 1 or 8, got '${TP}'" >&2
+    exit 1
+fi
+
+# --- Pull image ---------------------------------------------------------------
+
+echo "Pulling ${IMAGE_TAG} ..."
+docker pull "${IMAGE_TAG}"
+
+# --- Stop any existing container ----------------------------------------------
+
+if docker ps -a --format '{{.Names}}' | grep -qx "${CONTAINER_NAME}"; then
+    echo "Stopping existing container '${CONTAINER_NAME}' ..."
+    docker rm -f "${CONTAINER_NAME}" >/dev/null 2>&1
+fi
+
+# --- Launch server ------------------------------------------------------------
+
+echo "Launching SGLang server: model=${MODEL}, tp=${TP}, port=${PORT}"
+docker run -d \
+    --name "${CONTAINER_NAME}" \
+    --gpus all \
+    --ipc=host \
+    --network=host \
+    -v "${HF_HOME}:/root/.cache/huggingface" \
+    -e HF_HOME=/root/.cache/huggingface \
+    "${IMAGE_TAG}" \
+    python -m sglang.launch_server \
+        --model "${MODEL}" \
+        --tp "${TP}" \
+        --port "${PORT}" \
+        --mem-fraction-static 0.85 \
+        --trust-remote-code
+
+echo ""
+echo "Container '${CONTAINER_NAME}' started."
+echo "Waiting for server to be ready on port ${PORT} ..."
+
+# --- Wait for server readiness ------------------------------------------------
+
+MAX_WAIT=300
+ELAPSED=0
+while ! curl -sf "http://localhost:${PORT}/health" >/dev/null 2>&1; do
+    sleep 5
+    ELAPSED=$((ELAPSED + 5))
+    if [ "${ELAPSED}" -ge "${MAX_WAIT}" ]; then
+        echo "Error: server did not become ready within ${MAX_WAIT}s." >&2
+        echo "Check logs: docker logs ${CONTAINER_NAME}" >&2
+        exit 1
+    fi
+done
+
+echo "Server is ready (waited ${ELAPSED}s)."
+echo ""
+echo "Run the benchmark:"
+echo "  python measure_baseline.py --port ${PORT}"