Two vLLM serve failures that only occur outside datacenter assumptions, hit back-to-back launching nytopop/Qwen3-8B.w8a8 on a 16GB desktop GPU (Artix/KDE):
-
Pre-startup free-memory refusal: fixed gpu_memory_utilization 0.9 = 14.03GiB, but plasmashell/kwin/the app hold ~1.9GiB, so only 13.51GiB is free. vLLM refuses to start; no diagnosis pattern matched, so the card showed a raw traceback. Now diagnosed on all three surfaces (cookbook_helpers, routes-local shadow copy, cookbook-diagnosis.js) with one-click retry at 0.80 / context 8192.
-
flashinfer JIT-compiles its sampling kernel at startup and aborts the engine when nvcc is missing (assumes /usr/local/cuda; Arch-family uses /opt/cuda; most inference boxes have no toolkit). Sampling needs no compiler. The vLLM serve runner now probes /opt/cuda for CUDA_HOME and otherwise exports VLLM_USE_FLASHINFER_SAMPLER=0 with a log line. This fallback previously existed only inside the MiniMax-M3 command normalizer — generalized to every vLLM launch.
Develop commits: 66713b0e (free-VRAM diagnosis), cd3f182 (nvcc fallback). Known debt: the diagnosis table exists twice (helpers + shadowing routes-local copy) — dedup tracked separately. Upstream-candidate.
Two vLLM serve failures that only occur outside datacenter assumptions, hit back-to-back launching nytopop/Qwen3-8B.w8a8 on a 16GB desktop GPU (Artix/KDE):
Pre-startup free-memory refusal: fixed gpu_memory_utilization 0.9 = 14.03GiB, but plasmashell/kwin/the app hold ~1.9GiB, so only 13.51GiB is free. vLLM refuses to start; no diagnosis pattern matched, so the card showed a raw traceback. Now diagnosed on all three surfaces (cookbook_helpers, routes-local shadow copy, cookbook-diagnosis.js) with one-click retry at 0.80 / context 8192.
flashinfer JIT-compiles its sampling kernel at startup and aborts the engine when nvcc is missing (assumes /usr/local/cuda; Arch-family uses /opt/cuda; most inference boxes have no toolkit). Sampling needs no compiler. The vLLM serve runner now probes /opt/cuda for CUDA_HOME and otherwise exports VLLM_USE_FLASHINFER_SAMPLER=0 with a log line. This fallback previously existed only inside the MiniMax-M3 command normalizer — generalized to every vLLM launch.
Develop commits: 66713b0e (free-VRAM diagnosis), cd3f182 (nvcc fallback). Known debt: the diagnosis table exists twice (helpers + shadowing routes-local copy) — dedup tracked separately. Upstream-candidate.