perf(cpu): isolate Raspberry Pi 5 polling contention

.agents/NOW.md

@@ -22,7 +22,7 @@ Work: 7 external PRs landed (6 merged + #227 repaired).
 | Kimi-Linear-48B | 122/128 held; e2e NOT ESTABLISHED | tiktoken-only ckpt: no warm server |
 | 35B mid-band | **canonical 0.918-0.972x** c1-c32 (@`348c265d`, first c16/c32) | Decode-only window, ONE tool |
 | Qwen3.5-4B sm_120 | tput **1.0283x** `PENDING`; TTFT/TPOT/E2E 1.085/1.017/1.029x | Profile wave |
-| RPi5 A76 CPU | **R5 asm GREEN; llama NOT MET**: 0.461x pf, 0.653x dec | W6: BF16 GEMM |
+| RPi5 A76 | **#293 C1 GATED**: T4 decode 2.149x; T3 neutral | C2 event; F16 profile |
 | SERVE-ASYNC-MIRROR | **#323 FIXED** (mitigation): graph declines while the mirror is live; 7/7 async gates | Graph read ids at REPLAY |
 | CPU levers (`QUANT-GGUF-CIQ-GEMM`) | Profile DONE: decode **47% threadpool sync**, prefill **~39% paged attn** | Parakeet encoder; attn dtype hoist |
 | `SERVE-METRICS` async (#277) | **`/metrics` was DEAD on the shipped server**: AsyncLLM folded nothing. Now live, ctest 366/366 | Config-gated families |

.agents/coordination.md

@@ -1633,8 +1633,42 @@ and the config/CLI plumbing to select a mode from outside the library are both s
 Records a spec-after-code ordering deviation in the spec preamble rather than hiding it behind
 commit order.
 
+**Benchmark thread-budget control (#293 C1, 2026-08-10).** Codex
+(gpt-5.6-sol), fresh helper implementer, isolated worktree
+`/tmp/vllm-a76-c1-upstream`, branch
+`helper/SERVE-CLI-BENCH-293-C1-UPSTREAM`, base `9ec8cae23`; corrected W0/C0
+evidence `0a07c53db`. NOT in the claims table below: that table keys whole
+`SPIKE`/`ACTIVE` rows, while the broad `SERVE-CLI-BENCH` row must stay
+`PARTIAL`; the committed [#293 spec](specs/cpu-bench-thread-budget.md) tracks
+this leaf as `ACTIVE`, and the helper role marker provides the worktree lock.
+Scope: same-binary `poll` vs concurrency-1-only `blocking-c1`, RED mutation,
+native/QEMU gates, and interleaved Pi T4/T3 correctness/profile measurement.
+Owns `examples/bench/bench_core.h`, `examples/bench/main.cpp`,
+`tests/examples/test_bench.cpp`, and scoped #293/spec/evidence projections. C2
+AsyncLLM/output-event files are excluded even if C1 is decisively positive.
+Also excludes kernels, model/loader, CUDA/Vulkan, C ABI, assembly, Pi builds,
+service changes and remote Git operations.
+
+Implementer checkpoint `bdbfffcbf`: native 6/6 (47 assertions), scoped CTest
+5/5, dispatch mutation killed by the three intended assertions, AArch64 QEMU
+quant 23/23 (150,350 assertions), QEMU bench 6/6 (47 assertions), and A76
+assembly smoke green. The hash-verified same-binary Pi C1 is positive: T4
+blocking/poll is 2.193x decode and 0.458x E2E latency; T3 is neutral; all 12
+token files match. The pathological T4 poll spread is above 3%, so no binding
+floor moves. Raw evidence and exact hashes are indexed in
+[`rpi5-a76-output-wait-c1-20260810.md`](../docs/bench-evidence/rpi5-a76-output-wait-c1-20260810.md).
+After fresh review and rebase onto `upstream/main` `4ba051406`, immutable head
+`960647bf5` passed the operator's native and QEMU gates and a second
+hash-verified Pi series. T4 blocking/poll is 2.149x decode and 0.465x E2E
+latency; T3 remains neutral; exact tokens hold. T4 poll spread is 13.33-15.96%,
+so no binding floor moves. The
+[current-main evidence](../docs/bench-evidence/rpi5-a76-output-wait-c1-main-20260811.md)
+preserves that binary separately from the original run. C1 is complete; C2 was
+not started.
+
 | Claim | Row IDs | Agent | Worktree / remote dir | Branch | Owned scope | State | Last update |
 |---|---|---|---|---|---|---|---|
+| `CLAIM-KERNEL-GEMM-CPU-ELEM-A76` | `KERNEL-GEMM-CPU-ELEM-A76` (`ACTIVE`) | Codex (gpt-5.6-sol), fresh helper implementer | isolated local worktree `/tmp/vllm-a76-bf16-impl`; build/test under buildx/QEMU only; execute/PMU on user-authorized disposable `rich@rpi5fan.lan` | `helper/KERNEL-GEMM-CPU-ELEM-A76-W0-W2`, base spec commit `999e1221e` plus this claim commit | Issue #284 W0-W2 only: refresh both-engine baseline/profile; add the BF16 elementwise benchmark/selection mutation proof; audit disassembly and retain only byte-exact C++/NEON changes with recursive measurements. Owns the new spec/row and scoped edits to `examples/cpu_kernel_bench/main.cpp`, its CLI test/CMake only if needed, `src/vt/cpu/cpu_matmul_elem*`, `src/vt/cpu/cpu_ops.cpp` only if profiling proves partitioning, focused tests, and indexed Pi evidence/record projections. Excludes assembly, model/loader/CUDA/Vulkan/C ABI changes, building on the Pi, service changes and PR/push. | `DONE` | 2026-08-10 — W0 complete and reranked before code: current AsyncLLM T4/T3 proves a runnable-thread/core-budget interaction but does not isolate polling; the old binary is historical only. W1-W4 await the #293 discriminator and fresh profile; no kernel/fixture change retained; row stays `ACTIVE`. |
 | `CLAIM-SAMPLE-PROMPT-LOGPROBS-W1` | `SAMPLE-PROMPT-LOGPROBS` (`ACTIVE`; W1 runner source only) | Claude Code (claude-opus-5) | `/home/mudler/_git/vllm.cpp-prompt-logprobs`; CPU-only `build-gate` (`-DVLLM_CPP_CUDA=OFF` Release) — NO GPU, NO download, claims no benchmark device | `row/SAMPLE-PROMPT-LOGPROBS`, opened on `origin/main` `bd6b3936`, rebased onto `8a6704a2` for the 2026-08-10 review repair and onto `e63d11d3` to land, issue #223 | The runner-side prompt-logits source ported from `_get_prompt_logprobs_dict`. Owns ONLY: the `prompt_logprob_rows`/`prompt_logprob_indices` block in `prepare_inputs.{h,cpp}`, `collect_prompt_logprobs` + `in_progress_prompt_logprobs_` in `runner.{h,cpp}`, `Sampler::compute_prompt_logprobs` in `sampler.{h,cpp}`, `num_prompt_logprobs` in `input_batch.{h,cpp}`, section 9 of `tests/vllm/v1/test_llm_engine.cpp`, this claim, the row cells, `.agents/NOW.md`, `.agents/roadmap_v1.md` issue rows, one `docs/STATUS.md` sentence, and the `prompt_logprobs` paragraph in `docs/USAGE.md`. **NON-COLLISION:** NO kernel, NO vt op, NO ABI, NO CMake, NO model file — the full-logits route reuses the existing `VT_LOGITS_GATHER=0` path rather than widening any model's gather contract. EXCLUDED: the OpenAI `echo` serialization (W2) and issue #231, both split out. | `ACTIVE` | 2026-08-10 — W1 landed then REPAIRED after a fresh review returned `VERDICT: FAIL`. Review found a REAL bug: the inertness guard keyed on `prompt_logprob_indices` while the full-logits assertion keyed on `prompt_logprob_rows`, so a zero-row final chunk (the exact-prefill edge, `:5668-5673`) beside another multi-token request threw `VT_CHECK` out of `engine.step()` and killed the whole batch. Fixed by moving the check inside the `num_rows > 0` slice; regression §8(h), RED-first (the throw). Second finding: the on-vs-off inertness case could not see a change to the SHARED route (forcing full logits on every step left it 17/17·346 green), so §8(g) now asserts the route DECISION through a const runner seam and fails under that exact mutation. Third finding recorded, NOT fixed and NOT claimed: the full-logits route hands the sampler a host pointer wearing the device label — sound on unified memory, unverified on CUDA; spec risk 4 + a `PENDING` CUDA smoke gate, and `docs/USAGE.md` narrowed to say so. Three latent lifetime divergences recorded as risks 5-7. Gates on the rebased tree (`e63d11d3`): clean CPU Release build 0 warnings under `-Werror`, `test_llm_engine` **21/21 · 384**, full `ctest -j 6` **365/366** with `test_openai_conformance` (parallel starvation — connection `-1`s) passing serially 1/1 in 0.39 s. Row stays `ACTIVE`: W2 `echo` serialization is the named residual, and the CUDA smoke gate is `PENDING`. |
 | `CLAIM-MOONCAKE-STORE` | `KV-MOONCAKE-STORE` (new, `SPIKE`) | Claude Code (opus-5), helper role | isolated worktree `/home/mudler/_git/vllm.cpp-mooncake`; records-only — NO build, NO GPU, NO download, NO external dependency installed | `row/KV-MOONCAKE-STORE`, base `origin/main` `848d4a87` | The `MooncakeStoreConnector` spike, records-only in this commit: NEW [`.agents/specs/mooncake-store-connector.md`](specs/mooncake-store-connector.md), the NEW `KV-MOONCAKE-STORE` engine-matrix row + section/total counters, the `ENGINE_ROWS` 146→147 bump in `scripts/check-agent-record.py` with its justification comment, a PROSE-ONLY Mooncake-disposition correction inside the `KV-CONNECTORS` row (both NOT-SCHEDULED sentences; that row's `ANCHOR-BACKFILL` state, tier, anchors, evidence, spec link and owner are UNCHANGED, so it is not claimed here), the `ROAD-V1-D4` portfolio note + canonical-table link, the roadmap issue-table row for [#287](https://github.com/mudler/vllm.cpp/issues/287), and this claim. **NON-COLLISION:** touches NO `src/`, `include/`, `tests/`, `examples/` or CMake path; the only script touched is the record checker's own row-count constant. | `ACTIVE` | 2026-08-10 — spec committed. The `KV-CONNECTORS` spike's blanket "Mooncake NOT SCHEDULED" conflated TWO connectors: `MooncakeConnector` (P2P prefill/decode over the Transfer Engine — two nodes, fabric, proxy) KEEPS that verdict; `MooncakeStoreConnector` (shared KV object store, the LMCache analogue) is reopened because (a) Mooncake is NATIVE C++ — `mooncake::Client` in `client_service.h`, and the `MooncakeDistributedStore` vLLM imports is a pybind wrapper over it — so we LINK instead of reimplementing a wire (the inverse of the LMCache cost shape), and (b) its single-node `protocol: "tcp"` + `mooncake_master` config is gateable on one box with NO RDMA NIC. The landed W5 `KVConnector` seam needs NO change. NEXT: W0, the go/no-go link spike (build Mooncake from source, pin the revision, drive `mooncake::Client` from a standalone C++ TU against a local master over TCP) — a genuine stop point per spec §S1. Speed is recorded as an OPEN axis: the RDMA/GPUDirect path that motivates the connector is unmeasurable for want of a fabric on any box we own. PENDING developer authority: the Mooncake source build, push/PR. |
 | `CLAIM-MUSE-GLIMMER-SPEC` | `MODEL-MM-muse-glimmer-muse-glimmer-for-conditional-generation` (`SPIKE`) | Claude Code (opus-5), helper role | isolated worktree `/home/mudler/_git/vllm.cpp-muse-glimmer`; CPU-only, records+spec+W0 scaffold — NO GPU, weights on the shared NAS | `row/MODEL-MUSE-GLIMMER`, base `origin/main`; PR [#279](https://github.com/mudler/vllm.cpp/pull/279) OPEN | The Muse Glimmer port. Spec `.agents/specs/muse-glimmer.md`, the `MODEL-MM-muse-glimmer-*` matrix row + checklist + rollup, issue-table entry for [#268](https://github.com/mudler/vllm.cpp/issues/268), porting-inventory §9 deviation 16 (the OFF-PIN anchor), `check-agent-record.py` MODEL count, the W0 scaffold (registry + config parse + weight name map + refuse-by-name forward, gate 11/11), `CHECKPOINT_ROOT` in `.env.example` + `.agents/environment.md`. Those helpers ALL LANDED onto the row branch: W1 the 52-layer text forward, W3 the 50-layer perception encoder, W4 the mm wiring (tower to adapter to projection to `perception_emb_norm` to the masked scatter; released-30B accounting 1436/1436), W7 the ATEM reasoning + tool parsers. The branch is no longer "records + spec + W0 scaffold" and its forward no longer refuses. `CLAIM-MUSE-GLIMMER-FIX` (`row/MODEL-MUSE-GLIMMER-FIX`) then closed the PR #279 review findings: three COVERAGE HOLES that mutation testing found green (`perception_emb_norm` untested, the whole non-FusedChain fallback arm unexercised, RoPE theta ungated at a realistic magnitude), the stale "OPEN FINDING" that had disarmed the real-weights accounting assertion, and the false claim in both parser headers that the seam has no `skip_special_tokens` (it does; channel scoping is an OPEN GAP at server defaults, spec §6.7). WHAT IS STILL NOT ESTABLISHED, and must not be implied: the full-depth 52-layer arm never ran (evidence is reduced depth 4/52, 5 prefill argmax positions, no generated tokens), multi-step decode is untested, the perception encoder has no reference check, and nothing has run through the server. **Anchored to the UNMERGED [vllm#51655](https://github.com/vllm-project/vllm/pull/51655) head `075d645af`** on explicit developer direction 2026-08-10: Muse Glimmer exists neither at the pin `555967922` nor on vLLM `main`. **No speed axis is claimable** until #51655 merges and the pin advances — the pinned oracle cannot load the model. | `ACTIVE` | 2026-08-10 |