Skip to content

perf(cpu): isolate Raspberry Pi 5 polling contention - #361

Draft
richiejp wants to merge 11 commits into
mudler:mainfrom
richiejp:row/KERNEL-GEMM-CPU-ELEM-A76
Draft

perf(cpu): isolate Raspberry Pi 5 polling contention#361
richiejp wants to merge 11 commits into
mudler:mainfrom
richiejp:row/KERNEL-GEMM-CPU-ELEM-A76

Conversation

@richiejp

@richiejp richiejp commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Row

KERNEL-GEMM-CPU-ELEM-A76 — tracked by #284. The benchmark-control prerequisite is tracked by #293.

Scope

This draft contains the reviewed concurrency-one blocking-c1 benchmark control and the Raspberry Pi 5 W0/C1 records. It preserves poll as the default and rejects blocking-c1 above concurrency one.

It also carries the latest-main integration repairs needed to preserve #206 prompt pretokenization, exact prompt IDs, and atomic ordered wave admission after rebasing.

This PR does not contain the general C2 event wait, the F16 neon-zip / neon-p8 candidates, a default kernel change, or new assembly.

What changed

  • Added the explicit poll / blocking-c1 output-wait selector, CLI validation, JSON/text reporting, and exact-token tests.
  • Bound the real unconditional get_output_nowait evaluation before the C1 blocking call.
  • Preserved current-main pretokenization and wave-admission behavior through the rebase.
  • Added mutation-sensitive prompt-ID and checker tests required by the exact PR-size gate.
  • Updated STATUS, BENCHMARKS, USAGE, NOW, the owning specs/matrices, and indexed Pi evidence. The llama.cpp floor remains unmet.

Evidence

  • Current-head native benchmark suite: 15/15 cases, 124/124 assertions.
  • Current-head scoped native CTest: 5/5 (test_engine_core_proc, test_output_processor, test_async_llm, test_bench, test_cpu_threadpool).
  • Prompt-ID assignment deletion, nowait bypass, blocking dispatch inversion, report lies, special-token removal, and wave-publication mutations turn RED and restore byte-for-byte.
  • Exact PR-size semantic-evidence gate passes in an environment where its sanitized PATH can resolve Git.
  • Historical same-binary Pi C1 A/B: 12/12 clean, token-identical, unthrottled. At four compute threads, blocking improved prefill 1.973x, per-stream decode 2.149x, and output throughput 2.151x; T3 was neutral. Four-thread polling was unstable, so this is a causal discriminator rather than a new binding floor.
  • Exact-current-head AArch64/QEMU and Pi rebenchmark are owed after the next main rebase.

Review state

  • Latest-main C1 conflict integration and its repair passed fresh mutation-driven review. The reviewed repair tree is byte-identical to this PR head.
  • Full repository preflight still reports inherited stale live rows and environment/history fixture failures; all change-specific record, document, trailer, registration, PR-size, and scoped test gates pass.

Follow-up work not in this PR

  • C2 event wait candidate 518316436: product, native gates, and nine mutations passed review; a record-only anchor correction is required before it can pass final review.
  • F16 C++ candidate b120a71b5: native/cross-build/disassembly checks passed; review found that the physical-selection fixture needs an independent correctness golden before Pi benchmarking.
  • After those repairs: combine reviewed heads, build under AArch64 QEMU, run same-binary Pi baseline / neon-zip / neon-p8 A/B, retain only a token-exact physical winner, and compare the full model against llama.cpp. Assembly remains forbidden until the C++ path clears that floor.

Rebase status

This branch was rebased through upstream 11d45330b; upstream main is now 6dbedf9f3, and GitHub currently reports merge conflicts. Rebase, fresh conflict review, and exact-head QEMU/Pi evidence remain required before marking the PR ready.

FOLLOWING_AGENTS_PROTOCOL

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Transplant the reviewed Pi profile onto current upstream without treating the
historical binary or T4/T3 arm as a one-variable polling control. Record the
prompt-content mismatch and link the separately tracked mudler#293 discriminator.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the isolated current-upstream implementation worktree and move the
scoped polling discriminator from READY to ACTIVE without moving the broad
SERVE-CLI-BENCH row.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Reapply the same-binary concurrency-one polling discriminator unchanged onto
current upstream. Keep poll as the default, reject blocking-c1 above c1, and
record both the selected mode and blocking-call count in text and JSON output.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the hash-verified current-source Pi control: blocking the sole collector
restores the four-thread arm while the three-thread control stays neutral and
exact tokens hold. Keep the accepted llama.cpp floor unchanged because the
pathological polling arm exceeds the binding spread limit, and leave C2 to
fresh review and implementation.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Add a test-only wait trace that proves each blocking-c1 wait follows an empty
nowait probe, cover the human-readable report, and repair the row anchors. The
trace compiles out of production; native and AArch64 vllm-bench artifacts retain
the reviewed executable bytes.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Make the output-wait test observer inspect the result only after the same
unconditional AsyncLLM call used by production. Replace the source-text guard
with a deterministic runtime probe whose bypass mutation fails normally, while
preserving the measured AArch64 Release binary byte-for-byte.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Record the reviewed post-rebase operator gate without replacing the older binary's provenance. The current-main same-binary Pi control confirms four-core polling contention while T3 remains neutral, but the unstable poll arm still forbids a binding floor change.

Correct the low-severity evidence precision finding by reporting the raw perf-stat T3 CPU-utilization medians to two decimal places.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
Require both benchmark admission arms to expose populated, exact per-request prompt IDs, so dropping the production assignment fails normally. Add semantic mutation evidence for the A76 inventory and STATUS ratchet changes, re-anchor the SERVE-CLI-BENCH row, and project the confirmed C1 polling cause without changing the llama.cpp floor.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:gpt-5.6-sol [Codex]
@richiejp
richiejp force-pushed the row/KERNEL-GEMM-CPU-ELEM-A76 branch from c53700e to cc4b195 Compare August 11, 2026 13:25
@localai-bot

Copy link
Copy Markdown
Collaborator

This branch conflicts with main because #364 and #374 removed three shared record surfaces. It is a one-time cost for every branch that was in flight, and the resolution is mechanical — nothing this PR measured or decided needs revisiting. I can't push to your fork, so here is the exact recipe:

git fetch origin main && git merge origin/main

Then, per conflicted file:

File Resolution Why
scripts/check-public-doc-tables.py git checkout origin/main -- scripts/check-public-doc-tables.py The STATUS_RATCHET["chars"] key is gone. It was a byte count of docs/STATUS.md stored in the checker, so every PR that touched STATUS had to re-pin it. Your re-pin only existed to pay for your own STATUS edit; dropping it loses nothing.
.agents/NOW.md git checkout origin/main -- .agents/NOW.md The per-row live-claims table is gone. The live position is derived now — run scripts/now.py. Your row's next step belongs in your row's own spec under a ## Now heading, which has one writer and cannot conflict.
.agents/coordination.md keep both sides New claims go in .agents/claims/CLAIM-<ID>.md, one file per claim. The legacy table still parses, so an existing row there is still valid.
.agents/roadmap_v1.md keep both sides' rows The issue table is now sorted by row ID. Union both sides and re-sort; do not drop anyone's row.
.agents/benchmark-record.md union both sides Append-only log — taking either side alone silently drops a measurement.
tests/scripts/test_check_public_doc_tables.py git checkout origin/main -- <path> Follows the checker above.

Anything under src/, include/ or tests/*.cpp is not part of this and is a real conflict worth reading.

Sorry for the churn — removing a shared surface makes every in-flight branch pay once. After this merge your PR stops competing with other PRs for these files entirely, which is the point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants