feat(yoco): YOCO × frozen-KV MTP interaction bench on E4B-it (PR-D)

[pyc96]

Summary

Follow-up to PR-C #13. Answers the reviewer question "will YOCO be exportable to Gemma-4 MTP?" with a live measurement on gemma-4-E4B-it + gemma-4-E4B-it-assistant + NEXTN(3/4/1) on H100 TP=2.

TL;DR: yes for the target prefill; the assistant seed / draft loop / target verify are correctly skipped by the existing predicate; no worker-side changes needed.

Per-phase YOCO applicability

Phase	Worker entry	Mode	Q tokens/req	YOCO fires?
Target prefill	`forward_target_extend`	EXTEND	1..L	Yes (same path PR-C bench'd)
Assistant seed step	`_run_assistant_seed_step`	EXTEND seq_len=1	1	No (predicate: (N−1)=0)
Recurrent draft step	`draft_forward`	DECODE	1	No (predicate: not is_extend)
Target verify	`verify`	TARGET_VERIFY	1+spec(=4)	No (predicate: is_target_verify)

Correctness under MTP

• Per-prompt parity (20 prompts, greedy, MTP both sides): 20/20 byte-identical
• MMLU N=500: MTP YOCO off 0.588 (294/500) vs MTP YOCO on 0.590 (295/500) = +0.2 pp (within ±1 pp)
• accept_length: 2.10 summ / 2.20 chat — identical between YOCO-off and YOCO-on (direct consequence of byte-identical parity; YOCO does not change which tokens the verify loop accepts)

Performance under MTP — summ 8000/1000 n=80

Metric	MTP YOCO off	MTP YOCO on	Δ
output tok/s	1469.5	1559.0	+6.1 %
duration (s)	54.4	51.3	−5.7 %
median TTFT (ms)	6414.2	4555.5	−29.0 %
p99 TTFT (ms)	31526.7	28237.7	−10.4 %
median TPOT (ms)	22.5	21.1	−6.2 %
accept_length	2.10	2.10	tied

Performance under MTP — chat 1000/1000 n=80

Metric	MTP YOCO off	MTP YOCO on	Δ
output tok/s	2932.1	2887.9	−1.5 % (noise — break-even point)
median TTFT (ms)	1001.6	1012.3	+1.1 % (noise)
median TPOT (ms)	10.3	10.5	+1.9 % (noise)
accept_length	2.20	2.20	tied

When prefill is short (1 k tokens), the YOCO gather/scatter+metadata-rebuild overhead roughly matches the saved back-half compute. The summ scenario is unambiguous; chat is noise-floor neutral.

Comparison to PR-C (no-MTP) numbers

Scenario	Config	tok/s	Δ vs YOCO-off
chat	no-MTP YOCO off	9327.5	baseline
chat	no-MTP YOCO on	9520.2	+2.1 %
chat	MTP YOCO off	2932.1	(MTP is a 3× loss on E4B)
chat	MTP YOCO on	2887.9	−1.5 % (noise)
summ	no-MTP YOCO off	3469.6	baseline
summ	no-MTP YOCO on	4078.5	+17.6 %
summ	MTP YOCO off	1469.5	(MTP is a 2.4× loss on E4B)
summ	MTP YOCO on	1559.0	+6.1 %

Two independent effects:

1. MTP is a net loss on E4B at this concurrency due to the pre-existing `max_running_requests=48` auto-cap from `_handle_frozen_kv_mtp` (`arg_groups/speculative_hook.py:233-250`) plus the `disable_overlap_schedule` penalty. Same bottleneck that bit the 31B-it campaign. Orthogonal to YOCO.
2. YOCO is a real win on top of MTP for prefill-bound workloads. Summ median TTFT drops 29.0 % under MTP — essentially the same −27.8 % drop under no-MTP — because TTFT is purely target-prefill cost.

Rollout recommendation

• Ship YOCO as-is for any Gemma-4 model with `num_kv_shared_layers > 0`, MTP-on or MTP-off.
• The predicate already routes YOCO correctly; the MTP integration is automatic.
• No worker-side changes needed.
• A separate follow-up campaign should investigate why MTP is a net loss on small models (max_running_requests cap + disable_overlap_schedule).

What's in PR-D

One doc:

• `.humanize/yoco-gemma4/bench-results-e4b-mtp-h100.md` (~170 lines): full interaction matrix, per-phase YOCO-applicability table, headline numbers, rollout recommendation, reproducer.

Raw bench JSONLs (4 files) and MMLU rows (4 rows) are saved at the artifact root `runs/20260524_gemma4_e4b_yoco_h100/` and cited in the doc.

Reproducer

```bash

YOCO-off + MTP (GPUs 0,1)

bash runs/20260524_gemma4_e4b_yoco_h100/benchmark/launch_sglang_e4b_mtp.sh \
sglang_e4b_mtp_yoco_off 0,1 30100 yoco_off

YOCO-on + MTP (GPUs 2,3)

bash runs/20260524_gemma4_e4b_yoco_h100/benchmark/launch_sglang_e4b_mtp.sh \
sglang_e4b_mtp_yoco_on 2,3 30101 yoco_on

Parity

python runs/20260524_gemma4_e4b_yoco_h100/quality/parity_check.py \
--url-off http://127.0.0.1:30100 --url-on http://127.0.0.1:30101 --num-prompts 20

Bench

bash runs/20260524_gemma4_e4b_yoco_h100/benchmark/run_benchmark.sh \
e4b_mtp_yoco_on sglang-oai-chat http://127.0.0.1:30101 summ 8000 1000 80

MMLU

python runs/20260524_gemma4_e4b_yoco_h100/quality/run_mmlu.py \
--url http://127.0.0.1:30101 --label e4b_mtp_yoco_on --num-questions 500 --seed 0
```

Stack

• PR-A feat(yoco): add --kv-sharing-fast-prefill server arg + validators (PR-A/3) #11: flag plumbing
• PR-B feat(yoco): implement YOCO fast-prefill branch in Gemma4TextModel (PR-B/3) #12: model branch + 16 unit tests
• PR-C feat(yoco): E4B-it H100 bench results + per-prompt parity (PR-C/3) #13: no-MTP E4B bench + parity + MMLU
• PR-D (this PR): MTP-interaction bench + parity + MMLU

Stack base: `pyc/yoco-fast-prefill-bench` @ `5cb630913`

Plan: `.humanize/yoco-gemma4/refined-plan.md`

---

CI States

Latest PR Test (Base): ❌ Missing run-ci label -- add it to run CI tests.
Latest PR Test (Extra): ❌ Blocked -- run-ci is required first.