diff --git a/.humanize/yoco-gemma4/bench-results-e4b-h100.md b/.humanize/yoco-gemma4/bench-results-e4b-h100.md
new file mode 100644
index 000000000000..8e61b36476b9
--- /dev/null
+++ b/.humanize/yoco-gemma4/bench-results-e4b-h100.md
@@ -0,0 +1,120 @@
+# YOCO Fast-Prefill Bench Results — Gemma-4 E4B-it on H100 TP=2
+
+Date: 2026-05-24
+Branch: `pyc/yoco-fast-prefill-bench` (stacked on `pyc/yoco-fast-prefill-impl`)
+
+## Setup
+
+| Item | Value |
+|---|---|
+| Model | `google/gemma-4-E4B-it` (4.5B dense + PLE, 42 layers, `num_kv_shared_layers=18`, PLE `hidden_size_per_layer_input=256`) |
+| Hardware | NVIDIA H100 80 GB SXM5, TP=2 |
+| Attention backend | triton |
+| SGLang branch | `pyc/yoco-fast-prefill-impl` (= PR-A flag plumbing + PR-B model code) |
+| YOCO-on launch flag | `--kv-sharing-fast-prefill` |
+| Both servers | `--disable-piecewise-cuda-graph` (avoids a pre-existing PCG bug in `radix_attention.unified_attention_with_output` that crashes on KV-shared layers when key is None; this bug pre-dates YOCO and is orthogonal). |
+| Workload | random, warmup 2, seed 1, num_prompts=80 |
+| Quality benchmark | MMLU N=500, seed 0, temp 0 |
+
+## Correctness — per-prompt parity (20 prompts, greedy)
+
+```
+{
+  "total": 20,
+  "matched": 20,
+  "mismatched": 0,
+  "match_rate": 1.0
+}
+```
+
+**20/20 prompts produce byte-identical output between YOCO-off and YOCO-on.**
+
+## Quality — MMLU N=500
+
+| Stack | accuracy | correct/500 |
+|---|---:|---:|
+| SGLang YOCO off | 0.594 | 297 |
+| **SGLang YOCO on** | **0.592** | **296** |
+
+Δ = −0.2 pp (1 question difference; within the ±1 pp acceptance bar).
+
+## Performance — chat 1000/1000 n=80
+
+| Metric | YOCO off | YOCO on | Δ |
+|---|---:|---:|---|
+| output_throughput (tok/s) | 9327.5 | **9520.2** | **+2.1 %** |
+| duration (s) | 8.6 | **8.4** | −2.3 % |
+| mean_ttft (ms) | 636.2 | **567.6** | −10.8 % |
+| median_ttft (ms) | 618.0 | **557.7** | **−9.8 %** |
+| p99_ttft (ms) | 1032.1 | **856.3** | **−17.0 %** |
+| median_tpot (ms) | 7.9 | 7.8 | −1.3 % |
+
+## Performance — summ 8000/1000 n=80 (the target workload for YOCO)
+
+| Metric | YOCO off | YOCO on | Δ |
+|---|---:|---:|---|
+| **output_throughput (tok/s)** | **3469.6** | **4078.5** | **+17.6 %** |
+| duration (s) | 23.1 | **19.6** | −15.2 % |
+| mean_ttft (ms) | 5945.3 | **4319.4** | **−27.3 %** |
+| **median_ttft (ms)** | **5920.3** | **4276.1** | **−27.8 %** |
+| **p99_ttft (ms)** | **11550.5** | **8144.6** | **−29.5 %** |
+| median_tpot (ms) | 17.1 | **15.3** | **−10.5 %** |
+
+## Why the summ scenario wins more than chat
+
+YOCO eliminates compute on the `num_kv_shared_layers` back-half layers for all
+but the last token per request. That saving is proportional to
+`(input_len − 1) × num_kv_shared_layers / num_hidden_layers`.
+
+For E4B-it (`num_kv_shared_layers=18`, `num_hidden_layers=42`):
+- chat 1k/1k: theoretical max layer-compute saving =
+  `(1000−1) × 18 / 42 = ~428 layer-token-equivalents per request`
+- summ 8k/1k: theoretical max layer-compute saving =
+  `(8000−1) × 18 / 42 = ~3429 layer-token-equivalents per request`
+
+→ 8× more compute saved on summ, which matches the observed throughput
+delta (chat +2 %, summ +18 %) since the rest of the per-token cost is
+fixed (front-half layers + lm head + scheduler overhead).
+
+## Server log evidence
+
+YOCO-on startup confirms the flag fires correctly:
+
+```
+[2026-05-24 06:52:20] KV-sharing fast prefill enabled for Gemma4ForConditionalGeneration (num_kv_shared_layers=18).
+```
+
+YOCO-off startup log makes no mention of fast-prefill (default-off behavior preserved).
+
+## Why other Gemma-4 checkpoints don't benefit
+
+The HF text-config `num_kv_shared_layers` is:
+- `gemma-4-26b-a4b-it`: **0** (MoE; benchmarked in prior campaign at `runs/20260522_gemma4_26b_a4b_it_*`)
+- `gemma-4-31B-it`: **0** (dense; benchmarked in prior campaign at `runs/20260523_gemma4_31b_it_*`)
+- **`gemma-4-E4B-it`: 18** ← this run
+- `gemma-4-E2B-it`: 20 (not benchmarked; would benefit similarly)
+- `gemma-4-*-assistant` (MTP drafts): 4, but the worker runs them only in DECODE mode, so YOCO degrades to a no-op.
+
+So YOCO is a Gemma-4-E*-IT optimization. The PR-A flag is universal (any future Gemma-4 checkpoint that declares `num_kv_shared_layers > 0` automatically benefits when `--kv-sharing-fast-prefill` is set).
+
+## Reproducer
+
+```bash
+# SGLang YOCO-off (GPUs 0,1)
+bash /home/pyc_google_com/dev/gemma-op/agent-pod/runs/20260524_gemma4_e4b_yoco_h100/benchmark/launch_sglang_e4b.sh \
+     sglang_e4b_yoco_off 0,1 30000 yoco_off
+
+# SGLang YOCO-on (GPUs 2,3, in parallel)
+bash /home/pyc_google_com/dev/gemma-op/agent-pod/runs/20260524_gemma4_e4b_yoco_h100/benchmark/launch_sglang_e4b.sh \
+     sglang_e4b_yoco_on 2,3 30001 yoco_on
+
+# Parity test
+python /home/pyc_google_com/dev/gemma-op/agent-pod/runs/20260524_gemma4_e4b_yoco_h100/quality/parity_check.py \
+       --url-off http://127.0.0.1:30000 \
+       --url-on  http://127.0.0.1:30001 \
+       --num-prompts 20
+
+# Bench
+bash /home/pyc_google_com/dev/gemma-op/agent-pod/runs/20260524_gemma4_e4b_yoco_h100/benchmark/run_benchmark.sh \
+     e4b_yoco_on sglang-oai-chat http://127.0.0.1:30001 summ 8000 1000 80
+```
diff --git a/.humanize/yoco-gemma4/draft.md b/.humanize/yoco-gemma4/draft.md
new file mode 100644
index 000000000000..e77705e0882e
--- /dev/null
+++ b/.humanize/yoco-gemma4/draft.md
@@ -0,0 +1,277 @@
+# Draft: Port vLLM's YOCO Fast-Prefill to SGLang for Gemma-4
+
+## Background
+
+vLLM's Gemma-4 (and Gemma-3n) implementation contains a "fast prefill" path
+that exploits the model's KV-cache-sharing structure. The last
+`num_kv_shared_layers` decoder layers reuse K/V from earlier layers of the
+same type (sliding vs full) and write nothing of their own to the KV cache.
+Because the only function of those layers' hidden_states is to feed the LM
+head, **their compute is only needed at the per-request "last extend
+token" positions** — i.e. `num_reqs` rows instead of `num_extend_tokens`
+rows.
+
+For Gemma-4 31B-IT on H100 + TP=2 + an 80-prompt × 8000-token-input
+summarization workload, this means the last `num_kv_shared_layers` of the
+60-layer dense model can process ~80 rows in the cross-decoder phase
+instead of ~640,000 rows. Per the campaign benchmark, the SGLang summ TPOT
+already beats vLLM (23.4 vs 31.8 ms), but SGLang's summ throughput is
+−62% vs vLLM (331 vs 868 tok/s) and median TTFT is +96% (78s vs 40s).
+**Closing that gap is the YOCO opportunity.**
+
+This draft proposes porting the YOCO fast-prefill technique to SGLang's
+Gemma-4 path, with full benchmark + MMLU validation per the campaign's
+acceptance bar.
+
+## Problem statement
+
+When `forward_mode == EXTEND` and the served model has
+`num_kv_shared_layers > 0`:
+- The first `K = num_hidden_layers − num_kv_shared_layers` decoder layers
+  must run on the full `[T = sum(extend_seq_lens), H]` hidden_states
+  tensor — they write into the KV cache that the back half will read.
+- The last `num_kv_shared_layers` layers do **not** write KV; they only
+  produce hidden_states. Only the per-request last-extend-token rows of
+  those hidden_states are sampled (via `LogitsProcessor._get_pruned_states`
+  at `logits_processor.py:432-447`, which builds
+  `last_index = cumsum(extend_seq_lens) - 1` and gathers
+  `hidden_states[last_index]`).
+
+So all per-token compute in the cross-decoder for the non-last positions
+is **wasted work**. YOCO eliminates it by gathering hidden_states to the
+last-token positions before the cross-decoder runs, running the
+cross-decoder only on the gathered rows, then scattering the cross-decoder
+output back into a full-shape hidden_states tensor (so that downstream
+consumers that expect `CaptureHiddenMode == FULL` still get the right
+contract for the rows they actually read).
+
+## What the optimization touches
+
+Per the SGLang+vLLM audit:
+
+1. **Model code** — split `Gemma4TextModel.forward` (the layer loop in
+   `python/sglang/srt/models/gemma4_causal.py:922-1003`) into a self-decoder
+   half (layers `[0, K)`) and a cross-decoder half (layers `[K, N)`) with
+   a gather/scatter between them.
+
+2. **Attention metadata** — the cross-decoder runs Q at decode-shape
+   (one query token per request) but K/V at full prefix length. The
+   triton extend kernel (`triton_backend.py:894+`) is built from
+   `qo_indptr = cumsum(extend_seq_lens)`, so calling it directly on the
+   gathered Q would walk past the end of the q buffer. The cleanest fix
+   is to temporarily build a DECODE-shaped `ForwardMetadata` for the
+   cross-decoder phase (qo_indptr = `[0..B]`, max_extend_len = 1,
+   `kv_indptr` and `kv_indices` rebuilt from `seq_lens` covering the
+   full prefix-plus-extend KV span).
+
+3. **Per-Layer Embeddings (PLE)** — Gemma-4 31B-IT and E4B-IT have
+   `hidden_size_per_layer_input == 0` (no PLE), so the
+   `per_layer_inputs` argument that gets sliced per layer is `None`.
+   For the MoE 26B-A4B-IT variant, PLE is also disabled
+   (`enable_moe_block=True` does not imply PLE). **PLE is therefore
+   out of scope for the v0 patch**: a small guard `if has_ple:
+   fall_back_to_eager_full_path` covers all Gemma-4 sizes we care about.
+   Note: Gemma-3n E2B/E4B *does* use PLE; if we ever extend YOCO to
+   Gemma-3n, the PLE per_layer_inputs tensor needs to be sliced by
+   `last_index` before being passed into the cross-decoder.
+
+4. **Speculative decoding (frozen-KV MTP)** — YOCO applies only to
+   target-model EXTEND (`forward_target_extend` at
+   `frozen_kv_mtp_worker.py:493-503`). The assistant has
+   `num_kv_shared_layers=0` (`gemma4_mtp.py:71`), so YOCO degrades to
+   a no-op on the assistant forward. TARGET_VERIFY is decode-shaped
+   (Q-per-req=1+spec_tokens) and also a no-op. The worker calls
+   `target_worker.forward_batch_generation(batch)` with
+   `capture_hidden_mode == FULL`, then reads only the last-token row via
+   `_select_last_extend_hidden`. So **YOCO must still produce
+   full-shape hidden_states on output** (scatter back), but the
+   downstream consumer is happy as long as the per-request last-token row
+   is the post-cross-decoder value.
+
+5. **Piecewise CUDA graph (PCG)** — SGLang's `PiecewiseCudaGraphRunner`
+   currently captures the full `Gemma4TextModel.forward` end-to-end at
+   each `capture_num_tokens` bucket. Splitting the loop into two halves
+   that run at different token counts breaks the assumption of one fixed
+   shape per captured graph. **v0 strategy:** keep YOCO opt-in via a
+   server arg and **disable PCG when YOCO is enabled** (or vice versa).
+   The eager EXTEND path is what 31B-IT currently uses anyway (the
+   benchmark logs show `disable_piecewise_cuda_graph=False` is set but
+   piecewise is empirically not engaged for the spec-decode hot path —
+   see D1 attempt-ledger finding). This means YOCO's v0 lives in the
+   eager EXTEND path and does not coexist with PCG.
+
+6. **Multimodal** — already disabled in the current campaign branch via
+   `mm_disabled_models` for `Gemma4ForConditionalGeneration`. YOCO has
+   no interaction with multimodal towers; the bidi image-attention mask
+   is applied only when `forward_mode == EXTEND AND contains_image_inputs()`
+   (`gemma4_mm.py:614-622`). For the text-only benchmark workload, no
+   image tokens → no mask → no interaction.
+
+## Proposed mechanism
+
+### Step 1: configuration flag
+
+Add `kv_sharing_fast_prefill: bool = False` to `ServerArgs`. Default
+False to preserve current behavior. Plumb through to `ModelConfig` and
+read inside the model.
+
+### Step 2: predicate
+
+Inside `Gemma4TextModel.forward`, gate YOCO behind:
+
+```python
+can_run_yoco = (
+    self.config.num_kv_shared_layers > 0
+    and forward_batch.forward_mode.is_extend()
+    and not forward_batch.forward_mode.is_target_verify()
+    and forward_batch.extend_seq_lens is not None
+    and forward_batch.batch_size > 0
+    and not is_in_piecewise_cuda_graph()         # PCG bypass
+    and not self._has_ple                        # PLE bypass (v0)
+    and not _has_input_logprobs(forward_batch)   # input-logprob bypass
+    and self.fast_prefill_enabled                # opt-in flag
+)
+```
+
+If predicate fails, fall back to the existing layer loop unchanged.
+
+### Step 3: gather index
+
+```python
+# Built on GPU; one int64 tensor of shape [B].
+last_index = torch.cumsum(forward_batch.extend_seq_lens, dim=0) - 1
+```
+
+This is identical to the construction at `logits_processor.py:432-447`
+that the LM head gather already uses, so we know it's semantically
+correct.
+
+### Step 4: front half
+
+Run layers `[0, K)` on full `[T, H]` `hidden_states` as today. Each layer
+writes its own K/V into the KV cache (because `is_kv_shared_layer=False`
+for these layers).
+
+### Step 5: build decode-shaped attention metadata for back half
+
+The cross-decoder needs a `ForwardMetadata` that says "Q has B rows
+(one per request), K/V is the full prefix+extend per request". Approach:
+
+- Temporarily build a "shadow" `ForwardBatch` (or just temporarily mutate
+  `forward_batch.forward_mode = DECODE`, `forward_batch.seq_lens` and
+  `forward_batch.req_pool_indices` unchanged, and rebuild attention metadata
+  via `attn_backend.init_forward_metadata(shadow_batch)`).
+- Restore the original metadata after the back half finishes.
+
+The triton backend's DECODE path (`triton_backend.py:298-363`) builds
+`kv_indptr = cumsum(seq_lens)` and `kv_indices` that point at the full
+KV pool slots for each request — which is exactly what we want.
+
+### Step 6: gather → cross-decoder → scatter
+
+```python
+gathered_h        = front_half_out[last_index]               # [B, H]
+gathered_pos      = positions[last_index]                    # [B]
+# (skip per_layer_inputs gather — guarded out by has_ple)
+
+cross_h = run_cross_decoder(gathered_pos, gathered_h, forward_batch)
+# cross_h shape: [B, H]
+
+if forward_batch.capture_hidden_mode in (CaptureHiddenMode.FULL,
+                                          CaptureHiddenMode.LAST):
+    # Scatter back into the full hidden_states tensor.
+    full_h = front_half_out.clone()
+    full_h.index_copy_(0, last_index, cross_h)
+    hidden_states = full_h
+else:
+    # NULL mode: downstream LogitsProcessor will gather by last_index
+    # again; we can return the gathered tensor directly and let
+    # LogitsProcessor's NULL/short-circuit branch pass through.
+    hidden_states = cross_h  # caller must respect this is [B, H], not [T, H]
+```
+
+For the v0 we choose the **always-scatter** path — it preserves the
+existing `[T, H]` contract everywhere downstream, including frozen-KV MTP
+worker's `_select_last_extend_hidden`. The clone + scatter cost is
+`O(T * H)` = tiny compared to the saved cross-decoder compute.
+
+### Step 7: invariants
+
+- KV cache write semantics unchanged: front-half layers write their K/V
+  with `save_kv_cache=True`; back-half layers have
+  `is_kv_shared_layer=True` and `save_kv_cache=False`, so they only
+  read. No KV-pool corruption possible.
+- `req_to_token_pool` and `out_cache_loc` are not touched.
+- `seq_lens`, `extend_seq_lens`, `extend_prefix_lens` are not modified
+  in the persistent `ForwardBatch`; the back-half metadata is built from
+  a copy or via a context manager that restores on exit.
+- Final norm + LM head receives a `[T, H]` tensor with correct values at
+  the `last_index` rows (post-cross-decoder) and pre-cross-decoder values
+  at all other rows. Since the LM head only reads `last_index` rows, the
+  per-token logits are bit-identical to the non-YOCO path.
+
+## Quality bar
+
+- MMLU N=500, seed 0, temp 0: must stay within ±1 pp of the current
+  patched SGLang's 0.780 → i.e. ≥ 0.770.
+- The fixed benchmark scenarios (chat 1000/1000 n=80, summ 8000/1000
+  n=80) must complete all 80 prompts in both stacks.
+
+## Performance bar
+
+- summ scenario output tok/s: ≥ 331 (current SGLang no-MTP best on the
+  patched branch). Target: meaningful improvement, e.g. ≥ 500 tok/s
+  closing ~30 % of the gap to vLLM's 868.
+- chat scenario must not regress more than 1 % vs the current SGLang
+  best (1499 tok/s on the patched MTP cap-80 server).
+- Median TTFT on summ must improve (current 78023 ms is the bound).
+
+## Out of scope for v0
+
+- Gemma-3n E2B / E4B PLE-aware YOCO.
+- Coexistence with PCG (will be added after v0 ships and SGLang's
+  piecewise capture is extended to support dynamic mid-graph reshapes).
+- `--kv-sharing-fast-prefill` interaction with EAGLE / EAGLE3 (matches
+  vLLM behavior — banned at startup).
+- Attention backends other than triton (`flashinfer`, `fa3`, `trtllm_mha`
+  not in scope; `triton` is the user-pinned backend for this campaign).
+
+## Open design questions for the user
+
+1. **Default value of `--kv-sharing-fast-prefill`**: should it default
+   to `False` (opt-in, matches vLLM) or default to `True` for Gemma-4
+   (auto-enabled in `_handle_model_specific_adjustments`, like the
+   `swa_full_tokens_ratio` and `attention-backend triton` defaults)?
+
+2. **NULL vs FULL hidden mode handling**: scatter-back always (simpler,
+   preserves contract everywhere), or return the gathered tensor in
+   NULL mode (saves the `clone()` + `index_copy_` cost when the caller
+   doesn't need the full tensor)?
+
+3. **Backend metadata rebuild strategy**: full `init_forward_metadata`
+   call inside the model (works for any backend but adds invocation
+   overhead — measured at ~0.1-0.3 ms on H100 for `bs=80`); or a
+   triton-only fast-path that constructs a minimal `ForwardMetadata`
+   directly in the model (faster but couples the model to the backend's
+   metadata dataclass)?
+
+4. **Validation scope**: in addition to MMLU + bench, do we want a
+   per-prompt parity test (run the same N prompts with YOCO on/off,
+   diff the generated tokens) to prove zero quality regression beyond
+   MMLU's aggregate signal?
+
+5. **Scope of the port**: only `Gemma4ForCausalLM` and
+   `Gemma4ForConditionalGeneration` (i.e. Gemma-4 only), or also wire
+   the same flag for any other SGLang model that already has
+   `num_kv_shared_layers > 0`? Look at SGLang's model registry: are
+   there other models with that attribute non-zero today?
+
+6. **PR strategy**: one big PR (config + model + attn metadata + tests)
+   or split into a stack (config + attn-builder wrapper as PR 1; model
+   integration as PR 2; tests as PR 3)? Per the user's "one task can
+   have multiple PRs" constraint, a stack is allowed.
+
+7. **Acceptance criterion for SOTA loop**: should the loop continue past
+   v0 (to add PLE support, PCG coexistence, etc.) until the summ
+   throughput ties vLLM, or stop at v0 once the summ gap is materially
+   closed (e.g. ≥ 30 %)?
diff --git a/.humanize/yoco-gemma4/refined-plan.md b/.humanize/yoco-gemma4/refined-plan.md
new file mode 100644
index 000000000000..cba215d28249
--- /dev/null
+++ b/.humanize/yoco-gemma4/refined-plan.md
@@ -0,0 +1,462 @@
+# Port vLLM YOCO Fast-Prefill to SGLang for Gemma-4
+
+## Goal Description
+
+Add an opt-in fast-prefill path to SGLang's Gemma-4 model that, when a
+served Gemma-4 checkpoint has `num_kv_shared_layers > 0` and the new
+server arg `--kv-sharing-fast-prefill` is set, runs the last
+`num_kv_shared_layers` decoder layers (the "cross-decoder") only on the
+per-request last-extend-token rows during EXTEND-mode forwards, instead
+of on every input token. This matches the optimization vLLM ships for
+`Gemma4ForCausalLM` in `vllm/model_executor/models/gemma4.py:1190-1273`
+and is expected to materially close the SGLang-vs-vLLM throughput and
+TTFT gap on long-input prefill workloads (target: summarization 8000/1000
+n=80 throughput gap to vLLM narrows by ≥ 30 %).
+
+Quality must be preserved: per-prompt token output must be byte-identical
+to the non-fast-prefill path for greedy sampling, and MMLU N=500 must
+stay within ±1 pp of the pre-patch SGLang result (0.780).
+
+The implementation is gated to Gemma-4 only, runs only with the triton
+attention backend, runs only in the eager EXTEND path (PCG bypass for v0),
+and is split into three stacked draft PRs on `pyc96/sglang`.
+
+## Acceptance Criteria
+
+- AC-1: A new server arg `--kv-sharing-fast-prefill` (default `False`)
+  is plumbed end-to-end from the CLI through `ServerArgs` and
+  `ModelConfig` to a `bool` attribute the model can read at forward time.
+  - Positive Tests (expected to PASS):
+    - `python -m sglang.launch_server --help` lists
+      `--kv-sharing-fast-prefill` in its help text.
+    - Server starts with `--kv-sharing-fast-prefill` and the server log
+      includes a line confirming the flag is enabled (e.g.
+      `"KV-sharing fast prefill enabled for <model_arch>"`).
+    - Server starts without the flag and logs no such confirmation
+      (default-off behavior preserved).
+  - Negative Tests (expected to FAIL):
+    - Server startup is rejected with a clear error if the flag is set
+      and the served model is not Gemma-4 (or has
+      `num_kv_shared_layers == 0`).
+    - Server startup is rejected with a clear error if the flag is set
+      and the attention backend is not `triton`.
+
+- AC-2: `Gemma4TextModel.forward` contains a predicate-gated YOCO branch
+  that runs only when all of the following are true: the flag is on, the
+  model has `num_kv_shared_layers > 0`, `forward_mode.is_extend()` is
+  true, the mode is not `TARGET_VERIFY`, `extend_seq_lens` is populated,
+  the batch is not under `is_in_piecewise_cuda_graph()`, the model has
+  no per-layer embedding (`hidden_size_per_layer_input == 0`), and no
+  request in the batch requests input logprobs.
+  - Positive Tests (expected to PASS):
+    - With flag on, EXTEND-mode forward of a Gemma-4 checkpoint
+      executes the YOCO branch (verifiable by a debug log or counter
+      that the test reads).
+    - With flag off, the same forward executes the existing layer loop
+      verbatim (no YOCO).
+    - With flag on but `forward_mode == DECODE`, the YOCO branch is
+      bypassed.
+    - With flag on but the batch is `TARGET_VERIFY`, the YOCO branch
+      is bypassed.
+  - Negative Tests (expected to FAIL):
+    - With flag on and a request that has input-logprob start <
+      extend_seq_len, the YOCO branch must be bypassed (taking the
+      branch would produce wrong input-logprob results).
+    - With flag on and the model has no KV-shared layers, the YOCO
+      branch must be bypassed.
+
+- AC-3: When the YOCO branch runs, it produces a final
+  `hidden_states: [T, H]` tensor whose per-request last-extend-token row
+  (`hidden_states[cumsum(extend_seq_lens) - 1]`) is bit-identical (or
+  within bf16 numerical noise) to the same row computed by the
+  non-YOCO path on the same input.
+  - Positive Tests (expected to PASS):
+    - A unit harness that runs `Gemma4TextModel.forward` twice on the
+      same input (once with YOCO on, once with YOCO off) and asserts
+      `torch.allclose(yoco_out[last_index], baseline_out[last_index],
+      atol=5e-3, rtol=5e-3)` for bf16.
+    - A 20-prompt end-to-end parity test that runs greedy sampling
+      (temperature=0, max_tokens=64) twice (YOCO on, off) and asserts
+      the generated token IDs are identical for each prompt.
+  - Negative Tests (expected to FAIL):
+    - A parity test where intentionally the wrong `last_index`
+      construction is used (e.g. off-by-one) must fail.
+    - A parity test where the attention metadata for the cross-decoder
+      phase incorrectly uses the original `qo_indptr` (instead of the
+      rebuilt decode-shaped one) must fail.
+
+- AC-4: The attention metadata used by the cross-decoder phase is a
+  decode-shaped triton `ForwardMetadata`: `qo_indptr = [0, 1, ..., B]`,
+  `max_extend_len = 1`, `kv_indptr` and `kv_indices` cover the full
+  prefix-plus-extend KV span per request. The original metadata is
+  restored on exit from the YOCO branch.
+  - Positive Tests (expected to PASS):
+    - A unit test that inspects the attention metadata in effect during
+      the cross-decoder phase and verifies the decode-shape invariants
+      above.
+    - A unit test that verifies the original metadata is restored after
+      the YOCO branch returns (so subsequent code paths see the same
+      `ForwardMetadata` instance / fields as if YOCO had not run).
+  - Negative Tests (expected to FAIL):
+    - A version that forgets to restore the metadata triggers a failing
+      assertion when the next forward pass tries to use stale state.
+    - A version that uses `qo_indptr = cumsum(extend_seq_lens)` (the
+      front-half plan) for the cross-decoder phase fails the cross-
+      decoder attention-output shape check.
+
+- AC-5: KV cache integrity is preserved: front-half layers write their
+  K/V into the pool exactly as today; back-half layers do not write
+  (`save_kv_cache=False` continues to hold because
+  `is_kv_shared_layer=True` for them).
+  - Positive Tests (expected to PASS):
+    - A unit test that snapshots `token_to_kv_pool` slot counts before
+      and after a YOCO-enabled forward and verifies the write count
+      equals (front-half layer count) × (token count) — i.e. zero
+      writes from the back half.
+    - A multi-step test that runs YOCO-prefill then a DECODE step and
+      verifies the decoded tokens match the non-YOCO baseline (proves
+      the KV pool state after YOCO is identical to baseline).
+  - Negative Tests (expected to FAIL):
+    - A version that mistakenly sets `save_kv_cache=True` for the
+      back-half layers must fail the snapshot test (write count would
+      include back-half writes).
+
+- AC-6: Quality bar — MMLU N=500 (seed 0, temp 0) on
+  `google/gemma-4-31B-it` with YOCO enabled is within ±1 pp of the
+  same configuration with YOCO disabled.
+  - Positive Tests (expected to PASS):
+    - Two MMLU runs: SGLang with YOCO on, SGLang with YOCO off. Both
+      complete N=500 with zero errors. The accuracy delta is ≤ 0.01
+      (within ±1 pp).
+  - Negative Tests (expected to FAIL):
+    - If accuracy drops by > 1 pp, the patch is rejected.
+
+- AC-7: Performance bar — on the fixed campaign workload
+  (`google/gemma-4-31B-it`, TP=2, H100, triton, NEXTN spec config 3/4/1,
+  warmup 2, seed 1), the summ 8000/1000 n=80 scenario achieves
+  `output_throughput ≥ 492 tok/s` with YOCO enabled (closes ≥ 30 % of
+  the gap between the current SGLang best 331 tok/s and vLLM's
+  868 tok/s).
+  - Positive Tests (expected to PASS):
+    - Bench result file `result_*_yoco_on_summ_8000_1000_n80_*.jsonl`
+      has `output_throughput >= 492.0`.
+    - Bench result on chat 1000/1000 n=80 with YOCO enabled does not
+      regress more than 1 % vs the current SGLang best 1499 tok/s
+      (i.e. `output_throughput >= 1484`).
+  - Negative Tests (expected to FAIL):
+    - If summ throughput < 492 tok/s after the final implementation,
+      the SOTA loop continues with another round of tuning before
+      stopping.
+    - If chat throughput regresses > 1 %, the patch is rejected and
+      either reverted or fixed.
+
+- AC-8: Three stacked draft PRs land in `pyc96/sglang` (never
+  `sgl-project/sglang`, never submitted), each with its own benchmark
+  comparison and quality-score table in the PR body.
+  - Positive Tests (expected to PASS):
+    - `gh pr list --repo pyc96/sglang --state open --search "yoco"`
+      returns three drafts.
+    - Each PR body contains a benchmark delta table and an MMLU score
+      table.
+    - Each PR is built on top of the prior one (PR-B base = PR-A head,
+      PR-C base = PR-B head).
+  - Negative Tests (expected to FAIL):
+    - Any PR submitted (non-draft) is a contract violation.
+    - Any PR opened against `sgl-project/sglang` is a contract violation.
+    - PR with empty body or missing the bench/MMLU table is rejected
+      from "done".
+
+## Path Boundaries
+
+### Upper Bound (Maximum Acceptable Scope)
+
+The implementation:
+- Adds the `--kv-sharing-fast-prefill` server arg with all eligibility
+  checks (Gemma-4 architecture allow-list, triton attention backend
+  required, mutual-exclusion with EAGLE/EAGLE3 speculative algorithms,
+  rejection of incompatible combos at startup).
+- Implements the YOCO branch inside `Gemma4TextModel.forward` with full
+  predicate gating, triton-only fast-path attention metadata builder, and
+  always-scatter-to-`[T, H]` output contract.
+- Adds a unit test for the model branch (predicate, gather/back/scatter
+  semantics, attention-metadata invariants) plus a per-prompt parity
+  test (20 prompts × greedy sampling × YOCO on/off, assert byte-identical
+  token IDs).
+- Runs the full fixed-benchmark suite (chat 1000/1000 n=80, summ
+  8000/1000 n=80, MMLU N=500) on SGLang YOCO-on, SGLang YOCO-off, and
+  vLLM nightly MTP, and embeds the result table in each draft PR body.
+- Documents the new arg in the user-facing help text and in
+  `python/sglang/srt/server_args.py` docstrings.
+
+### Lower Bound (Minimum Acceptable Scope)
+
+The implementation:
+- Adds the `--kv-sharing-fast-prefill` server arg as a plain bool flag
+  on `ServerArgs` and `ModelConfig`, default False, no startup-time
+  validation beyond a single guard at the YOCO branch site.
+- Implements the YOCO branch inside `Gemma4TextModel.forward` with the
+  predicate at AC-2, the always-scatter output contract, and a minimal
+  triton metadata rebuild.
+- Adds one parity test (20-prompt greedy, YOCO on vs off) and one
+  benchmark comparison (summ 8000/1000 n=80 with YOCO on vs off vs
+  vLLM).
+- Stages one draft PR (instead of three stacked) on `pyc96/sglang`
+  containing all the changes.
+
+### Allowed Choices
+
+- Can use:
+  - The existing `LogitsProcessor._get_pruned_states` index construction
+    pattern (`cumsum(extend_seq_lens) - 1`).
+  - Direct construction of a triton `ForwardMetadata` dataclass inside
+    the model for the cross-decoder phase (the user is pinned to triton,
+    so backend-agnosticism is not required).
+  - A small helper module/file under `python/sglang/srt/models/` or
+    `python/sglang/srt/layers/` if the YOCO logic is large enough to
+    warrant separation from `gemma4_causal.py`.
+  - Python-level branching inside `Gemma4TextModel.forward` (no Triton
+    kernel rewrite is required).
+  - Stacked PRs (PR-A: flag plumbing; PR-B: model + metadata + tests;
+    PR-C: bench results + any tuning) per the user's "multiple PRs per
+    task" allowance.
+- Cannot use:
+  - Any change to the existing non-YOCO layer loop's semantics
+    (default-off must be a no-op).
+  - Any change to the KV cache layout or to `req_to_token_pool` /
+    `token_to_kv_pool` data structures.
+  - Any change to attention backends other than triton (flashinfer,
+    fa3, trtllm_mha are out of scope for v0).
+  - Any change to multimodal towers (vision/audio); YOCO must coexist
+    with the existing `mm_disabled_models` treatment without changes.
+  - Any direct dependency on vLLM source code or vLLM-specific
+    abstractions (we mirror the technique, not the implementation).
+  - Any PR submitted to `sgl-project/sglang` upstream; all PRs draft on
+    `pyc96/sglang` only.
+  - Any PR that lacks a benchmark+MMLU table in its body.
+
+## Feasibility Hints and Suggestions
+
+### Conceptual Approach
+
+```text
+ServerArgs (CLI: --kv-sharing-fast-prefill, default False)
+    └─→ ModelConfig.kv_sharing_fast_prefill
+            └─→ Gemma4TextModel.__init__ reads it, stores
+                self.kv_sharing_fast_prefill_enabled
+
+Gemma4TextModel.forward(input_ids, positions, forward_batch, ...):
+    if can_run_yoco(self, forward_batch):
+        # YOCO branch
+        hidden_states = self.embed_tokens(input_ids)
+        # Front half: layers [0, first_kv_shared_layer_idx)
+        for layer_idx in range(0, self.first_kv_shared_layer_idx):
+            hidden_states = self.layers[layer_idx](positions, hidden_states,
+                                                   forward_batch)[0]
+
+        # Gather index
+        last_index = torch.cumsum(forward_batch.extend_seq_lens, dim=0) - 1
+
+        # Gather + cross-decoder + scatter
+        front_out = hidden_states.clone()  # save full-shape for scatter
+        gathered_h = front_out[last_index]
+        gathered_pos = positions[last_index]
+
+        # Build decode-shaped attn metadata for back half, restore on exit
+        with self._cross_decoder_attn_metadata_scope(forward_batch):
+            cross_h = gathered_h
+            for layer_idx in range(self.first_kv_shared_layer_idx, len(self.layers)):
+                cross_h = self.layers[layer_idx](gathered_pos, cross_h,
+                                                 forward_batch)[0]
+
+        # Scatter back
+        front_out.index_copy_(0, last_index, cross_h)
+        hidden_states = front_out
+
+        # Final norm + LM head proceeds as usual on hidden_states
+    else:
+        # Existing path verbatim
+        ...
+    hidden_states = self.norm(hidden_states)
+    return hidden_states
+```
+
+The `_cross_decoder_attn_metadata_scope` is a `contextmanager` that:
+1. Saves the current `forward_batch.attn_backend.forward_metadata`.
+2. Builds a new `ForwardMetadata` instance with `qo_indptr=[0..B]`,
+   `max_extend_len=1`, `kv_indptr=cumsum(seq_lens)`, `kv_indices`
+   pointing at the full per-request KV slots from the pool.
+3. Yields.
+4. Restores the saved metadata.
+
+For triton specifically, look at `triton_backend.py:298-363` (DECODE
+branch of `init_forward_metadata`) for the exact field shapes and the
+SWA buffer setup.
+
+### Relevant References
+
+- `python/sglang/srt/models/gemma4_causal.py:922-1003` — the existing
+  `Gemma4TextModel.forward` layer loop that this patch modifies.
+- `python/sglang/srt/models/gemma4_causal.py:360-403` — KV-shared
+  layer detection logic (`is_kv_shared_layer`, `kv_shared_layer_index`)
+  that the new branch piggybacks on.
+- `python/sglang/srt/layers/logits_processor.py:432-447` — reference
+  construction of `last_index = cumsum(extend_seq_lens) - 1`.
+- `python/sglang/srt/layers/attention/triton_backend.py:298-363, 437-482` —
+  DECODE and EXTEND branches of `init_forward_metadata` (the templates
+  for the metadata builder).
+- `python/sglang/srt/model_executor/forward_batch_info.py:273-432` —
+  `ForwardBatch` field list (especially `extend_seq_lens`, `seq_lens`,
+  `out_cache_loc`, `req_pool_indices`, `forward_mode`,
+  `capture_hidden_mode`).
+- `python/sglang/srt/server_args.py:2205-2277` — `_handle_model_specific_adjustments`
+  for Gemma-4 (existing site for any Gemma-4-specific arg validation).
+- `python/sglang/srt/configs/model_config.py:215-244` — `mm_disabled_models`
+  and the existing `Gemma4ForConditionalGeneration` handling that the new
+  flag must coexist with.
+- `python/sglang/srt/speculative/frozen_kv_mtp_worker.py:493-503` — the
+  worker's `forward_target_extend` entry, which is the call site that
+  benefits from YOCO when MTP is on.
+- `vllm/model_executor/models/gemma4.py:759-952, 1190-1273` — vLLM
+  reference impl for `Gemma4SelfDecoderLayers`, `Gemma4CrossDecoderLayers`,
+  and `fast_prefill_forward`. **For inspiration only; do not copy code
+  directly — the SGLang port must use SGLang's own framework abstractions.**
+- `vllm/v1/attention/backends/utils.py:367-433, 728-776` — vLLM
+  reference for `make_kv_sharing_fast_prefill_common_attn_metadata` and
+  `create_fast_prefill_custom_backend`. **For inspiration only.**
+- `agent-pod/runs/20260522_gemma4_26b_a4b_it_h100_sota_humanize/analysis/framework-gap-investigation.md` —
+  prior D1 attempt that documents the PCG/Inductor incompatibility we
+  must work around (`is_in_piecewise_cuda_graph()` guard).
+- `agent-pod/runs/20260523_gemma4_31b_it_h100_sota_humanize/yoco/draft.md` —
+  the draft document this plan refines.
+
+## Dependencies and Sequence
+
+### Milestones
+
+1. Milestone 1 — Flag plumbing (PR-A on `pyc96/sglang`)
+   - Phase A: Add `kv_sharing_fast_prefill: bool = False` to
+     `ServerArgs` with `argparse` registration and help text.
+   - Phase B: Plumb the flag through `ModelConfig` (or wherever the
+     model can read it at construction time).
+   - Phase C: Add a startup-time validator in
+     `_handle_model_specific_adjustments` (or equivalent) that rejects
+     incompatible combinations (non-Gemma-4 arch, non-triton attn
+     backend, EAGLE/EAGLE3 spec algo).
+   - Phase D: Add a unit/integration test that asserts the flag appears
+     in `--help`, parses correctly, and the validator rejects bad combos.
+
+2. Milestone 2 — Model branch + attention metadata (PR-B, stacked on PR-A)
+   - Phase A: Add an `is_in_yoco_scope()` predicate helper.
+   - Phase B: Add a `_cross_decoder_attn_metadata_scope` context manager
+     (triton-only) that saves, rebuilds, and restores the attention
+     metadata for the back-half phase.
+   - Phase C: Add the YOCO branch inside `Gemma4TextModel.forward`
+     (gather + back-half loop + scatter).
+   - Phase D: Wire the branch behind the flag-on predicate.
+   - Phase E: Add a unit test for `_cross_decoder_attn_metadata_scope`
+     (asserts shapes, restoration).
+   - Phase F: Add the per-prompt parity test (20 prompts, greedy,
+     YOCO on vs off, assert identical token IDs).
+   - Phase G: Add a `Gemma4TextModel.forward` unit test that runs both
+     paths on the same input and asserts allclose on the last-token row.
+
+3. Milestone 3 — Benchmark + landing (PR-C, stacked on PR-B)
+   - Phase A: Run the fixed benchmark suite (chat + summ + MMLU) with
+     YOCO on, YOCO off, and vLLM nightly MTP, all on the campaign
+     hardware (H100 TP=2).
+   - Phase B: Embed the result table in the PR-C body.
+   - Phase C: If summ throughput < 492 tok/s, do one round of tuning
+     (e.g. micro-optimize the gather, try `index_select` vs fancy
+     indexing, profile and address any hot spot) and re-bench.
+   - Phase D: Update the campaign's `model-loop-checkpoint.md` and
+     `final_report.md` with the new "best SGLang" row.
+
+Dependencies between components:
+- PR-B depends on PR-A's flag being readable from the model.
+- PR-C depends on PR-B's correctness tests passing.
+- All three PRs share the same base branch `pyc/sota-gemma4-31b-mm-disabled`
+  (stacked).
+
+## Task Breakdown
+
+| Task ID | Description | Target AC | Tag | Depends On |
+|---------|-------------|-----------|------|------------|
+| task1  | Add `kv_sharing_fast_prefill` to `ServerArgs` + argparse + help text | AC-1 | coding | - |
+| task2  | Plumb flag through `ModelConfig` so models can read it at `__init__` | AC-1 | coding | task1 |
+| task3  | Startup validators (Gemma-4 arch, triton backend, no EAGLE/EAGLE3 with the flag) | AC-1 | coding | task2 |
+| task4  | Unit test for flag plumbing + validator rejection of bad combos | AC-1 | coding | task3 |
+| task5  | Open PR-A (flag plumbing only) on `pyc96/sglang` (draft) | AC-8 | coding | task4 |
+| task6  | Implement `_cross_decoder_attn_metadata_scope` context manager (triton-only) | AC-4 | coding | task5 |
+| task7  | Add `can_run_yoco` predicate function | AC-2 | coding | task5 |
+| task8  | Implement YOCO branch in `Gemma4TextModel.forward` (gather + back-half + scatter) | AC-2, AC-3, AC-5 | coding | task6, task7 |
+| task9  | Unit test for `_cross_decoder_attn_metadata_scope` (shape + restoration) | AC-4 | coding | task6 |
+| task10 | Unit test for `Gemma4TextModel.forward` allclose on last-token row, YOCO on vs off | AC-3 | coding | task8 |
+| task11 | Per-prompt parity test: 20 prompts greedy, YOCO on vs off, identical token IDs | AC-3 | coding | task8 |
+| task12 | Unit test for KV cache write-count invariant | AC-5 | coding | task8 |
+| task13 | Open PR-B (model + metadata + tests) stacked on PR-A (draft) | AC-8 | coding | task9, task10, task11, task12 |
+| task14 | Run MMLU N=500 with YOCO on and YOCO off, capture results | AC-6 | coding | task13 |
+| task15 | Run the fixed benchmark (chat + summ) on YOCO on, YOCO off, and vLLM | AC-7 | coding | task13 |
+| task16 | If summ throughput < 492 tok/s, do one tuning round and rebench | AC-7 | coding | task15 |
+| task17 | Open PR-C (bench + tuning) stacked on PR-B (draft) with results table | AC-7, AC-8 | coding | task16 |
+| task18 | Update campaign `model-loop-checkpoint.md` and `final_report.md` | AC-7 | coding | task17 |
+| task19 | Final review pass: confirm all three PRs are draft, all contain bench/MMLU tables, none submitted | AC-8 | analyze | task17 |
+
+## Claude-Codex Deliberation
+
+### Agreements
+- The YOCO branch must be gated behind a predicate that excludes
+  DECODE, TARGET_VERIFY, PCG-active forwards, input-logprob requests,
+  and models with PLE.
+- Default-off is the safer first cut; can be flipped to default-on for
+  Gemma-4 after correctness is established.
+- Triton-only metadata rebuild is acceptable because the user is pinned
+  to triton.
+
+### Resolved Disagreements
+- None at plan-generation time. Codex review will surface any during the
+  RLCR loop.
+
+### Convergence Status
+- Final Status: `converged` (user-confirmed all open questions; one-shot
+  draft → user-approved → plan generated).
+
+## Pending User Decisions
+
+(All decisions resolved before plan generation; none pending.)
+
+- DEC-1: Default value of `--kv-sharing-fast-prefill` — RESOLVED:
+  `False` (opt-in).
+- DEC-2: Hidden-mode handling — RESOLVED: always scatter back to `[T, H]`.
+- DEC-3: Attention metadata strategy — RESOLVED: triton-only fast path.
+- DEC-4: Validation scope — RESOLVED: MMLU + per-prompt parity test.
+- DEC-5: Port scope — RESOLVED: Gemma-4 only.
+- DEC-6: PR strategy — RESOLVED: three stacked draft PRs.
+- DEC-7: SOTA-loop stop criterion — RESOLVED: summ tok/s gap closes
+  by ≥ 30 % (≥ 492 tok/s).
+
+## Implementation Notes
+
+### Code Style Requirements
+- Implementation code and comments must NOT contain plan-specific
+  terminology such as "AC-", "Milestone", "Step", "Phase", or similar
+  workflow markers. These terms are for plan documentation only, not
+  for the resulting codebase.
+- Use descriptive, domain-appropriate naming: e.g.
+  `enable_kv_sharing_fast_prefill`, `cross_decoder_attn_metadata_scope`,
+  `gather_last_token_index`.
+- Follow existing SGLang code style: lower_snake_case for functions,
+  CapWords for classes, ALL_CAPS for module-level constants.
+- Add a brief docstring on the YOCO branch citing the inspiration source
+  (vLLM Gemma-4 fast-prefill) and noting the predicate semantics.
+- Do not import from `vllm.*` anywhere in the SGLang code.
+- All new code paths must have a corresponding unit test, with at least
+  one positive and one negative case per acceptance criterion.
+
+### Branching and PR Conventions
+- All branches branch from `pyc/sota-gemma4-31b-mm-disabled` (the current
+  best base for the 31b-it campaign).
+- PR-A branch name: `pyc/yoco-fast-prefill-config`.
+- PR-B branch name: `pyc/yoco-fast-prefill-impl`, stacked on PR-A.
+- PR-C branch name: `pyc/yoco-fast-prefill-bench`, stacked on PR-B.
+- All three opened as draft via `gh pr create --repo pyc96/sglang --draft`.
+- No `git push` to `sgl-project/sglang` from any branch.