Skip to content

worker: chunked-prefill interleaving — +25-35% batched throughput, up to 60% lower TTFT under load#37

Merged
hvasconcelos merged 2 commits into
masterfrom
chunked-prefill-experiment
Jun 11, 2026
Merged

worker: chunked-prefill interleaving — +25-35% batched throughput, up to 60% lower TTFT under load#37
hvasconcelos merged 2 commits into
masterfrom
chunked-prefill-experiment

doc: whitepaper covers quantized KV storage and the prefix cache

eb44b71
Select commit
Loading
Failed to load commit list.
Sign in for the full log view

Annotations

1 error and 1 warning
Build & test (Apple Silicon)
failed Jun 11, 2026 in 43m 8s