feat: HuggingFace Space live demo (Gradio, compress-as-you-type)

[OnlyTerp]

Summary

Adds spaces/turboquant-demo/, a ready-to-deploy Gradio Space. The single
biggest driver of retweets for inference/quant tools is "someone types their
model + context length, instantly sees GB saved, and has a copy-pasteable
install to act on it." That's exactly this.

Two tabs:

🧮 KV memory calculator — Pick a model (Llama-3.1-8B/70B, Llama-4 Scout,
Qwen3-72B, Qwen3.5-27B, DeepSeek-V3 MLA, Mistral-Nemo-12B, Gemma-2-27B), set
a context length (1K..1M) and batch size, see FP16 vs TurboQuant 3.5-bit vs
2.5-bit KV cache memory side-by-side.

Example results (verified locally):

• Llama-3.1-8B @ 32K × 1 → 4.29 GB FP16 → 1.24 GB @ 3.5-bit (3.46×)
• Qwen3-72B @ 256K × 1 → 85.90 GB FP16 → 24.83 GB @ 3.5-bit

🔬 Live compression demo — Real TurboQuantCache.store +
compute_attention on random KV vectors, sliders for b_mse / b_outlier /
n_outlier / seq_len / n_queries / seed. Reports avg cosine against FP16
scaled_dot_product_attention as ground truth. 3.5-bit default: ~0.976 avg
cosine on random Gaussians (hardest case). 2.5-bit: ~0.921.

Files

• spaces/turboquant-demo/app.py (new, ~280 LOC) — full Gradio Blocks UI,
  both tabs, eager-compute on load so the page isn't blank.
• spaces/turboquant-demo/requirements.txt (new) — turboquant>=0.2.0,
  torch>=2.1, numpy, gradio>=5.0.
• spaces/turboquant-demo/README.md (new) — HF Space metadata front-matter
  (sdk=gradio, emoji=⚡, pinned, mit), tags (iclr2026, long-context),
  deploy instructions (huggingface-cli repo create --type space → push),
  local-run snippet.
• README.md — top-of-TOC link so the main repo landing page surfaces the
  Space prominently.

Verified locally

python -c "from app import memory_calculator, live_demo; ..."
# Llama-3.1-8B @ 32K × 1 -> 4.29 GB FP16 / 1.24 GB 3.5-bit (3.46×) / 0.97 GB 2.5-bit
# Qwen3-72B @ 256K × 1   -> 85.90 GB FP16 / 24.83 GB 3.5-bit / 19.46 GB 2.5-bit
# 3.5-bit live demo      -> 0.9763 avg cosine (min 0.9654)
# 2.5-bit live demo      -> 0.9212 avg cosine (min 0.8936)
python app.py  # -> http://127.0.0.1:7860 returns HTTP 200

Review & Testing Checklist for Human

Yellow risk — the Gradio app is a standalone directory, no production path
is affected. But deployment is manual and the model-zoo numbers are
public-spec estimates.

• Confirm the HF Space namespace — README.md links to
  huggingface.co/spaces/OnlyTerp/turboquant-demo. If you'd rather deploy
  under a personal username, update that link and the git remote add origin
  line in spaces/turboquant-demo/README.md.
• Deploy to HuggingFace Spaces (≤3 min): follow the snippet at the
  bottom of spaces/turboquant-demo/README.md. Uses your existing
  huggingface-cli login — no new secrets.
• Cross-check a few model specs in the model-zoo table inside
  app.py (n_kv_heads, n_layers, d_head), especially for Llama-4 Scout
  and Qwen3.5 where a few public specs are still ambiguous.

Test plan (end-to-end, ≤5 min):

git checkout devin/1776403661-live-demo
pip install gradio
pip install -e .
python spaces/turboquant-demo/app.py
# -> open http://127.0.0.1:7860
#   Tab 1: change model / context / batch, click Compute; numbers update.
#   Tab 2: drag b_mse / b_outlier sliders, click Run TurboQuant;
#          verify cosine similarity ≥ 0.95 at b_mse=3, b_outlier=4.

Notes

• Does not depend on PR feat: PyPI-ready packaging — pyproject.toml, OIDC publish workflow, v0.2.0 #3 being on PyPI yet. The Space
  requirements.txt pins turboquant>=0.2.0; once v0.2.0 is tagged and
  published, the Space install is pip install turboquant gradio. In the
  interim, the Space can pin to the git URL
  (git+https://github.com/OnlyTerp/turboquant) if you want to deploy before
  PyPI.
• Not a replacement for the notebook — notebooks/demo.ipynb is still
  the read-through-the-algorithm path. The Space is "I just want the number
  for my model."
• Zero new backend CI surface — the Space is a flat static directory,
  not wired into the main repo's test.yml. Deploy is a one-shot push.

Link to Devin session: https://app.devin.ai/sessions/6e4d375a49bd47568b372d133d960daf
Requested by: @OnlyTerp

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[devin-ai-integration]

End-to-end runtime test — all 4 scenarios passed

Tested spaces/turboquant-demo/app.py at eef4577 by booting python app.py locally against http://127.0.0.1:7860 and driving both tabs in a real browser. Recording + full report attached below.

Session: https://app.devin.ai/sessions/6e4d375a49bd47568b372d133d960daf
Recording: https://app.devin.ai/attachments/c1830e75-d54d-4a3a-9ae0-0125f55f64ff/rec-057b6036-2e02-415f-8443-dc15f149e3c8-edited.mp4

Escalations: none.

#	Test	Result
1	KV calculator pre-populates on page load (Llama-3.1-8B @ 32K × 1)	✅ passed
2	Switching to Qwen3-72B @ 256K recomputes memory	✅ passed
3	Live TurboQuant demo runs at 3.5-bit defaults without hanging	✅ passed
4	n_outlier=4 lower bound does not crash (d=1 degenerate-density fix)	✅ passed

Test 3 + 4 — Live compression demo (the actual thing being tested)

Test 3 — defaults (b_mse=3, b_outlier=4, n_outlier=32, seq=64, q=16, seed=42):

• Avg cosine similarity: 0.9763 (≥ 0.95 paper threshold)
• Min/Max cosine: 0.9654 / 0.9833
• Avg MSE: 1.93e-03
• Effective bpv: 4.62 · Compression: 3.46× · Bytes/128-d vector: 74

Test 4 — n_outlier=4 boundary (directly exercises commit 3bfb8d4):

• No crash, no Gradio error toast
• Avg cosine: 0.9672 (in [0.90, 1.00])
• Min cosine: 0.9548 · bpv: 4.44 · ratio: 3.61×

Before the fix this path hit _beta_pdf(grid, d=1) → lgamma(0) = -∞ → ValueError: Degenerate density for d=1 and 500'd the Gradio call. Now cleanly bounded.

Test 1 — Memory calculator pre-populates on page load

• FP16: 4.29 GB (1.00×) · 3.5-bit: 1.24 GB (3.46×) · 2.5-bit: 0.97 GB (4.41×)
• Summary: "saves 3.1 GB of KV cache memory vs FP16"
• Confirms demo.load fix (commit fb33a64) — no Chrome "Page Unresponsive" hang on hydration.

Test 2 — Switch to Qwen3-72B @ 256K × 1

Dropdown showing the corrected Llama-4-Scout label ("16 experts, 1 active", commit eef4577):

After picking Qwen3-72B, typing 262144, and clicking Compute:

• FP16: 85.90 GB · 3.5-bit: 24.83 GB (3.46×) · 2.5-bit: 19.46 GB (4.41×)
• Summary: "At 262,144 tokens × batch 1, TurboQuant 3.5-bit saves 61.1 GB"
• Header: "Qwen3-72B" · Note: "GQA 8 KV heads. 256K context target."

Incidental regressions checked during the run

• ✅ Llama-4-Scout dropdown label reads "16 experts, 1 active" (commit eef4577).
• ✅ n_outlier slider label reads "# outlier channels; 4..124, d-head=128" (clamped, commit 3bfb8d4).
• ✅ b_outlier slider label reads "total bits for outlier channels" (original-PR Devin Review fix).
• ✅ Page loads cleanly with no "Page Unresponsive" dialog (commit fb33a64).
• ✅ Defensive int() casts on all sliders hold (commit 863cb20) — no range(2**3.0) TypeError path triggered.

Happy to dig deeper on any specific edge case before merge.