RTX PRO 6000 Blackwell LLM Wiki

This repository is a field wiki for running frontier LLMs on NVIDIA RTX PRO 6000 Blackwell / SM120 PCIe systems. It is more than a few launch snippets: it contains reproducible Docker builds, exact vLLM and SGLang runbooks, benchmark tables, KLD quality checks, quantization notes, DCP/MTP/DSpark/DFlash debugging, PCIe topology work, and regression history.

Community workbench for RTX PRO 6000 / SM120 serving: https://discord.gg/X54jjmcxWJ

Start Here

If you just want to run a model, use these stable hub pages first:

Model family	Start here	Current focus
GLM-5.2	GLM-5.2 Runbook Hub	Fathomless vLLM, NVFP4, online FP8/MXFP8, B12X, DCP, MTP, KLD.
DeepSeek-V4-Flash / DSpark	DeepSeek-V4-Flash Runbook Hub	Standard checkpoint, MTP, DSpark, B12X, Lucifer, CUTLASS.
Kimi	Kimi Runbook Hub	Kimi-K2.7-Code, DFlash, parser/tool-call runtime.
Xiaomi MiMo	MiMo Runbook Hub	MiMo V2.5 Pro FP4-DFlash.
GLM-5.1	GLM-5.1 Runbook Hub	Historical GLM-5.1, KLD methodology, older B12X/SGLang work.
Legacy / secondary models	Legacy Model Runbooks	DeepSeek-V4-Pro, GLM-4.7, Qwen, MiniMax, older Kimi pages.

Need the complete map of every Markdown page?

Index	Use
Full Wiki Index	Complete generated catalog of every page in this repository.
Glossary And Acronym Guide	Acronym expansions and writing rules for newcomer-friendly docs.
Newcomer Onboarding	How to ask useful questions without lowering the technical signal.

What Is In This Repository?

Need	Where
Copy/paste production launch commands	Model hubs and current versioned model pages.
Rebuild the Docker image	Eldritch Docker, current model image sections, and build scripts in scripts.
Compare backend speed	Model benchmark tables plus Benchmark Results.
Check quantization quality	GLM-5.2 KLD, KLD Evaluation, and model-specific KLD sections.
Understand MTP, DSpark, or DFlash	Speculative Decoding, DS4/Kimi/MiMo pages.
Debug topology or PCIe behavior	Topology, PCIe Bandwidth, GPU Configurations.
Avoid known runtime footguns	Common Issues, model caveats, and daily summaries.
Understand old measurements	Historical versioned pages and Daily Summaries.

Current Production-Style Pages

Area	Page	Why it matters
GLM-5.2 current stack	GLM-5.2 v20	Current Gilded Gnosis/SparkInfer image, source pins, TP6 fixes, Xid validation, and release gate.
GLM-5.2 MXFP4	GLM-5.2 FP8 + MXFP4 Experts	Native MXFP4 expert checkpoint path and A8 serving notes.
DS4 current quick start	DS4 DSpark v10	Recommended TP2 launch defaults and Fathomless validation.
DS4 full reference	DS4 DSpark v9	Full DSpark and standard MTP sweep reference.
Kimi-K2.7-Code	Kimi-K2.7-Code v3	Fathomless Kimi DFlash validation.
MiMo FP4-DFlash	MiMo FP4-DFlash v3	Current MiMo DFlash validation and fix notes.

Older pages are intentionally preserved. Prefer the hub page for each model family unless you are reproducing a specific old result.

Core Topics

Topic	Page
Docker images and release lines	Docker Images
PCIe oneshot all-reduce	PCIe oneshot all-reduce
NCCL tuning and empty graph-file failures	NCCL tuning
Speculative decoding	Speculative decoding
NVFP4 quantization	NVFP4 quantization
Hybrid NVFP4 assembly	Hybrid NVFP4 assembly
B12X FP8 / DeepGEMM comparison	B12X dense FP8 GEMM vs DeepGEMM
B12X W4A8 tiny decode	B12X W4A8 MX tiny decode
DSpark upstream consolidation	DSpark upstream consolidation
I/O tuning	I/O tuning

Benchmarks And Quality

Area	Page
Consolidated throughput	Benchmark Results
vLLM vs SGLang throughput	Inference throughput
GLM-5.2 KLD and quant quality	GLM-5.2 KLD Evaluation
General KLD methodology	KLD Evaluation
MTP quality checks	MTP Quality Evaluation
NVFP4 quantization comparison	NVFP4 Quantization Comparison

KLD is a regression and quantization-sanity tool, not a complete quality metric. Use it together with long-context decode, coding probes, acceptance-rate checks, and task-level benchmarks.

Hardware And Topology

Most current measurements target RTX PRO 6000 Blackwell / GB202 / SM120 cards: 96 GB GDDR7 per GPU, PCIe 5.0 x16, no NVLink, usually 4-GPU, 8-GPU, or 16-GPU PCIe-switch systems.

Area	Page
SM120 vs SM100	SM120 vs SM100 Architecture
PCIe topology	Topology
PCIe bandwidth	PCIe Bandwidth
GPU configs	GPU Configurations
ASUS ESC8000A-E13P	ASUS ESC8000A-E13P + Broadcom Switches
ASRockRack Turin 16 GPU	ASRockRack + EPYC Turin + 4x c-payne
ASRock WRX90 16 GPU	ASRock WRX90 + 4x c-payne
Power tuning	Blackwell power limit sweep

Inference Engines

Engine	Page	Current role
vLLM	vLLM	Primary runtime for current GLM-5.2, DS4, Kimi, and MiMo pages.
FlashInfer	FlashInfer	SM120 sparse MLA, CUTLASS MoE, sampler, and kernel integration notes.
SGLang	SGLang	Historical and alternate runtime notes, especially older GLM/MiMo paths.

Acronym Policy

The wiki uses many acronyms: DCP, MTP, DSpark, DFlash, MLA, MoE, KLD, TP, CC, P2P, NVFP4, MXFP8, and more. To make pages readable for newcomers:

• Expand important acronyms on first use: Decode Context Parallelism (DCP).
• Do not expand acronyms inside commands, Docker tags, environment variables, JSON, file paths, or raw logs.
• Use Glossary And Acronym Guide as the source of truth.
• Run python3 scripts/check-acronyms.py before polishing a major page.

Keeping The Community Useful

The wiki should reduce accidental gatekeeping: newcomers should be able to find the right runbook, decode acronyms, and reproduce a known-good launch without needing tribal knowledge. That does not mean every Discord question can be answered from memory.

Use Newcomer Onboarding as the support contract: bring the model page, Docker image, full launch command, GPU layout, TP/DCP/MTP or DSpark settings, client command, and logs. That keeps the server welcoming without turning it into an unstructured support queue.

Common Operational Rules

• Do not launch with NCCL_GRAPH_FILE= set to an empty string. Unset it if no real XML graph file is used.
• Reuse cache directories while debugging; otherwise TileLang, Triton, CuTe, and FlashInfer rebuilds dominate iteration time.
• For quick smoke tests, use small MAX_NUM_SEQS and graph caps. For published tables, use the graph sizes documented in the model page.
• For DFlash and DSpark, confirm backend markers and acceptance rates before trusting throughput numbers.
• For GLM-5.2, keep the exact index_topk_pattern and DCP policy from the relevant runbook; a truncated pattern can silently degrade output.

Maintaining The Wiki

When adding a page:

• Link it from the relevant model hub.
• Add a short status block if it is a current runbook.
• Keep exact Docker image tags, source commits, model snapshot IDs, GPU layout, backend choices, and benchmark commands.
• Regenerate the full index:

python3 scripts/generate-wiki-index.py > INDEX.md

For performance claims, include both the server launch config and the client command so results can be reproduced on another PCIe-only Blackwell host.

Maintained from community Discord experiments through July 2026.