Profile-guided layout optimization for llama.cpp GGUF MoE models — BOLT/PGO for LLM weights. Trace expert routing, rewrite the file so co-activated experts sit together, prefetch as merged reads.

Low-cost AMD/ROCm stationary-expert fabric for configurable trillion-parameter sparse MoE inference, with packed-INT4 HIP kernels, activation-only RoCE transport, hardware/BOM design, release gates, and reproducible roofline tooling.

Simulates Mixture-of-Experts routing strategies (TopK, NoisyTopK, ExpertChoice, Hash) across 432 configurations. Key findings: ExpertChoice achieves best practical balance (imbalance=1.40 vs TopK=5.52); NoisyTopK reduces drops by 40% on skewed distributions; capacity factor is the strongest control knob.

Hand-built MoE expert parallelism: routing-skew capture, fused grouped-GEMM Triton kernels, 2-all2all dispatch/combine, overlap ablations — proving by counter-example why DeepEP exists.