llama.cpp fork: TurboQuant KV cache (ktq+vtq, 1-4 bit, 83% smaller at f16 quality, 100k+ ctx on 12GB GPU) + sparse fine-tuning for hybrid MoE+SSM models (Qwen3.5/3.6-A3B, no Mamba backward needed). Turing-tuned Vulkan path.
vulkan cuda inference moe lora quantization ssm mamba fine-tuning kv-cache llm llama-cpp gguf trellis-quantization turboquant polarquant
-
Updated
May 18, 2026 - C++