SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.

Giant MoE models on a single consumer GPU by streaming experts from SSD. CUDA fork of antirez/ds4: runs GLM-5.2 (743B), Tencent Hy3 (295B), and DeepSeek 4 Flash, with io_uring expert streaming, LFU host cache, cross-layer expert prefetch, and the first MTP speculative decoding for GLM-5.2.

Open-source Metal-first inference engine for Apple Silicon with adaptive SSD streaming.

Run AI models too large for your Mac's memory — at near-full speed. Intelligent expert caching, speculative execution, and 15+ research techniques for MoE inference on Apple Silicon.

Run Qwen3.5-122B-A10B on a 16 GB MacBook Air via SSD-streamed MoE expert weights.

SSD-streaming MoE inference engine for consumer hardware. Run 80B parameter models on a 24GB Mac.

Profile-guided layout optimization for llama.cpp GGUF MoE models — BOLT/PGO for LLM weights. Trace expert routing, rewrite the file so co-activated experts sit together, prefetch as merged reads.

Run large MLX models on Apple Silicon with flash weight streaming, using native precision beyond RAM limits

Correctness-first DS4 fork with native dFlash speculative decoding, DeepSpec integration, CUDA low-memory inference, SSD streaming, continual learning, and a bilingual TUI.

Qwen3.6-35B-A3B inference engine for Vulkan