⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
-
Updated
May 19, 2026 - Swift
#
apple-sili
Here are 2 public repositories matching this topic...
Optimize LLM inference with near-optimal 4-bit weight quantization and on-the-fly dequantization for lower memory use and faster matmul
desktop-app python compression ai metal cuda pytorch transformer attention multi-model mlx rocm inference-optimization huggingface on-device-ai kv-cache llm vllm quansloth apple-sili
-
Updated
May 26, 2026 - Python
Improve this page
Add a description, image, and links to the apple-sili topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the apple-sili topic, visit your repo's landing page and select "manage topics."