Community benchmark database for running LLMs on Apple Silicon Macs

Pi extension that measures and displays model TPS in the status bar

Hands-on CPU vs GPU benchmarks for Apple Silicon (M-series): PyTorch MPS, TensorFlow-Metal, MLX, and llama.cpp to measure TFLOP/s & tokens/sec and learn why GPUs accelerate training.

Speedtest for AI. Test latency to every major AI provider from your terminal.

Local LLM inference benchmarker. Measures TPS, TTFT, and VRAM pressure across context sizes — from 2K to 256K tokens. Works with llama.cpp, Ollama, LM Studio, and any OpenAI-compatible server.

Dynamically benchmarks every model on NVIDIA NIM — type, context window, reasoning/cache, TTFT & token rate — and renders an interactive, dated HTML report.

Real-time terminal dashboard for Ollama — live output token counting (tokens/sec), CPU & thread usage, Apple Silicon GPU utilization, VRAM and loaded models. htop for your local LLM.

Simple tool for measuring inference engine performance under multi-user load

Lightweight shell script to benchmark token generation speed (tok/s) across Ollama models running in Docker. Auto-discovers all installed models or accepts a custom list via CLI. Uses Ollama's internal eval_duration timing for accurate results — no dependencies beyond curl and awk.

Test AI provider latency (TTFB, TTFT, TPS) in your CI/CD pipeline. Benchmark OpenAI, Anthropic, Google, and more.

The local-inference observability layer for your terminal: live tokens/sec, VRAM-spill & throttle warnings, the GPU metrics nvidia-smi can't show. Also a gorgeous htop/btop-class system monitor. Rust, zero deps.