Easy to launch LLM models 🚀
A CLI for launching LLMs on HPC clusters via SLURM directly or through FirecREST. Public serving endpoint: https://serving.swissai.svc.cscs.ch/.
Install from source with uv (requires Python 3.10–3.14):
git clone https://github.com/swiss-ai/model-launch.git
cd model-launch
uv venv --python 3.12
source .venv/bin/activate
uv pip install .Then set up credentials and launch a model:
sml init # one-time: configures credentials
sml # launch a model interactivelyEach new shell, re-activate the venv with source .venv/bin/activate before running sml.
Prefer a script you can copy? Browse examples/ and run any of them once sml is installed and initialized.
| Topic | When to read |
|---|---|
| Getting Started | First time here |
| Initialization | Setting up credentials, FirecREST vs SLURM |
| Using SML | Day-to-day launches via the interactive CLI |
| Advanced Usage | Full SLURM/framework control |
| How to Size a Model | Picking replica/node layout for a given model |
| Benchmarking | Measuring throughput and latency |
| MCP Server | Driving SML from Claude Desktop / Cursor |
| Architecture | How SML fits with serving-api and opentela |
| Development | Contributing to SML itself |
| CI/CD | Pipeline structure |
| FAQ | Always-on hosting, common gotchas |
A rendered docs site is built from the same files via MkDocs — run make docs for a local preview, or browse the published site at https://swiss-ai.github.io/model-launch/.
