Skip to content

README.md

Latest commit

 

History

History
66 lines (47 loc) · 1.95 KB

README.md

File metadata and controls

66 lines (47 loc) · 1.95 KB
Raw

dstack-images

Custom Docker images with dstack runner dependencies for faster cold starts.

Images

vllm

Based on vllm/vllm-openai:latest with pre-installed dstack runner prerequisites that eliminate the 2-4 minute apt-get install openssh-server delay at pod start.

  • Tag: ghcr.io/plumelo/dstack-images:vllm-latest
  • What's added: openssh-server, sudo, dstack user (uid/gid 1000), /dstack/run

vllm-nightly

Same as vllm but based on vllm/vllm-openai:nightly — bleeding edge vLLM builds with the latest features and fixes (may be less stable).

  • Tag: ghcr.io/plumelo/dstack-images:vllm-nightly-latest
  • What's added: openssh-server, sudo, dstack user (uid/gid 1000), /dstack/run

llamacpp

Based on ghcr.io/ggml-org/llama.cpp:server-cuda with pre-installed dstack runner prerequisites. Includes MTP (multi-token prediction) support and the Jinja-based autoparser for automatic tool call detection.

  • Tag: ghcr.io/plumelo/dstack-images:llamacpp-latest
  • What's added: openssh-server, sudo, dstack user (uid/gid 1000), /dstack/run

Usage

# vLLM (stable)
type: service
name: my-vllm
image: ghcr.io/plumelo/dstack-images:vllm-latest
commands:
  - |
    vllm serve $MODEL_ID --host 0.0.0.0 --port 8000 ...

# vLLM (nightly)
type: service
name: my-vllm
image: ghcr.io/plumelo/dstack-images:vllm-nightly-latest
commands:
  - |
    vllm serve $MODEL_ID --host 0.0.0.0 --port 8000 ...

# llama.cpp
type: service
name: my-llamacpp
image: ghcr.io/plumelo/dstack-images:llamacpp-latest
commands:
  - |
    /app/llama-server --hf-repo $MODEL_REPO --hf-file $MODEL_FILE --port 8000 ...

Cold start improvement

Phase What Time saved
This image Pre-installed sshd/sudo ~2-4 min
Phase 2 (planned) B2 compilation cache (rclone) ~2-3 min more

CI/CD

  • Push to main: builds and pushes all image tags + SHA tags
  • PR: builds and runs verification checks per image