Skip to content

feat: support QMD_LLAMA_RELEASE env var to use latest llama.cpp#658

Open
wzgrx wants to merge 1 commit into
tobi:mainfrom
wzgrx:feat/support-latest-llama
Open

feat: support QMD_LLAMA_RELEASE env var to use latest llama.cpp#658
wzgrx wants to merge 1 commit into
tobi:mainfrom
wzgrx:feat/support-latest-llama

Conversation

@wzgrx
Copy link
Copy Markdown

@wzgrx wzgrx commented May 17, 2026

Problem

QMD currently relies on node-llama-cpp's default behavior, which pins the llama.cpp version to an older release (e.g., b8390). This prevents users from accessing the latest features, model support, and performance improvements available in the master branch of llama.cpp.

Solution

This PR introduces the QMD_LLAMA_RELEASE environment variable.

When set (e.g., to master), QMD automatically patches node-llama-cpp's internal configuration (binariesGithubRelease.json) before initializing the LLM engine. This forces node-llama-cpp to download and build the specified llama.cpp release.

Usage

# Use the absolute latest llama.cpp (master branch)
QMD_LLAMA_RELEASE=master qmd status

# Use a specific commit hash
QMD_LLAMA_RELEASE=4f13cb7 qmd status

Changes

  1. src/utils/llama-version.ts: New utility function patchLlamaReleaseIfNeeded that handles the patching of node-llama-cpp's release config.
  2. src/llm.ts: Invokes the patch function before getLlama is called.
  3. CLAUDE.md: Documentation update for the new environment variable.

This allows power users to opt-in to the latest engine without waiting for a formal QMD release, while preserving the stable default behavior for everyone else.

Allows users to override the default pinned llama.cpp version (b8390)
by setting the QMD_LLAMA_RELEASE environment variable.
Example: QMD_LLAMA_RELEASE=master qmd status

This forces QMD to update node-llama-cpp's configuration to download
and build the specified release (e.g. master branch) on the next run.

Fixes: #0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant