llm-litellm

llm plugin for LiteLLM OpenAI-compatible proxies.

Dynamically registers all models exposed by a LiteLLM proxy and routes prompts through it.

Installation

llm install llm-litellm-proxy

Configuration

Variable	Required	Default	Description
LITELLM_BASE_URL	yes	—	Base URL of the LiteLLM proxy
LITELLM_API_KEY	yes	—	API key for the proxy
LITELLM_MODEL_PREFIX	no	""	Prefix prepended to each model ID
LITELLM_TIMEOUT_SECONDS	no	60	HTTP timeout in seconds
LITELLM_DEBUG	no	""	Set to 1 to log errors to stderr

LITELLM_BASE_URL may include or omit a trailing slash. The plugin appends /v1 internally.

Usage

export LITELLM_BASE_URL="https://your-litellm-proxy.example.com"
export LITELLM_API_KEY="your-key"

llm models
llm -m gpt-4o "say hello"
echo "explain this log" | llm -m claude-sonnet-4-bedrock
echo "raw output" | llm -m claude-sonnet-4-bedrock -s "Summarize as a health triage"

Model prefix

If your LiteLLM proxy exposes models with IDs that collide with other llm plugins, set a prefix to namespace them:

export LITELLM_MODEL_PREFIX="litellm/"
llm -m litellm/gpt-4o "hello"

Options

Option	Description
-o temperature <float>	Sampling temperature (0–2)
-o max_tokens <int>	Maximum output tokens

Error handling

If LITELLM_BASE_URL or LITELLM_API_KEY is not set, the plugin registers no models and produces no output. Set LITELLM_DEBUG=1 to log errors from model discovery to stderr.

Prompt execution errors surface the HTTP status code and response body.

See also

• llm documentation
• LiteLLM proxy documentation