Skip to content

fix: set model id for custom models in llamastack config - #223

Merged
sauagarwa merged 1 commit into
rh-ai-quickstart:mainfrom
ganeshmurthy:pr-208-against-main
Jul 16, 2026
Merged

fix: set model id for custom models in llamastack config#223
sauagarwa merged 1 commit into
rh-ai-quickstart:mainfrom
ganeshmurthy:pr-208-against-main

Conversation

@ganeshmurthy

Copy link
Copy Markdown
Collaborator

The llama-stack subchart requires each enabled model to have an id field for registered_resources.models.model_id. When using a custom model not predefined in the subchart, this field was never set, causing a pydantic ValidationError on startup. Default the id to the model key name, with an optional LLM_ID/SAFETY_ID override.

The llama-stack subchart requires each enabled model to have an `id`
field for `registered_resources.models.model_id`. When using a custom
model not predefined in the subchart, this field was never set, causing
a pydantic ValidationError on startup. Default the id to the model key
name, with an optional LLM_ID/SAFETY_ID override.

Made-with: Cursor
@ganeshmurthy

Copy link
Copy Markdown
Collaborator Author

Testing

Tested on OpenShift cluster with custom remote model:

  # Deployed with LLM_ID override
  make install \
    NAMESPACE=ai-virtual-agent \
    LLM=remote-llm \
    LLM_ID=Qwen3.6-35B-A3B \
    LLM_URL=https://litellm-litemaas.apps.prod.rhoai.rh-aiservices-bu.com/v1 \
    LLM_API_TOKEN=sk-***

Verification:

  1. Helm values show correct override:
    $ helm get values ai-virtual-agent -n ai-virtual-agent | grep -A 5 "remote-llm:"
  remote-llm:
    apiToken: sk-***
    enabled: true
    id: Qwen3.6-35B-A3B  # ✅ Override applied, not "remote-llm"
    url: https://litellm-litemaas.apps.prod.rhoai.rh-aiservices-bu.com/v1
  1. LlamaStack pod started successfully:
  $ oc get pods -n ai-virtual-agent | grep llamastack
  llamastack-79d47fb4f5-drnzv   1/1   Running   0   8m45s
- No validation errors (pod would crash if model_id field was missing or invalid)
- No restarts (clean startup with custom model configuration)

Result: The PR correctly allows LLM_ID to override the model identifier while LLM remains the YAML-safe config key. SAFETY model support also confirmed working.

@sauagarwa
sauagarwa merged commit 07638b5 into rh-ai-quickstart:main Jul 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants