Skip to content

Add LM Studio as a local model provider (and fix keyless-provider discovery)#206

Closed
hacksics wants to merge 3 commits into
andrewyng:mainfrom
hacksics:lmstudio-upstream
Closed

Add LM Studio as a local model provider (and fix keyless-provider discovery)#206
hacksics wants to merge 3 commits into
andrewyng:mainfrom
hacksics:lmstudio-upstream

Conversation

@hacksics

Copy link
Copy Markdown

Adds lmstudio alongside ollama as a first-class local runtime, and fixes a pre-existing bug that affects Ollama today.

The bug (independent of LM Studio, worth reading first)

Connecting a keyless provider without editing the endpoint stores no profile at all, and _ollama_models() reads that profile to decide the runtime was opted into — so the card goes green and the picker stays empty. Anyone who accepted the default http://localhost:11434 and clicked Detect got no model discovery.

Two halves:

  • useProviderSetup's Test-is-also-Save path skipped setProvider whenever a provider already reported configured, which keyless providers do from the first render — there is no key to be missing.
  • Discovery treated a stored-but-empty profile as absent. Connecting on the default endpoint has nothing to write, so {} is the correct representation of "connected" and is None is the right test.

The feature

LM Studio previously worked only by pointing the OpenAI provider at localhost, which burns the single provider:openai profile (making local and hosted OpenAI mutually exclusive), demands a key it ignores, and gets no liveness gating.

The new descriptor follows the existing pattern exactly — keyless, endpoint prefilled to http://localhost:1234, liveness-gated, live model discovery. Notable details:

  • Discovery prefers LM Studio's typed /api/v0/models over /v1/models: the stable route lists embedding models alongside chat models with nothing in the id to distinguish them. /api/v0 is beta, so any failure falls back to /v1 unfiltered.
  • Connecting a local runtime selects a model it actually has. A single named recommended_model nearly always misses when the catalogue is whatever the user downloaded, so local providers fall back to the first discovered model — otherwise a working local server still reads "No model".
  • The placeholder key is passed explicitly rather than via secrets, so a real OPENAI_API_KEY in the environment can never reach a local endpoint — the same rule _openai_compat already follows.
  • LOCAL_PROVIDERS / LOCAL_DEFAULT_URLS factor the shared behaviour; _normalize_ollama_url keeps its signature and tests.

Verification

Against a real LM Studio 0.3.x holding qwen/qwen3.5-9b, google/gemma-3-4b and text-embedding-nomic-embed-text-v1.5:

  • discovery returns the two LLMs and drops the embedder
  • a live tool-calling completion through ProviderRouter returned get_weather{'city': 'Colombo'}
  • the Settings flow end-to-end: card reads "✓ Running", the discovered model lands in the picker, and an already-working default is not stolen

902 passed (backend, +11 new), 68 passed (vitest), 154 passed (playwright), tsc --noEmit clean.

Maintainer calls I'd rather you make than me

  • lmstudio.webp is the only raster in providers/logos/ (13.6 KB) and is LM Studio's own app icon from lmstudio.ai — every other mark there is from the MIT-licensed lobe-icons set. I noted the provenance difference in logos.ts. Because it carries its own background it renders edge-to-edge via FULL_BLEED_LOGOS rather than floating at 60% on the light plate. Happy to drop the commit entirely if you'd rather not vendor it; it falls back to the neutral "L" monogram with no other change.
  • Capabilities stay conservative for lmstudio (vision=False, parallel_tool_calls=False), matching ollama — even though both test models are vision-capable. capabilities_for() is a pure function of the model string and can't see the live server.
  • Picker rows show the raw id (lmstudio:qwen/qwen3.5-9b). matrix.py deliberately curates no local entries, since which models exist is a property of the user's machine.

Screenshots

(to add)

🤖 Generated with Claude Code

hacksics and others added 3 commits July 26, 2026 23:05
Local inference was Ollama-only. LM Studio worked solely by pointing the
OpenAI provider at localhost — which burns the single provider:openai
profile (so local and hosted OpenAI become mutually exclusive), demands a
key it ignores, and gets no liveness gating, leaving dead entries in the
picker whenever the server is closed.

Model discovery prefers LM Studio's typed /api/v0/models over the stable
/v1/models: the latter lists embedding models alongside chat models with
nothing in the id to tell them apart. That route is beta, so any failure
falls back to /v1 unfiltered.

Connecting a local runtime now selects a model it actually has. A single
named recommendation nearly always misses when the catalogue is whatever
the user happened to download, and the old behaviour left a working local
server still reading as "No model".

The key is passed explicitly rather than via `secrets`, so a real
OPENAI_API_KEY in the environment can never reach a local endpoint.

Verified against LM Studio 0.3.x: discovery, embedding filtering, verify,
and a live tool-calling completion through the router.

Co-Authored-By: Claude <noreply@anthropic.com>
Connecting Ollama or LM Studio without editing the endpoint stored no
profile at all, and model discovery reads that profile to decide the
runtime was opted into — so the card went green while the picker stayed
empty.

Two halves. The GUI's Test-is-also-Save path skipped the save whenever a
provider already reported `configured`, which keyless providers do from
the first render (there is no key to be missing). And discovery treated a
stored-but-empty profile as absent; connecting on the default endpoint has
nothing to write, so `{}` is the correct representation of "connected" and
`is None` is the right test.

Pre-existing for Ollama; found while adding LM Studio.

Co-Authored-By: Claude <noreply@anthropic.com>
Replaces the "L" monogram fallback. Unlike the other marks this is a full
app icon — it brings its own background and corner radius — so ProviderMark
renders it edge-to-edge instead of floating it at 60% inside the light
plate, which would read as a rounded square inside a rounded square.

Shortens the title to "LM Studio (local)": the longer Ollama-style form
truncated in the gallery card, and the "No key needed" line underneath
already carries the rest.

Provenance is noted in logos.ts — the SVGs come from the MIT-licensed
lobe-icons set, this one is LM Studio's own icon from lmstudio.ai.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant