Add LM Studio as a local model provider (and fix keyless-provider discovery)#206
Closed
hacksics wants to merge 3 commits into
Closed
Add LM Studio as a local model provider (and fix keyless-provider discovery)#206hacksics wants to merge 3 commits into
hacksics wants to merge 3 commits into
Conversation
Local inference was Ollama-only. LM Studio worked solely by pointing the OpenAI provider at localhost — which burns the single provider:openai profile (so local and hosted OpenAI become mutually exclusive), demands a key it ignores, and gets no liveness gating, leaving dead entries in the picker whenever the server is closed. Model discovery prefers LM Studio's typed /api/v0/models over the stable /v1/models: the latter lists embedding models alongside chat models with nothing in the id to tell them apart. That route is beta, so any failure falls back to /v1 unfiltered. Connecting a local runtime now selects a model it actually has. A single named recommendation nearly always misses when the catalogue is whatever the user happened to download, and the old behaviour left a working local server still reading as "No model". The key is passed explicitly rather than via `secrets`, so a real OPENAI_API_KEY in the environment can never reach a local endpoint. Verified against LM Studio 0.3.x: discovery, embedding filtering, verify, and a live tool-calling completion through the router. Co-Authored-By: Claude <noreply@anthropic.com>
Connecting Ollama or LM Studio without editing the endpoint stored no
profile at all, and model discovery reads that profile to decide the
runtime was opted into — so the card went green while the picker stayed
empty.
Two halves. The GUI's Test-is-also-Save path skipped the save whenever a
provider already reported `configured`, which keyless providers do from
the first render (there is no key to be missing). And discovery treated a
stored-but-empty profile as absent; connecting on the default endpoint has
nothing to write, so `{}` is the correct representation of "connected" and
`is None` is the right test.
Pre-existing for Ollama; found while adding LM Studio.
Co-Authored-By: Claude <noreply@anthropic.com>
Replaces the "L" monogram fallback. Unlike the other marks this is a full app icon — it brings its own background and corner radius — so ProviderMark renders it edge-to-edge instead of floating it at 60% inside the light plate, which would read as a rounded square inside a rounded square. Shortens the title to "LM Studio (local)": the longer Ollama-style form truncated in the gallery card, and the "No key needed" line underneath already carries the rest. Provenance is noted in logos.ts — the SVGs come from the MIT-licensed lobe-icons set, this one is LM Studio's own icon from lmstudio.ai. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
lmstudioalongsideollamaas a first-class local runtime, and fixes a pre-existing bug that affects Ollama today.The bug (independent of LM Studio, worth reading first)
Connecting a keyless provider without editing the endpoint stores no profile at all, and
_ollama_models()reads that profile to decide the runtime was opted into — so the card goes green and the picker stays empty. Anyone who accepted the defaulthttp://localhost:11434and clicked Detect got no model discovery.Two halves:
useProviderSetup's Test-is-also-Save path skippedsetProviderwhenever a provider already reportedconfigured, which keyless providers do from the first render — there is no key to be missing.{}is the correct representation of "connected" andis Noneis the right test.The feature
LM Studio previously worked only by pointing the OpenAI provider at localhost, which burns the single
provider:openaiprofile (making local and hosted OpenAI mutually exclusive), demands a key it ignores, and gets no liveness gating.The new descriptor follows the existing pattern exactly — keyless, endpoint prefilled to
http://localhost:1234, liveness-gated, live model discovery. Notable details:/api/v0/modelsover/v1/models: the stable route lists embedding models alongside chat models with nothing in the id to distinguish them./api/v0is beta, so any failure falls back to/v1unfiltered.recommended_modelnearly always misses when the catalogue is whatever the user downloaded, so local providers fall back to the first discovered model — otherwise a working local server still reads "No model".secrets, so a realOPENAI_API_KEYin the environment can never reach a local endpoint — the same rule_openai_compatalready follows.LOCAL_PROVIDERS/LOCAL_DEFAULT_URLSfactor the shared behaviour;_normalize_ollama_urlkeeps its signature and tests.Verification
Against a real LM Studio 0.3.x holding
qwen/qwen3.5-9b,google/gemma-3-4bandtext-embedding-nomic-embed-text-v1.5:ProviderRouterreturnedget_weather{'city': 'Colombo'}902 passed(backend, +11 new),68 passed(vitest),154 passed(playwright),tsc --noEmitclean.Maintainer calls I'd rather you make than me
lmstudio.webpis the only raster inproviders/logos/(13.6 KB) and is LM Studio's own app icon from lmstudio.ai — every other mark there is from the MIT-licensed lobe-icons set. I noted the provenance difference inlogos.ts. Because it carries its own background it renders edge-to-edge viaFULL_BLEED_LOGOSrather than floating at 60% on the light plate. Happy to drop the commit entirely if you'd rather not vendor it; it falls back to the neutral "L" monogram with no other change.lmstudio(vision=False,parallel_tool_calls=False), matchingollama— even though both test models are vision-capable.capabilities_for()is a pure function of the model string and can't see the live server.lmstudio:qwen/qwen3.5-9b).matrix.pydeliberately curates no local entries, since which models exist is a property of the user's machine.Screenshots
(to add)
🤖 Generated with Claude Code