feat: warn when configured model belongs to a different provider#289
feat: warn when configured model belongs to a different provider#289Greg Land (bikeusaland) wants to merge 3 commits into
Conversation
isValidModelId only checks the shape of the model string, so a saved OPENWIKI_MODEL_ID left over from a previous provider (e.g. an Anthropic model while the provider is now OpenAI) passed validation and failed later with an opaque provider-side 400/404. Add isModelIdForOtherProvider / getProvidersForKnownModelId to constants.ts, which use exact matching against each provider's known model options (so namespaced overlaps like OpenRouter's "anthropic/claude-..." don't false-match, and custom/gateway model IDs are never flagged). resolveModelId now emits a non-fatal, actionable warning on mismatch; the run still proceeds since a custom endpoint may legitimately serve the model. Add unit tests for both helpers, including exact-match, shared openai/openai-chatgpt models, and custom-id cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tream langchain-ai#289) Adds getProvidersForKnownModelId / isModelIdForOtherProvider (constants.ts, exact-match, union-safe over PROVIDER_CONFIGS) and a non-fatal warnOnProviderModelMismatch emitted from resolveModelId: if OPENWIKI_MODEL_ID is a known model of another provider, warn instead of failing later with an opaque provider-side error. Run still proceeds (gateways may serve it). Fork guard: agent-cli aliases (opus/default/grok-4.5) stay known to their own provider and never trip the warning — covered by a regression test. Verified: typecheck, lint, format, pnpm test (344), build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tream langchain-ai#289) Adds getProvidersForKnownModelId / isModelIdForOtherProvider (constants.ts, exact-match, union-safe over PROVIDER_CONFIGS) and a non-fatal warnOnProviderModelMismatch emitted from resolveModelId: if OPENWIKI_MODEL_ID is a known model of another provider, warn instead of failing later with an opaque provider-side error. Run still proceeds (gateways may serve it). Fork guard: agent-cli aliases (opus/default/grok-4.5) stay known to their own provider and never trip the warning — covered by a regression test. Verified: typecheck, lint, format, pnpm test (344), build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…smatch-warning # Conflicts: # src/agent/index.ts # test/constants.test.ts
Merged
|
Problem
isValidModelIdonly validates the shape of the model string:So a saved
OPENWIKI_MODEL_IDleft over from a previous provider — e.g.claude-opus-4-8while the provider is nowopenai— passes validation and gets sent to the wrong backend, failing at request time with an opaque provider-side 400/404. Provider/model mismatch is the most common misconfiguration, and today it produces a cryptic error one turn later instead of an actionable local message.Fix
Add two pure helpers to
constants.ts:getProvidersForKnownModelId(modelId, excludeProvider)— providers (other than the one given) whose knownmodelOptionsinclude the model by exact match.isModelIdForOtherProvider(modelId, provider)— true when the model is a known model of some other provider and not known for the configured provider.resolveModelIdnow emits a non-fatal, actionable warning on mismatch (via the existingonEventtext channel) and still proceeds — a custom endpoint or gateway may legitimately serve the model.Why exact matching (not substring)
Provider model lists overlap heavily via namespacing:
claude-opus-4-8; OpenRouter listsanthropic/claude-opus-4-8.openai/gpt-oss-120b.Exact-match on the full normalized ID avoids false positives from these overlaps, and returns empty for custom/gateway IDs so
openai-compatiblemodel names are never flagged. Becauseopenaiandopenai-chatgptshareOPENAI_MODEL_OPTIONS, switching between those two never warns.Tests
Adds unit tests for both helpers covering: exact match, the
excludeProviderbehavior, namespaced-overlap non-matching, sharedopenai/openai-chatgptmodels, custom/unknown IDs, and whitespace trimming.Full gate passes:
typecheck,lint:check,format:check, test suite (204 tests, +8 new).🤖 Generated with Claude Code
Related PRs
Part of a batch from a single codebase-review pass. Independent,
main-based branches that can merge in any order.Security
Correctness
Feature
(This PR is #289.)