fix(platform): keep zero-cost fallback models after a credit-exhausted credential (#1454)#1937
fix(platform): keep zero-cost fallback models after a credit-exhausted credential (#1454)#1937larryro wants to merge 1 commit into
Conversation
📝 WalkthroughWalkthroughThe change addresses credit-exhaustion failures by splitting how model fallback retirement is handled. Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@services/platform/convex/providers/failure_scope.ts`:
- Around line 52-54: The condition checking for free models uses
includes(':free') which performs substring matching rather than suffix matching.
This causes false positives where model IDs like vendor/model:freedom would be
incorrectly classified as free. Replace the includes(':free') method call with
endsWith(':free') to ensure only model IDs that actually end with the :free
suffix are detected, preventing misclassification of similarly named models and
avoiding incorrect credit-scope retirement bypass.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1f7bfc7c-9e31-4631-a6c7-d8b118d9d794
📒 Files selected for processing (3)
services/platform/convex/lib/agent_chat/internal_actions.tsservices/platform/convex/providers/failure_scope.test.tsservices/platform/convex/providers/failure_scope.ts
| typeof data.modelId === 'string' && | ||
| data.modelId.toLowerCase().includes(':free') | ||
| ) { |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Use suffix matching for :free detection.
Line 53 uses includes(':free'), but this behavior is defined as a :free suffix check. A model ID like vendor/model:freedom would be misclassified as free and incorrectly bypass credit-scope retirement.
Proposed fix
export function isFreeModel(data: ScopeModelData): boolean {
if (
typeof data.modelId === 'string' &&
- data.modelId.toLowerCase().includes(':free')
+ data.modelId.toLowerCase().endsWith(':free')
) {
return true;
}
return data.inputCentsPerMillion === 0 && data.outputCentsPerMillion === 0;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| typeof data.modelId === 'string' && | |
| data.modelId.toLowerCase().includes(':free') | |
| ) { | |
| export function isFreeModel(data: ScopeModelData): boolean { | |
| if ( | |
| typeof data.modelId === 'string' && | |
| data.modelId.toLowerCase().endsWith(':free') | |
| ) { | |
| return true; | |
| } | |
| return data.inputCentsPerMillion === 0 && data.outputCentsPerMillion === 0; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@services/platform/convex/providers/failure_scope.ts` around lines 52 - 54,
The condition checking for free models uses includes(':free') which performs
substring matching rather than suffix matching. This causes false positives
where model IDs like vendor/model:freedom would be incorrectly classified as
free. Replace the includes(':free') method call with endsWith(':free') to ensure
only model IDs that actually end with the :free suffix are detected, preventing
misclassification of similarly named models and avoiding incorrect credit-scope
retirement bypass.
Desk Review — #1454 fix (keep zero-cost fallback models after a credit-exhausted credential)Verdict: READY TO MERGE. I reviewed the whole implementation (not just the diff), traced both consumers, ran the project's tests, and confirmed CI. What the fix does, and why it's correct
Verified:
Tests / CI
Scope note (non-blocking)
Optional polish (nits — not blocking)
None of these affect correctness. Approving. |
1 participant
Problem
Closes #1454.
Chat requests fail with "The AI provider's credit limit was exceeded" and no successful response is returned, even when fallback models are configured.
In the common single-credential setup (one OpenRouter key, several models on it), the adaptive fallback loop treats an out-of-funds (
credit_exhausted/ HTTP 402) failure as a property of the whole credential: it retirescredentialScopeKey(provider+key)and skips every remaining model that shares that key. That is correct for paid models — but it also skips zero-cost models (OpenRouter:freevariants, or models priced at 0) that draw no credits and would have answered. The turn ends with a failed message and nothing tried.Fix
Make credit retirement spare zero-cost models, while leaving auth / unreachable retirement untouched.
services/platform/convex/providers/failure_scope.ts:creditScopeKey()(distinct from the authcredentialScopeKey()) so the dead-set records why a credential died.retiredScopeKey('credit_exhausted', …)now retires the credit scope;auth_errorstill retires the auth credential;provider_unreachablestill retires the endpoint.isFreeModel()helper: a model is free when its id carries the OpenRouter:freesuffix or both token prices are explicitly0. Unconfigured (undefined) pricing is not treated as free (conservative).isModelScopeRetired()skips a model on a credit-dead credential only when it is not free. Auth-dead and endpoint-dead resources still skip every model (a bad key / down host kills free models too).The resolved
ResolvedModelDataalready carriesmodelId/inputCentsPerMillion/outputCentsPerMillion, so no call-site changes were needed.Tests
failure_scope.test.tsextended: paid sibling still skipped after credit death;:freeand zero-priced siblings now attempted; free models still skipped on auth / endpoint death;isFreeModeltruth table.vitest --project serverproviders + agent_chat suites: 230 passedtsc --noEmit: cleanoxlint --type-aware: 0 warnings / 0 errorsSummary by CodeRabbit