Skip to content

fix/refactor(models): make selected-model routing strict and fallbacks explicit #5625

Description

@RaresKeY

Prerequisites

Area

Chat / Agent

Problem or Motivation

Foreground Chat and Agent requests currently combine the session-selected model with the stored default_model_fallbacks list. Agent mode rebuilds that candidate chain on every tool-call round, so one round can fail over from model A to model B and the next round can silently retry model A. The final reply, metrics, and model label can therefore describe different providers or models.

The setting is also coupled directly into route code as an unqualified list. It does not express whether fallback is enabled for the current user, which failures are eligible, or whether a successful fallback remains pinned for the rest of an Agent run.

Existing nonempty default_model_fallbacks values should remain stored as legacy data, but they should not silently opt users into the replacement behavior and should no longer affect runtime routing.

Proposed Solution

Deliver this as three focused, stacked changes:

Order Issue Purpose Status
1 #4561 Make the selected foreground Chat/Agent model strict by default and disconnect the legacy default fallback list Draft PR #5632
2 #5626 Centralize an explicit foreground fallback policy, eligible failures, run stickiness, and model provenance Canonical draft PR #5683
3 #5627 Add per-user Settings controls for opting in and defining a new ordered fallback list Canonical draft PR #5684

Working rules:

  • A selected foreground model is strict unless the user explicitly enables the replacement fallback policy.
  • Existing default_model_fallbacks values remain stored but are not migrated or consulted.
  • Explicit fallback is limited to availability failures; request, authentication, authorization, and configuration failures do not switch providers.
  • Once a fallback produces substantive output or a tool call, Agent mode keeps that model for the remainder of the run.
  • The UI exposes concrete ordered models only. This work does not add inferred big/small model profiles.
  • Each implementation PR remains independently testable and is reviewed against its immediate stack base.

Scope

In scope:

  • normal streaming Chat and Agent model routing;
  • per-user explicit foreground fallback policy;
  • ordered concrete fallback candidates;
  • fallback eligibility and Agent-run stickiness;
  • accurate per-round model provenance and user-visible fallback state;
  • removal of confirmed-dead foreground fallback helpers/handlers;
  • focused backend, route, agent-loop, and frontend regressions.

Out of scope:

  • Utility, Vision, Deep Research, scheduled-task, teacher, image, embedding, or search-provider fallback policy;
  • inferred model tiers or automatic big/small routing;
  • deleting or migrating stored legacy fallback-list values;
  • changing default-model selection, missing-model repair, or endpoint discovery;
  • unrelated provider adapters or retry/backoff behavior.

Acceptance Criteria

  • With the new policy absent or disabled, Chat and every Agent round call only the selected session model.
  • Stored legacy default_model_fallbacks values do not affect foreground routing.
  • Enabling the replacement policy is a per-user action and requires a newly defined ordered list.
  • Explicit fallback occurs only for documented availability failures.
  • A fallback model that starts an Agent run remains pinned through later tool rounds.
  • Every emitted fallback/provenance event, final metric, and reply label identifies the model that produced that round.
  • Focused regressions and smoke tests cover strict Chat, strict multi-round Agent behavior, opt-in fallback, ineligible errors, run stickiness, and per-user isolation.
  • Each stacked PR receives a clean deep pre-review before publication or update.

Alternatives Considered

  • Treat existing nonempty fallback lists as automatic opt-in. Rejected because historical data should not silently enable a new routing policy.
  • Pass an empty list only to Agent mode. Rejected because Chat would retain a different implicit policy and the underlying policy would remain scattered.
  • Remove legacy values immediately. Rejected because preserving the stored values is safer for rollback and later manual reference.
  • Add automatic big/small model profiles. Rejected because concrete user-selected models and order are more predictable.

Prior Art / Related Issues

Are you willing to implement this?

Yes — the work is delivered as the focused stacked series above.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready for reviewDescription complete — ready for maintainer review

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions