Skip to content

Model-freshness: flag stale/retired models in cascades from live provider list #136

Description

@juniperbevensee

Task for the gardener. Model IDs change often (Anthropic retires older ones); the create-new wizard and existing-harness cascade editor can offer models that are retired → agent crash-loops on restart. Add live freshness checking. Anthropic-first, extensible.

Design (ready to build)

  1. lib/services/model-freshness.tsfetchLiveModels('anthropic'): find an Anthropic key via the keys service, call GET /v1/models (auth x-api-key + anthropic-version: 2023-06-01, no beta header; paginate), cache to disk with ~1h TTL. Fail-soft: no key / network error → return null, callers fall back to the static MODEL_CATALOG unflagged (never block the UI).
  2. GET /api/models/live?provider=anthropic{ models, fetchedAt }.
  3. Freshness signal = list membership. Retired models simply disappear from /v1/models (404 on retrieve) — there is no "deprecated" flag in the response. So: catalog entries absent from the live list → mark retired: true; live IDs not in MODEL_CATALOG → offer as suggestable/new.
  4. Consume in both the create-new wizard model step and the existing-harness cascade editor (app/api/harnesses/[id]/models) — show a "retired — pick a current model" warning on stale entries.
  5. Provider-keyed so openai/google can be added later; wire Anthropic only now.

Notes / guardrails

  • Verify model IDs against the live list, never hardcode from training data.
  • The models endpoint object carries id, display_name, created_at, and (since Mar 2026) max_input_tokens/max_tokens/capabilities.
  • RGTDD + independent audit before merge, per the repo's dev flow.

Context: raised 2026-07-02; deferred here from a live session so it lands as durable, gardener-pickup-able work rather than dying in chat.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions