feat(agent): unify model catalog queries#1231
Conversation
Signed-off-by: jomeswang <1551403343@qq.com>
2734399 to
9e65154
Compare
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| func (s *Service) resolveHiddenRuntimeModels(ctx context.Context, input agentcatalog.ResolveModelsInput) (agentcatalog.ModelSnapshot, error) { | ||
| resolverInput, ok := input.ResolverInput.(modelCatalogResolverInput) | ||
| if !ok { | ||
| return agentcatalog.ModelSnapshot{}, ErrInvalidArgument | ||
| } | ||
| scope := newComposerLiveModelScope(input.Key.ProviderID, resolverInput.ComposerInput.WorkspaceID, resolverInput.ComposerInput.Cwd, resolverInput.ComposerInput.AgentTargetID) | ||
| models, err := s.discoverLiveComposerModelsUncachedForScope(ctx, scope, resolverInput.ComposerInput.providerTargetRef, resolverInput.Settings) | ||
| if err != nil { | ||
| return agentcatalog.ModelSnapshot{}, err | ||
| } | ||
| snapshot := composerModelSnapshot(models, true) | ||
| snapshot.ResolverSource = runtimeLiveModelCatalogSource | ||
| return snapshot, nil | ||
| } |
There was a problem hiding this comment.
🔴 Hidden model-discovery sessions can be launched repeatedly instead of once
Model-list refresh reruns provider model discovery (discoverLiveComposerModelsUncachedForScope at services/tuttid/service/agent/catalog_query_service.go:63) without first consulting the "already attempted" marker, so a hidden provider session can be launched again each time the cached list goes stale rather than at most once per scope.
Impact: Repeatedly opening the composer for Claude/Cursor can keep spawning hidden credential-touching provider sessions and server-side artifacts, wasting resources and defeating the intended once-per-scope behavior.
How the attempt guard is bypassed on the coordinator path
The unified catalog coordinator routes hidden-probe providers through resolveHiddenRuntimeModels, which calls s.discoverLiveComposerModelsUncachedForScope(...) directly (services/tuttid/service/agent/catalog_query_service.go:57-70). The previously-used entry point discoverLiveComposerModels (services/tuttid/service/agent/composer_live_model_coordinator.go:20-92) is what enforces the once-per-key invariant: at composer_live_model_coordinator.go:43 it returns errLiveModelDiscoveryAlreadyAttempted when liveModelDiscoveryWasAttempted(cacheKey) is true, and it also serves an existing cache hit before spawning.
The uncached function only sets the marker (s.markLiveModelDiscoveryAttempted(scope.key()) at composer_live_model_discovery.go:297) but never checks it, and its internal errLiveModelDiscoveryAlreadyAttempted early-returns only cover the case where a running provider session still exists (composer_live_model_discovery.go:205-234). Once that hidden session has been auto-deleted and the coordinator snapshot has aged into the stale window (Claude: FreshTTL 600s, MaxStale 86400s), every interactive read hits the stale-while-revalidate branch in packages/agent/daemon/agentcatalog/coordinator.go:98-104, which calls c.startRefresh(input) and thus spawns another hidden session. If discovery keeps failing, nothing is cached and a new hidden session is spawned on essentially every cold composer open. The liveModelDiscoveryAttempted marker is therefore set and cleared on invalidation but has no effect on the coordinator path, breaking the documented "hidden discovery runs at most once per key" invariant.
Prompt for agents
The unified catalog coordinator's hidden-probe resolver resolveHiddenRuntimeModels (services/tuttid/service/agent/catalog_query_service.go) calls discoverLiveComposerModelsUncachedForScope directly, which skips the once-per-key attempt guard that discoverLiveComposerModels (composer_live_model_coordinator.go:43, liveModelDiscoveryWasAttempted) enforces. As a result, after the previously spawned hidden session is cleaned up and the coordinator snapshot goes stale (or if discovery keeps failing), each cold/stale interactive read triggers coordinator.startRefresh which spawns a fresh hidden provider session, repeatedly touching credentials and creating server-side artifacts, violating the intended 'hidden discovery runs at most once per key' invariant. Consider having the hidden-runtime resolver consult the liveModelDiscoveryWasAttempted marker (and return errLiveModelDiscoveryAlreadyAttempted / an empty completed-less result) before spawning, or route the coordinator resolver through the guarded discoverLiveComposerModels wrapper, so the attempt marker actually suppresses repeated hidden probes. Ensure invalidation still clears the marker so a legitimate re-probe can happen after auth/config changes.
Was this helpful? React with 👍 or 👎 to provide feedback.
Signed-off-by: Test <test@example.com>
Signed-off-by: Test <test@example.com>
Signed-off-by: Test <test@example.com>
| ) | ||
|
|
||
| const ( | ||
| defaultInteractiveWait = 10 * time.Second |
There was a problem hiding this comment.
🟡 First-time model list lookups can stall the composer far longer than intended
The wait limit for an uncached, first-time model lookup is fixed at 10 seconds (defaultInteractiveWait at packages/agent/daemon/agentcatalog/coordinator.go:13) and is never overridden, so opening the composer's model selector before results are cached can block the request for up to ten seconds instead of the short budget the change describes.
Impact: On the first model-picker read per provider, users can see the composer options request hang for up to ~10 seconds before it falls back and refreshes in the background.
How the 10s interactive budget reaches the composer request path
The coordinator is constructed in services/tuttid/service/agent/catalog_query_service.go:33-41 with no InteractiveWait, so NewCoordinator applies defaultInteractiveWait = 10s (packages/agent/daemon/agentcatalog/coordinator.go:68-71).
GetComposerOptions (a user-facing daemon endpoint via services/tuttid/api/daemon_agent_sessions.go) calls resolveModelsFromCatalog(..., "") for ModelDiscovery.Enabled providers (services/tuttid/service/agent/composer_options.go:168). For catalog providers such as Codex/OpenCode/Tutti Agent (not LiveModelDiscovery), the read policy stays ReadPolicyInteractive. On a cold cache ResolveModels starts a refresh and blocks on select { ... case <-timer.C } where timer := time.NewTimer(c.interactiveWait) (packages/agent/daemon/agentcatalog/coordinator.go:117-141), i.e. up to 10s, before returning the pending fallback. The PR description states cold interactive reads should use a ~1.5s budget, so the constant appears to be an oversight.
| defaultInteractiveWait = 10 * time.Second | |
| defaultInteractiveWait = 1500 * time.Millisecond |
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
What
agentcatalogcoordinator with structured keys, runtime/memory/durable priority, stale-while-revalidate, bounded detached single-flight, fallback, validation, ingestion, and invalidationModelDiscoveryDescriptorpolicy for Claude, Cursor, Codex, OpenCode, and Tutti AgentWhy
Composer reads and session validation previously used separate caches and discovery paths. Slow hidden/runtime discovery could block callers, and active conversation
configOptionswere not consistently reused.Impact
Risks / follow-ups
SnapshotStore, auth revision watchers, canonical OpenAPI routes, and GUI pending re-read are follow-up phasesChecks
go test -race ./agentcatalog ./providerregistry(frompackages/agent/daemon)go test ./service/agent -count=1(fromservices/tuttid)git diff --checkRelated