diff --git a/docs/bug-report/opencode-post-tool-output-gap/bug-report.md b/docs/bug-report/opencode-post-tool-output-gap/bug-report.md new file mode 100644 index 0000000000..36ca7ab20f --- /dev/null +++ b/docs/bug-report/opencode-post-tool-output-gap/bug-report.md @@ -0,0 +1,138 @@ +--- +title: OpenCode + DeepSeek no-output recovery +status: external-review +issue: https://github.com/zts212653/clowder-ai/issues/1341 +pr: https://github.com/zts212653/clowder-ai/pull/1342 +created: 2026-08-12 +--- + +# OpenCode + DeepSeek no-output recovery + +## Summary + +OpenCode + DeepSeek had two observed "no visible final answer" shapes. They share one root cause: the provider adapter treated CLI/tool completion as turn completion without verifying that the user-visible final assistant text had actually been produced. + +This is not a DeepSeek capability failure. The missing piece is a completion-state check in `OpenCodeAgentService`. + +## Phenotype A: pure silent completion + +Observed shape: + +1. OpenCode stdout emitted `step_start`. +2. stdout emitted no `text` and no `error`. +3. The assistant answer was still present in OpenCode local SQLite state. + +Direct local evidence: + +- The NDJSON sample contained only `type: "step_start"`. +- The `step_start.part` payload carried both `sessionID` and `messageID`. +- Querying OpenCode SQLite table `part` by that `sessionID + messageID` returned the assistant `text` part. + +Recovery rule: + +- When a completed OpenCode run has events but no text, no error, and no tool use, use the `step_start` `sessionID + messageID` as a read-only anchor into OpenCode SQLite. +- If assistant text is found, emit it as the visible response and do not surface `silent_completion`. +- If no text is found or SQLite is unavailable, preserve the existing `silent_completion` diagnostic path. + +## Phenotype B: post-tool gap + +Observed shape: + +1. OpenCode emitted a short prelude text, for example "I will check the actual config first." +2. OpenCode emitted `tool_use`; the tool completed and had output. +3. The run ended without a final assistant `text` after the tool. + +Teammate reproduction confirmation: + +- Ragdoll/OpenCode + DeepSeek observed the event sequence `text -> tool_use -> step_finish/end` with no final text. +- The frontend therefore showed only the incomplete prelude or appeared to have no useful output. + +Why the old logic missed it: + +- `textEventCount > 0` made the turn look like it had user-visible text. +- `toolUseEmitted = true` correctly suppressed the older `silent_completion` warning for pure tool-only turns. +- The adapter did not compare event order, so it missed the specific case where the last meaningful event was a tool and no later text existed. + +Recovery rule: + +- At normal CLI completion, detect `textEventCount > 0 && lastToolEventIndex > lastTextEventIndex && !errorAlreadyYielded`. +- Run exactly one no-tool finalizer in the same OpenCode session using a dedicated `cat-cafe-no-tool-finalizer` agent. +- Deny all tools in the finalizer config. The finalizer may use only existing session state and a sanitized latest-tool-output summary. +- Emit the first finalizer text with `textMode: "replace"` so it replaces the incomplete prelude. +- If the finalizer produces no text or attempts a tool, emit a deterministic diagnostic fallback with the sanitized latest-tool-output summary. + +## Non-goals and safety boundaries + +- Do not rerun the whole OpenCode turn. A full rerun can repeat tool side effects. +- Do not use word-count thresholds to decide whether a prelude is "too short." The stable signal is event order: last meaningful event is tool and no later text. +- Do not mark pure `tool_use` completions as silent failures. Existing AC-G3 behavior remains valid. +- Do not hide real provider or CLI errors behind no-output recovery. Error paths continue to surface their original diagnostics. +- Do not invent semantic conclusions if the finalizer cannot produce text. Fall back to an explicit recovery diagnostic. + +## Root cause + +The adapter had a completion判定缺口: + +- "A tool finished" is not the same as "the assistant finished its answer." +- "Some text appeared earlier" is not the same as "the final answer appeared after the last tool." +- "No stdout text" is not always "no answer exists," because OpenCode may have persisted the answer in its local SQLite session state. + +The correct completion contract is: + +1. If stdout contains final text after the last tool, stream it normally. +2. If stdout has no text but OpenCode persisted text for the current `sessionID + messageID`, recover that persisted text. +3. If stdout has prelude text, then a tool, then no final text, perform a single no-tool finalizer pass and replace the prelude. +4. If recovery cannot produce text, preserve explicit diagnostics instead of silently ending. + +## Implementation + +Code paths: + +- `packages/api/src/domains/cats/services/agents/providers/OpenCodeAgentService.ts` +- `packages/api/src/domains/cats/services/agents/providers/opencode-recovery.ts` + +- Tracks `lastTextEventIndex`, `lastToolEventIndex`, latest tool trace, and latest `step_start` message reference. +- Adds read-only SQLite recovery for phenotype A using `part.session_id + part.message_id`. +- Resolves OpenCode SQLite through an explicit override test seam, canonical `OPENCODE_DB`, XDG/platform data roots, and channel-named `opencode*.db` files. +- Adds a no-tool finalizer for phenotype B using `--session` and `--agent cat-cafe-no-tool-finalizer`. +- Denies all finalizer tool permissions through `OPENCODE_CONFIG_CONTENT` and treats any observed finalizer `tool_use` as a poisoned finalizer result. +- Buffers finalizer text until the finalizer completes without tool/error poison, then emits the first text with `textMode: "replace"`. +- Uses one central safe projector for latest tool output before including it in the finalizer prompt or deterministic fallback. +- Preserves deterministic fallback text only for finalizer failure, with secrets and absolute paths redacted. +- Serializes invocations for the same OpenCode session so a second turn cannot race a first turn's finalization window. + +Regression tests: + +- `packages/api/test/opencode-agent-service.test.js` +- `packages/api/test/opencode-recovery.test.js` + +- Adds a red/green SQLite recovery case for `step_start`-only NDJSON. +- Updates the post-tool gap case to require a second no-tool finalizer invocation, session resume, deny-all permissions, and `textMode: "replace"`. +- Adds fail-closed finalizer poisoning, sanitized fallback, and same-session single-flight regression cases. +- Adds SQLite path-resolution, schema-drift, malformed-part, multi-part, and redaction tests at the recovery boundary. +- Keeps the older AC-G3 cases for true silent diagnostics and pure tool-only completion. + +## Verification + +Red evidence before the SQLite recovery implementation: + +```text +node --import ./packages/api/test/helpers/setup-cat-registry.js --test --test-name-pattern "step_start-only NDJSON recovers" packages/api/test/opencode-agent-service.test.js +FAIL: expected "Recovered from OpenCode SQLite.", actual undefined +``` + +Green verification after the fix: + +```text +pnpm --dir packages/api run build +node --import ./packages/api/test/helpers/setup-cat-registry.js --test --test-name-pattern "step_start-only NDJSON recovers" packages/api/test/opencode-agent-service.test.js +node --import ./packages/api/test/helpers/setup-cat-registry.js --test --test-name-pattern "post-tool completion gap" packages/api/test/opencode-agent-service.test.js +node --import ./packages/api/test/helpers/setup-cat-registry.js --test packages/api/test/opencode-agent-service.test.js +``` + +Results: + +- `packages/api` build passed. +- Target SQLite recovery test passed. +- Target post-tool finalizer test passed. +- Full `opencode-agent-service.test.js` passed: 41/41. diff --git a/packages/api/src/config/env-registry.ts b/packages/api/src/config/env-registry.ts index 56e596f5f3..0621b2d137 100644 --- a/packages/api/src/config/env-registry.ts +++ b/packages/api/src/config/env-registry.ts @@ -835,6 +835,16 @@ export const ENV_VARS: EnvDefinition[] = [ label: 'CLI 超时', settingsGroup: 'runtime', }, + { + name: 'OPENCODE_DB', + defaultValue: '(OpenCode default data dir)', + description: + 'Optional OpenCode SQLite database override used only for read-only silent-completion recovery. Leave unset to use OpenCode data-dir discovery.', + category: 'cli', + sensitive: false, + runtimeEditable: false, + hubVisible: false, + }, { name: 'CAT_CAFE_SUPERVISOR_PARENT_PID', defaultValue: '(内部注入)', diff --git a/packages/api/src/domains/cats/services/agents/providers/OpenCodeAgentService.ts b/packages/api/src/domains/cats/services/agents/providers/OpenCodeAgentService.ts index c49dffd01f..a581959ba8 100644 --- a/packages/api/src/domains/cats/services/agents/providers/OpenCodeAgentService.ts +++ b/packages/api/src/domains/cats/services/agents/providers/OpenCodeAgentService.ts @@ -44,6 +44,20 @@ import { userControlsOpenCodeAutoApprove, } from './opencode-auto-approval.js'; import { transformOpenCodeEvent } from './opencode-event-transform.js'; +import { + buildOpenCodeNoToolFinalizerConfig, + buildOpenCodePostToolFallbackText, + buildOpenCodePostToolFinalizerPrompt, + extractOpenCodeMessageRef, + extractOpenCodeToolTrace, + hasOpenCodeManagedConfig, + identifierPrefix, + OPENCODE_CONFIG_CONTENT_ENV, + OPENCODE_NO_TOOL_FINALIZER_AGENT, + type OpenCodeToolTrace, + recoverOpenCodeSilentCompletion, + SessionSingleFlight, +} from './opencode-recovery.js'; const log = createModuleLogger('opencode-agent'); @@ -63,6 +77,10 @@ interface OpenCodeAgentServiceOptions { l0CompilerFn?: (options: { catId: string; userId?: string; dataDir?: string; outPath?: string }) => Promise; /** Test seam for the `opencode run --help` auto-approval capability probe. */ autoApproveProbeFn?: OpenCodeAutoApproveProbeFn; + /** Test seam for OpenCode's local SQLite state used to recover silent completions. */ + opencodeDbPath?: string; + /** Test seam for managed OpenCode config precedence detection. */ + opencodeManagedConfigPaths?: readonly string[]; } const OPENCODE_API_KEY_ENV = 'OPENCODE_API_KEY'; @@ -98,6 +116,28 @@ export interface OpenCodeEnvDebugSummary { catCafeOcBaseUrl: string; } +interface OpenCodePostToolFinalizerParams { + command: string; + cwd?: string; + childEnv: Record; + effectiveModel: string; + metadata: MessageMetadata; + sessionId?: string; + trace: OpenCodeToolTrace | null; + textMode: 'append' | 'replace'; + options?: AgentServiceOptions; +} + +function getOpenCodeStepFinishReason(event: unknown): string | undefined { + if (typeof event !== 'object' || event === null) return undefined; + const raw = event as Record; + if (raw.type !== 'step_finish') return undefined; + const part = raw.part; + if (typeof part !== 'object' || part === null) return undefined; + const reason = (part as Record).reason; + return typeof reason === 'string' ? reason : undefined; +} + function isPermanentOpenCodeProviderFailure(event: unknown, reasonCode: string | undefined): boolean { if (typeof event !== 'object' || event === null) return false; const rawError = (event as Record).error; @@ -171,6 +211,9 @@ export class OpenCodeAgentService implements L0InjectableAgentService { /** F203 Phase I: injectable L0 compiler (test seam, like Claude/Codex services). */ readonly l0CompilerFn: import('../../types.js').L0CompilerFn | undefined; private readonly autoApproveProbeFn: OpenCodeAutoApproveProbeFn | undefined; + private readonly opencodeDbPath: string | undefined; + private readonly opencodeManagedConfigPaths: readonly string[] | undefined; + private readonly sessionSingleFlight = new SessionSingleFlight(); private autoApproveProbe: Promise | undefined; constructor(options?: OpenCodeAgentServiceOptions) { @@ -182,6 +225,8 @@ export class OpenCodeAgentService implements L0InjectableAgentService { this.rawArchive = options?.rawArchive ?? new CliRawArchive(); this.l0CompilerFn = options?.l0CompilerFn; this.autoApproveProbeFn = options?.autoApproveProbeFn; + this.opencodeDbPath = options?.opencodeDbPath; + this.opencodeManagedConfigPaths = options?.opencodeManagedConfigPaths; } /** @@ -217,6 +262,10 @@ export class OpenCodeAgentService implements L0InjectableAgentService { } async *invoke(prompt: string, options?: AgentServiceOptions): AsyncIterable { + yield* this.sessionSingleFlight.run(options?.sessionId, () => this.invokeUnlocked(prompt, options)); + } + + private async *invokeUnlocked(prompt: string, options?: AgentServiceOptions): AsyncIterable { const readOnly = options?.toolExecutionPolicy?.mode === 'read_only'; // P1-2: runtime model override takes precedence over constructor model const effectiveModel = options?.callbackEnv?.CAT_CAFE_ANTHROPIC_MODEL_OVERRIDE ?? this.model; @@ -313,6 +362,12 @@ export class OpenCodeAgentService implements L0InjectableAgentService { let eventCount = 0; let textEventCount = 0; + let lastTextEventIndex = 0; + let lastToolEventIndex = 0; + let lastToolTrace: OpenCodeToolTrace | null = null; + let lastStepFinishReason: string | undefined; + let terminalStepFinishAfterLastTool = false; + let lastAssistantMessageId: string | undefined; // F212 Phase G (AC-G3, clowder-ai#875): track unique event types so the // silent_completion diagnostic can surface them when textEventCount===0. const uniqueEventTypes = new Set(); @@ -343,6 +398,9 @@ export class OpenCodeAgentService implements L0InjectableAgentService { ? String((event as Record).type) : '__unknown'; uniqueEventTypes.add(evtType); + const messageRef = extractOpenCodeMessageRef(event); + if (messageRef?.sessionId) metadata.sessionId = messageRef.sessionId; + if (messageRef?.messageId) lastAssistantMessageId = messageRef.messageId; log.debug({ catId: this.catId, eventIndex: eventCount, type: evtType }, 'CLI event received'); if (isCliTimeout(event)) { yield { @@ -410,8 +468,19 @@ export class OpenCodeAgentService implements L0InjectableAgentService { const result = transformOpenCodeEvent(event, this.catId); if (result !== null) { let terminateAfterYield = false; - if (result.type === 'text') textEventCount++; - if (result.type === 'tool_use') toolUseEmitted = true; + if (result.type === 'text') { + textEventCount++; + lastTextEventIndex = eventCount; + } + if (result.type === 'tool_use') { + toolUseEmitted = true; + lastToolEventIndex = eventCount; + terminalStepFinishAfterLastTool = false; + const toolTrace = extractOpenCodeToolTrace(event); + if (toolTrace !== null) { + lastToolTrace = toolTrace; + } + } // F212 Phase A AC-A8: enrich stream `error` event yield with cliDiagnostics so // frontend folded panel (Phase B) sees reasonCode / safeExcerpt / publicHint // even when CLI never exits non-zero (some providers emit error events then exit 0). @@ -481,12 +550,88 @@ export class OpenCodeAgentService implements L0InjectableAgentService { break; } } + const stepFinishReason = getOpenCodeStepFinishReason(event); + if (stepFinishReason) { + lastStepFinishReason = stepFinishReason; + if (lastToolEventIndex > lastTextEventIndex && eventCount > lastToolEventIndex) { + terminalStepFinishAfterLastTool = stepFinishReason !== 'tool-calls'; + } + } } log.info( - { catId: this.catId, totalEvents: eventCount, textEvents: textEventCount, sessionId: metadata.sessionId }, + { + catId: this.catId, + totalEvents: eventCount, + textEvents: textEventCount, + sessionIdPrefix: identifierPrefix(metadata.sessionId), + }, 'OpenCode CLI invocation completed', ); + if (eventCount > 0 && textEventCount === 0 && !errorAlreadyYielded && !toolUseEmitted) { + const recoveredText = this.recoverSilentCompletionText(metadata.sessionId, lastAssistantMessageId); + if (recoveredText) { + log.info( + { + catId: this.catId, + sessionIdPrefix: identifierPrefix(metadata.sessionId), + messageIdPrefix: identifierPrefix(lastAssistantMessageId), + textLength: recoveredText.length, + }, + 'Recovered OpenCode silent completion text from local SQLite state', + ); + textEventCount++; + yield { + type: 'text' as const, + catId: this.catId, + content: recoveredText, + metadata, + timestamp: Date.now(), + }; + } + } + if ( + textEventCount > 0 && + lastToolEventIndex > lastTextEventIndex && + !terminalStepFinishAfterLastTool && + !errorAlreadyYielded + ) { + log.warn( + { + catId: this.catId, + totalEvents: eventCount, + textEvents: textEventCount, + eventTypes: Array.from(uniqueEventTypes), + lastTextEventIndex, + lastToolEventIndex, + latestTool: lastToolTrace?.toolName, + lastStepFinishReason, + terminalStepFinishAfterLastTool, + textMode: 'replace', + }, + 'OpenCode CLI stopped after tool_use without final text - running no-tool finalizer', + ); + for await (const finalizerMsg of this.runPostToolFinalizer({ + command: opencodeCommand, + ...(cwd ? { cwd } : {}), + childEnv, + effectiveModel, + metadata, + ...((metadata.sessionId ?? options?.sessionId) + ? { sessionId: metadata.sessionId ?? options?.sessionId } + : {}), + trace: lastToolTrace, + textMode: 'replace', + options, + })) { + if (finalizerMsg.type === 'text') textEventCount++; + if (finalizerMsg.metadata?.usage != null && resolveCurrentContextUsage(finalizerMsg.metadata.usage) != null) { + usageTelemetryReceived = true; + } + yield finalizerMsg; + } + } + // F212 Phase G (AC-G3, clowder-ai#875): surface silent_completion via cliDiagnostics. // Only when eventCount > 0 (CLI actually produced events) AND no other diagnostic // already surfaced (don't double-yield on cli error / stream error / timeout — they @@ -555,6 +700,206 @@ export class OpenCodeAgentService implements L0InjectableAgentService { } } + private async *runPostToolFinalizer(params: OpenCodePostToolFinalizerParams): AsyncIterable { + const boundaryFailure = this.getNoToolFinalizerBoundaryFailure(); + if (boundaryFailure) { + log.warn( + { catId: this.catId, invocationId: params.options?.invocationId, reason: boundaryFailure }, + 'OpenCode no-tool finalizer blocked before spawn', + ); + yield { + type: 'text', + catId: this.catId, + content: buildOpenCodePostToolFallbackText(params.trace, boundaryFailure), + textMode: params.textMode, + metadata: params.metadata, + timestamp: Date.now(), + }; + return; + } + + const finalizerPrompt = buildOpenCodePostToolFinalizerPrompt(params.trace); + const finalizerArgs = this.buildNoToolFinalizerArgs(finalizerPrompt, params.sessionId, params.effectiveModel); + const finalizerEnv = this.buildNoToolFinalizerEnv(params.childEnv); + const cliOpts = { + command: params.command, + args: finalizerArgs, + ...(params.cwd ? { cwd: params.cwd } : {}), + env: finalizerEnv, + ...(params.options?.signal ? { signal: params.options.signal } : {}), + ...(params.options?.invocationId ? { invocationId: params.options.invocationId } : {}), + ...(params.options?.cliSessionId ? { cliSessionId: params.options.cliSessionId } : {}), + ...(params.options?.livenessProbe ? { livenessProbe: params.options.livenessProbe } : {}), + ...(params.options?.parentSpan ? { parentSpan: params.options.parentSpan } : {}), + ...(params.options?.invocationId && this.rawArchive.getPath + ? { rawArchivePath: this.rawArchive.getPath(params.options.invocationId) } + : {}), + }; + + const events = params.options?.spawnCliOverride + ? params.options.spawnCliOverride(cliOpts) + : spawnCli(cliOpts, this.spawnFn ? { spawnFn: this.spawnFn } : undefined); + + const finalizerTextBuffer: AgentMessage[] = []; + let finalizerPoisoned = false; + let finalizerErrorReason: string | undefined; + const finalizerEventTypes = new Set(); + + for await (const event of events) { + if (params.options?.invocationId) { + this.rawArchive.append(params.options.invocationId, sanitizeRawEvent(event)).catch((err) => { + log.warn( + { catId: this.catId, invocationId: params.options?.invocationId, err }, + 'Post-tool finalizer raw archive write failed', + ); + }); + } + const evtType = + typeof event === 'object' && event !== null && 'type' in event + ? String((event as Record).type) + : '__unknown'; + finalizerEventTypes.add(evtType); + + if (isCliTimeout(event)) { + finalizerPoisoned = true; + finalizerErrorReason = 'timeout'; + log.warn( + { catId: this.catId, invocationId: params.options?.invocationId, timeoutMs: event.timeoutMs }, + 'OpenCode no-tool finalizer timed out', + ); + continue; + } + if (isLivenessWarning(event)) { + continue; + } + if (isCliError(event)) { + finalizerPoisoned = true; + finalizerErrorReason = event.reasonCode ?? 'cli_error'; + log.warn( + { catId: this.catId, invocationId: params.options?.invocationId, reasonCode: event.reasonCode }, + 'OpenCode no-tool finalizer exited with an error', + ); + continue; + } + + const result = transformOpenCodeEvent(event, this.catId); + if (result === null) continue; + if (result.type === 'session_init') { + if (result.sessionId) params.metadata.sessionId = result.sessionId; + continue; + } + if (result.type === 'tool_use') { + finalizerPoisoned = true; + finalizerErrorReason = 'tool_use_blocked'; + log.warn( + { + catId: this.catId, + invocationId: params.options?.invocationId, + toolName: result.toolName, + }, + 'OpenCode no-tool finalizer attempted to use a tool', + ); + continue; + } + if (result.type === 'error') { + finalizerPoisoned = true; + finalizerErrorReason = 'provider_error'; + log.warn( + { catId: this.catId, invocationId: params.options?.invocationId, error: result.error }, + 'OpenCode no-tool finalizer returned an error event', + ); + continue; + } + if (result.type === 'text') { + if (finalizerPoisoned) continue; + finalizerTextBuffer.push({ + ...result, + metadata: params.metadata, + textMode: finalizerTextBuffer.length === 0 ? params.textMode : result.textMode, + }); + continue; + } + if (result.type === 'agent_loop') { + yield { + ...result, + metadata: + result.metadata?.usage != null ? { ...params.metadata, usage: result.metadata.usage } : params.metadata, + }; + } + } + + if (!finalizerPoisoned && finalizerTextBuffer.length > 0) { + for (const finalizerText of finalizerTextBuffer) { + yield finalizerText; + } + return; + } + + if (finalizerTextBuffer.length === 0 || finalizerPoisoned) { + log.warn( + { + catId: this.catId, + invocationId: params.options?.invocationId, + eventTypes: Array.from(finalizerEventTypes), + reason: finalizerErrorReason ?? 'no_text', + finalizerPoisoned, + }, + 'OpenCode no-tool finalizer produced no usable text - yielding deterministic recovery text', + ); + yield { + type: 'text', + catId: this.catId, + content: buildOpenCodePostToolFallbackText(params.trace, finalizerErrorReason ?? 'no_text'), + textMode: params.textMode, + metadata: params.metadata, + timestamp: Date.now(), + }; + } + } + + private recoverSilentCompletionText(sessionId: string | undefined, messageId: string | undefined): string | null { + const recovered = recoverOpenCodeSilentCompletion({ + sessionId, + messageId, + ...(this.opencodeDbPath ? { overridePath: this.opencodeDbPath } : {}), + }); + if (recovered.reason && recovered.reason !== 'missing_db' && recovered.reason !== 'no_text') { + log.warn( + { + catId: this.catId, + sessionIdPrefix: identifierPrefix(sessionId), + messageIdPrefix: identifierPrefix(messageId), + dbPathSource: recovered.source, + reason: recovered.reason, + }, + 'Failed to recover OpenCode silent completion text from local SQLite state', + ); + } + return recovered.text; + } + + private buildNoToolFinalizerArgs(prompt: string, sessionId: string | undefined, model: string): string[] { + const args = ['run', '--pure', '--agent', OPENCODE_NO_TOOL_FINALIZER_AGENT]; + if (sessionId) args.push('--session', sessionId); + if (model) args.push('-m', model); + args.push('--format', 'json', prompt); + return args; + } + + private buildNoToolFinalizerEnv(childEnv: Record): Record { + return { + ...childEnv, + [OPENCODE_CONFIG_CONTENT_ENV]: JSON.stringify(buildOpenCodeNoToolFinalizerConfig()), + }; + } + + private getNoToolFinalizerBoundaryFailure(): string | null { + if (hasOpenCodeManagedConfig({ managedConfigPaths: this.opencodeManagedConfigPaths })) { + return 'managed_config_present'; + } + return null; + } + private buildArgs( prompt: string, sessionId?: string, diff --git a/packages/api/src/domains/cats/services/agents/providers/opencode-recovery.ts b/packages/api/src/domains/cats/services/agents/providers/opencode-recovery.ts new file mode 100644 index 0000000000..0de6bbe043 --- /dev/null +++ b/packages/api/src/domains/cats/services/agents/providers/opencode-recovery.ts @@ -0,0 +1,436 @@ +import { existsSync, readdirSync } from 'node:fs'; +import { homedir, platform as osPlatform } from 'node:os'; +import { join, posix, win32 } from 'node:path'; +import Database from 'better-sqlite3'; +import { sanitizeCliStderr } from '../../../../../utils/sanitize-cli-stderr.js'; + +export const OPENCODE_DB_ENV = 'OPENCODE_DB'; +export const OPENCODE_CONFIG_CONTENT_ENV = 'OPENCODE_CONFIG_CONTENT'; +export const OPENCODE_NO_TOOL_FINALIZER_AGENT = 'cat-cafe-no-tool-finalizer'; +export const OPENCODE_NO_TOOL_PERMISSION = { + '*': 'deny', + read: 'deny', + list: 'deny', + glob: 'deny', + grep: 'deny', + lsp: 'deny', + skill: 'deny', + webfetch: 'deny', + websearch: 'deny', + edit: 'deny', + write: 'deny', + apply_patch: 'deny', + bash: 'deny', + shell: 'deny', + task: 'deny', + subagent: 'deny', + todowrite: 'deny', + todoread: 'deny', + question: 'deny', + external_directory: 'deny', + doom_loop: 'deny', +} as const; +export const OPENCODE_NO_TOOL_FLAGS = { + read: false, + list: false, + glob: false, + grep: false, + lsp: false, + skill: false, + webfetch: false, + websearch: false, + edit: false, + write: false, + apply_patch: false, + bash: false, + shell: false, + task: false, + subagent: false, + todowrite: false, + todoread: false, + question: false, +} as const; + +const MAX_OPENCODE_VISIBLE_TOOL_OUTPUT_CHARS = 4_000; +const DEFAULT_SAFE_TOOL_OUTPUT_CHARS = 1_000; +const OPENCODE_DB_FILE_PATTERN = /^opencode(?:[-_.][\w-]+)?\.db$/; +const OPENCODE_CONFIG_FILENAMES = ['opencode.json', 'opencode.jsonc'] as const; + +export interface OpenCodeToolTrace { + toolName: string; + status?: string; + output?: unknown; +} + +export interface OpenCodeMessageRef { + sessionId?: string; + messageId?: string; +} + +export interface OpenCodeDbCandidate { + path: string; + source: 'override' | 'OPENCODE_DB' | 'xdg' | 'localappdata' | 'darwin' | 'default'; +} + +export interface OpenCodeDbResolutionOptions { + overridePath?: string; + env?: Record; + homeDir?: string; + platform?: NodeJS.Platform; +} + +export interface OpenCodeManagedConfigDetectionOptions { + env?: Record; + homeDir?: string; + platform?: NodeJS.Platform; + managedConfigPaths?: readonly string[]; +} + +export interface OpenCodeSilentRecoveryOptions extends OpenCodeDbResolutionOptions { + sessionId?: string; + messageId?: string; +} + +export interface OpenCodeSilentRecoveryResult { + text: string | null; + source?: OpenCodeDbCandidate['source']; + reason?: 'missing_ref' | 'missing_db' | 'schema_unavailable' | 'no_text'; +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null; +} + +function truncateForVisibleText(value: string, maxChars: number): string { + if (value.length <= maxChars) return value; + return `${value.slice(0, maxChars)}\n...[truncated ${value.length - maxChars} chars]`; +} + +function stringifyForVisibleText(value: unknown, maxChars: number): string { + if (value == null) return ''; + if (typeof value === 'string') return truncateForVisibleText(value, maxChars); + const jsonText = JSON.stringify(value, null, 2); + return truncateForVisibleText(typeof jsonText === 'string' ? jsonText : String(value), maxChars); +} + +function redactGenericAbsolutePaths(value: string): string { + return value + .replace( + /(^|[\s"'(=[{,])\\\\(?:[^\s"'<>|\\]+\\)+[^\s"'<>|\\]+/g, + (_match, prefix: string) => `${prefix}[redacted path]`, + ) + .replace(/\b[A-Za-z]:\\(?:[^\s"'<>|]+\\)*[^\s"'<>|]+/g, '[redacted path]') + .replace( + /(^|[\s"'(=[{,])\/(?!\/)[^\s"'<>|:]+(?:\/[^\s"'<>|:]+)*/g, + (_match, prefix: string) => `${prefix}[redacted path]`, + ); +} + +function redactShortProviderSecrets(value: string): string { + return value.replace(/\bsk-[A-Za-z0-9_-]{8,}\b/g, '[redacted secret]'); +} + +export function projectSafeOpenCodeToolOutput(value: unknown, maxChars = DEFAULT_SAFE_TOOL_OUTPUT_CHARS): string { + const raw = stringifyForVisibleText(value, MAX_OPENCODE_VISIBLE_TOOL_OUTPUT_CHARS); + if (raw.trim().length === 0) return ''; + const sanitized = redactGenericAbsolutePaths(redactShortProviderSecrets(sanitizeCliStderr(raw))); + return truncateForVisibleText(sanitized, maxChars); +} + +export function extractOpenCodeToolTrace(event: unknown): OpenCodeToolTrace | null { + if (!isRecord(event)) return null; + if (event.type !== 'tool_use') return null; + const part = isRecord(event.part) ? event.part : {}; + const state = isRecord(part.state) ? part.state : {}; + const toolName = typeof part.tool === 'string' && part.tool.length > 0 ? part.tool : 'unknown'; + const trace: OpenCodeToolTrace = { toolName }; + if (typeof state.status === 'string' && state.status.length > 0) { + trace.status = state.status; + } + if ('output' in state) { + trace.output = state.output; + } + return trace; +} + +export function buildOpenCodePostToolFinalizerPrompt(trace: OpenCodeToolTrace | null): string { + const toolName = trace ? trace.toolName : 'a tool'; + const outputText = projectSafeOpenCodeToolOutput( + trace ? trace.output : undefined, + MAX_OPENCODE_VISIBLE_TOOL_OUTPUT_CHARS, + ); + return [ + 'The previous OpenCode turn stopped immediately after a tool call and did not produce the final assistant text.', + 'Do not call any tools. Use only the existing session state and the sanitized latest tool result below to write the final answer to the user.', + `Latest tool: ${toolName}${trace?.status ? ` (${trace.status})` : ''}.`, + outputText.length > 0 ? `Latest tool output (sanitized):\n${outputText}` : 'No safe tool output was captured.', + 'If the available tool result is insufficient, state the limitation briefly instead of inventing details.', + ].join('\n\n'); +} + +export function buildOpenCodePostToolFallbackText(trace: OpenCodeToolTrace | null, reason: string): string { + const toolName = trace ? trace.toolName : 'a tool'; + const outputText = projectSafeOpenCodeToolOutput(trace ? trace.output : undefined); + const lines = [ + `OpenCode stopped after running \`${toolName}\` but did not produce a final text response.`, + `No-tool finalizer recovery did not produce text: ${reason}.`, + trace?.status ? `Tool status: ${trace.status}.` : undefined, + outputText.length > 0 ? `Latest tool output (sanitized):\n${outputText}` : 'No safe tool output was captured.', + 'This is a recovery message; review the sanitized tool output before treating the task as complete.', + ]; + return lines.filter((line): line is string => Boolean(line)).join('\n\n'); +} + +export function extractOpenCodeMessageRef(event: unknown): OpenCodeMessageRef | null { + if (!isRecord(event)) return null; + if (event.type !== 'step_start') return null; + const part = isRecord(event.part) ? event.part : {}; + const sessionId = + typeof part.sessionID === 'string' + ? part.sessionID + : typeof event.sessionID === 'string' + ? event.sessionID + : undefined; + const messageId = + typeof part.messageID === 'string' + ? part.messageID + : typeof event.messageID === 'string' + ? event.messageID + : undefined; + if (!sessionId && !messageId) return null; + return { ...(sessionId ? { sessionId } : {}), ...(messageId ? { messageId } : {}) }; +} + +function extractOpenCodePartText(data: string): string | null { + try { + const parsed = JSON.parse(data) as unknown; + if (!isRecord(parsed)) return null; + if (parsed.type !== 'text' || typeof parsed.text !== 'string') return null; + const text = parsed.text; + return text.trim().length > 0 ? text : null; + } catch { + return null; + } +} + +function addCandidate( + candidates: OpenCodeDbCandidate[], + path: string | undefined, + source: OpenCodeDbCandidate['source'], +) { + if (!path) return; + if (candidates.some((candidate) => candidate.path === path)) return; + candidates.push({ path, source }); +} + +function addDbFilesFromDirectory( + candidates: OpenCodeDbCandidate[], + dir: string | undefined, + source: OpenCodeDbCandidate['source'], +) { + if (!dir || !existsSync(dir)) return; + try { + for (const entry of readdirSync(dir).sort()) { + if (OPENCODE_DB_FILE_PATTERN.test(entry)) { + addCandidate(candidates, join(dir, entry), source); + } + } + } catch { + return; + } +} + +function openCodeDataDir( + env: Record, + homeDir: string, + runtimePlatform: NodeJS.Platform, +): string { + if (runtimePlatform === 'win32') { + return env.LOCALAPPDATA ? join(env.LOCALAPPDATA, 'opencode') : join(homeDir, 'AppData', 'Local', 'opencode'); + } + if (runtimePlatform === 'darwin') { + return join(homeDir, 'Library', 'Application Support', 'opencode'); + } + return env.XDG_DATA_HOME ? join(env.XDG_DATA_HOME, 'opencode') : join(homeDir, '.local', 'share', 'opencode'); +} + +function isAbsoluteForPlatform(path: string, runtimePlatform: NodeJS.Platform): boolean { + if (runtimePlatform === 'win32') return win32.isAbsolute(path) || path.startsWith('/'); + return posix.isAbsolute(path) || /^[A-Za-z]:[\\/]/.test(path); +} + +function resolveOpenCodeDbEnvPath( + value: string | undefined, + env: Record, + homeDir: string, + runtimePlatform: NodeJS.Platform, +): string | undefined { + if (!value) return undefined; + return isAbsoluteForPlatform(value, runtimePlatform) + ? value + : join(openCodeDataDir(env, homeDir, runtimePlatform), value); +} + +export function resolveOpenCodeDbCandidates(options: OpenCodeDbResolutionOptions = {}): OpenCodeDbCandidate[] { + const env = options.env ?? process.env; + const homeDir = options.homeDir ?? homedir(); + const runtimePlatform = options.platform ?? osPlatform(); + const candidates: OpenCodeDbCandidate[] = []; + + addCandidate(candidates, options.overridePath, 'override'); + addCandidate( + candidates, + resolveOpenCodeDbEnvPath(env[OPENCODE_DB_ENV], env, homeDir, runtimePlatform), + 'OPENCODE_DB', + ); + + const xdgOpenCodeDir = env.XDG_DATA_HOME ? join(env.XDG_DATA_HOME, 'opencode') : undefined; + addCandidate(candidates, xdgOpenCodeDir ? join(xdgOpenCodeDir, 'opencode.db') : undefined, 'xdg'); + addDbFilesFromDirectory(candidates, xdgOpenCodeDir, 'xdg'); + + const localAppDataOpenCodeDir = env.LOCALAPPDATA ? join(env.LOCALAPPDATA, 'opencode') : undefined; + if (runtimePlatform === 'win32' || localAppDataOpenCodeDir) { + addCandidate( + candidates, + localAppDataOpenCodeDir ? join(localAppDataOpenCodeDir, 'opencode.db') : undefined, + 'localappdata', + ); + addDbFilesFromDirectory(candidates, localAppDataOpenCodeDir, 'localappdata'); + } + + if (runtimePlatform === 'darwin') { + const darwinOpenCodeDir = join(homeDir, 'Library', 'Application Support', 'opencode'); + addCandidate(candidates, join(darwinOpenCodeDir, 'opencode.db'), 'darwin'); + addDbFilesFromDirectory(candidates, darwinOpenCodeDir, 'darwin'); + } + + const defaultOpenCodeDir = join(homeDir, '.local', 'share', 'opencode'); + addCandidate(candidates, join(defaultOpenCodeDir, 'opencode.db'), 'default'); + addDbFilesFromDirectory(candidates, defaultOpenCodeDir, 'default'); + + return candidates; +} + +function defaultManagedConfigPaths(options: OpenCodeManagedConfigDetectionOptions): string[] { + const env = options.env ?? process.env; + const runtimePlatform = options.platform ?? osPlatform(); + const paths: string[] = []; + const addConfigDir = (dir: string | undefined) => { + if (!dir) return; + for (const filename of OPENCODE_CONFIG_FILENAMES) paths.push(join(dir, filename)); + }; + + if (runtimePlatform === 'win32') { + addConfigDir(env.ProgramData ? join(env.ProgramData, 'opencode') : undefined); + } else if (runtimePlatform === 'darwin') { + addConfigDir('/Library/Application Support/opencode'); + paths.push('/Library/Managed Preferences/ai.opencode.managed.plist'); + if (env.USER) paths.push(join('/Library/Managed Preferences', env.USER, 'ai.opencode.managed.plist')); + } else { + addConfigDir('/etc/opencode'); + } + + if (options.managedConfigPaths) paths.push(...options.managedConfigPaths); + return paths.filter((path) => path.length > 0); +} + +export function hasOpenCodeManagedConfig(options: OpenCodeManagedConfigDetectionOptions = {}): boolean { + return defaultManagedConfigPaths(options).some((path) => existsSync(path)); +} + +export function buildOpenCodeNoToolFinalizerConfig(): Record { + return { + $schema: 'https://opencode.ai/config.json', + default_agent: OPENCODE_NO_TOOL_FINALIZER_AGENT, + permission: OPENCODE_NO_TOOL_PERMISSION, + tools: OPENCODE_NO_TOOL_FLAGS, + agent: { + [OPENCODE_NO_TOOL_FINALIZER_AGENT]: { + mode: 'primary', + permission: OPENCODE_NO_TOOL_PERMISSION, + tools: OPENCODE_NO_TOOL_FLAGS, + }, + }, + }; +} + +export function recoverOpenCodeSilentCompletion(options: OpenCodeSilentRecoveryOptions): OpenCodeSilentRecoveryResult { + if (!options.sessionId || !options.messageId) return { text: null, reason: 'missing_ref' }; + + let sawExistingDb = false; + let sawSchemaError = false; + let schemaErrorSource: OpenCodeDbCandidate['source'] | undefined; + for (const candidate of resolveOpenCodeDbCandidates(options)) { + if (!existsSync(candidate.path)) continue; + sawExistingDb = true; + + let db: Database.Database | undefined; + try { + db = new Database(candidate.path, { readonly: true, fileMustExist: true }); + const rows = db + .prepare( + ` + SELECT data + FROM part + WHERE session_id = ? AND message_id = ? + ORDER BY time_created ASC + `, + ) + .all(options.sessionId, options.messageId) as Array<{ data: unknown }>; + const text = rows + .map((row) => (typeof row.data === 'string' ? extractOpenCodePartText(row.data) : null)) + .filter((partText): partText is string => partText !== null) + .join(''); + if (text.trim().length > 0) return { text, source: candidate.source }; + } catch { + sawSchemaError = true; + schemaErrorSource = candidate.source; + } finally { + db?.close(); + } + } + + if (!sawExistingDb) return { text: null, reason: 'missing_db' }; + if (sawSchemaError) return { text: null, source: schemaErrorSource, reason: 'schema_unavailable' }; + return { text: null, reason: 'no_text' }; +} + +export function identifierPrefix(value: string | undefined): string | undefined { + if (!value) return undefined; + return value.slice(0, 8); +} + +export class SessionSingleFlight { + private readonly tails = new Map>(); + + async *run(key: string | undefined, factory: () => AsyncIterable): AsyncIterable { + const release = await this.acquire(key); + try { + yield* factory(); + } finally { + release(); + } + } + + private async acquire(key: string | undefined): Promise<() => void> { + if (!key) return () => {}; + + const previous = this.tails.get(key) ?? Promise.resolve(); + let releaseCurrent!: () => void; + const current = new Promise((resolve) => { + releaseCurrent = resolve; + }); + const tail = previous.catch(() => undefined).then(() => current); + this.tails.set(key, tail); + + await previous.catch(() => undefined); + + let released = false; + return () => { + if (released) return; + released = true; + releaseCurrent(); + if (this.tails.get(key) === tail) this.tails.delete(key); + }; + } +} diff --git a/packages/api/test/opencode-agent-service.test.js b/packages/api/test/opencode-agent-service.test.js index 160f3b781c..6e6bd83586 100644 --- a/packages/api/test/opencode-agent-service.test.js +++ b/packages/api/test/opencode-agent-service.test.js @@ -1,7 +1,11 @@ import assert from 'node:assert/strict'; import { EventEmitter } from 'node:events'; +import { mkdtempSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; import { PassThrough } from 'node:stream'; import { describe, mock, test } from 'node:test'; +import Database from 'better-sqlite3'; import { OpenCodeAgentService, summarizeOpenCodeEnvForDebug, @@ -70,6 +74,46 @@ async function collect(iterable) { return messages; } +async function delay(ms) { + await new Promise((resolve) => setTimeout(resolve, ms)); +} + +async function waitFor(predicate, description) { + for (let i = 0; i < 50; i++) { + if (predicate()) return; + await delay(5); + } + assert.fail(`timed out waiting for ${description}`); +} + +function createOpenCodeRecoveryDb({ sessionId, messageId, text }) { + const dir = mkdtempSync(join(tmpdir(), 'cat-cafe-opencode-db-')); + const dbPath = join(dir, 'opencode.db'); + const db = new Database(dbPath); + db.exec(` + CREATE TABLE part ( + id text PRIMARY KEY, + message_id text NOT NULL, + session_id text NOT NULL, + time_created integer NOT NULL, + time_updated integer NOT NULL, + data text NOT NULL + ); + `); + db.prepare( + 'INSERT INTO part (id, message_id, session_id, time_created, time_updated, data) VALUES (?, ?, ?, ?, ?, ?)', + ).run( + 'prt_recovered_text', + messageId, + sessionId, + 1780915410601, + 1780915410687, + JSON.stringify({ type: 'text', text }), + ); + db.close(); + return dbPath; +} + // ── opencode JSON event fixtures ── const STEP_START = { @@ -116,6 +160,11 @@ const STEP_FINISH = { }, }; +const STEP_FINISH_TOOL_CALLS = { + ...STEP_FINISH, + part: { ...STEP_FINISH.part, reason: 'tool-calls' }, +}; + // CodeAgent 3.0 → OpenCode facade: translate script may drop usage, yielding // step_finish without tokens. This fixture pins the missing-usage alert path. const STEP_FINISH_NO_TOKENS = { @@ -944,6 +993,33 @@ describe('OpenCodeAgentService', () => { ); }); + test('step_start-only NDJSON recovers assistant text from OpenCode SQLite by session/message id', async () => { + const proc = createMockProcess(); + const spawnFn = mock.fn(() => proc); + const dbPath = createOpenCodeRecoveryDb({ + sessionId: STEP_START.sessionID, + messageId: STEP_START.part.messageID, + text: 'Recovered from OpenCode SQLite.', + }); + const service = new OpenCodeAgentService({ + catId: 'opencode', + spawnFn, + model: 'deepseek-chat', + opencodeDbPath: dbPath, + }); + const promise = collect(service.invoke('Test silent', { invocationId: 'inv-silent-sqlite-recovery' })); + + emitOpenCodeEvents(proc, [STEP_START]); + const messages = await promise; + + const recovered = messages.find((m) => m.type === 'text'); + assert.equal(recovered?.content, 'Recovered from OpenCode SQLite.'); + assert.ok( + !messages.some((m) => m.type === 'system_info' && m.metadata?.cliDiagnostics?.reasonCode === 'silent_completion'), + 'SQLite recovery supplies text, so silent_completion must not fire', + ); + }); + test('AC-G3: text event present → does NOT yield silent_completion (no false positive)', async () => { const proc = createMockProcess(); const spawnFn = mock.fn(() => proc); @@ -963,17 +1039,33 @@ describe('OpenCodeAgentService', () => { }); // F212 Phase G R1 P1 (cloud codex catch on 1d519e7f2): tool-only turns are legitimate - // task completions per F215 AC-B3. Tool events that complete the user's request without - // a text response MUST NOT be flagged as silent_completion. - test('AC-G3 R1 P1: tool_use event without text → does NOT yield silent_completion', async () => { + // task completions per F215 AC-B3. Tool events must not be mislabeled as + // silent_completion and must not be routed through the post-tool finalizer. + test('AC-G3 R1 P1: pure tool_use completion is preserved without silent_completion or finalizer', async () => { const proc = createMockProcess(); - const spawnFn = mock.fn(() => proc); + const finalizerProc = createMockProcess(); + let spawnCalls = 0; + const spawnFn = mock.fn(() => { + spawnCalls++; + if (spawnCalls === 2) { + process.nextTick(() => { + emitOpenCodeEvents(finalizerProc, [ + STEP_START, + { ...TEXT_RESPONSE, part: { ...TEXT_RESPONSE.part, text: 'unexpected finalizer text' } }, + STEP_FINISH, + ]); + }); + return finalizerProc; + } + return proc; + }); const service = new OpenCodeAgentService({ catId: 'opencode', spawnFn, model: 'claude-haiku-4-5' }); const promise = collect(service.invoke('Use tools')); - // step_start + tool_use only — no TEXT_RESPONSE. Per F215 AC-B3 this is a valid - // tool-only completion path. silent_completion would mislabel it as a provider error. + // step_start + tool_use only — no TEXT_RESPONSE from the main CLI invocation. + // This is not a provider error, but users still need final assistant text. emitOpenCodeEvents(proc, [STEP_START, TOOL_USE, STEP_FINISH]); const messages = await promise; + const textMsgs = messages.filter((m) => m.type === 'text'); const silentError = messages.find( (m) => m.type === 'error' && m.metadata?.cliDiagnostics?.reasonCode === 'silent_completion', @@ -994,12 +1086,279 @@ describe('OpenCodeAgentService', () => { messages.some((m) => m.type === 'tool_use'), 'tool_use yield confirms event reached transformer', ); + assert.equal(spawnCalls, 1, 'pure tool-only completion must not start a no-tool finalizer invocation'); + assert.equal(textMsgs.length, 0, 'pure tool-only completion remains non-user-visible text'); }); // Issue #1208: CodeAgent 3.0 → OpenCode translate may drop token usage. // When the CLI produces events but no step_finish carries tokens, auto-handoff // based on context fill cannot fire. The service must surface a persistent // visible alert so users know automatic handoff is unavailable. + test('post-tool completion gap runs a no-tool finalizer and replaces the incomplete prelude', async () => { + const proc = createMockProcess(); + const finalizerProc = createMockProcess(); + let spawnCalls = 0; + const spawnFn = mock.fn(() => { + spawnCalls++; + if (spawnCalls === 2) { + process.nextTick(() => { + emitOpenCodeEvents(finalizerProc, [ + STEP_START, + { + ...TEXT_RESPONSE, + part: { ...TEXT_RESPONSE.part, text: 'The active model is deepseek-v4-pro.' }, + }, + STEP_FINISH, + ]); + }); + return finalizerProc; + } + return proc; + }); + const service = new OpenCodeAgentService({ catId: 'opencode', spawnFn, model: 'deepseek-v4-pro' }); + const promise = collect(service.invoke('Check the active model and report it')); + + emitOpenCodeEvents(proc, [ + STEP_START, + { + ...TEXT_RESPONSE, + part: { ...TEXT_RESPONSE.part, text: 'Let me verify from the actual config rather than guessing.' }, + }, + { + ...TOOL_USE, + part: { + ...TOOL_USE.part, + tool: 'read', + state: { + status: 'completed', + input: { filePath: '.cat-cafe/cat-catalog.json', offset: 210, limit: 50 }, + output: '"defaultModel": "deepseek-v4-pro"', + }, + }, + }, + STEP_FINISH_TOOL_CALLS, + ]); + + const messages = await promise; + const textMsgs = messages.filter((m) => m.type === 'text'); + + assert.equal(spawnCalls, 2, 'post-tool gap should start exactly one no-tool finalizer invocation'); + const finalizerArgs = spawnFn.mock.calls[1].arguments[1]; + assert.ok(finalizerArgs.includes('--session'), `finalizer must resume the same OpenCode session: ${finalizerArgs}`); + assert.equal(finalizerArgs[finalizerArgs.indexOf('--session') + 1], 'ses_test123'); + assert.ok(finalizerArgs.includes('--agent'), `finalizer must use a dedicated no-tool agent: ${finalizerArgs}`); + assert.equal(finalizerArgs[finalizerArgs.indexOf('--agent') + 1], 'cat-cafe-no-tool-finalizer'); + const finalizerConfig = JSON.parse(spawnFn.mock.calls[1].arguments[2].env.OPENCODE_CONFIG_CONTENT); + assert.equal(finalizerConfig.permission['*'], 'deny', 'finalizer env must deny tool execution'); + assert.equal( + finalizerConfig.agent['cat-cafe-no-tool-finalizer'].permission['*'], + 'deny', + 'dedicated finalizer agent must deny tool execution', + ); + assert.equal(textMsgs.length, 2, 'finalizer should add one replacement text after the incomplete prelude'); + assert.equal(textMsgs[0].content, 'Let me verify from the actual config rather than guessing.'); + assert.equal(textMsgs.at(-1)?.textMode, 'replace'); + assert.equal(textMsgs.at(-1)?.content, 'The active model is deepseek-v4-pro.'); + assert.ok( + !messages.some((m) => m.type === 'system_info' && m.metadata?.cliDiagnostics?.reasonCode === 'silent_completion'), + 'post-tool finalizer is a recovery, not silent_completion', + ); + }); + + test('post-tool finalizer tool attempt poisons later text and falls back safely', async () => { + const proc = createMockProcess(); + const finalizerProc = createMockProcess(); + let spawnCalls = 0; + const spawnFn = mock.fn(() => { + spawnCalls++; + if (spawnCalls === 2) { + process.nextTick(() => { + emitOpenCodeEvents(finalizerProc, [ + STEP_START, + { + ...TOOL_USE, + part: { ...TOOL_USE.part, tool: 'read', state: { status: 'completed', input: {}, output: 'forbidden' } }, + }, + { + ...TEXT_RESPONSE, + part: { ...TEXT_RESPONSE.part, text: 'accepted-after-tool' }, + }, + STEP_FINISH, + ]); + }); + return finalizerProc; + } + return proc; + }); + const service = new OpenCodeAgentService({ catId: 'opencode', spawnFn, model: 'deepseek-v4-pro' }); + const promise = collect(service.invoke('Check the active model and report it')); + + emitOpenCodeEvents(proc, [ + STEP_START, + { + ...TEXT_RESPONSE, + part: { ...TEXT_RESPONSE.part, text: 'Let me check that.' }, + }, + { + ...TOOL_USE, + part: { ...TOOL_USE.part, tool: 'read', state: { status: 'completed', output: 'model=deepseek-v4-pro' } }, + }, + STEP_FINISH_TOOL_CALLS, + ]); + + const messages = await promise; + const textMsgs = messages.filter((m) => m.type === 'text'); + + assert.equal(spawnCalls, 2); + assert.ok( + !textMsgs.some((m) => m.content === 'accepted-after-tool'), + 'any finalizer tool_use must poison the whole finalizer result before later text is accepted', + ); + assert.equal(textMsgs.at(-1)?.textMode, 'replace'); + assert.match(String(textMsgs.at(-1)?.content), /tool_use_blocked/); + }); + + test('post-tool finalizer fails closed before spawn when managed OpenCode config can override permissions', async () => { + const proc = createMockProcess(); + const finalizerProc = createMockProcess(); + const managedConfigPath = join(mkdtempSync(join(tmpdir(), 'cat-cafe-opencode-managed-')), 'opencode.json'); + writeFileSync(managedConfigPath, JSON.stringify({ permission: { '*': 'allow' } }), 'utf8'); + let spawnCalls = 0; + const spawnFn = mock.fn(() => { + spawnCalls++; + if (spawnCalls === 2) { + process.nextTick(() => { + emitOpenCodeEvents(finalizerProc, [ + STEP_START, + { ...TEXT_RESPONSE, part: { ...TEXT_RESPONSE.part, text: 'managed override would have run' } }, + STEP_FINISH, + ]); + }); + return finalizerProc; + } + return proc; + }); + const service = new OpenCodeAgentService({ + catId: 'opencode', + spawnFn, + model: 'deepseek-v4-pro', + opencodeManagedConfigPaths: [managedConfigPath], + }); + const promise = collect(service.invoke('Check the active model and report it')); + + emitOpenCodeEvents(proc, [ + STEP_START, + { + ...TEXT_RESPONSE, + part: { ...TEXT_RESPONSE.part, text: 'Let me check that.' }, + }, + { + ...TOOL_USE, + part: { ...TOOL_USE.part, tool: 'read', state: { status: 'completed', output: 'model=deepseek-v4-pro' } }, + }, + STEP_FINISH_TOOL_CALLS, + ]); + + const messages = await promise; + const textMsgs = messages.filter((m) => m.type === 'text'); + + assert.equal( + spawnCalls, + 1, + 'managed config precedence must block finalizer before a second OpenCode process starts', + ); + assert.equal(textMsgs.at(-1)?.textMode, 'replace'); + assert.match(String(textMsgs.at(-1)?.content), /managed_config_present/); + }); + + test('post-tool deterministic fallback redacts raw tool output secrets and absolute paths', async () => { + const proc = createMockProcess(); + const finalizerProc = createMockProcess(); + let spawnCalls = 0; + const spawnFn = mock.fn(() => { + spawnCalls++; + if (spawnCalls === 2) { + process.nextTick(() => { + emitOpenCodeEvents(finalizerProc, [STEP_START, STEP_FINISH]); + }); + return finalizerProc; + } + return proc; + }); + const service = new OpenCodeAgentService({ catId: 'opencode', spawnFn, model: 'deepseek-v4-pro' }); + const promise = collect(service.invoke('Read config')); + + emitOpenCodeEvents(proc, [ + STEP_START, + { + ...TEXT_RESPONSE, + part: { ...TEXT_RESPONSE.part, text: 'Reading config.' }, + }, + { + ...TOOL_USE, + part: { + ...TOOL_USE.part, + tool: 'read', + state: { + status: 'completed', + output: + 'token=sk-review-secret-123 path=C:\\Users\\Alice\\secrets\\config.json also /Users/alice/.ssh/id_rsa', + }, + }, + }, + STEP_FINISH_TOOL_CALLS, + ]); + + const messages = await promise; + const fallback = messages.filter((m) => m.type === 'text').at(-1); + const content = String(fallback?.content); + + assert.equal(spawnCalls, 2); + assert.equal(fallback?.textMode, 'replace'); + assert.doesNotMatch(content, /sk-review-secret/); + assert.doesNotMatch(content, /C:\\Users\\Alice/); + assert.doesNotMatch(content, /\/Users\/alice/); + assert.match(content, /\[redacted/); + }); + + test('same OpenCode session invocations are serialized through finalization', async () => { + let active = 0; + let maxActive = 0; + let started = 0; + const releases = []; + const spawnCliOverride = async function* () { + active++; + started++; + maxActive = Math.max(maxActive, active); + await new Promise((resolve) => releases.push(resolve)); + yield STEP_START; + yield TEXT_RESPONSE; + yield STEP_FINISH; + active--; + }; + const service = new OpenCodeAgentService({ + catId: 'opencode', + model: 'deepseek-v4-pro', + spawnFn: mock.fn(() => { + throw new Error('spawnCliOverride must own this test'); + }), + }); + const first = collect(service.invoke('first', { sessionId: 'ses_shared', spawnCliOverride })); + const second = collect(service.invoke('second', { sessionId: 'ses_shared', spawnCliOverride })); + + await waitFor(() => started >= 1, 'first session process to start'); + await delay(20); + assert.equal(maxActive, 1, 'second same-session invoke must not spawn before first completes'); + releases[0](); + await first; + + await waitFor(() => started === 2, 'second session process to start after first completes'); + releases[1](); + await second; + + assert.equal(maxActive, 1, 'same-session invokes must remain single-flight'); + }); + test('issue #1208: opencode events without usage telemetry yield warning alert', async () => { const proc = createMockProcess(); const spawnFn = mock.fn(() => proc); diff --git a/packages/api/test/opencode-omoc-isolation.test.js b/packages/api/test/opencode-omoc-isolation.test.js index 6510da7381..8dc718e413 100644 --- a/packages/api/test/opencode-omoc-isolation.test.js +++ b/packages/api/test/opencode-omoc-isolation.test.js @@ -61,7 +61,18 @@ describe('OMOC Sisyphus Isolation (AC-9)', () => { test('full OMOC session: all events stay within opencode boundary', async () => { const proc = createMockProcess(); - const spawnFn = mock.fn(() => proc); + const finalizerProc = createMockProcess(); + let spawnCalls = 0; + const spawnFn = mock.fn(() => { + spawnCalls++; + if (spawnCalls === 2) { + process.nextTick(() => { + emitOpenCodeEvents(finalizerProc, [OMOC_STEP_START, OMOC_SISYPHUS_TEXT, OMOC_STEP_FINISH]); + }); + return finalizerProc; + } + return proc; + }); const service = new OpenCodeAgentService({ catId: 'opencode', spawnFn, model: 'claude-sonnet-4-6' }); const promise = collect(service.invoke('Analyze and fix the auth module')); @@ -76,6 +87,8 @@ describe('OMOC Sisyphus Isolation (AC-9)', () => { ]); const messages = await promise; + assert.strictEqual(spawnCalls, 1, 'valid OMOC completion must not start a no-tool finalizer'); + const toolUses = messages.filter((m) => m.type === 'tool_use'); assert.ok(toolUses.length >= 4, `expected >=4 tool_use, got ${toolUses.length}`); for (const tu of toolUses) { diff --git a/packages/api/test/opencode-recovery.test.js b/packages/api/test/opencode-recovery.test.js new file mode 100644 index 0000000000..6b8e3e0e06 --- /dev/null +++ b/packages/api/test/opencode-recovery.test.js @@ -0,0 +1,202 @@ +import assert from 'node:assert/strict'; +import { mkdirSync, mkdtempSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { describe, test } from 'node:test'; +import Database from 'better-sqlite3'; +import { + projectSafeOpenCodeToolOutput, + recoverOpenCodeSilentCompletion, + resolveOpenCodeDbCandidates, +} from '../dist/domains/cats/services/agents/providers/opencode-recovery.js'; + +function createPartDb(dbPath, rows) { + mkdirSync(dirname(dbPath), { recursive: true }); + const db = new Database(dbPath); + db.exec(` + CREATE TABLE part ( + id text PRIMARY KEY, + message_id text NOT NULL, + session_id text NOT NULL, + time_created integer NOT NULL, + time_updated integer NOT NULL, + data text NOT NULL + ); + `); + const insert = db.prepare( + 'INSERT INTO part (id, message_id, session_id, time_created, time_updated, data) VALUES (?, ?, ?, ?, ?, ?)', + ); + rows.forEach((row, index) => { + insert.run(`prt_${index}`, row.messageId, row.sessionId, 1780915410601 + index, 1780915410687 + index, row.data); + }); + db.close(); +} + +describe('opencode recovery boundary', () => { + test('recovers from OPENCODE_DB before default locations', () => { + const root = mkdtempSync(join(tmpdir(), 'cat-cafe-opencode-recovery-')); + const dbPath = join(root, 'custom', 'opencode-custom.db'); + createPartDb(dbPath, [ + { sessionId: 'ses_env', messageId: 'msg_env', data: JSON.stringify({ type: 'text', text: 'from env db' }) }, + ]); + + const recovered = recoverOpenCodeSilentCompletion({ + sessionId: 'ses_env', + messageId: 'msg_env', + env: { OPENCODE_DB: dbPath }, + homeDir: join(root, 'home'), + platform: 'linux', + }); + + assert.equal(recovered.text, 'from env db'); + assert.equal(recovered.source, 'OPENCODE_DB'); + }); + + test('resolves relative OPENCODE_DB under the OpenCode data root', () => { + const root = mkdtempSync(join(tmpdir(), 'cat-cafe-opencode-relative-db-')); + const xdgRoot = join(root, 'xdg'); + const dbPath = join(xdgRoot, 'opencode', 'custom.db'); + createPartDb(dbPath, [ + { + sessionId: 'ses_relative', + messageId: 'msg_relative', + data: JSON.stringify({ type: 'text', text: 'from relative env db' }), + }, + ]); + + const candidates = resolveOpenCodeDbCandidates({ + env: { OPENCODE_DB: 'custom.db', XDG_DATA_HOME: xdgRoot }, + homeDir: join(root, 'home'), + platform: 'linux', + }); + assert.equal(candidates[0].path, dbPath); + assert.equal(candidates[0].source, 'OPENCODE_DB'); + + const recovered = recoverOpenCodeSilentCompletion({ + sessionId: 'ses_relative', + messageId: 'msg_relative', + env: { OPENCODE_DB: 'custom.db', XDG_DATA_HOME: xdgRoot }, + homeDir: join(root, 'home'), + platform: 'linux', + }); + + assert.equal(recovered.text, 'from relative env db'); + assert.equal(recovered.source, 'OPENCODE_DB'); + }); + + test('discovers channel database files under XDG data root', () => { + const root = mkdtempSync(join(tmpdir(), 'cat-cafe-opencode-xdg-')); + const xdgRoot = join(root, 'xdg'); + const dbPath = join(xdgRoot, 'opencode', 'opencode-beta.db'); + createPartDb(dbPath, [ + { sessionId: 'ses_beta', messageId: 'msg_beta', data: JSON.stringify({ type: 'text', text: 'from beta db' }) }, + ]); + + const candidates = resolveOpenCodeDbCandidates({ + env: { XDG_DATA_HOME: xdgRoot }, + homeDir: join(root, 'home'), + platform: 'linux', + }); + assert.ok(candidates.some((candidate) => candidate.path === dbPath && candidate.source === 'xdg')); + + const recovered = recoverOpenCodeSilentCompletion({ + sessionId: 'ses_beta', + messageId: 'msg_beta', + env: { XDG_DATA_HOME: xdgRoot }, + homeDir: join(root, 'home'), + platform: 'linux', + }); + + assert.equal(recovered.text, 'from beta db'); + assert.equal(recovered.source, 'xdg'); + }); + + test('fails closed when db is missing or schema drifts', () => { + const root = mkdtempSync(join(tmpdir(), 'cat-cafe-opencode-schema-')); + const missing = recoverOpenCodeSilentCompletion({ + sessionId: 'ses_missing', + messageId: 'msg_missing', + env: {}, + homeDir: join(root, 'home'), + platform: 'linux', + }); + assert.equal(missing.text, null); + assert.equal(missing.reason, 'missing_db'); + + const driftDbPath = join(root, 'drift', 'opencode.db'); + mkdirSync(dirname(driftDbPath), { recursive: true }); + const db = new Database(driftDbPath); + db.exec('CREATE TABLE part_v2 (data text NOT NULL);'); + db.close(); + + const drift = recoverOpenCodeSilentCompletion({ + sessionId: 'ses_missing', + messageId: 'msg_missing', + overridePath: driftDbPath, + env: {}, + homeDir: join(root, 'home'), + platform: 'linux', + }); + assert.equal(drift.text, null); + assert.equal(drift.source, 'override'); + assert.equal(drift.reason, 'schema_unavailable'); + }); + + test('skips malformed and non-text parts while joining same-message text parts', () => { + const root = mkdtempSync(join(tmpdir(), 'cat-cafe-opencode-parts-')); + const dbPath = join(root, 'opencode.db'); + createPartDb(dbPath, [ + { sessionId: 'ses_parts', messageId: 'msg_parts', data: '{not-json' }, + { + sessionId: 'ses_parts', + messageId: 'msg_parts', + data: JSON.stringify({ type: 'reasoning', text: 'hidden reasoning' }), + }, + { sessionId: 'ses_parts', messageId: 'msg_parts', data: JSON.stringify({ type: 'text', text: 'part A ' }) }, + { sessionId: 'ses_parts', messageId: 'msg_parts', data: JSON.stringify({ type: 'text', text: 'part B' }) }, + { sessionId: 'other', messageId: 'msg_parts', data: JSON.stringify({ type: 'text', text: 'wrong session' }) }, + ]); + + const recovered = recoverOpenCodeSilentCompletion({ + sessionId: 'ses_parts', + messageId: 'msg_parts', + overridePath: dbPath, + env: {}, + homeDir: join(root, 'home'), + platform: 'linux', + }); + + assert.equal(recovered.text, 'part A part B'); + }); + + test('safe tool-output projection redacts provider tokens and absolute paths', () => { + const projected = projectSafeOpenCodeToolOutput( + [ + 'token=sk-review-secret-123', + 'C:\\Users\\Alice\\secrets\\config.json', + '/Users/alice/.ssh/id_rsa', + '/usr/local/bin/opencode', + '/root/.ssh/id_rsa', + '/data/app/config.env', + '/srv/opencode/runtime.log', + '/secrets/acme/private.pem', + '/custom/private/file.txt', + '/single-segment-secret', + '\\\\server\\share\\private.txt', + ].join(' '), + ); + + assert.doesNotMatch(projected, /sk-review-secret/); + assert.doesNotMatch(projected, /C:\\Users\\Alice/); + assert.doesNotMatch(projected, /\/Users\/alice/); + assert.doesNotMatch(projected, /\/usr\/local/); + assert.doesNotMatch(projected, /\/root\/\.ssh/); + assert.doesNotMatch(projected, /\/data\/app/); + assert.doesNotMatch(projected, /\/srv\/opencode/); + assert.doesNotMatch(projected, /\/secrets\/acme/); + assert.doesNotMatch(projected, /\/custom\/private/); + assert.doesNotMatch(projected, /\/single-segment-secret/); + assert.doesNotMatch(projected, /\\\\server\\share/); + assert.match(projected, /\[redacted/); + }); +});