Skip to content

Commit 6c6c72d

Browse files
authored
πŸš€ feat: Decouple File Attachment Persistence from Preview Rendering (danny-avila#12957)
* πŸ—‚οΈ feat: add `status` lifecycle to file records for two-phase previews Schema and model foundation for decoupling the agent's final response from CPU-heavy office-format HTML extraction. - `MongoFile.status: 'pending' | 'ready' | 'failed'` (indexed) and `previewError?: string` mirror the lifecycle: phase-1 emits the file record at `pending` so the response is unblocked; phase-2 transitions to `ready` (with text/textFormat) or `failed` (with previewError) in the background. Absent for legacy records β€” clients treat that as `ready` for back-compat. - Mirror types added to `TFile` in data-provider so frontend cache consumers see the new fields. - New `sweepOrphanedPreviews(maxAgeMs)` method on the file model recovers stale `pending` records left behind by a process restart mid-extraction; transitions them to `failed` with `previewError: 'orphaned'`. Cheap because `status` is indexed. * ⚑ feat: two-phase code-execution preview flow (unblocks final response) The agent's final response no longer waits on CPU-heavy office HTML extraction. Phase-1 (download + storage save + DB record at `status: 'pending'`) is awaited as before; phase-2 (extract + `updateFile`) runs in the background with a hard 60s ceiling. Three flows, all funneling through `processCodeOutput` and updated to the new `{ file, finalize? }` return shape: - `callbacks.js` (chat-completions + Open Responses streaming): emit the phase-1 attachment immediately (carries `status: 'pending'` for office buckets so the UI shows "preparing preview…"), then fire-and-forget `finalize()`. If the SSE stream is still open when phase-2 lands, push an `attachment` update event with the same `file_id` so the client merges over the placeholder in place. - `tools.js` direct endpoint: same split β€” return the phase-1 metadata immediately, run extraction in the background. Client polls for the resolved record. `finalize()` wraps the existing 12s per-render timeout in a 60s outer `withTimeout`. The HTML-or-null contract from danny-avila#12934 is preserved: office types that fail extraction transition to `status: 'failed'` with `previewError: 'parser-error' | 'timeout'` rather than falling back to plain text (would be an XSS vector). Promises continue running after the HTTP response closes (Node doesn't kill them). The boot-time orphan sweep covers the only case that loses progress β€” actual process restart mid-extraction. `primeFiles` annotates the agent's `toolContext` line for prior-turn files: `(preview not yet generated)` for pending, `(preview unavailable: <reason>)` for failed. The model can volunteer "you can still download it" instead of pretending the preview is fine. `hasOfficeHtmlPath` exported from `@librechat/api` so `processCodeOutput` can decide whether a file expects a preview at all. * πŸ” feat: `GET /api/files/:file_id/preview` endpoint and boot orphan sweep - New `GET /api/files/:file_id/preview` route returns `{ status, text?, textFormat?, previewError? }`. The frontend's `useFilePreview` React Query hook polls this while phase-2 is in flight, then auto-stops on terminal status. ACL identical to the download route (reuses `fileAccess` middleware). Defaults `status` to `'ready'` for legacy records so back-compat is implicit. `text` only included when `status === 'ready'` and non-null β€” preserves the HTML-or-null security contract from danny-avila#12934. - `sweepOrphanedPreviews()` invoked on boot in both `server/index.js` and `server/experimental.js`. Recovers any `pending` records left behind by a process restart mid-extraction (the only case the in-process two-phase flow can't handle on its own). Fire-and-forget so a transient sweep failure doesn't block startup. * πŸ–₯️ feat: frontend two-phase preview consumer (polling + UI states) Wires the React side to the new lifecycle so the user sees what's happening with their file while phase-2 extraction runs in the background and after the response stream closes. - `useAttachmentHandler` upserts by `file_id` (was append-only) so the phase-2 SSE update event merges over the pending placeholder in place. Lightweight attachments without a `file_id` (web_search / file_search citations) keep the legacy append path. - `useFilePreview(file_id)` React Query hook with `refetchInterval: (data) => data?.status === 'pending' ? 2500 : false` so polling auto-stops on the first terminal response without the caller having to flip `enabled`. - `useAttachmentPreviewSync(attachment)` bridges polled data into `messageAttachmentsMap`. Polling enabled iff `status === 'pending' && isAnySubmitting` β€” per the design ask: active polling while the LLM is still generating, then quiet. Process-restart and post-stream cases are covered by polling on the next interaction. - `Attachment.tsx` renders a small `PreviewStatusIndicator` (spinner + "Preparing preview…" for pending, alert icon + "Preview unavailable" for failed) inside `FileAttachment`. Download button stays fully functional in both states. Two new English locale keys. - Data-provider scaffolding: `TFilePreview` type, `endpoints.filePreview`, `dataService.getFilePreview`, `QueryKeys.filePreview`. * πŸ§ͺ fix: stub `useAttachmentPreviewSync` in pre-existing Attachment test mocks The new `useAttachmentPreviewSync` hook is called unconditionally inside `FileAttachment` (added in the prior commit). Two pre-existing test files mock `~/hooks` to provide `useLocalize` only β€” the un-mocked preview hook reference resolved to undefined and crashed render with `(0 , _hooks.useAttachmentPreviewSync) is not a function` on the Ubuntu/Windows CI runners. Fix is local to the test mocks: add a no-op stub that returns `{ status: 'ready' }` so the component renders the legacy chip path. The two-phase preview behavior itself has its own dedicated suites (`useAttachmentHandler.spec.tsx`, `useAttachmentPreviewSync.spec.tsx`). * πŸ› fix: route phase-2 attachment update to current-run messageId Codex P1 review on PR danny-avila#12957. `processCodeOutput` intentionally preserves the original DB `messageId` across cross-turn filename reuse so `getCodeGeneratedFiles` can still trace a file back to the assistant message that originally produced it. The phase-1 SSE emit already routes by the current run's messageId β€” `processCodeOutput` runtime-overlays it via `Object.assign(file, { messageId, toolCallId })` and the callback writes `result.file` directly. Phase-2 was passing the raw `updateFile` return through `attachmentFromFileMetadata`, which read `messageId` straight off the DB record. On a turn-N run that re-emitted a filename from turn-1 (e.g. agent writes `output.csv` again), the phase-2 SSE update routed to `turn-1-msg` instead of `turn-N-msg`. Frontend's `useAttachmentHandler` upserts under the wrong messageAttachmentsMap slot β€” turn-N's pending chip stays stuck at "preparing preview…" while turn-1's already-resolved attachment gets re-merged. Fix: thread `runtimeMessageId` through `attachmentFromFileMetadata` and pass `metadata.run_id` from the phase-2 emit site. Mirrors how phase-1 sources its messageId. Tests cover the cross-turn reuse case plus the writableEnded / null-finalize / no-finalize paths to lock in the broader phase-2 emit contract. * πŸ› οΈ refactor: address codex audit findings (wire-shape parity, DRY, defensive catch) Comprehensive audit on PR danny-avila#12957. Resolves all valid findings: - **MAJOR #1 β€” Wire-shape parity**: phase-1 ships the full `fileMetadata` record over SSE; phase-2 was using a tight `attachmentFromFileMetadata` projection. Drop the projection and have phase-2 spread `{...updated, messageId, toolCallId}` so both events match the long-standing legacy phase-1 shape clients depend on. - **MAJOR danny-avila#2 β€” DRY**: extract `runPhase2Finalize({ finalize, fileId, onResolved })` into `process.js` (alongside `processCodeOutput` whose contract it pairs with). Both `callbacks.js` paths and `tools.js` now flow through it. Single catch path eliminates divergence surface β€” the fix landed in 01704d4f0 (cross-turn messageId routing) was a symptom of this duplication risk. - **MINOR danny-avila#3 β€” JSDoc accuracy**: `finalizePreview`'s buffer is bounded by `fileSizeLimit`, not the 1MB extractor cap. Updated and added a note about peak heap from queued buffers. - **MINOR danny-avila#4 β€” Defensive catch**: `runPhase2Finalize`'s catch attempts a best-effort `updateFile({ status: 'failed', previewError: 'unexpected' })` for the file_id, so a programming bug in `finalizePreview` doesn't leave the record stuck `'pending'` until the next boot-time orphan sweep. - **NIT danny-avila#6 β€” Stale PR refs**: 12952 β†’ 12957 in 3 places. - **NIT danny-avila#7 β€” Schema bound**: `previewError` capped at `maxlength: 200` to prevent a future codepath from accidentally persisting a stack trace. Skipped per audit verdict (non-blocking): - danny-avila#5 (memory pressure): documented in JSDoc; impl change was reviewer's "consider", not actionable. - danny-avila#8 (double DB query per poll): low cost, indexed by_id, polling is gated narrow. - danny-avila#9 (TAttachment cast): the union type is intentional; the casts are safe widening, refactoring TAttachment is invasive and out of scope. Tests: 11 new (7 `runPhase2Finalize` unit tests covering happy path, null-finalize, throws, double-fail, no-fileId, no-onResolved; +4 wire-shape parity assertions in the existing cross-turn test). 328 backend tests pass; 528 frontend tests pass; lint and typecheck clean. * πŸ›‘οΈ refactor: address codex P1+P2 + rename to drop phase-1/2 jargon Codex round 2 review on PR danny-avila#12957 caught two race conditions and one recovery gap, all triggered by cross-turn filename reuse (`claimCodeFile` intentionally returns the same `file_id` for the same `(filename, conversationId)` across turns). Plus naming cleanup the user requested β€” internal "phase 1 / phase 2" vocabulary leaks across sprints, replace it everywhere with terms describing what's actually happening. P1 β€” stale render overwrites newer revision (process.js) Two turns reusing `output.csv` share a `file_id`. If turn-1's background render resolves AFTER turn-2's persist step, the unconditional `updateFile` writes turn-1's stale text/status over turn-2's pending placeholder. Fix: stamp a fresh `previewRevision` UUID on every emit, thread it through `finalizePreview`, and make the commit conditional via a new optional `extraFilter` argument on `updateFile` (`{ previewRevision: <expected> }`). The defensive `updateFile` in `runPreviewFinalize`'s catch uses the same guard so a programming error from an older render also can't override a newer turn. P1 β€” stale React Query cache on pending remount (queries.ts) Same root cause from the frontend side. Cache key `[QueryKeys.filePreview, file_id]` may hold a prior turn's `'ready'` payload; with `refetchOnMount: false` and the polling gate on `pending`, polling never starts for the new placeholder. Fix: `useAttachmentHandler` invalidates that query whenever an attachment with a `file_id` arrives. Both initial-emit and update events trigger invalidation β€” uniform gate. P2 β€” quick-restart orphans skipped by boot sweep (files.js) Boot `sweepOrphanedPreviews` uses a 5-min cutoff for multi-instance safety. A crash + restart inside the cutoff leaves `pending` records that never get touched again. Fix: lazy sweep inside the preview endpoint β€” if a polled record is `pending` and `updatedAt` is older than 5 min, mark it `failed:orphaned` on the spot before responding. Conditional on the same `updatedAt` we observed so a concurrent legitimate update wins. Cheap, bounded by user activity. Naming cleanup - `runPhase2Finalize` β†’ `runPreviewFinalize` - `PHASE_TWO_TIMEOUT_MS` β†’ `PREVIEW_FINALIZE_TIMEOUT_MS` - All `phase-1` / `phase-2` / `two-phase` prose replaced with "the immediate emit", "the deferred render", "the persist step", "the deferred preview", etc. Skill-feature `phase 1/2` references (different feature) left alone. Tests: 10 new (4 lazy-sweep Γ— preview endpoint, 3 cache-invalidation Γ— useAttachmentHandler, 3 extraFilter Γ— updateFile data-schemas). Backend 332/332, frontend 531/531, data-schemas 37/37, lint clean. * πŸ› οΈ refactor: address comprehensive review (round 3) β€” stale-cache MAJOR + 3 minors Comprehensive review on PR danny-avila#12957 caught a P1 follow-on bug from the prior `invalidateQueries` fix, plus 3 maintainability findings. MAJOR: stale React Query cache not actually fixed by `invalidateQueries` The previous fix called `invalidateQueries` to flush stale cached preview data on cross-turn filename reuse. But `useFilePreview` had `refetchOnMount: false`, which made the new observer read the stale-marked 'ready' data without refetching. The polling `refetchInterval` then evaluated against stale 'ready' β†’ returned `false` β†’ polling never started β†’ user stuck on stale content. Fix (belt-and-suspenders): a) `useAttachmentHandler` switched to `removeQueries` β€” drops the cache entry entirely so the next mount has nothing to read and must fetch. b) `useFilePreview` no longer sets `refetchOnMount: false`, so the React Query default (`true`) kicks in β€” second line of defense if any future codepath observes stale data before the handler has a chance to evict. MINOR: `finalizePreview` JSDoc missing `previewRevision` param Added with explanation of the conditional update guard. MINOR: asymmetric stream-writable guard between SSE protocols Chat-completions delegated the gate to `writeAttachmentUpdate`; Open Responses inlined `!res.writableEnded && res.headersSent`. Extracted `isStreamWritable(res, streamId)` predicate; both paths + `writeAttachmentUpdate` now share the single source of truth. NIT: `(data as Partial<TFile>).file_id` cast repeated 4 times Extracted to a `fileId` local at the top of the handler. Tests: existing 9 invalidate-tests rewritten as remove-tests; +1 new lock-in test asserts removeQueries is called and invalidateQueries is NOT (regression guard against round-3 finding). 332 backend pass, 532 frontend pass, lint clean. Skipped findings (deferred / acceptable): - MINOR: post-submission pending state has no auto-recovery β€” the `isAnySubmitting` polling gate was the user's explicit design; LLM context surfaces failed/pending so the model can volunteer. Worth a follow-up if real users hit it. - NIT: double DB query per preview poll β€” reviewer marked acceptable; changing `fileAccess` middleware is out of scope. * πŸ›‘οΈ test: address comprehensive review NITs (initial-emit guard + isStreamWritable coverage) NIT β€” chat-completions initial emit skips writableEnded check The Open Responses initial emit was switched to use the new `isStreamWritable` predicate in the round-3 commit, but the chat-completions initial emit kept the older narrower check (`streamId || res.headersSent`). On a client disconnect mid-stream (`writableEnded === true`) it would still hit `res.write` and raise `ERR_STREAM_WRITE_AFTER_END` β€” caught by the outer IIFE catch but logged as noise. Switch this site to `isStreamWritable` too so both initial-emit paths share the same gate as the deferred update emits. NIT β€” `isStreamWritable` not directly unit-tested The predicate was only covered indirectly via the deferred-preview SSE tests (writableEnded skip, headersSent check). Export from `callbacks.js` and add 5 parametric tests pinning down each branch (streamId truthy, res null, !headersSent, writableEnded, happy path) so a future condition addition can't silently regress. * πŸ› fix: stuck "Preparing preview…" + inline the chip subtitle Two related fixes for a stuck-spinner bug a user reported in manual testing of PR danny-avila#12957. **Stuck spinner (the bug)** The deferred preview render can complete a few seconds AFTER the SSE stream closes (typical case: PPTX render finishes ~3s after the LLM emits FINAL). When that happens, the SSE update is silently dropped (`isStreamWritable` returns false on a closed stream) and polling is the only recovery path. The earlier polling gate was `status === 'pending' && isAnySubmitting`, which mirrored the original design intent ("only query while the LLM is still generating"). But `isAnySubmitting` flips false the moment the model emits FINAL β€” milliseconds before the deferred render commits. Polling never runs, the chip stays "Preparing preview…" forever even though the DB has `status: 'ready'` with valid HTML. Drop the `isAnySubmitting` part of the gate. `useFilePreview`'s `refetchInterval` is already a function-form that returns `false` on the first terminal response, so polling auto-stops within one tick of resolution. The server-side render ceiling (60s) plus the lazy sweep in the preview endpoint cap the worst case to ~24 polls per pending attachment. Polling itself never blocks UX β€” the gate's purpose was "don't waste cycles", and capping by terminal status is the correct expression of that. **Inline the chip subtitle (the visual)** The previous design rendered "Preparing preview…" as a loose-feeling spinner+text BELOW the file chip. The chip itself looked done while a floating annotation said it wasn't. `FileContainer` gains an optional `subtitle?: ReactNode` prop that overrides the default file-type label. `Attachment.tsx` passes a `PreviewStatusSubtitle` (spinner + "Preparing preview…" / alert + "Preview unavailable") into that slot when the file's preview is pending or failed. The chip footprint stays identical to its `'ready'` form β€” just the second row swaps from "PowerPoint Presentation" to the status indicator. No floating element, no layout shift. Tests: regression test pinning down "polling stays enabled after the LLM finishes" so a future revert can't reintroduce the stuck-spinner bug. Existing FileContainer tests pass unchanged (subtitle override is opt-in). 522 frontend tests pass; lint clean. * πŸ› fix: deferred-preview survives reload + matches artifact card chrome Fixes the remaining stuck-pending case after the polling gate fix: on a reloaded conversation, message.attachments come from the DB frozen at the immediate-persist `status: 'pending'`, but `messageAttachmentsMap` is empty because no SSE handler ever fired for that messageId. Polling now INSERTS a new live entry when no record matches the file_id, and `useAttachments` merges live entries onto DB entries by file_id so the resolved text/textFormat reach `artifactTypeForAttachment` and the chip routes through the proper PanelArtifact card. Also replaces the small file chip used during the pending state with a PreviewPlaceholderCard that mirrors ToolArtifactCard chrome, so the transition to the resolved PanelArtifact no longer reshapes the UI. * ✨ feat: auto-open panel when deferred preview resolves pendingβ†’ready The legacy auto-open path is gated only on `isSubmitting`, so an office-file preview that resolves *after* the SSE stream closes would render in place but never auto-open the panel β€” even though that's exactly the moment the result becomes meaningful to the user. Adds a per-file_id one-shot signal that `useAttachmentPreviewSync` flips on the pendingβ†’ready edge; `ToolArtifactCard` consumes it on mount and auto-opens regardless of submission state. The signal is *only* set on the actual transition (history loads of pre-resolved files don't trigger it) and is consumed once (panel close + reopen on the same card stays user-controlled). * πŸ› fix: drop placeholder Terminal overlay + scope auto-open to fresh resolutions Two fixes for issues spotted in manual testing of the deferred-preview auto-open feature: 1. PreviewPlaceholderCard was passing `file={attachment}` to FilePreview, which triggered SourceIcon's Terminal overlay (`metadata.fileIdentifier` is set on every code-execution file). The artifact card itself doesn't show that overlay; the placeholder shouldn't either, so the pendingβ†’resolved transition is visually seamless. 2. The `previewJustResolved` flag flipped on every pendingβ†’ready transition observed by the polling hook β€” including stale-pending DB records that resolve via the first poll on a *history load*. Conversations whose immediate-persist snapshot left attachments at `status: 'pending'` would yank the panel open every revisit. Adds `mountedDuringStreamRef` to the hook (mirroring ToolArtifactCard) so the flag fires only when the hook itself was mounted during an active turn β€” preserving the pre-PR contract that the panel only auto-opens for results the user is actively waiting on, never for history. * πŸ› fix: don't downgrade preview to failed when only the SSE emit throws Codex P2 finding on PR danny-avila#12957: the original chain placed `.catch` after `.then(onResolved)`, so a throw inside `onResolved` (transport-side errors β€” SSE write race after stream close, an emitter listener throwing) would propagate into the finalize catch and persist `status: 'failed'` / `previewError: 'unexpected'`. That surfaced "preview unavailable" in the UI for a perfectly valid file, and degraded next-turn LLM context to reflect a non-existent failure. Wraps `onResolved` in its own try/catch so emit errors are logged but do not affect the file's persisted status. Extraction success and emit success are now independent: if extraction succeeds and `finalizePreview` writes the terminal status, the polling layer / next page load surfaces the resolved preview even if this turn's SSE emit didn't land. * πŸ›‘οΈ fix: run boot-time orphan sweep under system tenant context Codex P2 finding on PR danny-avila#12957: `File` is tenant-isolated, so under `TENANT_ISOLATION_STRICT=true` the boot-time `sweepOrphanedPreviews` threw `[TenantIsolation] Query attempted without tenant context in strict mode` and the recovery path silently failed every restart. Stale `status: 'pending'` records would be stuck until a user happened to poll the preview endpoint and trigger the lazy sweep β€” which only covers the file the user is currently looking at, not the bulk candidate set the boot sweep is designed to recover. Wraps the sweep in `runAsSystem(...)` in both boot paths (`api/server/index.js` and `api/server/experimental.js`) and pins the contract with regression tests in `file.spec.ts` β€” one test asserts the bare call throws under strict mode, the other asserts the `runAsSystem`-wrapped call succeeds. * 🧹 chore: trim verbose comments from previous commit * 🧹 chore: address review findings (dead branch, lazy-sweep cutoff, stale JSDoc) - finalizePreview: drop unreachable !isOfficeBucket branch (caller already gates on hasOfficeHtmlPath, so this path is always office) - preview endpoint: drop lazy-sweep cutoff from 5min to 2min β€” anything past the 60s render ceiling is definitively orphaned, and per-request sweep can be tighter than the per-instance boot sweep - strip stale `isSubmitting` references from JSDoc in 3 spots (the client-side gate was removed in 9a65840) Skipped: function-length (danny-avila#3) and client-side polling cap (danny-avila#4) β€” refactors without correctness/perf wins; remaining NITs. * 🧹 fix: trim 1 query off pending polls + clear stale lifecycle on cross-shape updates - Preview endpoint: reuse fileAccess middleware's record for the lifecycle check; only re-fetch with text on the terminal ready response. Cuts the typical poll lifecycle from 2(N+1) to N+1 queries, since the vast majority of polls hit while pending and don't need text at all. - processCodeOutput non-office branch: explicitly null out status, previewError, previewRevision (codex P2). Without this, an update at the same (filename, conversationId) where the prior emit was an office file leaves stale lifecycle fields and the client renders the wrong state for the now non-office artifact. - Tests: rewire preview.spec mocks for the new shape, add boundary test pinning the 2min cutoff, add regression test for the cross-shape update. * πŸ› fix: keep polling on transient errors but cap permanently-broken endpoint Codex P2: the previous `data?.status === 'pending' ? 2500 : false` gate killed polling on the first transient error. With `retry: false`, a 500 left `data` undefined, the callback returned false, and the chip was stuck "Preparing preview…" forever β€” exactly the bug the polling layer was supposed to recover from. Inverts the gate: stop on terminal success (`ready`/`failed`) or after 5 consecutive errors. Transient errors keep retrying; a permanently broken endpoint caps at ~12.5s instead of polling forever. Predicate extracted as `previewRefetchInterval` for direct unit testing without fighting React Query's timer machinery. * ✨ feat: render pending-preview files in their own row Pending deferred-preview chips now bucket into a separate row above the resolved attachments β€” reads as "this is still happening" rather than mixing with completed downloads. Once status flips to ready, the chip re-buckets into panelArtifacts; failed re-buckets into the file row alongside other downloads. * 🎨 fix: render pending-preview chips in the panel-artifact row, not the file row Previous bucketing put pending chips in the file row (since `artifactTypeForAttachment` returns null for empty-text records). The pending placeholder is a future panel artifact β€” sharing the row keeps the chip in place when it resolves instead of jumping rows. Plain files still get their own row. * πŸ› fix: phase-1 SSE replay must not regress a resolved attachment Codex P1: useEventHandlers.finalHandler iterates responseMessage.attachments at stream end and dispatches each through the attachment handler. Those records are the immediate-persist snapshot (status:pending, text:null) β€” if a deferred update has already moved the same file_id to ready/failed, the existing merge let the pending fields win and downgraded the resolved record. Result: chip flickers back to pending and polling restarts until the lazy sweep corrects. Pin the terminal lifecycle fields (status, text, textFormat, previewError) when existing is ready/failed and incoming is pending. Other field updates still go through. * πŸ› fix: track preview-poll error cap outside React Query state Codex P2: the previous cap relied on `query.state.fetchFailureCount`, but React Query v4's reducer resets that to 0 on every fetch dispatch (the `'fetch'` action). With `retry: false`, each failed poll left count at 1 and the next dispatch reset it back to 0, so the `>= 5` branch never fired and a permanently-broken endpoint polled forever. Track consecutive errors in a module-level Map keyed by file_id, incremented in a thin `fetchFilePreview` wrapper around the data service call. The Map is cleared on success and on cap-stop, so memory is bounded by in-flight pending file_ids per session.
1 parent cf06575 commit 6c6c72d

35 files changed

Lines changed: 3861 additions & 142 deletions

File tree

β€Žapi/server/controllers/agents/__tests__/callbacks.spec.jsβ€Ž

Lines changed: 284 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,28 @@ jest.mock('~/server/services/Files/Citations', () => ({
2828

2929
jest.mock('~/server/services/Files/Code/process', () => ({
3030
processCodeOutput: jest.fn(),
31+
/* `runPreviewFinalize` is the runtime pairing for `finalize` (defined
32+
* alongside processCodeOutput in process.js). The callback wires
33+
* the deferred render through it; reproduce the basic happy-path here so the
34+
* SSE-emit assertions still work. The catch/defensive-updateFile
35+
* branch is unit-tested directly against the real helper in
36+
* process.spec.js β€” exercising it here would add test coupling
37+
* without coverage benefit. */
38+
runPreviewFinalize: ({ finalize, onResolved }) => {
39+
if (typeof finalize !== 'function') {
40+
return;
41+
}
42+
finalize()
43+
.then((updated) => {
44+
if (!updated || !onResolved) {
45+
return;
46+
}
47+
onResolved(updated);
48+
})
49+
.catch(() => {
50+
/* swallowed in the mock β€” see process.spec.js for catch coverage */
51+
});
52+
},
3153
}));
3254

3355
jest.mock('~/server/services/Tools/credentials', () => ({
@@ -326,4 +348,266 @@ describe('createToolEndCallback', () => {
326348
expect(res.write).not.toHaveBeenCalled();
327349
});
328350
});
351+
352+
describe('code execution deferred-preview emit', () => {
353+
/* The deferred-preview code-execution flow emits the attachment twice over
354+
* SSE: the initial emit with `status: 'pending'` and the current run's
355+
* messageId, the deferred render with the resolved record. The preview update emit
356+
* must use the CURRENT run's messageId (not the persisted DB one)
357+
* because `processCodeOutput` intentionally preserves the original
358+
* `messageId` on cross-turn filename reuse β€” `getCodeGeneratedFiles`
359+
* needs that for prior-turn priming.
360+
*
361+
* Codex P1 review on PR #12957: shipping `updated.messageId`
362+
* straight from the DB record routed preview-update patches to the wrong
363+
* message slot, leaving the current turn's pending chip stuck. */
364+
365+
const { processCodeOutput } = require('~/server/services/Files/Code/process');
366+
367+
function makeCodeExecutionEvent({ runId, threadId, toolCallId, fileId, name }) {
368+
return {
369+
output: {
370+
name: 'execute_code',
371+
tool_call_id: toolCallId,
372+
artifact: {
373+
session_id: 'sess-1',
374+
files: [{ id: fileId, name, session_id: 'sess-1' }],
375+
},
376+
},
377+
metadata: { run_id: runId, thread_id: threadId },
378+
};
379+
}
380+
381+
/** Parse the SSE frame `res.write` produces back to a payload object. */
382+
function parseSseAttachment(call) {
383+
const frame = call[0];
384+
const dataLine = frame.split('\n').find((l) => l.startsWith('data: '));
385+
return JSON.parse(dataLine.slice('data: '.length));
386+
}
387+
388+
it('the preview update emit uses the current run messageId, not the persisted DB messageId (cross-turn filename reuse)', async () => {
389+
/* Simulate turn-2 reusing `output.csv` from turn-1. The DB record
390+
* surfaced by `updateFile` carries the original `turn-1-msg`
391+
* messageId; the runtime emit must rewrite to `turn-2-msg`. */
392+
res.headersSent = true;
393+
const finalize = jest.fn().mockResolvedValue({
394+
file_id: 'fid-shared',
395+
filename: 'output.csv',
396+
filepath: '/uploads/output.csv',
397+
type: 'text/csv',
398+
conversationId: 'thread789',
399+
messageId: 'turn-1-original-msg', // persisted DB id (older turn)
400+
status: 'ready',
401+
text: '<table></table>',
402+
textFormat: 'html',
403+
});
404+
processCodeOutput.mockResolvedValue({
405+
file: {
406+
file_id: 'fid-shared',
407+
filename: 'output.csv',
408+
filepath: '/uploads/output.csv',
409+
type: 'text/csv',
410+
conversationId: 'thread789',
411+
messageId: 'turn-2-current-run', // runtime overlay (current turn)
412+
toolCallId: 'tool-2',
413+
status: 'pending',
414+
text: null,
415+
textFormat: null,
416+
},
417+
finalize,
418+
});
419+
420+
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises });
421+
const event = makeCodeExecutionEvent({
422+
runId: 'turn-2-current-run',
423+
threadId: 'thread789',
424+
toolCallId: 'tool-2',
425+
fileId: 'fid-shared',
426+
name: 'output.csv',
427+
});
428+
await toolEndCallback({ output: event.output }, event.metadata);
429+
await Promise.all(artifactPromises);
430+
// Wait one more tick so the fire-and-forget finalize() chain settles.
431+
await new Promise((resolve) => setImmediate(resolve));
432+
433+
// Two SSE writes: the initial emit (pending) and the deferred render (ready).
434+
expect(res.write).toHaveBeenCalledTimes(2);
435+
const phase1 = parseSseAttachment(res.write.mock.calls[0]);
436+
const phase2 = parseSseAttachment(res.write.mock.calls[1]);
437+
438+
// Initial emit already used the runtime messageId (sourced from result.file).
439+
expect(phase1.messageId).toBe('turn-2-current-run');
440+
expect(phase1.status).toBe('pending');
441+
442+
/* The preview update MUST also route to the current run's messageId so the
443+
* frontend's `useAttachmentHandler` upserts under the same
444+
* messageAttachmentsMap slot as the initial emit. Routing to
445+
* `turn-1-original-msg` would land the patch on a stale message
446+
* and leave turn-2's pending chip stuck. */
447+
expect(phase2.messageId).toBe('turn-2-current-run');
448+
expect(phase2.file_id).toBe('fid-shared');
449+
expect(phase2.status).toBe('ready');
450+
expect(phase2.text).toBe('<table></table>');
451+
expect(phase2.toolCallId).toBe('tool-2');
452+
/* Wire-shape parity with the initial emit: preview update emits the full updated
453+
* record so the client doesn't see one shape on the initial emit and a
454+
* narrower projection on the deferred render. (Codex audit on PR #12957
455+
* Finding 1.) */
456+
expect(phase2.filename).toBe('output.csv');
457+
expect(phase2.filepath).toBe('/uploads/output.csv');
458+
expect(phase2.type).toBe('text/csv');
459+
expect(phase2.conversationId).toBe('thread789');
460+
expect(phase2.textFormat).toBe('html');
461+
});
462+
463+
it('the preview update emit is skipped when finalize resolves to null (no DB update happened)', async () => {
464+
res.headersSent = true;
465+
processCodeOutput.mockResolvedValue({
466+
file: {
467+
file_id: 'fid-1',
468+
filename: 'data.xlsx',
469+
filepath: '/uploads/data.xlsx',
470+
type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
471+
messageId: 'run-1',
472+
toolCallId: 'tool-1',
473+
status: 'pending',
474+
},
475+
finalize: jest.fn().mockResolvedValue(null),
476+
});
477+
478+
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises });
479+
const event = makeCodeExecutionEvent({
480+
runId: 'run-1',
481+
threadId: 'thread-1',
482+
toolCallId: 'tool-1',
483+
fileId: 'fid-1',
484+
name: 'data.xlsx',
485+
});
486+
await toolEndCallback({ output: event.output }, event.metadata);
487+
await Promise.all(artifactPromises);
488+
await new Promise((resolve) => setImmediate(resolve));
489+
490+
// Only the initial emit fired; preview update noop'd because finalize returned null.
491+
expect(res.write).toHaveBeenCalledTimes(1);
492+
});
493+
494+
it('the preview update emit is skipped when the response stream has already closed', async () => {
495+
res.headersSent = true;
496+
/* Hand-rolled deferred so we can hold finalize() open until
497+
* AFTER setting `res.writableEnded = true`. Otherwise the mock
498+
* resolves synchronously, the .then() runs in the same microtask
499+
* queue as the artifactPromises await, and writableEnded is set
500+
* too late. */
501+
let resolveFinalize;
502+
const finalizeDeferred = new Promise((resolve) => {
503+
resolveFinalize = resolve;
504+
});
505+
processCodeOutput.mockResolvedValue({
506+
file: {
507+
file_id: 'fid-1',
508+
filename: 'data.xlsx',
509+
filepath: '/uploads/data.xlsx',
510+
type: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
511+
messageId: 'run-1',
512+
toolCallId: 'tool-1',
513+
status: 'pending',
514+
},
515+
finalize: jest.fn().mockReturnValue(finalizeDeferred),
516+
});
517+
518+
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises });
519+
const event = makeCodeExecutionEvent({
520+
runId: 'run-1',
521+
threadId: 'thread-1',
522+
toolCallId: 'tool-1',
523+
fileId: 'fid-1',
524+
name: 'data.xlsx',
525+
});
526+
await toolEndCallback({ output: event.output }, event.metadata);
527+
await Promise.all(artifactPromises);
528+
// Simulate the response closing AFTER the initial emit fires but BEFORE
529+
// the deferred render lands. The frontend's polling path will catch the
530+
// resolved record on its next tick.
531+
res.writableEnded = true;
532+
// Now resolve finalize and let the .then() chain run.
533+
resolveFinalize({
534+
file_id: 'fid-1',
535+
filename: 'data.xlsx',
536+
messageId: 'run-1',
537+
status: 'ready',
538+
text: '<x/>',
539+
textFormat: 'html',
540+
});
541+
await new Promise((resolve) => setImmediate(resolve));
542+
543+
// Initial emit wrote; preview update noop'd because writableEnded.
544+
expect(res.write).toHaveBeenCalledTimes(1);
545+
});
546+
547+
it('does not call finalize for a non-office file (no preview expected)', async () => {
548+
res.headersSent = true;
549+
processCodeOutput.mockResolvedValue({
550+
file: {
551+
file_id: 'fid-txt',
552+
filename: 'note.txt',
553+
filepath: '/uploads/note.txt',
554+
type: 'text/plain',
555+
messageId: 'run-1',
556+
toolCallId: 'tool-1',
557+
// No status β€” non-office files skip the deferred render entirely.
558+
},
559+
// No finalize key β€” caller should not call anything.
560+
});
561+
562+
const toolEndCallback = createToolEndCallback({ req, res, artifactPromises });
563+
const event = makeCodeExecutionEvent({
564+
runId: 'run-1',
565+
threadId: 'thread-1',
566+
toolCallId: 'tool-1',
567+
fileId: 'fid-txt',
568+
name: 'note.txt',
569+
});
570+
await toolEndCallback({ output: event.output }, event.metadata);
571+
await Promise.all(artifactPromises);
572+
await new Promise((resolve) => setImmediate(resolve));
573+
574+
expect(res.write).toHaveBeenCalledTimes(1);
575+
});
576+
});
577+
});
578+
579+
describe('isStreamWritable', () => {
580+
/* Direct parametric coverage of the predicate that gates SSE writes
581+
* in both the chat-completions and Open Responses callbacks. The
582+
* existing deferred-preview tests cover this indirectly via the
583+
* `writeAttachmentUpdate` writableEnded path; these tests pin down
584+
* each individual branch so a future modification (e.g. adding a
585+
* new condition) can't silently regress.
586+
* (Comprehensive review NIT on PR #12957.) */
587+
const { isStreamWritable } = require('../callbacks');
588+
589+
it('returns true when streamId is truthy regardless of res state', () => {
590+
/* Resumable mode writes go to the job emitter; res state is
591+
* irrelevant. Even a closed res with no headers should not block. */
592+
expect(isStreamWritable(null, 'stream-1')).toBe(true);
593+
expect(isStreamWritable({ headersSent: false, writableEnded: true }, 'stream-1')).toBe(true);
594+
expect(isStreamWritable(undefined, 'stream-1')).toBe(true);
595+
});
596+
597+
it('returns false when streamId is falsy and res is null/undefined', () => {
598+
expect(isStreamWritable(null, null)).toBe(false);
599+
expect(isStreamWritable(undefined, null)).toBe(false);
600+
});
601+
602+
it('returns false when headers have not been sent yet', () => {
603+
expect(isStreamWritable({ headersSent: false, writableEnded: false }, null)).toBe(false);
604+
});
605+
606+
it('returns false when the stream has already ended', () => {
607+
expect(isStreamWritable({ headersSent: true, writableEnded: true }, null)).toBe(false);
608+
});
609+
610+
it('returns true on the happy path: headers sent, not ended, no streamId', () => {
611+
expect(isStreamWritable({ headersSent: true, writableEnded: false }, null)).toBe(true);
612+
});
329613
});

0 commit comments

Comments
Β (0)