Skip to content

fix(workspaces): stream kernel file reads to avoid Durable Object resets#623

Draft
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-code/stream-workspace-file-reads
Draft

fix(workspaces): stream kernel file reads to avoid Durable Object resets#623
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-code/stream-workspace-file-reads

Conversation

@posthog

@posthog posthog Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Why

A user hit two full-page crashes in one workspace session: the WorkspaceKernel Durable Object kept blowing past Cloudflare's isolate-memory and storage-operation-timeout limits and resetting itself, which dropped the client's live connection and blanked the whole page. Both flavors appeared in the same session — a storage-operation timeout, then an isolate-memory reset right as the user opened a PDF, exactly the large-file case the buffer-everything design couldn't handle.

What changed

  • Stream file reads instead of buffering. readFileContent loaded whole blobs into a single in-memory Uint8Array inside the DO isolate. The content route now uses a new streamed read (readFileStream) so R2-backed bytes flow straight through and are piped directly into the Response, never materializing the full file in the isolate.
  • Keep large blobs out of inline DO storage. Lowered the inline SQLite threshold from 1.5MB to 128KB so anything substantial spills to R2 rather than bloating a single SQLite row (base64 inflates ~33%), which was the source of the storage-operation timeout. This only governs new writes — existing files keep whatever backend they were written with.
  • Safety net. Wrapped the workspace file viewer in a scoped error boundary so a transient content-load failure (e.g. a DO reset mid-request) degrades to a pane-level message with Retry instead of bubbling to the root boundary and taking the page down.

Notes / verification

  • vp check (format + lint + typecheck) passes.
  • The streamed read returns a ReadableStream across native Durable Object RPC (the agents/partyserver stub is a native DurableObjectStub), so bytes transfer without buffering in either isolate.
  • The background extraction workflow still reads full bytes (readFileContent) because OCR/extraction inherently needs the whole file; that path runs in a retrying Workflow and is not the reported crash. Left unchanged to keep this focused on the file-viewing flow.
  • The error boundary is a defense-in-depth addition; its runtime UX could not be exercised in this environment (no live Cloudflare runtime), but the root-cause streaming + threshold changes are what stop the DO from resetting.

Created with PostHog Code from an inbox report.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

Review in cubic

The WorkspaceKernel Durable Object read whole file blobs into a single in-memory
Uint8Array when serving content, and stored blobs inline in the DO's SQLite up to a
1.5MB threshold. Under that load the DO tripped Cloudflare's isolate memory limit and
storage-operation timeout and reset itself, dropping the client's live connection and
taking the whole workspace page down (notably when opening large files/PDFs).

- Serve file content via a streamed read (readFileStream) so R2-backed bytes flow
  straight through the isolate instead of being buffered into one Uint8Array. The
  content route now streams the ReadableStream directly in its Response.
- Lower the inline SQLite storage threshold from 1.5MB to 128KB so large files always
  spill to R2 and never bloat a single SQLite row. This only governs new writes;
  existing files keep whatever backend they were written with.
- Add a scoped error boundary around the workspace file viewer so a transient
  content-load failure degrades to a pane-level message and Retry instead of bubbling
  to the root boundary and blanking the page.

Generated-By: PostHog Code
Task-Id: d1f9df85-3348-44d1-8c3b-59f448133a7a
@github-actions

Copy link
Copy Markdown

React Doctor found no new issues. 🎉

Reviewed by React Doctor for commit d2e91d1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

0 participants