fix(workspaces): stream kernel file reads to avoid Durable Object resets#623
Draft
posthog[bot] wants to merge 1 commit into
Draft
fix(workspaces): stream kernel file reads to avoid Durable Object resets#623posthog[bot] wants to merge 1 commit into
posthog[bot] wants to merge 1 commit into
Conversation
The WorkspaceKernel Durable Object read whole file blobs into a single in-memory Uint8Array when serving content, and stored blobs inline in the DO's SQLite up to a 1.5MB threshold. Under that load the DO tripped Cloudflare's isolate memory limit and storage-operation timeout and reset itself, dropping the client's live connection and taking the whole workspace page down (notably when opening large files/PDFs). - Serve file content via a streamed read (readFileStream) so R2-backed bytes flow straight through the isolate instead of being buffered into one Uint8Array. The content route now streams the ReadableStream directly in its Response. - Lower the inline SQLite storage threshold from 1.5MB to 128KB so large files always spill to R2 and never bloat a single SQLite row. This only governs new writes; existing files keep whatever backend they were written with. - Add a scoped error boundary around the workspace file viewer so a transient content-load failure degrades to a pane-level message and Retry instead of bubbling to the root boundary and blanking the page. Generated-By: PostHog Code Task-Id: d1f9df85-3348-44d1-8c3b-59f448133a7a
|
React Doctor found no new issues. 🎉 Reviewed by React Doctor for commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A user hit two full-page crashes in one workspace session: the
WorkspaceKernelDurable Object kept blowing past Cloudflare's isolate-memory and storage-operation-timeout limits and resetting itself, which dropped the client's live connection and blanked the whole page. Both flavors appeared in the same session — a storage-operation timeout, then an isolate-memory reset right as the user opened a PDF, exactly the large-file case the buffer-everything design couldn't handle.What changed
readFileContentloaded whole blobs into a single in-memoryUint8Arrayinside the DO isolate. The content route now uses a new streamed read (readFileStream) so R2-backed bytes flow straight through and are piped directly into theResponse, never materializing the full file in the isolate.Notes / verification
vp check(format + lint + typecheck) passes.ReadableStreamacross native Durable Object RPC (theagents/partyserverstub is a nativeDurableObjectStub), so bytes transfer without buffering in either isolate.readFileContent) because OCR/extraction inherently needs the whole file; that path runs in a retrying Workflow and is not the reported crash. Left unchanged to keep this focused on the file-viewing flow.Created with PostHog Code from an inbox report.
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.