fix(sftp): rewrite drag-out via SFTP stream + zip archival#6
Merged
Conversation
The SFTP drag-out cache could silently accumulate hundreds of
gigabytes in %TEMP%/hypershell-drag — a user reported 458 GB on
one machine. Three overlapping bugs caused this:
1. Selecting a directory in the SFTP file list fired
`sftpDragOut({ prepareOnly: true })`, which triggered a recursive
`scp -r` of the entire remote tree into the temp cache — the user
never dragged anything, just clicked a row.
2. Nothing ever deleted cached files: no per-drag cleanup, no startup
prune, no size cap.
3. The deterministic cache filename helped *within* a session, but
the in-memory map was not persisted, so across restarts the same
path still landed in a fresh unique filename — producing duplicate
`home/` and `home-1a8e5ac74672/` copies.
Fixes:
- FileList.tsx: skip `prepareOnly` pre-caching for directories. An
explicit drag still works because handleDragStart runs separately.
- sftpIpc.ts: after `event.sender.startDrag`, schedule a 5-minute
best-effort deletion of the temp file. The OS has copied to the
drop target well before then; if it hasn't (slow network share),
the startup prune will catch it.
- sftpIpc.ts: on registerSftpIpc init, prune anything in the temp
dir older than 24h. Mirrors the existing 7-day prune on
transferManifest.
- Extract `pruneDragOutCache` and export it, with unit tests
covering stale file removal, recursive directory removal, and the
missing-directory no-op.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to the earlier drag-out cache fix, surfaced by
running the feature branch against a 458 GB stale cache:
1. Startup prune was synchronous — rmSync on hundreds of GB
blocked the Electron main process long enough to crash the
renderer (ACCESS_VIOLATION). Switch to fs.promises and
fire-and-forget so startup never awaits filesystem cleanup.
2. event.sender.startDrag({ file: directoryPath }) is unreliable
on Windows — large directories crash outright, small ones
don't consistently land at the drop target. Skip drag-out for
directories entirely. Internal SFTP-to-SFTP drag still works
via the HTML5 dataTransfer channel; users who want to download
a remote directory should use the transfer panel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the spawn-scp-subprocess pre-cache approach with a streaming
pipeline that runs through the already-open SFTP transport:
- Single-file drag: stream the remote file into a deterministically
named staging path in %TEMP%/hypershell-drag, then hand it to the
OS via event.sender.startDrag.
- Directory drag: walk the remote tree via the SFTP transport, stream
each entry into a yazl ZipFile, and drag the resulting .zip.
(Previously a directory drag either triggered `scp -r` of the whole
tree into temp with no cleanup, or crashed Electron outright because
startDrag({ file: directoryPath }) is unreliable on Windows.)
- New fire-and-forget IPC `sftp:drag-out:start-native` (via
ipcRenderer.send / ipcMain.on) for the two-phase directory drag:
first drag triggers the zip build and shows a "preparing" toast;
once the "ready" toast fires, subsequent drags hit the cached zip
and initiate the native drag immediately.
- No more click-based pre-staging. Selecting a row in the SFTP pane
is now free — staging only runs in response to an actual dragstart.
(The previous design silently scp'd every row the user clicked;
one user accumulated over a million cached files before noticing.)
- startDrag icon: use app.getFileIcon on the staged file (returns the
real Windows shell icon for .zip etc.); fall back to a stub only
when the shell lookup fails. Windows rejects startDrag with an
empty NativeImage, which was silently killing the drag ghost.
- Startup prune: async fs.promises.rm, 1-hour TTL (drag cache has no
long-term value), fire-and-forget so app launch never blocks on
multi-GB rm.
- Per-drag cleanup: 5-minute best-effort removal after startDrag;
handles both the .zip and the underlying staged directory.
- Extract testable helpers (stageSftpDragOutItem, shouldStartNativeDragOut,
pruneDragOutCache, resolveSafeDragOutPath) and add 18 unit tests
covering staging, archival of mixed-readable trees, prune TTL, and
path-traversal rejection.
479/479 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tomertec
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The SFTP drag-out cache at
%TEMP%/hypershell-draghad three independent failure modes that combined to cause a ~458 GB + 1M-file disk leak and a crashing directory drag:%TEMP%— for directories, a fullscp -rof the remote tree. Users just browsing the pane accumulated GBs without ever dragging.event.sender.startDrag({ file: directoryPath })is unreliable on Windows — the drag would either silently fail or segfault the renderer.What this PR does
yazlZipFile, then the.zipis what gets dragged.dragstartevent. The first drag on a directory triggers the zip build and shows a "preparing" toast; once the "ready" toast fires, subsequent drags initiate the native drag immediately.sftp:drag-out:start-nativeIPC (viaipcRenderer.send/ipcMain.on) for the two-phase directory drag.app.getFileIcon(stagedPath, { size: "small" }). Windows rejectsstartDragwith an empty NativeImage — the stub was silently killing the drag ghost.rm.startDragresolves; cleans both the.zipand the underlying staged directory.stageSftpDragOutItem,shouldStartNativeDragOut,pruneDragOutCache,resolveSafeDragOutPath) with 18 new unit tests covering staging, archival of mixed-readable trees, TTL pruning, and path-traversal rejection.Users already carrying a large stale
hypershell-dragcache will have it drained automatically within an hour of next app launch.Test plan
pnpm buildclean across all workspacespnpm test— 479/479 passscp -r, no disk growth%TEMP%/hypershell-drag/drains within 5 minutesPreparing …toast, then… ready to drag out, second drag produces a.zipon desktop🤖 Generated with Claude Code