Make GUI remote pulls fast (one-time cluster load) + keep fitting local

[matt-pharr]

Reframed (2026-06-30, after measurement)

The original framing — "run the analysis on the cluster and stream fits back per request" — is wrong for the interactive case, and measurement shows why:

• A local re-fit is ~65 ms (compute_spectrogram on a 2M-sample full-rate channel; an fmin/fmax crop is 0.05 ms). Once the raw data is local, parameter tweaks are already fully interactive.
• The "~10 s per tweak" worry only existed if each re-fit went to the cluster. It shouldn't. Fitting stays local.

So the cluster's job is a one-time fast data loader, not a per-fit engine. The remote-compute + service-ingest-of-precomputed-results work is dropped for interactive use (see "Deferred" below).

What's the win, measured

path	one-time load	per re-fit
laptop mdsthin (+-C, #14)	~60 s	65 ms
remote fetch → ship compressed h5	~21–37 s	65 ms

Only the compressed ~80 MB h5 crosses the tunnel, not ~370 MB of raw float over mdsip. (Only the cluster can compress-before-tunnel; the laptop mdsip stream can't.)

Done (PRs)

• Fix toksearch backend: use toksearch_d3d.PtDataSignal + validate identical HDF5 #16 — toksearch backend fixed (toksearch_d3d.PtDataSignal), produces HDF5 identical to mdsthin to float32 precision. On-cluster fetch ~110 MB/s.
• remote_run: direct env python (no conda/module) + ssh-config aliases #19 — remote_run runs the cluster env's python directly (no module load/conda, ~4–5 s saved), uses ssh-config aliases, silences index-dir spam. Verified end-to-end via run_remote().

Remaining work

1. Surface it in the GUI. Drop the "(WIP)" label on the remote backend option (it works). Make clear creds are optional with a key-based alias. Decide default: keep mdsthin (works from any laptop) but make remote a first-class, trusted choice; consider auto-selecting remote when the user has cluster access.
2. Progress granularity for the remote phase. The remote pull currently reports coarse progress (connect -> fetch -> ship). Stream the per-channel/per-phase progress back so the bar is smooth during the ~20–35 s load.
3. (Optional) Spectrogram result cache in the service. Re-fits are 65 ms now; caching the base STFT makes crop/gate/cursor/denoise tweaks 0.05 ms (only an FFT-window change recomputes). Nice-to-have, not required.

Deferred (separate, low priority)

Remote compute (run compute_spectrogram on the cluster, ship the small result + ingest into nodes.py) only helps headless/batch multi-shot runs, where you never want the raw data local. Not needed for the interactive single-shot loop. Spin out as its own issue if/when batch analysis is on the table.

Acceptance

• GUI "remote" pull of a real shot -> local h5 in ~20–35 s with a smooth progress bar -> subsequent parameter tweaks re-fit locally at ~65 ms.

All numbers measured against shots 174446 / 184927 / 147131 on omega, 2026-06-30.