Sub-100ms event-loop liveness for large initial sync (follow-up to #104)

[AlexJeffcott]

Background

Polly #104 closed against the fragmentation + observability + yield-only scope. That fix removed the silent-drop wedge (werift's 64 KiB cap dropping every 65 630-byte fragment) and added the inFlightSync snapshot field, the sync-progress event, and the receiver-side macrotask yield. The contract the ticket asked for is partially held:

• Sentinel propagates end-to-end on a throttled in-process transport with a 5.5 MB seeded snapshot.
• inFlightSync is observable mid-run.
• Most of the receive-side sync time is spent off the wire-callback stack frame.

What is NOT held: the strict <100ms event-loop tick-gap bar during the bulk-apply step. The single applyChanges call for a multi-megabyte initial sync is one synchronous Automerge call. The existing setTimeout(0) yield moves dispatch off the wire frame but cannot split that call. The harness in examples/mesh-large-initial-sync/ reports a residual ~300 ms tick spike for a 5.5 MB snapshot and the example's NOTE line documents this.

Burden of proof

Same shape as #104. This ticket closes only when polly demonstrates with a captured run that:

• The harness exercises the failing-shape diff (≥ 5 MB compacted snapshot, throttled transport, tick-gap probe running before sync starts).
• The probe records NO tick > 100 ms during the sync window.
• The sentinel still propagates end-to-end.
• A falsification path (pre-fix-emulated mode) reproduces the residual spike and exits non-zero with a named reason that points at the apply step, not at a generic timeout.

Two candidate approaches

A. Receive-side change-batching apply

Intercept reassembled sync messages, decode the embedded changes array, apply them in batches of N changes with setTimeout yields between batches. Re-emit the synchroniser's outbound sync message only after the full chain has applied so the sender doesn't see partial state.

Risk: Automerge's sync state machine isn't designed to be paused mid-apply. The decode path is implementation-private; relying on it pins polly to an Automerge version. The synchroniser's own state-tracking would need to match the slow apply to avoid sending stale have heads while we're still applying.

B. Web Worker apply

Run the Automerge Repo in a Worker. The receive-side mesh adapter forwards inbound sync messages to the worker via postMessage; the worker applies them and posts state deltas back. The main thread never sees the multi-megabyte apply call.

Risk / scope: storage adapter bridging (the worker needs to own its own storage subsystem, or proxy through postMessage), state subscription bridging ($meshState consumers expect synchronous reads of the latest doc state), and the document handle lifecycle (whenReady, change) all need worker-aware variants. Tests and examples need to be redone for the worker target. Multi-week scope.

Approach (B) is more invasive but more durable; (A) is less invasive but more fragile against Automerge upstream churn.

What I will not accept as proof

Same exclusions as #104. In particular: "all unit tests pass" does not imply the contract is held; the load-bearing signal is the tick-gap probe running against the failing-shape diff.

Out of scope

• Optimising fairfox or any other consumer's document shape (a Large initial sync to fresh peer hangs the receiver on bandwidth-constrained transports #104 out-of-scope line that applies here too).
• Reducing the threshold below 100ms — the bar is the ticket's own.

Tracking

Follow-up to polly#104. The harness examples/mesh-large-initial-sync/ is the proof surface; the residual tick-gap spike it reports under the post-fix mode is the failure this ticket closes.

[AlexJeffcott]

Tried approach A. The mechanical implementation is sound; the contract is not met. Reverting and recommending approach B.

What was implemented

The split landed in MeshNetworkAdapter.emitMessageMaybeSplit, the first place in the receive pipeline that sees decrypted Automerge sync messages. For every inbound sync message:

1. Automerge.decodeSyncMessage(message.data) → { heads, need, have, changes }.
2. When changes.length > syncReceiveBatchSize (default 25), split changes into N sub-batches.
3. Re-encode each sub-batch as a separate sync message under the same heads/need/have. The receiver's DocSynchronizer sees a sequence of valid sync messages and advances its sync state on every applied sub-message; the cumulative effect on the doc is identical to the monolithic apply.
4. setTimeout(emitNext, 0) between sub-message emits so the JS event loop drains its timer queue between Automerge applyChanges calls.

The split lives in MeshNetworkAdapter, not MeshWebRTCAdapter, because the WebRTC layer sees only encrypted bytes — Automerge.decodeSyncMessage throws on those and the splitter falls through. Plumbed syncReceiveBatchSize through createMeshClient and added POLLY_111_DISABLE_FIX=1 semantics (set the option to 0 to disable splitting).

What the harness reported

Configuration	max-tick-gap-ms	sync-elapsed-ms	sentinel propagated
Pre-fix (no split)	3,336	4,067	yes
Split, batch=25	3,374 (averaged over 3 runs)	4,131	yes
Split, batch=10	3,471 (averaged over 3 runs)	6,015	yes

The split does not move the tick-gap needle in either direction. One run at batch=25 reported 342ms (the only result that looked like the fix worked) but did not reproduce; it was variance, not signal.

Why it doesn't help here

Instrumentation on the splitter showed the smoking gun:

[#111] decode bytes=5791126 changes=1

The 5.79 MB sync message carries one Automerge change. The harness builds its load shape with a single Automerge.change call against a 7 MB random payload, then Automerge.save packs the whole history into one change. The splitter has nothing to split — changes.length === 1 falls through to the unchanged emit path, and the receiver's A.receiveSyncMessage still sits inside a single multi-megabyte applyChanges call for 3.3 seconds.

This is the fragility the issue's approach-A description named explicitly: "Automerge's sync state machine isn't designed to be paused mid-apply." Splitting at the change boundary is the only granularity polly can introduce without forking Automerge's apply path; when a single change holds the whole history, that granularity is wrong.

A workload with many small changes (the realistic shape — fairfox's mesh:users accumulating per-key writes over days) would carry multiple changes per sync message and the split would help. That case is untested in this harness, and even there the largest-individual-change apply still bounds the tick gap.

What I am proposing

Reverting the split. The change is in git history at the rewind-point for anyone who wants to revisit the mechanically-correct shape, but it is not on main.

Approach B (Web Worker with the Automerge Repo off the main thread) is the path that can hold the tick-gap contract for compacted-snapshot loads, because the bottleneck is one CPU-bound applyChanges call that no API in current Automerge can interleave. Worker isolation moves that call off the consumer's main thread by construction, which is the property the harness's probe is measuring. Scope as the original ticket called it: multi-week — storage-subsystem bridging, handle-lifecycle proxies, $meshState synchronous-read contract preservation.

Leaving #111 open; the harness's NOTE continues to document the residual spike as a known limitation against approach A.