fix(encoder): heal a diverged shared-structures dictionary on decode (durable-wins reload)

[kriszyp]

Status: defense-in-depth (held) — root cause now fixed by #1520

The v4→v5 structure-dictionary fork's root cause is the migration not persisting the canonical seed: copyDb's shapeForStructure stubbed the RecordObjects the migration reads (value.constructor === Object gate), so the observer minted no structure and the seed was skipped — every v5 worker then minted the dictionary from an empty durable and forked it. That's fixed at the source by #1520. With the seed persisted, workers adopt one canonical dictionary and don't diverge.

This PR is therefore no longer the primary fix — it's defense-in-depth:

• decode-heal (durable-wins reload on a present-but-wrong decode) — recovers any residual in-memory divergence in-process, without a restart: a worker that somehow still strands a divergent dictionary self-heals on the next decode instead of returning nulls until a restart.
• The isFaithfulExtension saveStructures guard here is not load-bearing — the optimistic-transaction retry + durable-wins merge already converge concurrent appends (thanks @kzyp for pushing on this). It's kept only as cheap belt-and-suspenders; if this PR is promoted it should likely be trimmed to just the decode-heal.

Holding in draft pending evidence it's needed beyond #1520. If residual divergence shows up in the field after #1520 ships — or we want no-restart recovery as insurance — promote (and trim) this; otherwise close it.

Superseded as the primary fix by #1520. Refs #1508, #1453.

— repositioned by KrAIs (Claude Opus 4.8)

[gemini-code-assist]

Code Review

This pull request introduces a mechanism to detect and recover from 'present-but-wrong' structure mismatch errors during decoding in RecordEncoder.ts by reloading the canonical durable structures and retrying. It also enforces an append-only invariant on structures to prevent silent corruption. The feedback recommends hardening namedStructures to guarantee it always returns an array to prevent bypassing the append-only check, and improving error handling and logging within the recovery block of decode() to prevent silent failures if _mergeStructures is unavailable or throws an error.

[gemini-code-assist]

The namedStructures function extracts the shared-structures array from the dictionary. However, if dict.named or dict.get('named') is not an array (e.g., due to corrupted data or an unexpected structure format), it will return that non-array value. This would cause isFaithfulExtension to silently return true because a.length would be undefined, bypassing the append-only check.

We should harden namedStructures to guarantee it always returns an array.

Suggested change

	function namedStructures(dict: any): any[] {
	if (!dict) return [];
	if (Array.isArray(dict)) return dict;
	if (dict instanceof Map) return dict.get('named') ?? [];
	return dict.named ?? [];
	}
	function namedStructures(dict: any): any[] {
	if (!dict) return [];
	if (Array.isArray(dict)) return dict;
	if (dict instanceof Map) {
	const named = dict.get('named');
	return Array.isArray(named) ? named : [];
	}
	return Array.isArray(dict.named) ? dict.named : [];
	}

[gemini-code-assist]

In the recovery block, if this._mergeStructures is not a function (e.g., due to an unexpected runtime environment or dependency version mismatch), or if getStructures throws an error, the error is silently caught and swallowed. This makes troubleshooting extremely difficult because the actual cause of the recovery failure is hidden, and it just falls through to the generic "Error decoding record" log with the original mismatch error.

We should check if _mergeStructures is a function before calling it, and log any unexpected errors thrown during the recovery attempt.

			if (!this._reloadingStructures && isStructureMismatchError(error)) {
				this._reloadingStructures = true;
				try {
					if (typeof this._mergeStructures === 'function') {
						this._mergeStructures(this.getStructures());
						const recovered = this.decode(buffer, options);
						recordAction(true, STRUCTURE_RELOAD_RECOVERED_METRIC, this.name ?? this.rootStore?.name);
						return recovered;
					} else {
						harperLogger.warn('Cannot recover from structure mismatch: _mergeStructures is not a function');
					}
				} catch (recoveryError) {
					harperLogger.warn('Structure reload recovery failed', recoveryError);
				} finally {
					this._reloadingStructures = false;
				}
			}

[claude]

Reviewed; no blockers found.

[kriszyp]

Repositioned as defense-in-depth and holding in draft: the root cause is fixed at the source by #1520 (the migration now persists the canonical seed; shapeForStructure was stubbing the RecordObjects it reads). This PR's decode-heal stays as optional no-restart recovery for any residual divergence; isFaithfulExtension here isn't load-bearing. Will promote (and trim to just the heal) only if the field shows it's needed beyond #1520. — KrAIs