Fix: stream could emit nothing when a write raced its initial query (+ exp 256: never emit stale)

[danReynolds]

Hypothesis

Not a hypothesis — a bug, found by finally investigating the write_coalescing_test failure that had been dismissed as a flake (by me, three times in one session, while shepherding unrelated PRs).

db.stream(sql) promises to emit current results immediately and re-emit when writes change them. It could do neither: a stream whose initial query was still in flight when a write landed went permanently silent, never emitting even its first value.

The failure output said so plainly, once read:

Which: emitted x Stream closed.
       which never did emit an event that <8>

emitted x — nothing at all. Not a stale value: no value.

Mechanism

StreamEngine._createStream stored the initial result's hash as the comparison baseline, then — if a write had dirtied the entry mid-flight — discarded those rows unemitted in favour of a catch-up re-query. But that re-query's selectIfChanged compares against the baseline taken from the very rows it replaced, finds them identical, and returns rows == null ("unchanged, nothing to emit").

Initial emission suppressed as stale. Replacement suppressed as unchanged. Stream silent, permanently.

The race window is the initial query's duration — trivial on a 4-row table, reliable on a 40k-row one. Opening a stream and immediately writing (seeding a list on first paint) is ordinary usage.

Approach

Always emit the initial result — it is a valid observation and the documented contract — then re-query if the entry was dirtied:

entry.emit(initialRows);

if (entry.dirty) {
  _requeryQueue.add(entry);
  _flushQueue();
}

No duplicate risk: the catch-up still runs through selectIfChanged, which self-suppresses when nothing changed.

Two earlier hypotheses were tried and discarded (both recorded in the writeup): a test-side registration race, and a stranded re-query queue. Neither moved the failure rate. Reading the actual failure text instead of theorising would have gone straight to the answer.

Results

New regression test seeds 40k rows so the write reliably races the initial query:

	without fix	with fix
new regression test	0 / 5 pass	6 / 6 pass
write_coalescing_test (the old "flake")	3 / 6 pass	8 / 8 pass

Standalone reproduction, main vs this branch:

main:        ✗ stream emitted NOTHING in 4s (permanently silent)
this branch: ✓ emissions: [40008] → final 40008

Also corrects that test file's comment asserting the shape "can't flake under load" — a false confidence marker is worse than none, since it actively redirects investigation.

Outcome

Accepted — user-visible correctness fix. Recorded as exp 255 with claims 255.1 (the emission guarantee) and 255.2 (a 1-in-3 reproduction is a bug, not a flake), so the knowledge graph carries both the mechanism and the methodology lesson.

Unblocks #273, which was only ever failing on this.

Test plan

• dart analyze --fatal-infos clean on changed lib + tests
• Regression test: 0/5 without fix, 6/6 with
• write_coalescing_test: 8/8 (was 3/6)
• 140 tests pass across stream, write-coalescing, trigger-cascade, database, transaction suites
• finalize_experiment.dart + check_knowledge_links.dart green

[danReynolds]

Updated: folded in exp 256, which changes the fix to its better form.

The original commit fixed the silence by always emitting the initial rows, then correcting. That worked, but left a design question the bug had obscured: once emission is guaranteed, which rows should the first one carry — the known-superseded initial rows (emit-then-correct), or only the corrected result (suppress-stale)?

Measured both, across race windows widened by seed size:

shape	metric	A emit-then-correct	B suppress-stale
racing · 20k	time to correct (p50)	8.1 ms	9.5 ms
racing · 20k	stale frames per stream	1.00	0.00
racing · 60k	time to correct (p50)	18.5 ms	18.9 ms
racing · 60k	stale frames per stream	1.00	0.00
no race · 20k (control)	correct / emissions	13.3 ms / 2.00	13.3 ms / 2.00

A's faster first emission measures the wrong clock — it delivers rows already known superseded, so the subscriber has to disregard them anyway. On time-to-correct-value the two are within ~1 ms, and B removes the stale frame entirely (the list-jump artifact a reactive stream exists to prevent). Unraced streams are identical.

Shipping B: poison the comparison baseline when the entry is dirtied, so the catch-up re-query is guaranteed to emit and that emission becomes the stream's first.

This is not a return to the bug. The exp 255 defect was suppression without poisoning — the re-query compared against the very rows it replaced, reported "unchanged", and both emissions cancelled. The sentinel guarantees a mismatch, and _requery propagates errors to subscribers, so a stream always resolves to a value or an error.

Exp 255's writeup and claim are restated for the shipped semantics; 255.1 now refines 256.1 in the knowledge graph.