fix(tryout): let a host with no working browser skip the retina gate#882
Open
aelefebv wants to merge 3 commits into
Open
fix(tryout): let a host with no working browser skip the retina gate#882aelefebv wants to merge 3 commits into
aelefebv wants to merge 3 commits into
Conversation
The live-usage harness only ever drove the SPA at deviceScaleFactor 1, which
is the easy half of the matrix: a retina backing store is 4x the pixels per
frame, and when that crosses the frame-completion budget the failure is
silent and total — a black canvas with no exception, no console error, and a
frame counter still climbing. That is exactly the defect class the wiki
gotcha describes, and the harness could not see it.
The ceiling now drives a DPR render matrix (2 then 1) in one browser, one
context per arm, and the retina arm gates the surface. Each arm is judged on
three things rather than "the driver didn't throw":
* readiness — window.__lucidaCaptureReady, checked but never trusted alone,
because it is published from the JS side of a WebGPU submit and so reports
ready on a frame the GPU never presented;
* DPR fidelity — the observed devicePixelRatio and the captured image's
scale versus its CSS box, so an arm that silently degrades to DPR 1 fails
loudly instead of manufacturing confidence about the untested half;
* content — the centre 60% x 60% of the main canvas must not be one flat
colour. Neither cheaper check works: the full page is richly coloured
because the SPA chrome renders fine while the viewer is black, and an
element-clipped canvas shot composites the corner-anchored overlays (FPS
badge, orientation cube, minimap) that supply a spurious second colour.
Verified against a deliberately broken render: the canvas crop had 27 distinct
colours and scripts/assert_png_nonblank.py passed it, while the centre had 1
and the gate failed the surface. A healthy page passes both arms.
A host with no browser reports the gate as NOT ENFORCED rather than passing;
LUCIDA_TRYOUT_REQUIRE_DPR2=1 makes that fatal. Scenario UI captures default to
DPR 2 for the same reason. The policy is pure functions over measurements, so
the 17 new tests run in CI without a browser.
Still uncovered and out of scope: the harness floor (the product CLI's own
capture path) hardcodes deviceScaleFactor 1 in its CDP calls.
Review found both of the gate's decline-to-answer paths resolving to pass, and one of them reachable by the exact failure shape the gate exists to catch. An errored retina arm was indistinguishable from "no browser on this host". `ran` was the last statement of the driver's per-arm try, so any throw — a goto timeout, a renderer or GPU process death under the 4x backing store — left `ran: false`, no gating arm was found, and the gate reported NOT ENFORCED and passed. Reproduced against the published commit with a page that wedges only at DPR >= 2: DPR 2 timed out, DPR 1 was healthy, gate ok=True gated=False, with the self-contradictory summary "drove the SPA at deviceScaleFactor 2 and 1". `ran` now means "attempted against a live browser" and is set before the work; the new `completed` carries "reached its measurements". An arm that ran but did not complete is judged and fails, every check behind it failing closed. The gate treats a present-but-unanswered retina arm as a failure, using the signal that already distinguished the cases: a genuine provisioning skip never builds an arms list at all. The driver's summary no longer claims arms it did not drive. A driver that died without printing a result also failed open, inconsistently with the timeout branch right above it, which was already fail-closed and said why. By that point node was found, Playwright resolved and a browser path resolved, so this is not an environment fact. It now matches. A failed gate changed no exit code: only `not result.ran` fed the run verdict, so the gate printed FAIL under an OK headline and exited 0. The verdict now also reads each surface's declared `render_gate`. Deliberately narrow — it reads the gate, not `surface.ok`, and tests `ok is False`, so ordinary per-step failures and an unenforced gate keep their existing semantics. `report --human` printed "FAIL (viewer non-blank)" for a gate failure, which is the reassuring-but-irrelevant signal this whole change exists to stop showing. The one-liner now leads with the gate, distinguishes held from NOT ENFORCED, and `drive --human` shares it so the two cannot disagree. A wedged arm no longer vanishes from the HTML and Markdown matrix for want of a screenshot. Also said plainly what CI covers: it runs the judging policy and renders nothing, so a green tick is not evidence that lucida renders at retina. The README no longer tells readers to set LUCIDA_TRYOUT_REQUIRE_DPR2 "in CI" when no CI job drives a browser. 29 policy tests (up from 17), including the retina-only wedge, the silent driver death, and the run-verdict and summary-line paths.
The previous fix leaned on an invariant the code does not have: that a non-empty `arms` list means a browser launched. It does not — `capture_real_spa` builds a placeholder arm for every *requested* scale factor, so a host whose browser cannot start (Chrome on disk, system libraries missing; a sandbox it cannot enter) got a full-length list of arms that never ran, and the gate hard-failed it while claiming the arm "was attempted in a live browser". That branch also fired only on those environment early exits, never on the wedged-arm shape it was written for. The discriminator is now `ran`, which the driver sets before an arm's work: the gate fails a missing retina verdict only when an arm actually ran, and it prefers the unrun arm's own reason so the failure explains itself. And "did the browser run anything" is read from the driver's raw records, not from the paired arms — otherwise a pairing failure erases the very fact the gate needs, and the run reports a tolerated skip over a payload that plainly says two arms were driven. Arms are paired to requested factors by the scale factor the record *names*, never by list position, so a truncated or reordered driver list cannot relabel one arm's pixels as the retina arm's. (Latent today: the shipped driver emits arms in the order Python requested them. It stops being latent the first time those arms are parallelised.) Two records claiming one factor are refused rather than silently resolved — keeping the first would drop a black second arm and pass. The gate no longer flips the surface's `ok`. `ok` is the floor, and only the floor; the gate travels in `render_gate`, from where it fails the run and sets every verdict word a human reads — the `--human` summaries and the web headings in report.html and report.md, which otherwise printed `PASS (viewer non-blank)` directly above a gate section reading FAIL. Long gate reasons are clipped in the one-line headline, which prints the same text in full two rows below, and a wedged arm's "no screenshot" note in the Markdown report gets its trailing blank line so it stops merging into the next arm. The report's verdict word asks `surface_render_gate_failed` instead of re-deriving "did the gate fail" — the copy it had was already inconsistent with the run verdict (it also honoured a gate under `real_spa.gate`, which the run verdict does not read), so a record of that shape would have printed FAIL headings over a run that exited 0. Unreachable today, since the surface emits `render_gate` on every record it can produce; removed because this change is about a rule copied into too many places. The README's `report` acceptance line no longer says the web surface's `ok` must be true, which under floor-only semantics is no longer the test. The same false invariant is corrected in the four places that repeated it (the docstring, the harness README, the wiki gotcha, and the test that guarded it with a payload the shipped driver cannot emit). Tests: 33 -> 43. Each new test that guards a fix was run against the previous candidate and fails there; one pins the invariant that makes the report's delegation safe, and passes on both sides by design.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes
lucida-sai.The problem
The pre-release harness reported the viewer as passing when no browser had actually run. If the machine had no working browser the render check produced no result, and "no result" was read as "nothing failed". Three separate places printed a pass on that basis — the CLI summary line, the HTML report, and the Markdown report, the last of which printed
## Web surface — PASS (viewer non-blank)as a heading directly above a failed gate.That is the exact false confidence the gate exists to prevent.
The fix
A host with no browser is now reported as not enforced — distinct from both pass and fail, because "the gate never ran" and "the gate passed" are different facts and a reader must be able to tell them apart.
The surface's own
okis now the floor and only the floor, which is what a black viewer passes; the gate travels separately inrender_gate. The question "did the gate fail?" is asked of exactly one function and never re-derived, so the report headings, the CLI, and the run's exit code cannot disagree.This also adds the retina requirement that prompted the work: the harness drives both
deviceScaleFactor1 and 2, perwiki/gotchas/retina-dpr2-render-verification.md.How it was verified
Independent review ran a 12-case probe through the real capture path. Both previously fail-quiet rows now fail closed: a pairing that matches nothing, and records that claim to have run but name only non-requested scale factors. A conflicting duplicate scale factor is refused rather than quietly resolved, and does not catch any legitimate payload — healthy, black-retina, reordered, single-factor and string-valued inputs all behave as before. The no-browser and opt-out paths still reach not enforced rather than fail.
An independent sweep of every reader of the surface
okfield across the repository found no fourth instance of the defect.Deliberate-failure trials: the new tests were run against the previous modules and fail there — two behavioural failures plus four signature errors on the capture module, and one behavioural failure on the report module with the literal bad heading string. 43 tests green through both the Makefile and CI's own discovery.
Known and not fixed
The tests pin the record shape rather than the producer, so reintroducing the old propagation upstream would not be caught. The display-layer guard is the load-bearing part. Four unreachable
render_gate or real_spa.gateselection expressions remain and are tracked separately aslucida-y9cp.🤖 Generated with Claude Code