Central list of tests that fail non-deterministically because of the environment (slow CI runners, timing, resource contention) rather than a product bug. Use it to spot patterns and decide when a flake is frequent enough to be worth hardening.
Monitoring
When a known flake fails CI, re-run the failed job. If the rerun passes, add a comment here with the run URL, so the recurrence frequency stays visible in one place. When a test recurs often enough to be annoying, harden it (raise its retry count or relax the brittle assertion) and record the fix here.
The stress steps run under nextest with retries = 1 (2 attempts total). A real regression fails every attempt; a flake usually passes on a fresh rerun.
Known flaky tests
stability::test_error_recovery_stress (crates/playwright/tests/integration/stability.rs), macOS stress step.
Signature: Too few successful navigations: 1 (expected at least 2).
Cause: the test requires at least ~30% of its navigations to succeed (a floor of 2 of 4); a slow runner managed only 1 and failed both nextest attempts. Durable fix when needed: lower the floor to 1, or give the stress step more retries.
First recorded: 2026-06-05 (commit 5d846ed; passed on rerun).
test_context_browser_cross_browser, Windows stress step timeout. Detail and history in #99.
Central list of tests that fail non-deterministically because of the environment (slow CI runners, timing, resource contention) rather than a product bug. Use it to spot patterns and decide when a flake is frequent enough to be worth hardening.
Monitoring
When a known flake fails CI, re-run the failed job. If the rerun passes, add a comment here with the run URL, so the recurrence frequency stays visible in one place. When a test recurs often enough to be annoying, harden it (raise its retry count or relax the brittle assertion) and record the fix here.
The stress steps run under nextest with
retries = 1(2 attempts total). A real regression fails every attempt; a flake usually passes on a fresh rerun.Known flaky tests
stability::test_error_recovery_stress(crates/playwright/tests/integration/stability.rs), macOS stress step.Signature:
Too few successful navigations: 1 (expected at least 2).Cause: the test requires at least ~30% of its navigations to succeed (a floor of 2 of 4); a slow runner managed only 1 and failed both nextest attempts. Durable fix when needed: lower the floor to 1, or give the stress step more retries.
First recorded: 2026-06-05 (commit 5d846ed; passed on rerun).
test_context_browser_cross_browser, Windows stress step timeout. Detail and history in #99.