test: de-flake whole-group teardown; make CI failures diagnosable#2
Merged
Conversation
… message The whole-group-teardown fixture starts two node processes (server + fanned-out child) before it binds. An 8s readiness wait is marginal on a loaded ubuntu runner where vitest runs 31 files in parallel next to a 9s stress test; it flaked once on main (ubuntu/22 only — macos/22 passed). - Raise that wait to 20s. A successful wait costs nothing; only failures wait. - On timeout, dump the server's own log instead of asserting 'expected false to be true', which said nothing about the cause. - CI: fail-fast: false. The first failure cancelled the sibling jobs and hid whether the flake was platform-specific.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mainwent red immediately after the 0.4.0 merge, on a tree identical to one that had just passed CI.What actually happened
Only
ubuntu-latest, 22failed.macos-latest, 22passed; the other two jobs were cancelled by fail-fast, which made it look like a four-job failure. The failing assertion wastest/process.test.ts:48—expected false to be true— i.e.waitForPort(GROUP_PORT, 8000)timed out after 8047ms.It does not reproduce locally (12/12 clean runs of that file in isolation).
Cause
The
multiproc-serverfixture starts two node processes — the server plus a fanned-out child that mimics vite→esbuild — before it binds the port. An 8s readiness wait is marginal on a loaded 4-core ubuntu runner, where vitest runs all 31 test files in parallel alongside a 9s rapid-switch stress test. This is a timing flake, not a product bug: the group-teardown behaviour it guards is unchanged and passes everywhere else.Changes
expected false to be truetold us nothing about the cause; the test now raisesserver never bound port 39518. Its log: …. Verified by forcing the failure path (binding a different port) and confirming the message prints.fail-fast: falsein CI. Cancelling sibling jobs on the first failure destroyed the evidence needed to tell a platform-specific flake from a real regression.Test-only + CI config. No
src/changes. 114/114 pass locally, typecheck clean.