ci: add cross-browser workflow using twd-runner#1
Open
kevinccbsg wants to merge 14 commits into
Open
Conversation
Runs the TWD tests across Chromium, Firefox, and WebKit via the twd-runner composite action, complementing the existing twd-cli contract-validation CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
6 passed · 0 failed · 1 warnings |
twd-runner is now published, so the composite action's npx call resolves it from npm directly — drop the GitHub install step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
6 passed · 0 failed · 1 warnings |
… flakiness Pin twd-runner to 0.1.1 in devDependencies so `npx twd-runner run` resolves the exact published version in CI. Duplicate the Todo List scenarios into 40 generated *.twd.test.ts files (~124 tests) to mirror a larger real-world suite (web-checkout). All tests run in a single page per browser and each navigation re-registers service-worker mocks; with this many tests Firefox intermittently fails (Chromium/WebKit stay green), reproducing the reported cross-browser flakiness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
Experiment: keep all 3 browsers running concurrently (concurrency: 0) but opt into waitForServiceWorker so the runner waits for the SW to control the page before running tests. If Firefox goes green under unchanged memory contention, the root cause is the service-worker control race, not the runner memory ceiling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
249 passed · 0 failed · 41 warnings |
Experiment: concurrency=1 (one engine at a time) + timeout=60s, keeping waitForServiceWorker=true. Removes cross-engine memory/CPU contention to test whether the Firefox mid-run SW evictions and WebKit SW-wait timeouts are driven by resource pressure or are a genuine SW-control bug in headless Firefox/WebKit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
Replace the single-job all-browsers-at-once run with a matrix over chromium/firefox/webkit. Each engine gets a fresh runner with full CPU/RAM (no cross-engine contention) and the jobs run in parallel, so wall-clock is the slowest single browser, not the sum. twd-runner reads the browser list only from twd.config.json, so each job patches .browsers to its matrix engine via jq before running. Keep waitForServiceWorker=true + timeout=60s in the config: isolation alone does not fix the SW registration-before-control race — those flags do. Drop the now-meaningless concurrency setting (each job runs a single browser). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
252 passed · 0 failed · 41 warnings |
Root cause of the parallel-only flakiness: a cold Vite dev server runs optimizeDeps on first page load and forces a full reload, disrupting service worker registration. Chromium tolerates it; Firefox/WebKit lose the race and the SW never controls the page (the runner's SW wait is also capped at Playwright's default 30s, so it can't be extended via config.timeout). In a sequential run Chromium accidentally warms the shared dev server for the slower engines — which is why series passed but parallel/matrix failed. Each parallel job has its own cold server, so pre-bundle deps with `vite optimize` before starting it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
Point twd-runner at BRIKEV/twd-runner#fix/sw-wait-timeout instead of the published 0.1.1 so CI picks up the fix that makes the service-worker wait honor config.timeout. With timeout=60s now actually applied, Firefox/WebKit get enough runway to claim the SW in headless parallel jobs. Uses an https git URL so CI can clone the public repo without SSH credentials. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
The vite-optimize warm-up only pre-bundled deps, so the app module graph and SW path were still cold on first load — enough for WebKit (with the timeout fix) but Firefox still timed out claiming the SW in parallel. Replace it with a real headless Chromium page load that waits for the sidebar + SW controller, fully compiling the module graph and activating the SW. This mirrors what a sequential all-browsers run gets for free (Chromium warms the shared server first), so each parallel job's test engine starts against a warm server. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
twd-runner now warms the dev server itself whenever waitForServiceWorker is set (BRIKEV/twd-runner#fix/sw-wait-timeout, commit 03d8b1a). Remove the bespoke warmup.mjs script and the "Install Chromium for warm-up" / "Warm up dev server" steps — each parallel job gets a hot server with no extra CI plumbing, just `npx twd-runner run` via the action. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
249 passed · 0 failed · 41 warnings |
…ertion Chromium is already exercised by twd-cli (Puppeteer) in ci.yml, so the twd-runner matrix only needs the engines Puppeteer can't reach — drop chromium from the matrix and the browsers list. Also fix the create-todo assertion to use findAllByText (async, retries) instead of getAllByText (sync): the list re-renders a tick after the post-create fetch resolves, so the synchronous query could run before the DOM updated and flake. The mock was landing fine — only the assertion timing was wrong. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
Temporary workflow to check whether the built-in warm-up makes a single job running both engines concurrently viable, vs requiring the matrix. To be removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
twd-runner runs firefox + webkit in parallel within one runner via `npx twd-runner run`, so the matrix and per-job jq browser-pinning are unnecessary. A single job keeps the workflow portable — the same two steps (start dev server, run the runner) work on any CI, not just GitHub Actions. Verified green across 4 consecutive runs. Remove the throwaway single-job experiment workflow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
Switch from the git branch ref to the released npm version now that 0.1.2 is published (SW-wait timeout fix + automatic warm-up). Pins cleanly to a registry version instead of a moving branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TWD Contract Validation
246 passed · 0 failed · 41 warnings |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new GitHub Actions workflow (
.github/workflows/cross-browser.yml) that runs this app's TWD tests across Chromium, Firefox, and WebKit usingtwd-runner's reusable composite action.This complements the existing
ci.yml(which usestwd-clifor Chromium + contract validation). The same*.twd.test.tstests run unchanged — they usetwd-jscommands, not engine-specific APIs, so they're portable across runners/engines.How it works
BRIKEV/twd-runner/.github/actions/run@main, which installs the Playwright browsers (cached) and runsnpx twd-runner run.twd-runnerreads the existingtwd.config.jsonand runs the full browser matrix in parallel (it ignores thecontractskeys — contract validation stays withtwd-cli).Note
twd-runnerisn't published to npm yet, so the workflow installs it from GitHub (npm install --no-save github:BRIKEV/twd-runner). Once it's published, that step can be dropped andtwd-runneradded todevDependencies.Test Plan
cross-browserworkflow passes on this PR (chromium + firefox + webkit, headless on Ubuntu)🤖 Generated with Claude Code