chore(ci): surface pass/fail + failure reasons on Actions run page - #92
Merged
Conversation
Adds Playwright's built-in `github` reporter (CI-only, additive alongside the existing html/json reporters) for inline ::error:: annotations on the Checks tab, plus a step that parses test-results/json/run.json into a markdown pass/fail table + per-failure reason on $GITHUB_STEP_SUMMARY — so a run's outcome and its "why" are visible without downloading the HTML report artifact.
5 tasks
Operator-requested defense-in-depth for the infra-crash bucket seen on run
28658801255 ("Target page, context or browser has been closed" + empty
.fill() reads). Verified this flag is already Playwright's own
unconditional chromium default (playwright-core's chromiumSwitches), so
this is a harmless duplicate rather than a fix for something that was
missing -- noted inline. Real root cause of the infra-crash bucket, if it
persists on the next run, is still open.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
githubreporter toplaywright.config.ts, gated toprocess.env.CIonly (kept out of local runs) — additive, alongside the existinghtml/jsonreporter entries which are unchanged. Emits::error::workflow commands (file:line + message) for every failure, rendered by GitHub as inline annotations on the Checks tab..github/scripts/summarize-results.jsscript + a newif: always()workflow step in.github/workflows/playwright.ymlthat parses the existingtest-results/json/run.json(json reporter output, unchanged) into a markdown pass/fail/flaky/skipped table + per-failed-test title/file:line/reason, appended to$GITHUB_STEP_SUMMARY.workflow_dispatchonly, per.agents/testing.md§ CI integration.Test plan
npx playwright test --listlocally (noCIenv) —githubreporter correctly absent, no errors.CI=1 npx playwright test --list --grep "TC-001"—githubreporter active, emitted a real::notice::workflow command, exit 0..github/scripts/summarize-results.jsunit-verified against a realrun.jsoncaptured from an actual (intentionally-pointed-at-a-wrong-URL) local run — confirmed: JSON reporter's exact schema (stats.expected/unexpected/flaky/skipped, nestedsuites[].specs[].tests[]), ANSI color codes inerror.messageare stripped, top-level file-suite title excluded from the failure breadcrumb (would've duplicated the Location column).run.json, malformed JSON, zero failures, failures present.https://claude.ai/code/session_01Le9JiPjQB7349wghx16Ghv