Found while working #127 (ELITEA-1739 automation, PR #130)
The situation
PR #130 (ELITEA-1739 "Search Skills by Name" automation) is fully ready:
- Implementer: green 3/3 local runs.
- A fresh, independent reviewer approved it after triangulating the original test case, the automation spec, and the code, and re-running it live twice.
- I additionally triggered this repo's CI workflow (
Playwright Tests, run 29105930248) against the PR's branch to get an independent, full-suite confirmation. Downloaded the JSON test report: the new test (ELITEA-1739: search skills by name) passed. The run's overall result is "failure," but that's from 12 pre-existing, already-documented failures in unrelated modules (agents/pipelines/modal-handling/lazy-loading/artifacts — e.g. the known GH#29 defect) — not from anything in this PR.
The actual blocker
My working rule for this task is: merge only on a green gh pr checks exit — never on my own reading of test output. But gh pr checks 130 reports no checks at all, and always will under the current CI config: .github/workflows/playwright.yml is deliberately workflow_dispatch-only (manual trigger, no pull_request trigger — see the workflow file's own header comment, and .agents/testing.md § CI integration, which says not to add a pull_request trigger "without a separate, explicit decision"). Manually running the workflow against the PR's branch (as I did) produces a run, but GitHub does not attach it to the PR as a check — gh pr checks still reports nothing, regardless.
So as currently configured, no PR on this repo can ever satisfy "green gh pr checks" — not because anything is broken, but because there's structurally no PR-attached check to be green. This isn't something I can safely decide unilaterally: adding a pull_request trigger is exactly the "separate, explicit decision" the workflow file's own comment says to wait for, and it's out of scope for the ELITEA-1739 case I was dispatched to automate.
Options I see
- Add a
pull_request trigger to .github/workflows/playwright.yml (framework-scale CI change) so future PRs get a real, PR-attached check. Tradeoff: every PR would then run the entire suite (currently ~43 minutes, workers:1) including the 12 known-defect reds, so gh pr checks would likely never go fully green anyway unless those known-defects are marked in a way CI treats as non-blocking (e.g. a required-checks allowlist, or splitting the workflow to run only changed-module tests).
- Redefine the merge-gate mechanic for this repo specifically: since a PR-attached check isn't available, treat "reviewer
APPROVED + a manually-triggered full-suite run showing the new test passing, with no new failures beyond the existing known-defect baseline" as the accepted merge evidence for unattended runs — i.e., amend this project's merge-gate rule/seed to match what's actually achievable here, rather than a rule written for a repo with PR-triggered CI.
- Something else a human sees that I don't.
My recommendation
Option 2, scoped narrowly: treat a manually-triggered workflow_dispatch run (like the one I already have for PR #130) as valid merge evidence specifically when (a) the new/changed test(s) pass, and (b) the only failures present already match this repo's documented known-defect baseline (.agents/testing.md § Known issues / § CI integration's open infra-crash bucket) with no new failures introduced. Option 1 is a legitimate future improvement but a bigger, separate framework-scale decision (would need a "known-defects don't block" mechanism to avoid making PRs permanently red by design).
Parking #127 as Blocked pending this decision — PR #130 itself is otherwise done (implementer green, reviewer approved, manually-verified CI pass on the specific test).
Found while working #127 (ELITEA-1739 automation, PR #130)
The situation
PR #130 (ELITEA-1739 "Search Skills by Name" automation) is fully ready:
Playwright Tests, run 29105930248) against the PR's branch to get an independent, full-suite confirmation. Downloaded the JSON test report: the new test (ELITEA-1739: search skills by name) passed. The run's overall result is "failure," but that's from 12 pre-existing, already-documented failures in unrelated modules (agents/pipelines/modal-handling/lazy-loading/artifacts — e.g. the knownGH#29defect) — not from anything in this PR.The actual blocker
My working rule for this task is: merge only on a green
gh pr checksexit — never on my own reading of test output. Butgh pr checks 130reports no checks at all, and always will under the current CI config:.github/workflows/playwright.ymlis deliberatelyworkflow_dispatch-only (manual trigger, nopull_requesttrigger — see the workflow file's own header comment, and.agents/testing.md§ CI integration, which says not to add apull_requesttrigger "without a separate, explicit decision"). Manually running the workflow against the PR's branch (as I did) produces a run, but GitHub does not attach it to the PR as a check —gh pr checksstill reports nothing, regardless.So as currently configured, no PR on this repo can ever satisfy "green
gh pr checks" — not because anything is broken, but because there's structurally no PR-attached check to be green. This isn't something I can safely decide unilaterally: adding apull_requesttrigger is exactly the "separate, explicit decision" the workflow file's own comment says to wait for, and it's out of scope for the ELITEA-1739 case I was dispatched to automate.Options I see
pull_requesttrigger to.github/workflows/playwright.yml(framework-scale CI change) so future PRs get a real, PR-attached check. Tradeoff: every PR would then run the entire suite (currently ~43 minutes,workers:1) including the 12 known-defect reds, sogh pr checkswould likely never go fully green anyway unless those known-defects are marked in a way CI treats as non-blocking (e.g. a required-checks allowlist, or splitting the workflow to run only changed-module tests).APPROVED+ a manually-triggered full-suite run showing the new test passing, with no new failures beyond the existing known-defect baseline" as the accepted merge evidence for unattended runs — i.e., amend this project's merge-gate rule/seed to match what's actually achievable here, rather than a rule written for a repo with PR-triggered CI.My recommendation
Option 2, scoped narrowly: treat a manually-triggered
workflow_dispatchrun (like the one I already have for PR #130) as valid merge evidence specifically when (a) the new/changed test(s) pass, and (b) the only failures present already match this repo's documented known-defect baseline (.agents/testing.md§ Known issues / § CI integration's open infra-crash bucket) with no new failures introduced. Option 1 is a legitimate future improvement but a bigger, separate framework-scale decision (would need a "known-defects don't block" mechanism to avoid making PRs permanently red by design).Parking #127 as
Blockedpending this decision — PR #130 itself is otherwise done (implementer green, reviewer approved, manually-verified CI pass on the specific test).