Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/superpowers/baselines/2026-04-17/BASELINE.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,19 @@ Ordered by severity. Each item cites the raw artifact it was derived from.

- **Playwright E2E: 0 passed / 575 failed.** 100% failure rate. Almost certainly environment/config rather than regressions — the audit script runs `npm run test:e2e` without starting the backend (`java -jar ... serve`), so any test that hits `/api/*` will fail. Needs a harness that spins up the server (or mocks it) before running Playwright, or a `webServer` entry in `playwright.config.ts`.
- Raw: `raw/frontend/playwright.log`, `raw/frontend/playwright-summary.json`.
- **RESOLVED — config + system-deps + run (2026-04-17, branch `phase-a/fix-playwright-webserver`)**: added a `webServer` block to `src/main/frontend/playwright.config.ts` that boots `java -jar target/code-iq-*-cli.jar serve .seeds/spring-petclinic --port 8080` and gates readiness on `/api/stats` (HTTP 200). Installed system libs via `sudo npx playwright install-deps chromium`. Ran full chromium suite against the live backend (19.2 min wall time).

**New chromium baseline**: `33 passed / 94 failed / 4 skipped` out of **131 tests**. Huge improvement from "0 passed / 575 failed" (the 575 was the sum across 7 Playwright projects — chromium/firefox/edge/webkit + 3 responsive breakpoints). The 94 chromium failures are **not environmental** — they're real test ↔ UI divergence. Signature histogram of the 22 detailed failure blocks analyzed:

| Pattern | Interpretation |
|---|---|
| `expect(locator).toBeVisible() failed` / `element(s) not found` | Selectors targeting UI elements that no longer render (renamed / removed). |
| `strict mode violation: getByText(/0/) resolved to 8 elements` | Tests assumed unique stats values; spring-petclinic's 691-node graph produces many zero-valued stat cards. |
| `expect(received).toHaveLength(expected)` | List count mismatches (menu items, nav links, stats rows). |
| `Error: Button ... has no aria-label` | Real a11y regressions in `accessibility.spec.ts`. |
| `locator.focus: Test timeout 30000ms exceeded` | UI interactions hanging on elements that never appear. |

Test-suite maintenance / fixture generation is a follow-up (not a Phase A blocker). Candidate approaches: (1) rebuild the Ant-Design selectors to match current UI taxonomy; (2) pin a synthetic fixture with known, deterministic stats counts; (3) relax strict-mode selectors to `.first()` where tests genuinely target "any such element". 22 of the 94 failures have full error blocks captured in `raw/frontend/playwright.log`.

### High

Expand Down
34 changes: 31 additions & 3 deletions src/main/frontend/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,14 @@ import { defineConfig, devices } from '@playwright/test';
* Playwright E2E test configuration for Code IQ UI Redesign (Phase 7).
*
* Prerequisites:
* - `code-iq serve` running on localhost:8080 (Neo4j graph loaded)
* - Or use `webServer` below to start the Vite dev server pointing at a real backend
* - A pre-built CLI jar under ../../../target/code-iq-*-cli.jar
* (run `mvn -DskipTests package` from the repo root if missing).
* - A pre-populated fixture at <repo-root>/.seeds/spring-petclinic
* with .code-iq/cache + .code-iq/graph populated by
* `./scripts/baseline/run-pipeline.sh spring-petclinic`. The webServer
* block below boots `code-iq serve` against this fixture automatically.
* - Set `BASE_URL` / `E2E_FIXTURE` to override defaults when running against
* a different backend or fixture.
*
* Run all tests: npm run test:e2e
* Run headed: npm run test:e2e:headed
Expand All @@ -22,8 +28,30 @@ export default defineConfig({
workers: process.env.CI ? 1 : undefined,
reporter: [['html', { outputFolder: 'playwright-report' }], ['line']],

// Boot the real code-iq backend against a pre-enriched fixture before any
// spec runs. Skipped if BASE_URL points elsewhere (developer has their own
// backend) or if a server is already listening on the target port locally.
webServer: process.env.BASE_URL ? undefined : {
command: [
'bash -c',
// Use shell so the target/*-cli.jar glob expands. `exec` hands the PID
// to Playwright so it can kill the process cleanly on test teardown.
`"exec java -jar ../../../target/code-iq-*-cli.jar serve `
+ `${process.env.E2E_FIXTURE ?? '../../../.seeds/spring-petclinic'} `
+ `--port ${process.env.E2E_PORT ?? '8080'}"`,
].join(' '),
// /actuator/health is not a reliable readiness signal today (see
// BASELINE.md §GraphHealthIndicator). /api/stats returns 200 iff the
// server has finished starting and the graph has loaded.
url: `http://localhost:${process.env.E2E_PORT ?? '8080'}/api/stats`,
timeout: 120_000,
reuseExistingServer: !process.env.CI,
stdout: 'pipe',
stderr: 'pipe',
},

use: {
baseURL: process.env.BASE_URL ?? 'http://localhost:8080',
baseURL: process.env.BASE_URL ?? `http://localhost:${process.env.E2E_PORT ?? '8080'}`,
trace: 'on-first-retry',
screenshot: 'only-on-failure',
video: 'retain-on-failure',
Expand Down
Loading