diff --git a/docs/superpowers/baselines/2026-04-17/BASELINE.md b/docs/superpowers/baselines/2026-04-17/BASELINE.md index 71260720..73438119 100644 --- a/docs/superpowers/baselines/2026-04-17/BASELINE.md +++ b/docs/superpowers/baselines/2026-04-17/BASELINE.md @@ -222,6 +222,19 @@ Ordered by severity. Each item cites the raw artifact it was derived from. - **Playwright E2E: 0 passed / 575 failed.** 100% failure rate. Almost certainly environment/config rather than regressions — the audit script runs `npm run test:e2e` without starting the backend (`java -jar ... serve`), so any test that hits `/api/*` will fail. Needs a harness that spins up the server (or mocks it) before running Playwright, or a `webServer` entry in `playwright.config.ts`. - Raw: `raw/frontend/playwright.log`, `raw/frontend/playwright-summary.json`. + - **RESOLVED — config + system-deps + run (2026-04-17, branch `phase-a/fix-playwright-webserver`)**: added a `webServer` block to `src/main/frontend/playwright.config.ts` that boots `java -jar target/code-iq-*-cli.jar serve .seeds/spring-petclinic --port 8080` and gates readiness on `/api/stats` (HTTP 200). Installed system libs via `sudo npx playwright install-deps chromium`. Ran full chromium suite against the live backend (19.2 min wall time). + + **New chromium baseline**: `33 passed / 94 failed / 4 skipped` out of **131 tests**. Huge improvement from "0 passed / 575 failed" (the 575 was the sum across 7 Playwright projects — chromium/firefox/edge/webkit + 3 responsive breakpoints). The 94 chromium failures are **not environmental** — they're real test ↔ UI divergence. Signature histogram of the 22 detailed failure blocks analyzed: + + | Pattern | Interpretation | + |---|---| + | `expect(locator).toBeVisible() failed` / `element(s) not found` | Selectors targeting UI elements that no longer render (renamed / removed). | + | `strict mode violation: getByText(/0/) resolved to 8 elements` | Tests assumed unique stats values; spring-petclinic's 691-node graph produces many zero-valued stat cards. | + | `expect(received).toHaveLength(expected)` | List count mismatches (menu items, nav links, stats rows). | + | `Error: Button ... has no aria-label` | Real a11y regressions in `accessibility.spec.ts`. | + | `locator.focus: Test timeout 30000ms exceeded` | UI interactions hanging on elements that never appear. | + + Test-suite maintenance / fixture generation is a follow-up (not a Phase A blocker). Candidate approaches: (1) rebuild the Ant-Design selectors to match current UI taxonomy; (2) pin a synthetic fixture with known, deterministic stats counts; (3) relax strict-mode selectors to `.first()` where tests genuinely target "any such element". 22 of the 94 failures have full error blocks captured in `raw/frontend/playwright.log`. ### High diff --git a/src/main/frontend/playwright.config.ts b/src/main/frontend/playwright.config.ts index d9b67875..8d82b978 100644 --- a/src/main/frontend/playwright.config.ts +++ b/src/main/frontend/playwright.config.ts @@ -5,8 +5,14 @@ import { defineConfig, devices } from '@playwright/test'; * Playwright E2E test configuration for Code IQ UI Redesign (Phase 7). * * Prerequisites: - * - `code-iq serve` running on localhost:8080 (Neo4j graph loaded) - * - Or use `webServer` below to start the Vite dev server pointing at a real backend + * - A pre-built CLI jar under ../../../target/code-iq-*-cli.jar + * (run `mvn -DskipTests package` from the repo root if missing). + * - A pre-populated fixture at /.seeds/spring-petclinic + * with .code-iq/cache + .code-iq/graph populated by + * `./scripts/baseline/run-pipeline.sh spring-petclinic`. The webServer + * block below boots `code-iq serve` against this fixture automatically. + * - Set `BASE_URL` / `E2E_FIXTURE` to override defaults when running against + * a different backend or fixture. * * Run all tests: npm run test:e2e * Run headed: npm run test:e2e:headed @@ -22,8 +28,30 @@ export default defineConfig({ workers: process.env.CI ? 1 : undefined, reporter: [['html', { outputFolder: 'playwright-report' }], ['line']], + // Boot the real code-iq backend against a pre-enriched fixture before any + // spec runs. Skipped if BASE_URL points elsewhere (developer has their own + // backend) or if a server is already listening on the target port locally. + webServer: process.env.BASE_URL ? undefined : { + command: [ + 'bash -c', + // Use shell so the target/*-cli.jar glob expands. `exec` hands the PID + // to Playwright so it can kill the process cleanly on test teardown. + `"exec java -jar ../../../target/code-iq-*-cli.jar serve ` + + `${process.env.E2E_FIXTURE ?? '../../../.seeds/spring-petclinic'} ` + + `--port ${process.env.E2E_PORT ?? '8080'}"`, + ].join(' '), + // /actuator/health is not a reliable readiness signal today (see + // BASELINE.md §GraphHealthIndicator). /api/stats returns 200 iff the + // server has finished starting and the graph has loaded. + url: `http://localhost:${process.env.E2E_PORT ?? '8080'}/api/stats`, + timeout: 120_000, + reuseExistingServer: !process.env.CI, + stdout: 'pipe', + stderr: 'pipe', + }, + use: { - baseURL: process.env.BASE_URL ?? 'http://localhost:8080', + baseURL: process.env.BASE_URL ?? `http://localhost:${process.env.E2E_PORT ?? '8080'}`, trace: 'on-first-retry', screenshot: 'only-on-failure', video: 'retain-on-failure',