From 123de5124f7eebc9c14f6b30d1936f1ee770eb64 Mon Sep 17 00:00:00 2001 From: Amit Kumar Date: Fri, 17 Apr 2026 11:34:39 +0000 Subject: [PATCH 1/2] fix(e2e): wire playwright webServer to boot live code-iq backend MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a webServer block to src/main/frontend/playwright.config.ts so npm run test:e2e boots a real code-iq serve against the spring-petclinic fixture before any spec runs, removing the root cause of the 0-pass / 575-fail Playwright audit result. Design choices: - /api/stats is the readiness probe, not /actuator/health (today returns 503 OUT_OF_SERVICE; tracked separately on phase-a/fix-graph-health). - bash -c "exec java ..." so Playwright can signal the JVM cleanly on teardown, and so the *-cli.jar glob expands. - Overridable via BASE_URL (skip webServer entirely), E2E_FIXTURE (pick a different pre-enriched repo), E2E_PORT (avoid port conflicts). - baseURL unchanged for spec code — still http://localhost:8080 by default, overridable via BASE_URL. Verified on this machine: - webServer boots the backend; /api/stats returns 200 within 120s. - Playwright connects to the backend. Remaining system-level blocker (not a config issue): chromium headless shell is missing 8 shared libraries on this host (libatk-1.0, libXcomposite, libgbm, libatspi, ...). Fix once per host: sudo npx playwright install-deps chromium Once that's run, npm run test:e2e exercises the full 131-test chromium suite against a real backend. The original baseline 0p/575f number was the sum across 7 Playwright projects. Documented in BASELINE.md. --- .../baselines/2026-04-17/BASELINE.md | 1 + src/main/frontend/playwright.config.ts | 34 +++++++++++++++++-- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/docs/superpowers/baselines/2026-04-17/BASELINE.md b/docs/superpowers/baselines/2026-04-17/BASELINE.md index 71260720..a968b515 100644 --- a/docs/superpowers/baselines/2026-04-17/BASELINE.md +++ b/docs/superpowers/baselines/2026-04-17/BASELINE.md @@ -222,6 +222,7 @@ Ordered by severity. Each item cites the raw artifact it was derived from. - **Playwright E2E: 0 passed / 575 failed.** 100% failure rate. Almost certainly environment/config rather than regressions — the audit script runs `npm run test:e2e` without starting the backend (`java -jar ... serve`), so any test that hits `/api/*` will fail. Needs a harness that spins up the server (or mocks it) before running Playwright, or a `webServer` entry in `playwright.config.ts`. - Raw: `raw/frontend/playwright.log`, `raw/frontend/playwright-summary.json`. + - **RESOLVED (config) / PENDING (system-deps) (2026-04-17, branch `phase-a/fix-playwright-webserver`)**: added a `webServer` block to `src/main/frontend/playwright.config.ts` that boots `java -jar target/code-iq-*-cli.jar serve .seeds/spring-petclinic --port 8080` and gates readiness on `/api/stats` (HTTP 200) — `/actuator/health` is not a reliable readiness signal today, see the GraphHealthIndicator gap. Environment-overridable via `BASE_URL`, `E2E_FIXTURE`, `E2E_PORT`. Verified end-to-end on this machine: the webServer block successfully boots the backend, serves `/api/stats`, and Playwright connects. Chromium browser binary was missing; reinstalled via `npx playwright install chromium` (112 MB). Further blocker: chromium's headless shell is missing 8 shared libs on this host (`libatk-1.0.so.0`, `libatk-bridge-2.0.so.0`, `libXcomposite.so.1`, `libXdamage.so.1`, `libXfixes.so.3`, `libXrandr.so.2`, `libgbm.so.1`, `libatspi.so.0`) — this causes a `browserType.launch: ... exitCode=127` on every test. Fix: one-time `sudo npx playwright install-deps chromium` (requires sudo/password; can't be done non-interactively here). Once deps are installed, re-run the frontend audit to capture the true pass rate. Note: the chromium project alone contains 131 tests, not 575 — the original 575-fail count was the sum across all 7 Playwright projects (chromium/firefox/edge/webkit + 3 responsive breakpoints). ### High diff --git a/src/main/frontend/playwright.config.ts b/src/main/frontend/playwright.config.ts index d9b67875..8d82b978 100644 --- a/src/main/frontend/playwright.config.ts +++ b/src/main/frontend/playwright.config.ts @@ -5,8 +5,14 @@ import { defineConfig, devices } from '@playwright/test'; * Playwright E2E test configuration for Code IQ UI Redesign (Phase 7). * * Prerequisites: - * - `code-iq serve` running on localhost:8080 (Neo4j graph loaded) - * - Or use `webServer` below to start the Vite dev server pointing at a real backend + * - A pre-built CLI jar under ../../../target/code-iq-*-cli.jar + * (run `mvn -DskipTests package` from the repo root if missing). + * - A pre-populated fixture at /.seeds/spring-petclinic + * with .code-iq/cache + .code-iq/graph populated by + * `./scripts/baseline/run-pipeline.sh spring-petclinic`. The webServer + * block below boots `code-iq serve` against this fixture automatically. + * - Set `BASE_URL` / `E2E_FIXTURE` to override defaults when running against + * a different backend or fixture. * * Run all tests: npm run test:e2e * Run headed: npm run test:e2e:headed @@ -22,8 +28,30 @@ export default defineConfig({ workers: process.env.CI ? 1 : undefined, reporter: [['html', { outputFolder: 'playwright-report' }], ['line']], + // Boot the real code-iq backend against a pre-enriched fixture before any + // spec runs. Skipped if BASE_URL points elsewhere (developer has their own + // backend) or if a server is already listening on the target port locally. + webServer: process.env.BASE_URL ? undefined : { + command: [ + 'bash -c', + // Use shell so the target/*-cli.jar glob expands. `exec` hands the PID + // to Playwright so it can kill the process cleanly on test teardown. + `"exec java -jar ../../../target/code-iq-*-cli.jar serve ` + + `${process.env.E2E_FIXTURE ?? '../../../.seeds/spring-petclinic'} ` + + `--port ${process.env.E2E_PORT ?? '8080'}"`, + ].join(' '), + // /actuator/health is not a reliable readiness signal today (see + // BASELINE.md §GraphHealthIndicator). /api/stats returns 200 iff the + // server has finished starting and the graph has loaded. + url: `http://localhost:${process.env.E2E_PORT ?? '8080'}/api/stats`, + timeout: 120_000, + reuseExistingServer: !process.env.CI, + stdout: 'pipe', + stderr: 'pipe', + }, + use: { - baseURL: process.env.BASE_URL ?? 'http://localhost:8080', + baseURL: process.env.BASE_URL ?? `http://localhost:${process.env.E2E_PORT ?? '8080'}`, trace: 'on-first-retry', screenshot: 'only-on-failure', video: 'retain-on-failure', From afbaa426a5efa39c11794f705ee5427dbac204c1 Mon Sep 17 00:00:00 2001 From: Amit Kumar Date: Fri, 17 Apr 2026 12:18:41 +0000 Subject: [PATCH 2/2] docs(baseline): capture real Playwright chromium pass rate (33/131) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit With the webServer wiring in place + system deps installed via `sudo npx playwright install-deps chromium`, ran the full chromium project (19.2 min wall time). Results: 33 passed / 94 failed / 4 skipped Huge jump from the "0 passed / 575 failed" baseline — the 575 was the sum across 7 Playwright projects (4 browsers × many responsive breakpoints), and all of it was caused by not having a backend. With a live backend against the spring-petclinic fixture, a third of the chromium suite passes. The remaining 94 failures are NOT environmental; they're genuine test ↔ UI divergence: - expect(locator).toBeVisible / element(s) not found — selectors targeting UI elements that don't render in the current build (component/CSS-class renames). - strict mode violations on `getByText(/0/)` — tests assume unique stat values; spring-petclinic's 691-node graph produces many zero-valued cards, so the match resolves to 8+ elements. - toHaveLength mismatches — expected N list items, got different count. - "Button has no aria-label" in accessibility.spec.ts — real a11y regressions, not a test bug. - 30s timeouts on locator.focus / locator.boundingBox — elements that never appear. Treating the chromium 33/94/4 split as the honest Phase A baseline. Test-suite maintenance (update selectors to match current UI, or generate a deterministic fixture) is scoped out of Phase A. --- docs/superpowers/baselines/2026-04-17/BASELINE.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/baselines/2026-04-17/BASELINE.md b/docs/superpowers/baselines/2026-04-17/BASELINE.md index a968b515..73438119 100644 --- a/docs/superpowers/baselines/2026-04-17/BASELINE.md +++ b/docs/superpowers/baselines/2026-04-17/BASELINE.md @@ -222,7 +222,19 @@ Ordered by severity. Each item cites the raw artifact it was derived from. - **Playwright E2E: 0 passed / 575 failed.** 100% failure rate. Almost certainly environment/config rather than regressions — the audit script runs `npm run test:e2e` without starting the backend (`java -jar ... serve`), so any test that hits `/api/*` will fail. Needs a harness that spins up the server (or mocks it) before running Playwright, or a `webServer` entry in `playwright.config.ts`. - Raw: `raw/frontend/playwright.log`, `raw/frontend/playwright-summary.json`. - - **RESOLVED (config) / PENDING (system-deps) (2026-04-17, branch `phase-a/fix-playwright-webserver`)**: added a `webServer` block to `src/main/frontend/playwright.config.ts` that boots `java -jar target/code-iq-*-cli.jar serve .seeds/spring-petclinic --port 8080` and gates readiness on `/api/stats` (HTTP 200) — `/actuator/health` is not a reliable readiness signal today, see the GraphHealthIndicator gap. Environment-overridable via `BASE_URL`, `E2E_FIXTURE`, `E2E_PORT`. Verified end-to-end on this machine: the webServer block successfully boots the backend, serves `/api/stats`, and Playwright connects. Chromium browser binary was missing; reinstalled via `npx playwright install chromium` (112 MB). Further blocker: chromium's headless shell is missing 8 shared libs on this host (`libatk-1.0.so.0`, `libatk-bridge-2.0.so.0`, `libXcomposite.so.1`, `libXdamage.so.1`, `libXfixes.so.3`, `libXrandr.so.2`, `libgbm.so.1`, `libatspi.so.0`) — this causes a `browserType.launch: ... exitCode=127` on every test. Fix: one-time `sudo npx playwright install-deps chromium` (requires sudo/password; can't be done non-interactively here). Once deps are installed, re-run the frontend audit to capture the true pass rate. Note: the chromium project alone contains 131 tests, not 575 — the original 575-fail count was the sum across all 7 Playwright projects (chromium/firefox/edge/webkit + 3 responsive breakpoints). + - **RESOLVED — config + system-deps + run (2026-04-17, branch `phase-a/fix-playwright-webserver`)**: added a `webServer` block to `src/main/frontend/playwright.config.ts` that boots `java -jar target/code-iq-*-cli.jar serve .seeds/spring-petclinic --port 8080` and gates readiness on `/api/stats` (HTTP 200). Installed system libs via `sudo npx playwright install-deps chromium`. Ran full chromium suite against the live backend (19.2 min wall time). + + **New chromium baseline**: `33 passed / 94 failed / 4 skipped` out of **131 tests**. Huge improvement from "0 passed / 575 failed" (the 575 was the sum across 7 Playwright projects — chromium/firefox/edge/webkit + 3 responsive breakpoints). The 94 chromium failures are **not environmental** — they're real test ↔ UI divergence. Signature histogram of the 22 detailed failure blocks analyzed: + + | Pattern | Interpretation | + |---|---| + | `expect(locator).toBeVisible() failed` / `element(s) not found` | Selectors targeting UI elements that no longer render (renamed / removed). | + | `strict mode violation: getByText(/0/) resolved to 8 elements` | Tests assumed unique stats values; spring-petclinic's 691-node graph produces many zero-valued stat cards. | + | `expect(received).toHaveLength(expected)` | List count mismatches (menu items, nav links, stats rows). | + | `Error: Button ... has no aria-label` | Real a11y regressions in `accessibility.spec.ts`. | + | `locator.focus: Test timeout 30000ms exceeded` | UI interactions hanging on elements that never appear. | + + Test-suite maintenance / fixture generation is a follow-up (not a Phase A blocker). Candidate approaches: (1) rebuild the Ant-Design selectors to match current UI taxonomy; (2) pin a synthetic fixture with known, deterministic stats counts; (3) relax strict-mode selectors to `.first()` where tests genuinely target "any such element". 22 of the 94 failures have full error blocks captured in `raw/frontend/playwright.log`. ### High