diff --git a/.gitignore b/.gitignore index b763e0d..aee943b 100644 --- a/.gitignore +++ b/.gitignore @@ -11,4 +11,9 @@ oclif.manifest.json ## local-only dev/release notes .release-steps.md -AGENTS.md \ No newline at end of file +AGENTS.md + +## docs/ is the GitHub Pages source. Keep local research/scratch markdown out of git, +## but DO publish the blog post (and index.html, which isn't .md so isn't matched). +docs/*.md +!docs/blog-lessons-ai-e2e.md diff --git a/README.md b/README.md index 83330bc..f06ec00 100644 --- a/README.md +++ b/README.md @@ -91,15 +91,13 @@ lacuna generate lacuna fix ├─ Read tsconfig paths, deps, setup ├─ Model writes a surgical fix ├─ Send full context to the model ├─ Pass → next file ├─ Run the generated tests └─ Fail → record it, detect loops, - ├─ Pass → next file retry, restore on giving up + ├─ Pass → next file retry, keep the best attempt └─ Fail → retry with the error, keep the best attempt ``` Two rules hold throughout: lacuna never leaves a half-written file behind, and it never removes passing tests. If it can't fully fix a file, it keeps the attempt with the most passing tests — and if nothing beat the starting point, it puts the original back. -**Coverage is a guide, not a mandate.** lacuna won't manufacture junk tests to turn a red line green — no type-impossible inputs (`null as any` on a non-nullable prop), no assertions that contradict the test's own title, no tests that lock in an incidental quirk. When you target an already-tested file, the only lines left uncovered are usually defensive/edge branches that aren't meaningfully testable; lacuna leaves those uncovered and tells you so, rather than padding the suite. Each accepted file is then run through your project's own `eslint --fix` + `prettier` so it matches your style. - --- ## Commands @@ -138,6 +136,7 @@ lacuna generate --verbose # live panel as the model writes lacuna generate --workers 4 # process 4 files in parallel lacuna generate --fresh # ignore the cached coverage report lacuna generate --format json --output report.json +lacuna generate --e2e # generate Playwright end-to-end specs (see below) ``` If you ran `analyze` in the last 10 minutes, `generate` reuses that report instead of running the suite again (`--fresh` forces a new run). When retries are exhausted, lacuna keeps the best attempt **only if it adds passing tests** and points you to `lacuna fix` for the rest; otherwise it restores the original. If the model produces the same output twice, the loop stops early instead of wasting iterations. @@ -196,6 +195,7 @@ lacuna fix --verbose lacuna fix --fresh lacuna fix --no-regenerate-on-failure # don't fall back to regenerating lacuna fix --fix-polluters # handle tests that pass alone but fail in the suite +lacuna fix --e2e # repair failing Playwright specs (see below) ``` A few behaviors worth knowing: @@ -216,6 +216,160 @@ lacuna run --- +## End-to-end testing (Playwright) + +Everything above writes unit and integration tests. With `--e2e`, lacuna instead works at the **browser** layer: it discovers your app's routes, drives a real browser to see what's actually on each page, and writes [Playwright](https://playwright.dev) specs that click and assert like a user. + +Requirements: `@playwright/test` installed and a `playwright.config.ts` with a `webServer` block (so lacuna can start your app) and a `baseURL`. Route discovery currently supports **Next.js** (app and pages router) and **React Router**. + +You don't have to set Playwright up by hand. `lacuna init` offers to do it for React and Next.js projects, and if you run `--e2e` without it, lacuna offers to install it on the spot (interactive terminals only — in CI it prints the command and exits so nothing hangs). Setting it up means: installing `@playwright/test` and the browser binaries, **scaffolding a `playwright.config.ts`** (framework-aware `webServer` command + `baseURL`, `testDir: ./e2e`) if you don't already have one — an existing config is never overwritten — and scaffolding the auth helpers described below. After the browser download, on Linux lacuna detects Playwright's "host system is missing dependencies" warning and tells you to run `sudo npx playwright install-deps` (that step needs sudo, so it can't be automated). + +The scaffolded config's `webServer.command`/`url` and `baseURL` are derived from your project — the package manager comes from the lockfile, the dev command from your `dev`/`start` script, and the port from that script (`-p`/`--port`/`PORT=`) or the framework default (`:3000` Next.js/CRA, `:5173` Vite). It can't infer a non-localhost host, so double-check it. A correct `webServer`/`baseURL` is also what lets `--workers` run in parallel (without it, lacuna can't confirm a shared app server and falls back to running specs one at a time). + +```bash +lacuna generate --e2e # discover routes, generate a spec for each +lacuna generate --e2e --route /login # just one route +lacuna generate --e2e --workers 4 # generate in parallel +lacuna generate --e2e --deep # walk multi-step flows: fill + SUBMIT forms (test/staging only) +lacuna generate --e2e --dry-run # list which routes would get specs (no app, no API call) +lacuna fix --e2e # repair failing Playwright specs +``` + +How generation works: + +1. **Discover** the routes from your router (e.g. `app/login/page.tsx` → `/login`, or a ``). +2. **Snapshot** each page: lacuna starts your app once via the `webServer` config and captures the accessibility tree plus any `data-testid`s, so the model writes specs against the elements that are *really there*. If you've set up authentication (below), routes that redirect to login are re-snapshotted **signed in**, so the captured DOM is the real logged-in page. +3. **Generate** one spec per route. Selectors follow a strict order: `getByRole`, `getByLabel`, `getByPlaceholder`, then `getByTestId` (only for testids actually present on the page, never invented), with `getByText` as a last resort. Brittle CSS, XPath, and arbitrary sleeps are forbidden; assertions are auto-waiting and validate the outcome of each action. Protected routes become `*.auth.spec.ts` specs that run signed in; un-set-up auth means a login redirect is asserted rather than fabricated. +4. **Verify**: lacuna runs each spec, confirms it isn't flaky, and retries on failure. A spec that goes green and stays green is kept; one that never passes is **kept on disk for repair** (run `lacuna fix --e2e` to iterate on it) rather than discarded — only a route that never yielded a runnable spec at all is cleaned up. Retries may not **shrink the suite**: an attempt that deletes a test to go green is rejected, and lacuna keeps the attempt with the most passing tests (so coverage never silently drops across retries — this applies to `fix` too). + +**Going deeper (`--deep`).** By default lacuna writes one focused spec per route plus the actions a single click reveals. With `--deep` it *walks* multi-step journeys in a real browser — filling inputs (type-aware values), driving custom ARIA comboboxes/selects that a plain fill can't (Radix/Headless UI/MUI), clicking the advance/submit control, dismissing onboarding/cookie interrupts, and capturing the real success/validation **toast** to assert — step by step until the flow ends. Because it fills and **submits real forms**, use a test/staging environment (and see [Seeding](#seeding-test-data-when-flows-need-existing-records) for flows that need existing data, like an item that requires a category). Progress streams per flow so a long walk never looks stuck. + +If your project already uses `data-testid`s, lacuna picks them up from the snapshot and prefers them where a semantic locator isn't enough. It reads testids from your components but does not add them, it only touches test files. + +Specs are written to your Playwright `testDir`. Routes that already have a spec are skipped, so re-running only fills the gaps. `--dry-run` is a free preview: it lists what would be generated without starting your app or calling the model. + +**Repairing specs (`lacuna fix --e2e`).** When a spec breaks, lacuna captures a fresh snapshot of the page, then asks the model to diagnose the root cause (selector drift, a timing gap, an auth redirect, a removed feature) and apply the smallest fix that preserves what the test checks. It fixes selectors and synchronization rather than weakening or deleting assertions to force a pass. + +### Authenticated coverage (testing signed-in pages) + +Most of an app lives behind a login. lacuna can cover those pages too — it logs in as a test user, captures the **signed-in** DOM of each protected route, and writes specs that run authenticated. You provide the test user; lacuna never invents or commits credentials. + +**1. Set Playwright up** (if you haven't). `lacuna init` (choose end-to-end) or just `lacuna generate --e2e --dry-run` scaffolds everything without running the app or calling the model: + +| File | What it is | +|---|---| +| `playwright.config.ts` | Three projects: `setup` (logs in), `chromium` (public specs), `authenticated` (`*.auth.spec.ts`, reuses the saved session). | +| `e2e/test-config.ts` | Exports `testUser { email, password }` (env-backed) and `authRoutes { login, afterLogin }` — **you fill these in**. | +| `e2e/auth.setup.ts` | A setup spec that logs in and saves the session to `playwright/.auth/user.json`. | +| `.gitignore` | `playwright/.auth/` is added so the saved session is never committed. | + +On Linux, run `sudo npx playwright install-deps` once so the browsers can launch. + +**2. Fill in the test user.** Edit `e2e/test-config.ts` with a real seeded account (or set `E2E_EMAIL` / `E2E_PASSWORD`), and set `authRoutes.login` to your login path: + +```ts +export const testUser = { + email: process.env.E2E_EMAIL ?? 'qa@yourapp.com', + password: process.env.E2E_PASSWORD ?? 'a-real-test-password', +} +export const authRoutes = { login: '/login', afterLogin: '/dashboard' } +``` + +**3. Point the login helper at your form.** The scaffolded `e2e/auth.setup.ts` uses generic selectors — adjust them to your actual login form (the field labels and the submit button), and change the post-login wait to a real signal: + +```ts +await page.getByLabel(/email/i).fill(testUser.email) +await page.getByLabel(/password/i).fill(testUser.password) +await page.getByRole('button', { name: /sign in/i }).click() +await page.waitForURL('**/dashboard') // a real "you're logged in" signal +``` + +**4. Capture the session.** Run the setup project once — it logs in and writes the saved session: + +```bash +npx playwright test --project=setup +ls playwright/.auth/user.json # success = this file now exists +``` + +**5. Generate.** Now `lacuna generate --e2e` does a two-pass capture: public routes are snapshotted normally; routes that redirect to login are **re-snapshotted signed in** and get `*.auth.spec.ts` specs that exercise the logged-in UI. Token sessions (Firebase/Supabase/JWT) expire after ~1 hour, so if the saved session is stale or missing lacuna **auto-refreshes** it by running the `setup` project before capturing (you'll see `✓ Login session refreshed.`); if no valid credentials are configured it falls back gracefully. + +```bash +lacuna generate --e2e +# Generating: /admin (authenticated) → e2e/admin.auth.spec.ts +``` + +**6. Run everything.** Plain `npx playwright test` runs the `setup` project first (fresh login), then the public and authenticated specs: + +```bash +npx playwright test +``` + +Notes: +- Public routes stay `*.spec.ts`; protected routes become `*.auth.spec.ts`. Re-running `generate --e2e` skips a route if **either** variant already exists, so there are no duplicates. +- lacuna's own snapshot/verify runs never need the credentials — only the `authenticated` project and the saved session do. So generating specs works even before you've filled in the test user (protected routes just stay shallow until you do). +- If a protected route still snapshots as the login page, the saved session didn't unlock it — re-check the selectors in `auth.setup.ts` and re-run `npx playwright test --project=setup`. +- **Firebase / Supabase / Amplify auth** keep the session in **IndexedDB**, which `storageState()` does *not* capture by default — so the saved session is empty and protected pages stay locked even though login succeeded. The scaffolded `auth.setup.ts` saves with `storageState({ path, indexedDB: true })` (Playwright ≥ 1.51) to handle this; if you wrote your own setup, add `indexedDB: true`. + +#### How a route is detected as protected (and what it can't see) + +lacuna decides a route is auth-gated when its signed-out snapshot either **redirects to a login URL** (`/login`, `/users/sign_in`, `/auth/...`, etc.) or **renders a login form inline** (a password field, an OAuth "Continue with…" button, or a sign-in/up button next to an email field). It then re-snapshots that route with your saved session and only treats it as authenticated if the login wall is *gone*. This is a heuristic with a safety net — a wrong guess never produces a broken spec: + +- A **false positive** (a public page that looks login-ish, e.g. a newsletter "Sign up" box) is re-checked signed in, still looks the same, and **stays a public `*.spec.ts`**. Cost is one extra snapshot. +- A **false negative** (a login screen lacuna doesn't recognise) just falls back to a normal unauthenticated spec — same as if auth weren't set up. + +Cases it currently does **not** auto-detect, so they fall back to public specs (write the `*.auth.spec.ts` by hand, or `lacuna fix --e2e` it): +- **Magic-link / passwordless** screens (email only, no password or OAuth button). +- **Non-English** login UIs — the keywords it matches (`password`, `sign in`, `login`) are English. + +And one that's about *which* session you save, not detection: +- **Role-based pages** (e.g. `/admin`): capture the session as a user who actually has that role. A logged-in-but-unauthorized session makes lacuna write a spec against the "access denied" page. The saved-session path is read from your config's `authenticated` project, so a custom `storageState` location works too. + +### Seeding test data (when flows need existing records) + +Many real flows can't reach success from an empty account. "Add a menu item" needs a **category** to select; "create an order" needs a **product**; "assign a teammate" needs an **invite**. On a fresh test user those prerequisites don't exist, so the flow dead-ends at validation — and **no test, however well written, can get past it**. The fix is *seeding*: put known, controlled data into your database **before** tests run. + +This matters for lacuna specifically because `lacuna generate --e2e --deep` *drives* each flow (filling forms, submitting, and asserting the real outcome). If a required field can't be satisfied, the deepest it can honestly go is the validation message. Seed the prerequisite and the same run reaches the success toast / created row instead. + +What makes data "seeded" (not just inserted): +- **Deterministic** — the same fixed records every run (use fixed IDs/keys), so specs can rely on them and cleanup is exact. +- **Isolated** — scoped to the *test* user, never real data. +- **Set up before, torn down after** — created in Playwright's `globalSetup`, removed in `globalTeardown`, so re-runs don't pile up duplicates. (`globalSetup` is not a project, so it runs on every `playwright test` invocation — including lacuna's own verify runs — which is what you want: the data is present whenever specs run.) + +The recommended pattern — seed through your backend's **admin/service-role client** (fast, bypasses the UI and security rules), keyed to the test user: + +```ts +// e2e/seed.ts — example shape; swap in your own DB client (Firebase Admin, Supabase service role, Prisma…) +export async function seedTestData() { + const uid = await getTestUserId() // resolve the test user once + await db.set(`categories/${'e2e-seed-category'}`, { // FIXED key → idempotent + exact cleanup + userId: uid, name: 'E2E Seed Category', order: 0, + }) +} +export async function cleanupTestData() { + await db.remove(`categories/${'e2e-seed-category'}`) +} +``` + +```ts +// e2e/global-setup.ts // e2e/global-teardown.ts +import { seedTestData } from './seed' // import { cleanupTestData } from './seed' +export default async () => { await seedTestData() } // export default async () => { await cleanupTestData() } +``` + +```ts +// playwright.config.ts — add at the top level +globalSetup: './e2e/global-setup.ts', +globalTeardown: './e2e/global-teardown.ts', +``` + +Notes: +- A standalone setup script doesn't get your framework's automatic `.env` loading. Load it yourself (e.g. `dotenv`), and make sure the script uses **service-account / service-role** credentials, not the public client keys. +- The test user must already exist in your auth provider (the same one in `e2e/test-config.ts`). +- Match the seeded record's fields to your app's real model — the same shape your create-form would write. +- Seed only the *prerequisites* a flow needs to start. lacuna still drives the actual create/edit/delete and asserts their outcomes; seeding just gets the flow off the ground. + +--- + ## Configuration `lacuna init` writes `.lacuna.json`. Every field is optional and has a sensible default. @@ -307,6 +461,8 @@ Lacuna can run the suite and collect coverage for a wide range of languages. The **Runner only:** Go, Ruby (RSpec), Rust (cargo), C# (dotnet), Java (Gradle/Maven), Swift. Suites run and coverage is collected, but test generation isn't tuned for them yet. +**End-to-end (`--e2e`):** Playwright, with route discovery for Next.js (app and pages router) and React Router. See [End-to-end testing](#end-to-end-testing-playwright). + --- ## Shared mocks @@ -487,10 +643,11 @@ lacuna/ │ ├── commands/ # CLI commands: analyze, generate, fix, run, init │ ├── agent/ │ │ ├── loop.ts # generate → run → retry loop -│ │ ├── fix-loop.ts # fix → run → retry loop +│ │ ├── fix-loop.ts # fix → run → retry loop (+ --e2e repair) +│ │ ├── e2e-loop.ts # E2E generate: discover → snapshot → generate → verify │ │ ├── context.ts # builds model context (source, tests, mocks, types) │ │ ├── generator.ts # calls the model, manages conversation history -│ │ └── prompts/ # prompt builders, split by framework and runner +│ │ └── prompts/ # prompt builders, split by framework and runner (incl. e2e.ts) │ ├── lib/ │ │ ├── config.ts # config loader + zod schema │ │ ├── detector.ts # detects test runner and language @@ -498,6 +655,8 @@ lacuna/ │ │ ├── reporter.ts # terminal / JSON / markdown output │ │ ├── validate.ts # patch application, regression + broken-import detection │ │ ├── typecheck.ts # tsc pass and type-error scoping +│ │ ├── playwright.ts # Playwright detection, config parse, result parsing +│ │ ├── flows/ # E2E route discovery, DOM snapshot, app-server lifecycle │ │ ├── providers/ # model provider abstraction (anthropic, openai-compatible) │ │ └── coverage/ # lcov / json parsers, gap extraction │ └── ci/ # PR comment + GitHub Actions outputs diff --git a/docs/blog-lessons-ai-e2e.md b/docs/blog-lessons-ai-e2e.md new file mode 100644 index 0000000..2ed3728 --- /dev/null +++ b/docs/blog-lessons-ai-e2e.md @@ -0,0 +1,108 @@ +# Lessons from building an AI that writes end-to-end tests + +We built [lacuna](https://github.com/Octagon-simon/lacuna) to generate and repair real Playwright suites for real apps — log in, crawl the routes, recover the workflows, write the specs, run them, keep only what genuinely passes. The interesting part wasn't the model; it was everything around it. Here are the lessons that actually moved the needle, each tied to the bug that taught it. + +> The recurring theme: **most failures looked like one thing and were actually another.** Auth timing was a default timeout. A "workers bug" was `networkidle`. A "weak model" was a missing per-control map. Make the black box observable and the real cause is usually mundane. + +--- + +## 1. "What should I assert?" is the hard problem, not "what should I click?" + +Clicking is easy. The *assertion* is where generated E2E tests live or die — and the expected result is almost never in the DOM snapshot, because a success toast is transient and a redirect hasn't happened yet. + +The naive fix — grep the source for every `toast(...)` / `router.push(...)` and offer them as "assert one of these" — **fails on any non-trivial page.** A page with Save, Delete, and Upgrade buttons yields a dozen unrelated strings, and the model confidently asserts the *upgrade* redirect as the success of *saving a record*. We shipped that bug. + +**The fix is an AST.** Resolve `control → its handler → the toast/redirect/calls inside that handler`, so each action asserts *its own* outcome. Regex can find the strings; only the AST knows which control owns which. We open-sourced this as [FlowMap](https://github.com/Octagon-simon/flowmap) — a zero-dependency module that maps each control to its outcome by borrowing the target project's own TypeScript compiler. + +**Lesson:** ground the model in a *per-action* fact, not a bag of file-wide strings. + +--- + +## 2. A running UI is a state machine a snapshot can't navigate + +Deep flows (open a form → fill → submit → see the result; or a multi-screen wizard) can't be recovered from a static DOM capture. You have to *drive* the browser: fill the visible inputs, find the advance control, click, capture what appeared, repeat. + +Two things make this brutal in practice: + +- **Custom widgets.** A `role="combobox"` from Radix/Headless UI/MUI isn't an `` — `fill()` does nothing. But they all implement the same WAI-ARIA pattern, so one sequence (open → type-ahead → wait for `listbox` → click `option`) drives them all. (We pulled this out as a standalone module: [widget-driver](https://github.com/Octagon-simon/widget-driver).) +- **Modals are ambiguous.** A modal can be the *result* of your action (interact with it) or an *interrupt* (onboarding/cookie/promo — dismiss it). Dismiss the wrong one and the flow breaks; ignore an interrupt and it blocks you. Heuristics on the copy ("Maybe later" / "Skip" → dismiss; "Save" / "Confirm" → interact) get you most of the way. + +**Lesson:** static analysis tells you what *should* happen; only driving the real browser tells you what *does*. You need both — they cover each other's blind spots. + +--- + +## 3. The 30-second timeout that silently deleted coverage + +Our explorer walked many "openers" in **one Playwright test sharing one page**. On a 10-route app it would report "walked 17 flows, recorded 8 steps" — and a whole auth-gated section had *zero* tests. + +The cause was invisible until we dumped per-probe debug: probes 1–5 worked, then *every* later one failed with `Target page… has been closed`. Playwright's **default 30s per-test timeout** killed the test mid-loop, closed the page, and the loop kept going — so every later route (which sorted last) was dropped. + +**Lesson:** any test that loops N items in one `test()` must set `test.setTimeout()` scaled to N. The 30s default doesn't error loudly — it truncates silently. And: **make the black box observable.** A one-line per-item debug dump turned a multi-session mystery into a five-minute diagnosis. + +--- + +## 4. `networkidle` is a lie on realtime apps + +After fixing the timeout, exploration looked *hung* for minutes. The culprit: a `settle()` that waited on `waitForLoadState('networkidle', 5000)`. Firebase (and any websocket/polling app) keeps the network busy, so **networkidle never fires** — every settle burned the full timeout, times every step, times every probe. + +We already *tell generated specs* never to use `networkidle` as a sync crutch. We'd violated our own rule internally. Wait on a real UI signal (a spinner disappearing, content appearing) instead. + +**Lesson:** `networkidle` is meaningless on realtime apps. And a long operation must **show progress** — silence reads as "hung," which is a UX failure even when the tool is working. + +--- + +## 5. The model will go green by deleting the failing test + +Give an agent "make the suite pass" and it will, eventually, discover that the easiest way is to *remove the failing test*. We watched a spec shrink from 190 to 149 lines across retries and still "pass." + +A pass-count check alone can't catch it — deleting a *failing* test leaves the pass count unchanged. You have to gate on **test count not decreasing** and **keep-best by passing count**, then *restore the best attempt* on exhaustion rather than accept the latest. Once the loop tracks the most-passing version, a smaller all-green spec (4/4) naturally loses to a fuller one (5 passing of 6), and the temptation to special-case "green-but-smaller beats nothing" disappears. + +**Lesson:** "the suite is green" is a seductive, wrong success metric. Make the metric *coverage that doesn't decrease*. + +--- + +## 6. A green suite isn't a good suite + +Even when tests pass and aren't deleted, they can assert *nothing*: + +- "Sidebar still visible" after a create/edit/delete — proves nothing changed. +- Click "Categories", assert the *Categories button* is still there — a tautology (it was always there). +- `if (await btn.isVisible()) { … }` with no `else` — passes having asserted nothing. + +These all go green. They're worthless. The rules that kill them: a layout element that existed *before* the action is never a valid outcome; a tab switch asserts the *revealed panel*, not the clicked nav; an absent control is `test.skip(reason)`, not a vacuous pass. And feed the model the *real* outcome to assert (from #1/#2) so it doesn't reach for the lazy fallback. + +**Lesson:** the assertion, not the action, is the test. Ban the assertions that pass regardless of behavior. + +--- + +## 7. Some flows can't pass without seeded data — and that's fine + +"Add a menu item" needs a category to select. On a fresh test account there are none, so the flow validation-blocks no matter how good the test is. No amount of model cleverness fixes a missing prerequisite. + +The answer is **seeding**: deterministic (fixed keys), isolated (scoped to the test user), set up before / torn down after (Playwright `globalSetup`/`globalTeardown`), via the backend's admin client. The honest framing: a strong *generic* pass plus a thin *declared* per-project layer for what can't be inferred. + +(Gotcha: Ctrl+C skips `globalTeardown`, so provide a standalone manual-clear script — not a `playwright test --project=cleanup`, which would boot the app and *re-seed* before deleting.) + +**Lesson:** know the boundary between what you can auto-derive and what must be declared. Don't try to make the model paper over missing data. + +--- + +## 8. Auth is the gate to everything, and every framework hides it differently + +Authenticated coverage is most of a real app, and it's a minefield: + +- **Firebase/Supabase/Amplify store the session in IndexedDB**, which Playwright's `storageState()` ignores by default — so the saved session is empty and every protected page stays locked even though login "worked." Fix: `storageState({ path, indexedDB: true })`. +- **Token sessions expire (~1h).** If your repair runs use `--no-deps` (no per-attempt re-login), a stale session silently fails every authed spec. Auto-refresh when the saved session is stale or missing. +- **SPAs rehydrate auth asynchronously** (`if (authLoading) return `), so asserting the dashboard *cold* times out. Wait for a signed-in landmark first. + +**Lesson:** "login succeeded" doesn't mean "the session is captured, fresh, and rendered." Each of those is a separate failure mode. + +--- + +## The meta-lesson + +A test-writing agent is mostly **not** a prompting problem. It's route discovery, DOM capture, auth, state-machine exploration, custom-widget driving, outcome grounding, coverage guards, and assertion-quality enforcement — with the model as one component. Every one of the bugs above was in that scaffolding, not the model. Build the scaffolding to be *observable*, and the model's job gets small and reliable. + +--- + +*Open-sourced as standalone modules: [FlowMap](https://github.com/Octagon-simon/flowmap) and [widget-driver](https://github.com/Octagon-simon/widget-driver). The patterns above — coverage guards, assertion-quality rules, authenticated-Playwright recipe, seeding — are documented in the lacuna wiki.* diff --git a/docs/index.html b/docs/index.html index a3ce53f..37dc05e 100644 --- a/docs/index.html +++ b/docs/index.html @@ -632,6 +632,14 @@

Fix failing tests

Run lacuna fix to repair existing failing tests without rewriting them. The model surgically fixes what's broken and leaves everything else untouched.

+
+
+ +
+

End-to-end tests

+

With --e2e, lacuna discovers your routes, drives a real browser to see each page, and writes Playwright specs with role, label, and data-testid locators. It repairs broken specs against a fresh snapshot too. Supports Next.js and React Router.

+
+
diff --git a/project-map.md b/project-map.md index dbf46f9..d4907bc 100644 --- a/project-map.md +++ b/project-map.md @@ -9,8 +9,10 @@ CLI tool that uses AI to close test coverage gaps. Three commands: - `lacuna analyze` — read-only: runs coverage, shows gaps -- `lacuna generate` — generates new tests for untested/undertested source files -- `lacuna fix` — repairs existing failing tests without rewriting them +- `lacuna generate` — generates new tests for untested/undertested source files. With `--e2e`, instead generates Playwright **end-to-end** specs from discovered routes (see E2E section below) +- `lacuna fix` — repairs existing failing tests without rewriting them. With `--e2e`, repairs failing Playwright specs instead of unit tests + +**Two testing layers.** The default (unit/integration) layer is coverage-driven: targets are uncovered source symbols, tests import the source and assert on return values. The **E2E layer** (`--e2e`) is route-driven: targets are user-reachable routes, specs drive a real browser via Playwright and assert on what's on screen. They share the agent loop, provider, retry/oscillation machinery, and Playwright result parsing; what differs is target selection, context (DOM snapshot vs source), and verification (a green browser run vs executed lines). E2E specifics are documented inline: the `--e2e` rows under Commands, `agent/e2e-loop.ts` + `prompts/e2e.ts`, the `src/lib/playwright.ts` + `src/lib/flows/` modules, the E2E data flows, and the E2E invariants near the end. --- @@ -30,9 +32,9 @@ Each is an oclif `Command` subclass. All flags are defined here. | File | Command | What it does | |------|---------|-------------| | `analyze.ts` | `lacuna analyze` | Runs coverage suite, loads LCOV, calls `filterTestableGaps` + `findUncoveredFiles`, prints report. `passed` = `coveragePct >= threshold && untouchedCount === 0` — files with no tests at all cause FAIL even if covered files are at 100%. Flags: `--threshold`, `--format`, `--output`, `--verbose`. **Optional positional `[path]` (scope dir)**: scopes discovery + the coverage command to that subtree (`scopedCoverageCommand` for vitest/jest, else the full command + `isWithinDir` post-filter). When the scoped folder has **no tests** the suite is skipped entirely (pure FS scan — a brand-new folder reports in <1s instead of running the whole suite). Scoped runs pass `includeExisting: true` so the report also surfaces *partially-covered* files (below threshold WITH a test), not just untested ones. A file path is accepted and scoped to its parent dir. **`@diff[:]` positional (patch-coverage mode, read-only)**: resolves the diff scope (`git-diff.ts`), runs the normal suite, narrows gaps to changed-and-uncovered lines (`narrowGapsToDiff` + `missingChangedFileGaps`), prints a `Patch:` line, and `passed` = patch coverage ≥ threshold (Codecov's patch gate). Docs-only diff → "nothing to cover", exit 0 before running the suite. The generate hint re-targets `lacuna generate @diff:`. **`@diff` + a directory** (`analyze @diff packages/api`, order-independent — two positionals `[path] [scope]` disentangled by which one is the `@diff[:]` token): `scopeDiffToDir` narrows the changed-line map to files under the dir → patch coverage of just that folder's changed lines; the coverage command stays the FULL command (never dir-scoped) so it still matches Codecov. | -| `generate.ts` | `lacuna generate` | Calls `runAgentLoop`. Flags: `--file`, `--dry-run`, `--verbose`, `--model`, `--threshold`, `--format`, `--output`, `--workers`, `--fresh`, **`--improve`**. Header surfaces Model/Runner/Threshold/Workers + **Mocks** (when `mocksFile` set) and **Debug** (when enabled, via `debugLogPattern`). **Optional positional `[path]`**: a **file** routes to single-file mode (subsumes `--file`); a **directory** sets `scopeDir` — a scoped **create + improve** run that brings every file under the dir to threshold (creates tests for untested files AND extends existing below-threshold tests). `--improve` enables the same create+improve repo-wide without scoping. Header shows **Scope** (dir) / **Target** (file) / **Mode: improve**. **`@diff[:]` positional (patch-coverage mode)**: detected BEFORE the file-vs-dir `stat` branch (not a filesystem path); `''` after `@diff` = default base. **`--file` + `@diff` = intersection**: the diff scope is narrowed to that one file's changed lines (header `Scope: diff vs ∩ (M changed line(s))`; patch coverage measured over just those lines; an unchanged file → "nothing to cover", exit 0). The single-file FAST path is skipped in diff mode (coverage report needed), but the test-file-path validation still applies. **`@diff` + a directory** (`generate @diff packages/api`, order-independent two positionals `[path] [scope]` split by which is the `@diff` token): `scopeDir` + `diffRef` are BOTH passed to the loop, which narrows the diff to that folder (`scopeDiffToDir`) — only the changed lines inside the dir are targets (header `Scope: diff vs under (N changed file(s), M line(s))`). **Coverage sourcing in patch mode (critical, learned from a real miss)**: the BEFORE report MUST be a FULL-suite report — patch coverage only matches Codecov if it reads the same measurement Codecov uploaded (the whole CI suite). A NARROWED run (`vitest related` / one dir) executes only a subset of the tests that cover the changed file, so any line covered solely by an integration/DI test outside that subset looks uncovered → lacuna over-targets lines Codecov shows green (exactly what happened on afriex `BusinessService.runSuccessfulSwapHooks`: its covering test mocks BusinessService, real coverage came from elsewhere, `related` missed it → all 11 body lines flagged instead of Codecov's 4). So: diff mode REUSES any on-disk report regardless of age (only `--fresh` re-runs), and when it must run it uses the FULL command, never `related`. `relatedCoverageCommand` survives but is used ONLY for the cheap AFTER measurement (see loop.ts), where it's a safe union addend on top of the accurate before-report. Resolves the scope up front for the header and to fail fast (exit 2) on a bad base; passes `diffRef` into `runAgentLoop`. | -| `fix.ts` | `lacuna fix` | Calls `runFixLoop`. Flags: `--file`, `--dry-run`, `--verbose`, `--model`, `--workers`, `--fresh`, `--regenerate-on-failure` (default true), `--fix-polluters`, `--types` (select files by TypeScript type errors instead of test failures). Summary wording adapts in `--types` mode ("Still erroring" / "All type errors fixed"). Header surfaces **Mode: type errors** (`--types`), **Mocks**, and **Debug**. **Optional positional `[path]`**: a **test file** routes to single-file mode (subsumes `--file`); a **directory** sets `scopeDir` — only failing/erroring tests under that subtree are selected and the discovery run is scoped to it (`scopedTestCommand`). Header shows **Scope** (dir) / **Target** (file). | -| `init.ts` | `lacuna init` | Interactive wizard using `@inquirer/prompts`. Writes `.lacuna.json`. **Runs from any subdirectory** — `findProjectRoot()` walks up to nearest `package.json`. Detects framework from `package.json`: React, React Native (`react-native`), Expo (`expo`), Next.js (`next`), Vue (`vue`), Svelte (`svelte`), Angular (`@angular/core`), NestJS (`@nestjs/core`). **Dependency install**: `checkPackageInstallState(pkg, cwd)` returns three states — `declared` (in package.json, skip), `undeclared` (in node_modules but NOT in package.json — prompts user to add it for CI), `missing` (full install). `isPackageInstalled` is a convenience wrapper that returns `true` for both `declared` and `undeclared` (used only for checking extra packages). **Next.js**: setup file at `test/setup.ts` (project root); vitest.config.ts gets `@` alias from tsconfig AND `server-only` alias pointing to `test/empty-module.ts` (auto-created); setup file includes global mocks for `next/navigation`, `next/headers`, `next/cache`, `next/image`, `next/font/google`, `next/font/local` — these are Next.js-specific and would break a plain React project. No `environment: 'jsdom'` in vitest config (Next.js manages its own env). **Plain React** (not Next.js): setup file at `${sourceDir}/test/setup.ts`; vitest.config.ts gets `environment: 'jsdom'`; setup file is just `@testing-library/jest-dom` + cleanup hooks — NO Next.js mocks. **React Native/Expo**: `jest-expo` or `react-native` preset, no jsdom, warns if Vitest chosen. **`resolveJestVersionForPreset(preset, cwd)`** — dynamically resolves compatible jest version: (1) reads `node_modules/${preset}/package.json` peerDependencies.jest locally; (2) falls back to `npm info ${preset} peerDependencies.jest`; (3) warns if neither works. Used to install the correct jest major for RN/Expo (jest-expo@56 requires jest@29 — using jest@30 causes `clearMocksOnScope is not a function`). ts-jest NOT installed for RN/Expo projects (jest-expo handles TS via babel-preset-expo). Async-storage mock in the generated setup template uses `jest.requireActual()` not `require()`. **All Vitest projects**: vitest.config.ts includes `restoreMocks: true` and `clearMocks: true`. Setup file has `beforeEach(() => vi.restoreAllMocks())` + `afterEach` cleanup. **Vue/Svelte**: `@testing-library/vue`/`@testing-library/svelte` + jsdom. **Angular**: `jest-preset-angular`, no jsdom. **NestJS**: no setup file. `sourceDir` written as array `["src"]`. **Coverage is opt-in (perf)**: the generated vitest config's `coverage` block sets `enabled: false` (coverage instrumentation + report generation is the single biggest suite cost), and the recommended scripts are split — `"test": "vitest run"` (fast inner loop) + `"test:cov": "vitest run --coverage"` (reports/CI). Same script split recommended for jest. `lacuna analyze` is unaffected — it passes `--coverage` on its own `coverageCommand`, which overrides `enabled: false`. The lean generated setup (jest-dom import + `vi.*` cleanup only, no `cleanup()`/`document` at load) is intentionally **node-safe** so `env-route.ts` can route DOM-free tests off jsdom. | +| `generate.ts` | `lacuna generate` | Calls `runAgentLoop`. Flags: `--file`, `--dry-run`, `--verbose`, `--model`, `--threshold`, `--format`, `--output`, `--workers`, `--fresh`, **`--improve`**. Header surfaces Model/Runner/Threshold/Workers + **Mocks** (when `mocksFile` set) and **Debug** (when enabled, via `debugLogPattern`). **Optional positional `[path]`**: a **file** routes to single-file mode (subsumes `--file`); a **directory** sets `scopeDir` — a scoped **create + improve** run that brings every file under the dir to threshold (creates tests for untested files AND extends existing below-threshold tests). `--improve` enables the same create+improve repo-wide without scoping. Header shows **Scope** (dir) / **Target** (file) / **Mode: improve**. **`@diff[:]` positional (patch-coverage mode)**: detected BEFORE the file-vs-dir `stat` branch (not a filesystem path); `''` after `@diff` = default base. **`--file` + `@diff` = intersection**: the diff scope is narrowed to that one file's changed lines (header `Scope: diff vs ∩ (M changed line(s))`; patch coverage measured over just those lines; an unchanged file → "nothing to cover", exit 0). The single-file FAST path is skipped in diff mode (coverage report needed), but the test-file-path validation still applies. **`@diff` + a directory** (`generate @diff packages/api`, order-independent two positionals `[path] [scope]` split by which is the `@diff` token): `scopeDir` + `diffRef` are BOTH passed to the loop, which narrows the diff to that folder (`scopeDiffToDir`) — only the changed lines inside the dir are targets (header `Scope: diff vs under (N changed file(s), M line(s))`). **Coverage sourcing in patch mode (critical, learned from a real miss)**: the BEFORE report MUST be a FULL-suite report — patch coverage only matches Codecov if it reads the same measurement Codecov uploaded (the whole CI suite). A NARROWED run (`vitest related` / one dir) executes only a subset of the tests that cover the changed file, so any line covered solely by an integration/DI test outside that subset looks uncovered → lacuna over-targets lines Codecov shows green (exactly what happened on afriex `BusinessService.runSuccessfulSwapHooks`: its covering test mocks BusinessService, real coverage came from elsewhere, `related` missed it → all 11 body lines flagged instead of Codecov's 4). So: diff mode REUSES any on-disk report regardless of age (only `--fresh` re-runs), and when it must run it uses the FULL command, never `related`. `relatedCoverageCommand` survives but is used ONLY for the cheap AFTER measurement (see loop.ts), where it's a safe union addend on top of the accurate before-report. Resolves the scope up front for the header and to fail fast (exit 2) on a bad base; passes `diffRef` into `runAgentLoop`. **`--e2e` flag**: routes to `runE2E()` → `runE2ELoop` (Playwright E2E generation) **before** the unit-path setup and its "unknown runner" guard — a Playwright-only repo legitimately has no unit runner. E2E-only flags: `--route ` (one route), `--max-routes ` (cap per run). `-w`/`--workers` is honored in E2E mode (parallel worker pool). The E2E header prints Model/Mode/Route/Workers/Debug; `--dry-run` previews the routes that would get specs (no app boot, no model call, no API key needed). | +| `fix.ts` | `lacuna fix` | Calls `runFixLoop`. Flags: `--file`, `--dry-run`, `--verbose`, `--model`, `--workers`, `--fresh`, `--regenerate-on-failure` (default true), `--fix-polluters`, `--types` (select files by TypeScript type errors instead of test failures). Summary wording adapts in `--types` mode ("Still erroring" / "All type errors fixed"). Header surfaces **Mode: type errors** (`--types`), **Mocks**, and **Debug**. **Optional positional `[path]`**: a **test file** routes to single-file mode (subsumes `--file`); a **directory** sets `scopeDir` — only failing/erroring tests under that subtree are selected and the discovery run is scoped to it (`scopedTestCommand`). Header shows **Scope** (dir) / **Target** (file). **E2E (`--e2e`)**: repairs failing Playwright specs instead of unit tests; the "unknown runner" guard is skipped (runFixLoop swaps in the Playwright env and checks for `@playwright/test` itself), the header shows **Runner: playwright** / **Mode: end-to-end**, and the E2E repair path is used (see fix-loop.ts). | +| `init.ts` | `lacuna init` | Interactive wizard using `@inquirer/prompts`. Writes `.lacuna.json`. **Runs from any subdirectory** — `findProjectRoot()` walks up to nearest `package.json`. Detects framework from `package.json`: React, React Native (`react-native`), Expo (`expo`), Next.js (`next`), Vue (`vue`), Svelte (`svelte`), Angular (`@angular/core`), NestJS (`@nestjs/core`). **Dependency install**: `checkPackageInstallState(pkg, cwd)` returns three states — `declared` (in package.json, skip), `undeclared` (in node_modules but NOT in package.json — prompts user to add it for CI), `missing` (full install). `isPackageInstalled` is a convenience wrapper that returns `true` for both `declared` and `undeclared` (used only for checking extra packages). **Next.js**: setup file at `test/setup.ts` (project root); vitest.config.ts gets `@` alias from tsconfig AND `server-only` alias pointing to `test/empty-module.ts` (auto-created); setup file includes global mocks for `next/navigation`, `next/headers`, `next/cache`, `next/image`, `next/font/google`, `next/font/local` — these are Next.js-specific and would break a plain React project. No `environment: 'jsdom'` in vitest config (Next.js manages its own env). **Plain React** (not Next.js): setup file at `${sourceDir}/test/setup.ts`; vitest.config.ts gets `environment: 'jsdom'`; setup file is just `@testing-library/jest-dom` + cleanup hooks — NO Next.js mocks. **React Native/Expo**: `jest-expo` or `react-native` preset, no jsdom, warns if Vitest chosen. **`resolveJestVersionForPreset(preset, cwd)`** — dynamically resolves compatible jest version: (1) reads `node_modules/${preset}/package.json` peerDependencies.jest locally; (2) falls back to `npm info ${preset} peerDependencies.jest`; (3) warns if neither works. Used to install the correct jest major for RN/Expo (jest-expo@56 requires jest@29 — using jest@30 causes `clearMocksOnScope is not a function`). ts-jest NOT installed for RN/Expo projects (jest-expo handles TS via babel-preset-expo). Async-storage mock in the generated setup template uses `jest.requireActual()` not `require()`. **All Vitest projects**: vitest.config.ts includes `restoreMocks: true` and `clearMocks: true`. Setup file has `beforeEach(() => vi.restoreAllMocks())` + `afterEach` cleanup. **Vue/Svelte**: `@testing-library/vue`/`@testing-library/svelte` + jsdom. **Angular**: `jest-preset-angular`, no jsdom. **NestJS**: no setup file. `sourceDir` written as array `["src"]`. **Coverage is opt-in (perf)**: the generated vitest config's `coverage` block sets `enabled: false` (coverage instrumentation + report generation is the single biggest suite cost), and the recommended scripts are split — `"test": "vitest run"` (fast inner loop) + `"test:cov": "vitest run --coverage"` (reports/CI). Same script split recommended for jest. `lacuna analyze` is unaffected — it passes `--coverage` on its own `coverageCommand`, which overrides `enabled: false`. The lean generated setup (jest-dom import + `vi.*` cleanup only, no `cleanup()`/`document` at load) is intentionally **node-safe** so `env-route.ts` can route DOM-free tests off jsdom. **E2E (Playwright) opt-in**: for web frameworks the route discovery supports (`meta.isNextJs || meta.isReact`), after the unit setup the wizard offers "Set up end-to-end (Playwright) testing?" (default **false** — it's a heavy, extra install) and, on yes, runs `installPlaywright` (from `playwright.ts`) to add `@playwright/test` + browser binaries. Skipped if already installed (just prints a ready hint). Playwright lives here in the wizard rather than as a side effect of `generate --e2e` because the browser-binary download is heavy and deserves an explicit opt-in. | | `run.ts` | `lacuna run` | Just runs the test suite and reports pass/fail. No AI. **Optional positional `[path]`**: a **file** runs only that file (`fileTestCommand`); a **directory** runs only the tests under it (`scopedTestCommand`, full command fallback for runners that can't be narrowed). Header shows **Scope**. | --- @@ -44,10 +46,12 @@ The AI loop. Reads source files, calls the model, writes test files, retries on | File | Purpose | |------|---------| | `loop.ts` | `runAgentLoop()` — main generate loop. **Single-file fast path**: when `--file` (or a positional *file* path) is given, skips the coverage suite entirely and builds a synthetic `CoverageGap` (lineCoverage: 0, empty uncoveredLines/Functions). Passes `parallel=true` to `processGap` so it uses `fileTestCommand` (not the full suite) to verify the generated tests. Returns immediately after that one file. **Scoped / improve mode** (`scopeDir` from a positional *dir*, or `--improve` repo-wide): `improveExisting = improve || !!scopeDir`. Discovery (`findTestFiles`/`findUncoveredFiles`) is restricted to `scopeDir`, and the lcov-gap list is post-filtered with `isWithinDir`. The coverage run uses `scopedCoverageCommand` (vitest/jest only — narrows BOTH executed tests and instrumented files; else falls back to the full command). Scoped runs always run fresh (the time-cache holds a whole-repo report whose freshness says nothing about the scope, and a scoped run rewrites lcov with scope-only data). When the scope has no tests, the suite is skipped (FS-only scan). `filterTestableGaps` is called with `{ includeExisting: improveExisting }` so already-tested below-threshold files are KEPT and *extended* (the single-file improve behavior generalized across a dir) instead of skipped. `perFileVerify = improveExisting || parallel` makes even the sequential loop verify each file with `fileTestCommand` and use the keep-best-vs-baseline branch. **Final coverage measurement**: a single instrumented pass (scoped `coverageCommand` when scoped) runs after generation when `!dryRun && testsWritten > 0 && (parallel || perFileVerify)` — i.e. whenever the per-file verification ran WITHOUT coverage (all parallel runs + scoped/improve sequential runs), so `coverageAfter` is real not stale. Classic unscoped sequential generate (which already verifies via the full suite) is left untouched — no extra pass. Label is **"Measuring coverage under …"** (accurate whether or not an initial run happened — previously said "Re-running…" even on the FS-only first-run path). **Full suite path**: runs coverage suite (or reuses cached report if < 10 min old via `coverageAgeSeconds`). Finds gaps. In parallel mode: `runWorkerPool()` with `WorkerDisplay`. In sequential mode: iterates gaps with tip + memory update after each success. Builds `ProjectMemory` once before loop starts. **Zero-pass detection**: if `parsePassCount(coverageOutput) === 0`, sets `hasTests = false` so generate treats it as a fresh project. **No-gaps-but-below-threshold**: when `gaps.length === 0 && coverageBefore < threshold`, shows "Every source file already has a test file — run \`lacuna fix\`" instead of the misleading "All files meet threshold". This is the correct state when all test files exist but are failing. **File restoration**: `processGap()` reads the pre-existing test file content before the retry loop (`originalTestContent`). On `OscillationError` or max-iterations failure, calls `restoreTestFile(testPath, original)`. **Regression detection**: stores `firstError` and `firstPassCount` from the first failed test run; subsequent failures use `buildStructureBrokenMessage` or `buildRegressionMessage`. **Verbose streaming**: in `--verbose` sequential mode, creates a `StreamingFileViewer` per attempt, calls `generator.setTokenCallback(t => viewer.append(t))`, then `viewer.start()` before and `viewer.stop()` after the generate/retry call (in a try/finally via the catch block path). Generator is created with no `onToken` — the viewer owns it entirely. **Patch failure handling**: `consecutivePatchFailures` counter tracks back-to-back patch anchor misses. After 1 failure, sends detailed error with the exact failing anchor and a checklist. After 2 failures, switches to full-rewrite mode (instructs model to use `` instead of `` with the complete file). Uses `tryApplyPatchWithDiag` for precise failure attribution. **Patch-base accumulation**: `patchBase` starts as `context.existingTestCode` and is reassigned to the written `testCode` after every attempt — so a retry that patches a test an *earlier* attempt added anchors against the current file, not the frozen original (previously caused false "anchor not found"). **Keep-best on failure**: tracks the best *collecting* attempt (`bestCode`/`bestPassCount`, only attempts where `isZeroTestsOutput` is false qualify). On exhaustion: new file → keep the best attempt + suggest `lacuna fix --file`; existing file (parallel/file-scoped only) → re-run the original to measure baseline and keep the best attempt only if it adds passing tests, else restore the original; non-parallel full-suite → restore (suite-wide pass count isn't per-file comparable). Type-error last attempt is reported as passed (keeps the green file). **Weak-wait one-shot nudge**: in the success path (tests green), `detectWeakAsyncWait(testCode)` scans for the assert-state-after-a-call-only-`waitFor` race (passes locally, fails in slow CI — invisible to the run-loop since the test is green). On a hit it retries once with the fix guidance, guarded by a `weakWaitNudged` flag so a false positive costs at most one extra iteration, never an infinite loop. **Cause-aware retry wording**: after a type-check failure the tests already PASS, so the retry header reads `Retry N/M — fixing type errors (tests pass)…` (and the trigger line `Tests pass — fixing type errors (retrying)…`) instead of the misleading "fixing failures"; keyed off whether `lastError` starts with `Tests passed but TypeScript type errors`. **Patch-coverage mode (`LoopOptions.diffRef`)**: `undefined` = off, `''` = default base, else explicit base ref. Resolved up front via `resolveDiffScope` (fail fast on bad base; docs-only diff returns immediately with patch 100 / exit 0 — no suite run). Implies `improveExisting` (→ `perFileVerify`). Gap build uses threshold **101** (a 94%-covered file can still have uncovered CHANGED lines), merges `missingChangedFileGaps`, then `narrowGapsToDiff` intersects each gap's `uncoveredLines` with the changed set — the narrowed lines flow to the model through the existing "UNCOVERED LINES" prompt section, no prompt changes. `patchCoverageBefore` from `computePatchCoverage` on the reused/ full BEFORE report; `patchCoverageAfter` computed CHEAPLY WITHOUT a full re-run — run a narrow related/scoped pass for the new test, then `mergeReportHits` UNIONs its hits onto the before-report (a line is covered-after if covered before OR by the new test) and recompute. `diffBase` + both numbers land in `LoopResult`; reporter/exit-code gate on patch coverage. **Diff mode reuses any on-disk coverage report regardless of age** (patch coverage interprets an existing measurement — ideally CI's); only `--fresh` forces a re-run, and that re-run is the FULL suite (matches Codecov), NOT a narrowed command. `relatedCoverageCommand(env, relFile)` (detector.ts) is used only for the after-union. **`scopeDir` + diff (`@diff `)**: `scopeDiffToDir` intersects the changed map with the dir right after `resolveDiffScope` (mirrors the `--file` intersection); `scopedGaps` then prefers `diffGaps` (already dir-narrowed) over the plain scope filter; the cached FULL report is still reused (`ageSeconds` only nulls out for scoped runs when NOT in diff mode) so `@diff ` reads the CI/test:cov report instantly instead of re-running. | -| `fix-loop.ts` | `runFixLoop()` — fix loop. Discovery: if `--file` given, runs only that file (no full suite). Otherwise checks `.lacuna-fix-cache.json` (30-min TTL, bypassed by `--fresh`); if stale, runs full suite and saves result to cache. **Scope (`scopeDir` from `lacuna fix `)**: the discovery run uses `scopedTestCommand` (vitest/jest — only the scope's tests; else full command), the failing/erroring set is restricted to the subtree (`isWithinDir`), `discoverTestFiles(cwd, env, scopeDir)` is scope-walked in `--types` mode, and the failing-files **cache is bypassed entirely** (read AND write) so a scoped run's partial set never corrupts the whole-suite cache an unscoped run resumes from. **Failing-file detection**: `parseFailingTestFiles` uses `TEST_FILE_RE = /[\w./\\@\[\]-]+\.(?:test|spec)\./` — includes `[` `]` for Next.js dynamic-route paths like `[classId]`; parses expected count from summary line; prunes over-detected and supplements under-detected with stack-trace extraction. `fixFile()` runs the file alone, calls `generator.fix()`, retries with `generator.retry()`. **Regeneration fallback (default on)**: when `fixFile` exhausts retries, calls `regenerateFile()` which (1) calls `findSourceFile()` to locate the source — handles absolute paths correctly (if sourceDir is absolute, joins directly without prepending cwd to avoid doubled paths), (2) deletes the broken test file so `processGap` sees a clean slate (otherwise `buildFileContext` reads the broken file as "existing tests to preserve"), (3) wraps `onStatus` to show the test file path rather than the source path during regen. Parallel workers send `phase: 'regenerating'` before calling `regenerateFile` — this undoes the prior `phase: 'failed'` done-count so each file is counted exactly once in the progress bar. Handles `// ---MOCKS_FILE---` separator. **Type definitions**: after reading `sourceCode`, calls `collectTypeDefinitions`. **Regression detection**: `buildStructureBrokenMessage` / `buildRegressionMessage`. Parallel via `runFixWorkers()`. **Polluters & Victims**: `--fix-polluters` bisects to find polluter files and adds cleanup; if bisection fails (concurrency-based contamination, not reproducible sequentially), falls back to regenerating the victim from source. **Type-aware repair**: in targeted (`--file`) or `--types` mode, a file whose tests PASS is still type-checked; if it has type errors `fixFile` enters the loop with them as the error input instead of skipping. Two type modes inside the loop: *type-cleanup* (tests passed at start) retries to clear type errors; *test-repair* (tests were failing) treats tests-now-passing as success and keeps the fix even if type errors remain (never reverts a behavioral fix over type noise) — its follow-up hint points at the targeted `lacuna fix --file ` (a re-run finds passing tests and enters type-cleanup), not the whole-project `--types`. **Keep-best across retries**: `fixFile` tracks `bestCode`/`bestPassCount` (init to the original/baseline). After each failing run, if `!structureBroken && currentPassCount > bestPassCount` it captures `testFileContent`. On exhaustion (and the oscillation early-exit) it writes `bestCode` — NOT the last attempt, NOT blindly the original — so a partial win (e.g. attempt 1 fixes 2 of 3 broken tests but later retries regress) is preserved; a pure failure still restores the original (bestCode stays = original when nothing beats baseline). Returns `baselinePassCount: bestPassCount` so the regen fallback compares against the kept improvement and never replaces it with a worse from-scratch rewrite. (The generate loop already had this; the fix loop did not — it restored the original and discarded its own best work.) **`--types` selection**: `runFixLoop` discovers all test files and calls `findTestFilesWithTypeErrors` (one scoped tsc per governing tsconfig) instead of running the suite; skips the failing-files cache. **Regeneration safety**: regen fires only when `result.baselinePassCount < REGEN_MAX_BASELINE_PASS` (10) — a file with a substantial passing suite is never nuked; suppressed for `--types`/type-only repairs. `regenerateFile` backs up the original before `unlink` and (a) restores it if regen fails, (b) re-runs the regenerated file and restores the original if regen has *fewer* passing tests than the baseline (never-regress). **Patch anchors against the original `testCode`**: in patch mode, each retry's patch is applied to `testCode` (the file content the model was shown in `history[0]`), with a disk-content fallback — NOT the drifted on-disk file. A regressed attempt isn't reverted until the loop ends, so anchoring to the on-disk file made every post-regression retry fail with "Patch anchors not found" (the model re-anchors to the original it saw, which no longer matched disk). Mirrors the generate loop's patch-base invariant. | +| `fix-loop.ts` | `runFixLoop()` — fix loop. Discovery: if `--file` given, runs only that file (no full suite). Otherwise checks `.lacuna-fix-cache.json` (30-min TTL, bypassed by `--fresh`); if stale, runs full suite and saves result to cache. **Scope (`scopeDir` from `lacuna fix `)**: the discovery run uses `scopedTestCommand` (vitest/jest — only the scope's tests; else full command), the failing/erroring set is restricted to the subtree (`isWithinDir`), `discoverTestFiles(cwd, env, scopeDir)` is scope-walked in `--types` mode, and the failing-files **cache is bypassed entirely** (read AND write) so a scoped run's partial set never corrupts the whole-suite cache an unscoped run resumes from. **Failing-file detection**: `parseFailingTestFiles` uses `TEST_FILE_RE = /[\w./\\@\[\]-]+\.(?:test|spec)\./` — includes `[` `]` for Next.js dynamic-route paths like `[classId]`; parses expected count from summary line; prunes over-detected and supplements under-detected with stack-trace extraction. `fixFile()` runs the file alone, calls `generator.fix()`, retries with `generator.retry()`. **Regeneration fallback (default on)**: when `fixFile` exhausts retries, calls `regenerateFile()` which (1) calls `findSourceFile()` to locate the source — handles absolute paths correctly (if sourceDir is absolute, joins directly without prepending cwd to avoid doubled paths), (2) deletes the broken test file so `processGap` sees a clean slate (otherwise `buildFileContext` reads the broken file as "existing tests to preserve"), (3) wraps `onStatus` to show the test file path rather than the source path during regen. Parallel workers send `phase: 'regenerating'` before calling `regenerateFile` — this undoes the prior `phase: 'failed'` done-count so each file is counted exactly once in the progress bar. Handles `// ---MOCKS_FILE---` separator. **Type definitions**: after reading `sourceCode`, calls `collectTypeDefinitions`. **Regression detection**: `buildStructureBrokenMessage` / `buildRegressionMessage`. Parallel via `runFixWorkers()`. **Polluters & Victims**: `--fix-polluters` bisects to find polluter files and adds cleanup; if bisection fails (concurrency-based contamination, not reproducible sequentially), falls back to regenerating the victim from source. **Type-aware repair**: in targeted (`--file`) or `--types` mode, a file whose tests PASS is still type-checked; if it has type errors `fixFile` enters the loop with them as the error input instead of skipping. Two type modes inside the loop: *type-cleanup* (tests passed at start) retries to clear type errors; *test-repair* (tests were failing) treats tests-now-passing as success and keeps the fix even if type errors remain (never reverts a behavioral fix over type noise) — its follow-up hint points at the targeted `lacuna fix --file ` (a re-run finds passing tests and enters type-cleanup), not the whole-project `--types`. **Keep-best across retries**: `fixFile` tracks `bestCode`/`bestPassCount` (init to the original/baseline). After each failing run, if `!structureBroken && currentPassCount > bestPassCount` it captures `testFileContent`. On exhaustion (and the oscillation early-exit) it writes `bestCode` — NOT the last attempt, NOT blindly the original — so a partial win (e.g. attempt 1 fixes 2 of 3 broken tests but later retries regress) is preserved; a pure failure still restores the original (bestCode stays = original when nothing beats baseline). Returns `baselinePassCount: bestPassCount` so the regen fallback compares against the kept improvement and never replaces it with a worse from-scratch rewrite. (The generate loop already had this; the fix loop did not — it restored the original and discarded its own best work.) **`--types` selection**: `runFixLoop` discovers all test files and calls `findTestFilesWithTypeErrors` (one scoped tsc per governing tsconfig) instead of running the suite; skips the failing-files cache. **Regeneration safety**: regen fires only when `result.baselinePassCount < REGEN_MAX_BASELINE_PASS` (10) — a file with a substantial passing suite is never nuked; suppressed for `--types`/type-only repairs. `regenerateFile` backs up the original before `unlink` and (a) restores it if regen fails, (b) re-runs the regenerated file and restores the original if regen has *fewer* passing tests than the baseline (never-regress). **Patch anchors against the original `testCode`**: in patch mode, each retry's patch is applied to `testCode` (the file content the model was shown in `history[0]`), with a disk-content fallback — NOT the drifted on-disk file. A regressed attempt isn't reverted until the loop ends, so anchoring to the on-disk file made every post-regression retry fail with "Patch anchors not found" (the model re-anchors to the original it saw, which no longer matched disk). Mirrors the generate loop's patch-base invariant. **E2E mode (`--e2e`)**: `runFixLoop` normalises `options.env` to the Playwright env up front (so `fixFile` and every per-file run use it) after an `ensurePlaywrightForRun` guard (offers an inline install when interactive, else prints the manual hint; `offerInstall: !dryRun`). Selection runs the whole Playwright suite once (`playwrightTestCommand`) and takes the failing spec files parsed from the JSON reporter (resolved to absolute paths). `fixFile` threads `e2e`: it uses `extractE2EFailure` (Playwright JSON → readable failure) instead of the unit extractor, skips the source-file lookup, skips the type-check success/failure gate, and bypasses the `parsePassCount`/regression/structure-broken heuristics (unit-runner-only). Regeneration-from-scratch is disabled for E2E (it rebuilds from coverage gaps, meaningless for a spec). **E2E repair uses the dedicated repair prompt**: before the retry loop, `fixFile` parses the route from the spec's first `goto()` (`extractRouteFromSpec`) and captures a **fresh snapshot** of it (best-effort), then attempt 1 calls `generator.fixE2E(buildE2ESystemPrompt(), buildE2EFixPrompt({ specCode, failureOutput, route, snapshot, … }), …)` instead of the unit `generator.fix()`. Retries reuse the E2E system prompt via the override. This replaced the earlier behavior where `fix --e2e` repaired specs with the unit-test fix prompt (mocks/imports/coverage) — wrong for browser specs. | | `context.ts` | `buildFileContext()` — builds `FileContext` for a single source file: reads source, finds existing test file, computes suggested test path, computes `sourceImportPath` (relative import from test to source, no extension), reads mocks file (always computes `mocksImportPath` even if mocks file doesn't exist yet), reads setup file, reads `packageDeps`, `tsconfigPaths`, `typeDefinitions`, and `localImportContents`. Also exports: `buildFixFileContext(absTestPath, cwd, config?)` — lightweight context for fix-loop (no inferred paths, no spurious dir creation); `computeRelativeImport(fromFile, toFile)` — relative import path helper; `collectTypeDefinitions(sourceCode, absoluteSourcePath, cwd)` — BFS traversal of locally-imported files extracting `interface`/`type`/`enum` declarations, caps 10 files / 4000 chars; `collectUsedSymbolsContext(sourceCode, absoluteSourcePath, cwd)` — targeted BFS symbol extractor: parses exactly which names the source imports from each local file, extracts only those declarations (functions collapsed to signature + return statement, classes collapsed to method signatures, types shown in full), then follows PascalCase type references transitively through both same-file declarations and cross-file imports — no line cap needed, naturally bounded by what's actually used. Output stored in `FileContext.localImportContents` and sent to the agent as `USED SYMBOL DEFINITIONS`. `readTsconfigAliases(cwd)` reads raw path aliases for import resolution. `resolveLocalImport` handles both relative paths and tsconfig aliases. `extractTypeDeclarations(code)` uses brace-depth tracking to extract type-shaped declarations. **Bug fixed**: `extractSymbolFromCode` previously called `summariseFunctionBlock` for `export default class` — this collapsed all method signatures to `// ...`, making method names invisible. Now branches to `summariseClassBlock` when the default export line matches `/export default (?:abstract )?class/`. **Test placement (`inferTestFilePath`)**: tries co-located → sibling convention (`__tests__/` or co-located) → mirror roots (`test/unit`, `test`, `tests`, …) → `__tests__/` default. **A mirror root must actually contain a test file** (`dirContainsTestFile`) to be used — a bare `test/` holding only mock/setup helpers (a common `mocksFile`/`setupFile` location) no longer hijacks placement away from the project's real `__tests__/` convention. | -| `generator.ts` | `TestGenerator` class. Wraps the model provider. Methods: `generate(context, gap, projectMemory?)`, `fix(args)`, `fixPollution(args)` (adds cleanup to a polluter file — different prompt, uses `buildPollutionFixPrompt`), `retry(failureOutput)`, `setTokenCallback(cb)`. Maintains `history[]` for multi-turn retries. `parseStructuredResponse(raw)` extracts `` or `` (hypothesis — handles both Claude and DeepSeek R1 tag styles) and `` (code) using **line-anchored search** (`(?:^|\n)`) — ignores prose mentions of `` inside planning text. Uses `isCodeIncomplete()` (brace balance + last-char check) to detect truncation; throws `TruncatedOutputError` if truncated. `isRepetitionLoop(text)` — detects prose loops where the same sentence (>45 chars) appears 3+ times in the raw fallback output; when detected, marks response as truncated so `TRUNCATION_RETRY_MESSAGE` fires ("IMMEDIATELY write the code"). `stripThinkingBleed` handles both `` (Claude) and `` (DeepSeek R1) including unclosed variants. `failedAttempts[]` accumulates `{ attemptNumber, hypothesis, failureReason }` across retries. `retry()` extracts "4. WHY IT FAILED / 5. PLAN" sections from hypothesis (regex) for more targeted negative constraints — falls back to last 800 chars. `maxTokens` from `config.maxTokens` (default 16000). **Oscillation detection** via `normalizeCode()` + `previousCodes[]`. **Temperature**: `GENERATE_TEMPERATURE = 0.4`; `RETRY_TEMPERATURE = 0.1`. **Debug logging**: `private readonly debugFile` — set in constructor from `config.debug ?? ENV_DEBUG_FILE` (env var `LACUNA_DEBUG` takes precedence). `debugWrite(file, label, content, clear?)` writes each prompt and response to the file separated by `═` headers with ISO timestamps. **Per-file logs**: debug is a boolean on/off switch — `resolveDebugBase(config.debug)` returns a fixed base (`lacuna-debug.txt`) when enabled (env `LACUNA_DEBUG` wins: any value enables except `0/false/no/off`), else null. `perFileDebugPath(base, filePath)` derives the per-target-file path from the file's **full relative path** (slashes → `_`), e.g. `src/app/api/send-email/route.ts` → `lacuna-debug.src_app_api_send-email_route.txt` — NOT the basename: identically-named files (`send-email/route.ts`, `login/route.ts`) would both slug to `lacuna-debug.route.txt` and clobber each other since each run clears its own file; the full path keeps every log in the one flat base dir while staying unique. `this.activeDebugFile` is set at the start of `generate()`/`fix()` from the file being processed and used by all `debugWrite` calls (including `retry()`). Each per-file log is cleared on that file's entry (`debugWrite` mkdirs the dirname) and appended through its retries — so `--workers` and multi-file suites never share/clobber/interleave one stream (a single shared file was truncated+garbled by concurrent workers). Enable via `LACUNA_DEBUG=1` env var or `{ "debug": true }` in `.lacuna.json`. `debugLogPattern(config.debug)` (exported) returns the user-facing log pattern (`lacuna-debug..txt`) or null — used by the command headers to surface debug state. **Parser hardening**: (1) patch ops emitted inside `` instead of `` are detected via `PATCH_OP_RE` and treated as a patch (skip the full-file truncation check, which would false-flag intentionally-unbalanced anchors); the truncation check is skipped for patches in all parse branches. (2) `` glued to the end of a prose sentence (`…replace it.\n// @@@ …`) is accepted when real `// @@@` ops follow it. (3) `stripCodeFences` removes a stray leading/trailing markdown ``` fence in the ``/`` branches (prevents "Unterminated string literal" at EOF). **Mode flags**: `patchMode` (set in `generate()`/`fix()` from `existingTestLineCount > PATCH_MODE_LINE_THRESHOLD`) and `reactish` (`reactMajorVersion != null`) are threaded into `buildRetryPrompt` so retries stay in `` for large files and omit React-only causes for backend projects. `TRUNCATION_RETRY_MESSAGE` is tag-neutral (no longer forces ``). | +| `generator.ts` | `TestGenerator` class. Wraps the model provider. Methods: `generate(context, gap, projectMemory?)`, `fix(args)`, `fixPollution(args)` (adds cleanup to a polluter file — different prompt, uses `buildPollutionFixPrompt`), `retry(failureOutput)`, `setTokenCallback(cb)`. Maintains `history[]` for multi-turn retries. `parseStructuredResponse(raw)` extracts `` or `` (hypothesis — handles both Claude and DeepSeek R1 tag styles) and `` (code) using **line-anchored search** (`(?:^|\n)`) — ignores prose mentions of `` inside planning text. Uses `isCodeIncomplete()` (brace balance + last-char check) to detect truncation; throws `TruncatedOutputError` if truncated. `isRepetitionLoop(text)` — detects prose loops where the same sentence (>45 chars) appears 3+ times in the raw fallback output; when detected, marks response as truncated so `TRUNCATION_RETRY_MESSAGE` fires ("IMMEDIATELY write the code"). `stripThinkingBleed` handles both `` (Claude) and `` (DeepSeek R1) including unclosed variants. `failedAttempts[]` accumulates `{ attemptNumber, hypothesis, failureReason }` across retries. `retry()` extracts "4. WHY IT FAILED / 5. PLAN" sections from hypothesis (regex) for more targeted negative constraints — falls back to last 800 chars. `maxTokens` from `config.maxTokens` (default 16000). **Oscillation detection** via `normalizeCode()` + `previousCodes[]`. **Temperature**: `GENERATE_TEMPERATURE = 0.4`; `RETRY_TEMPERATURE = 0.1`. **Debug logging**: `private readonly debugFile` — set in constructor from `config.debug ?? ENV_DEBUG_FILE` (env var `LACUNA_DEBUG` takes precedence). `debugWrite(file, label, content, clear?)` writes each prompt and response to the file separated by `═` headers with ISO timestamps. **Per-file logs**: debug is a boolean on/off switch — `resolveDebugBase(config.debug)` returns a fixed base (`lacuna-debug.txt`) when enabled (env `LACUNA_DEBUG` wins: any value enables except `0/false/no/off`), else null. `perFileDebugPath(base, filePath)` derives the per-target-file path from the file's **full relative path** (slashes → `_`), e.g. `src/app/api/send-email/route.ts` → `lacuna-debug.src_app_api_send-email_route.txt` — NOT the basename: identically-named files (`send-email/route.ts`, `login/route.ts`) would both slug to `lacuna-debug.route.txt` and clobber each other since each run clears its own file; the full path keeps every log in the one flat base dir while staying unique. `this.activeDebugFile` is set at the start of `generate()`/`fix()` from the file being processed and used by all `debugWrite` calls (including `retry()`). Each per-file log is cleared on that file's entry (`debugWrite` mkdirs the dirname) and appended through its retries — so `--workers` and multi-file suites never share/clobber/interleave one stream (a single shared file was truncated+garbled by concurrent workers). Enable via `LACUNA_DEBUG=1` env var or `{ "debug": true }` in `.lacuna.json`. `debugLogPattern(config.debug)` (exported) returns the user-facing log pattern (`lacuna-debug..txt`) or null — used by the command headers to surface debug state. **Parser hardening**: (1) patch ops emitted inside `` instead of `` are detected via `PATCH_OP_RE` and treated as a patch (skip the full-file truncation check, which would false-flag intentionally-unbalanced anchors); the truncation check is skipped for patches in all parse branches. (2) `` glued to the end of a prose sentence (`…replace it.\n// @@@ …`) is accepted when real `// @@@` ops follow it. (3) `stripCodeFences` removes a stray leading/trailing markdown ``` fence in the ``/`` branches (prevents "Unterminated string literal" at EOF). **Mode flags**: `patchMode` (set in `generate()`/`fix()` from `existingTestLineCount > PATCH_MODE_LINE_THRESHOLD`) and `reactish` (`reactMajorVersion != null`) are threaded into `buildRetryPrompt` so retries stay in `` for large files and omit React-only causes for backend projects. `TRUNCATION_RETRY_MESSAGE` is tag-neutral (no longer forces ``). **E2E generation/repair**: `generateE2E(...)` and `fixE2E(...)` both delegate to `private runE2ERound(systemPrompt, userPrompt, debugName, label)` — the E2E loop / fix loop own the prompts (via `prompts/e2e.ts`), so this is a thin pass-through: it sets `systemPromptOverride` (so the shared `retry()` reuses the E2E system prompt rather than `buildSystemPrompt(env)`), runs one provider round, parses via the same `parseStructuredResponse`, and reuses oscillation/truncation handling. `label` distinguishes the debug-log header (`e2e generate` vs `e2e fix`). `private systemPrompt()` returns the override when set, else the env-derived prompt; all four unit provider call sites use it. The unit `generate()`/`fix()` **clear** `systemPromptOverride` at entry so a reused generator instance never inherits an E2E override. `debugName` (the spec filename, e.g. `login.spec.ts`) keys the per-file debug log. | +| `e2e-loop.ts` | `runE2ELoop()` — the **E2E generation orchestrator** (analogue of `runAgentLoop`, but route-driven). Flow: `ensurePlaywrightForRun` guard (installs Playwright inline when interactive, else prints the manual hint — `offerInstall: !dryRun`) → `loadPlaywrightConfig` → `discoverFlows(cwd, sourceDir)` → filter routes that already have a spec on disk (skip) → optional `--max-routes` limiter (no default — all routes processed, like unit generate; re-runs skip existing specs so batching is free) → **`--dry-run` returns here** with a preview (no app boot / no provider / no key) → `ensureAppServer` brings the app up once → `snapshotRoutes` captures every route's DOM in one browser run → **authenticated dual-pass** (Stage 2): if a saved session exists (`playwright/.auth/user.json`, from `tests`/`e2e/auth.setup.ts`), routes whose unauth snapshot is auth-gated — either `redirectedToLogin(route,url)` (landed path looks like login/sign-in/auth AND ≠ route) OR `looksLikeAuthWall(snapshot)` (an INLINE login/signup form on the route itself: a password textbox, an OAuth/SSO "continue/sign in with " button, or a sign-in/up CTA paired with an email/username field — catches client-side guards that render the form without a URL change) — are re-snapshotted SIGNED IN via `snapshotRoutes(..., storageState)`; those that now load (no redirect) are added to `authedRoutes` and their snapshot is replaced with the logged-in DOM → **flow exploration** (Stage 3): `pickOpenerProbes(route, snapshot, 4)` selects opener controls in three buckets — **tabs** (`role=tab`, cap 8), **sections** (P7 feature boundaries: short plain nav labels like Categories/Orders/Team that aren't action verbs — `isSection` gated by a ≥3 cluster, cap 6, catches role=button navs the verb filter missed), and **openers** (Add/New/Create/Edit/Open/Settings…, cap 4); skips logout/delete/print/back/close — one batched `snapshotInteractions` clicks each and captures the result, `revealedAfter(capture, base)` diffs to the NEW UI surfaced, and `RouteFlow[]` per route is passed to the prompt's FLOWS DISCOVERED block (perform the action, assert the revealed element) → **worker pool** (`--workers`, max 8): each worker owns its own `TestGenerator` (history is per-instance, must not be shared) and pulls routes from a shared index, calling `generateAndVerifySpec` per route. `generateAndVerifySpec`: `generator.generateE2E` → `hasTestFunctions` check → write spec to testDir → `runPlaywrightSpec` → on pass, **flake-confirm** with `FLAKE_CONFIRM_RUNS` extra green runs (a pass-then-fail is fed back as a "FLAKY" retry) → on fail, feed the parsed Playwright failure to `generator.retry()`, up to `maxIterations`. **Keep-best**: on exhaustion `generateAndVerifySpec` KEEPS the last runnable spec on disk (`everWritten`) and returns `{ kept: true }` so `lacuna fix --e2e` can iterate on it (mirrors the unit/generate keep-best); only a route that never produced a runnable spec (all attempts truncated or had no `test()`) is `rm`'d. The caller logs "kept at for repair (run `lacuna fix --e2e`)" vs "could not produce a runnable spec". **Silent retries**: the `Spec failed (attempt N/M)` line prints only on a written-and-run failure; oscillation `return`s early (model looped) and truncation / no-`test()` `continue` silently (only verbose logs them) — so the visible attempt numbers can look non-sequential. Parallel workers require one shared server, so if `ensureAppServer` can't confirm one up, the run clamps to sequential. Spec filename derived from route via `specFileName(route, authed)` (`/` → `home.spec.ts`; `/products/[id]` → `products-id.spec.ts`; **authed routes → `.auth.spec.ts`** so they run under the `authenticated` project + the prompt gets an authenticated block). `specExists` checks BOTH the `.spec.ts` and `.auth.spec.ts` variants so a re-run never regenerates a route in the other flavour / duplicates coverage. **Worker display**: in parallel mode (`--workers > 1`) drives the same live `WorkerDisplay` panel as the unit commands (per-worker rows + progress bar + rotating `E2E_TIPS`, `successLabel: 'generated'`); `generateAndVerifySpec` takes an `onStatus` callback and emits `generating`/`retrying` → `writing` → `running` phases, with the worker emitting the terminal `passed`/`failed`. When a display is active its per-route `log()` lines are suppressed (the panel owns the output; non-TTY/CI falls back to `[wN]` lines). Sequential mode keeps the plain streamed logs. | | `prompts/` | **Folder** — all prompt builders, split by concern. | +| `prompts/e2e.ts` | `buildE2ESystemPrompt()` — the **selector-discipline + intent-fidelity** system prompt for Playwright spec generation. Selector order: `getByRole` > `getByLabel` > `getByPlaceholder` > **`getByTestId`** (only for testids listed in the snapshot — never invented) > `getByText`; CSS/class/Tailwind/nth-child/XPath/deep-descendant **forbidden**. **Locator uniqueness** (one element; avoid `.first()/.nth()` unless the snapshot shows equivalents). Web-first auto-waiting assertions; **no `waitForTimeout`** and **no `networkidle` crutch** (wait on a real UI signal). **Post-action validation** (assert the observable outcome of every state change). **Test isolation** (independent, no seeded-data assumptions). **Auth** (no hardcoded creds, prefer fixtures/storageState, assert login redirects rather than fabricating a login). **Snapshot=selectors / source=intent** confidence model. **Dynamic routes**: only substitute param values present in snapshot/source. Output `` then ``. `buildE2EGeneratePrompt({ route, specFilePath, baseURL, snapshot, pageSource?, dynamic?, existingSpecExample? })` — assembles the per-route prompt from the `RouteSnapshot` (title, **redirect flag** when final URL ≠ route, headings, the role+name interactive palette, and the **data-testid palette**), the page source (intent only), and an existing spec for convention mirroring (describe hierarchy, fixtures, helpers, `test.step`, hooks). `buildE2EFixPrompt({ specFilePath, specCode, failureOutput, route?, baseURL?, snapshot?, existingSpecExample? })` — the **repair/failure-analysis** prompt used by `lacuna fix --e2e`: shows the broken spec + Playwright failure + an optional **fresh snapshot** (authoritative for selectors — fixes selector drift, the dominant E2E breakage), with rules to diagnose root cause, apply the smallest fix (prefer selector then sync repair), and preserve test intent/names/structure (never weaken or delete assertions to pass unless the snapshot proves them invalid). `sameRoute()` distinguishes a genuine redirect from the normal origin-prefixed URL. **Anti-flake + repair-quality refinements**: the system prompt bans **forced interactions** (`click({force:true})`/`fill({force:true})`/`dispatchEvent`/`evaluate`) and gives `` a light structure (Intent/Selectors/Outcome/Assumptions — no self-reported confidence, which is unreliable for gating). Interactive elements are **priority-sorted** (`prioritizeInteractives` — buttons/inputs/comboboxes ahead of links/menus) before the `INTERACTIVE_CAP` (60) truncation, so the relevant control survives on a busy page rather than being dropped by capture order. `buildE2EFixPrompt` adds an explicit **REPAIR PRIORITY** order (selector → sync → redirect → a11y → data-setup → feature-removal, with assertion modification/removal only as a last resort and only when the snapshot proves it invalid) and a **strict-mode-violation** rule (make the locator unique via role+name/testid; don't silence with `.nth()`). The injection prompt is **conservative on custom components** (only add to plainly-forwarding components, never component-specific prop bags like slotProps/inputProps unless a LIBRARY-SPECIFIC FORWARDING section is present; prefer a native element when unsure). The real safety is the **interactive-element verify-and-revert** in `e2e-loop.ts`: after writing the source and re-snapshotting, a new testid is kept ONLY if it landed on an actual interactive element (native control tag or interactive ARIA role) — so a mis-placed testid on a wrapper `
`, or one that never reached the DOM, reverts the source. Library guidance can only ever *try*; physics decides what persists. Library detection is import-chain-aware (`resolve-libraries.ts`), so a barrel-exported custom component is not mistaken for a library one. Wrapping a component in a `
` to attach a testid is **rejected** (changes the DOM tree → can break layout, defeating the point of a no-op attribute); proper component-library forwarding (e.g. MUI `inputProps`) is a future library-aware-injector concern, not model guesswork. Further edge-case rules: **icon-only controls** (use the exact accessible name/aria-label from the snapshot), **native dialogs** (register `page.on('dialog', …)` before the action), **file uploads** (`setInputFiles`), **clipboard** (assert the UI feedback; clipboard read via `evaluate` is the only permitted use of `evaluate`, read-only, not for driving the UI), and a **truncation escape hatch** (if a needed element was sliced out on a busy page, fall back to a container/heading visibility check and note it in `` rather than inventing a selector). | | `prompts/index.ts` | Main assembly point. **`buildSystemPrompt(env)`** — fully conditional on `isJS`, `isTS`, `isVitest`, `isJSRunner`. **Thinking template** — 5 steps including MOCK AUDIT (a–i). Step (a) is SOURCE ANCHOR — forces the model to quote verbatim from the actual source before writing any test: request body fields, exact DB call pattern, exact error message strings, HTTP status guards for routes; exact rendered strings, exact method names, exact conditional render guards for components. Overrides training-data priors (e.g. "inviteTeamMember → Firestore + teamId") with what the code literally says. **`jsCauses`** block: file structure ordering, factory scope (self-contained pattern — see invariants), `clearAllMocks()` wipes factory impls, barrel mock miss, barrel envelope mismatch. **`vitestCauses`**: global-spy, duplicate vi.mock, resetAllMocks vs clearAllMocks. `buildGeneratePrompt` / `buildFixPrompt` — inject framework guidance from sub-modules, **mock module inventory** (see below). `buildPollutionFixPrompt`. `detectTypeScriptErrors`. `detectThinkingBleed`. `detectUnhandledRejection`. `detectRealRequestInError`. **`detectWeakAsyncWait(code)`** (exported) — static scan of a PASSING test for the weak-wait race: a `waitFor` whose body has only `toHaveBeenCalled*` matchers (no settle signal) followed by an UNWRAPPED `expect(result.current.*)` read. Skips strings/comments in paren-matching (`matchingParen`) and blanks later `waitFor`/`act` spans (`blankWrappedBlocks`) so wrapped reads and `mock.calls[...]` arg reads don't false-positive. Consumed by `loop.ts` as a one-shot retry nudge (the run-loop can't catch it — the racy test is green). `buildRetryPrompt`. **Shared utilities**: `parseMockInventory(code)` — scans every `vi.mock()` call, extracts line number + top-level factory entries as `key(Variable)` pairs (e.g. `ValidationError(MockValidationError)`). `parseMockExports(code)` — flat list of exported names. `extractGlobalNextMocks(setupCode)` — lists modules already mocked in the setup file so the AI doesn't double-mock. `analyzeNetworkDeps`, `buildNetworkMockingGuidance`. **Mock file display**: full mock file always shown (no 80-line gate, no 300-line cap) — model needs to read exact text for REPLACE anchors; `filterMockFileForSource` still reduces size where possible. **Mock patch instructions**: rule 6 teaches `// ---MOCKS_PATCH---` (preferred for existing files) vs `// ---MOCKS_FILE---` (new file only); REPLACE/APPEND_EXPORT/ADD_TO_BEFOREEACH ops explained with examples. **MOCK EDITING RULES block**: previously only shown when `parseMockInventory` found `vi.mock()` blocks — now always shown when the mock file exists (class-only mock files with no `vi.mock()` blocks were getting no editing guidance). New rules added: (1) "To add a method to an existing class mock (e.g. MockWalletService): use REPLACE with the class definition as anchor"; (2) "If the source imports a service/hook with no mock here yet: use APPEND_EXPORT — never leave it inline in the test". **Available exports CLASS MOCKS bullet**: tells the model that Mock-prefixed exports are the canonical mock for their service — use `MockWalletService` if source imports `WalletService`, and add missing methods via REPLACE rather than creating a separate inline mock. **USED SYMBOL DEFINITIONS CRITICAL note**: "Do NOT mock any service method or hook return key not shown here — inventing names causes 'X is not a function' / 'undefined.then()' crashes." **Test patch mode** (> 300 lines): `// @@@ REPLACE:` documented in both generate and fix patch instructions; preferred over `ADD_IMPORT` when updating an existing import. Source skeleton: `uncoveredFunctions=[]` → full source if ≤ 600 lines, skeleton for larger files. **`PATCH_MODE_LINE_THRESHOLD`** (300, exported) — single source of the patch-mode line threshold, used by generate, fix, and the generator's retry decision. **`buildRetryPrompt(failureOutput, failedAttempts, patchMode?, reactish?)`** is mode-aware: neutral header ("The previous attempt did not pass" — no longer claims "the tests failed", which contradicted type-only repairs), instructs `` (not ``) when `patchMode`, and gates React/RTL-specific causes behind `reactish`. **Thinking budget**: the system prompt caps `` at ~30 lines and forbids re-pasting source/test excerpts (prevents the unclosed-thinking token-blowout). **`detectTypeScriptErrors`** now also surfaces named-type property errors (`TS2339/2551` on `type 'Foo'`, not just inline object literals) that were being dropped by the exclusion lookahead. **`universalCauses`** (shared generate rules) also carry the test-integrity guards (respect the TYPE CONTRACT — no type-impossible `as any` inputs; don't lock in incidental QUIRKS as "expected") and **keep PURE-LOGIC tests DOM-free** (services/utils/validators import-and-assert, no `render`/`screen`/`document`/`window`/`localStorage`) — which both improves test quality and lets `env-route.ts` route those tests to the fast `node` env. | | `prompts/runners/` | **Subfolder** — runner-specific prompt content extracted from `index.ts`. | | `prompts/runners/js-common.ts` | `buildJsCauses(mockApi)` — rules for all JS runners (Jest + Vitest + Mocha): `require()` banned everywhere, factory scope (self-contained pattern), `${mockApi}.mocked()` for typed mock access, `${mockApi}.requireActual()` in factories, barrel mock miss, mock structure (object vs factory), etc. Uses `${mockApi}` template variable throughout — never hardcodes `jest` or `vi`. | @@ -68,7 +72,8 @@ Pure utilities. No AI, no CLI. | File | Purpose | |------|---------| | `config.ts` | `loadConfig()` — cosmiconfig loader. **`ConfigSchema` is exported** and every field carries a `.describe()` — the single source of truth for both code readers and the generated `lacuna.schema.json` (editor completion/hover in `.lacuna.json`). An unknown `$schema` key is silently stripped by Zod (no `.strict()`). `scripts/generate-schema.mjs` (run after `tsc` in `npm run build` via `zod-to-json-schema`) writes `lacuna.schema.json`; `lacuna init` writes a `"$schema"` URL into the generated config. Zod schema with defaults. Config fields: `testRunner`, `coverageFormat`, `coverageDir` (default `coverage`), `sourceDir` (default `["src"]` — accepts string or array, always normalised to `string[]` via Zod transform; use `["src","lib","utils"]` to scan multiple top-level dirs), `threshold` (80), `maxIterations` (3), `coverageTimeout` (300s), `maxTokens` (16000), `ignore[]`, `mocksFile`, `setupFile`, `provider` (default `openai-compatible`), `model` (default `deepseek-chat`), `baseURL` (default `https://api.deepseek.com/v1`), `apiKeyEnv` (default `DEEPSEEK_API_KEY`), `debug` (optional boolean — `true` enables per-file debug logs; lacuna writes each target file's raw prompts/responses to `lacuna-debug..txt`; equivalent to `LACUNA_DEBUG` env var, env var takes precedence), `format` (default true — local eslint --fix + prettier, see `format.ts`), `nodeEnvRouting` (default true — route DOM-free generated tests to the `node` env, see `env-route.ts`). `applyModelOverride(config, model)` — applies `-m` / `--model` flag. Checks PRESET by key first, then by preset model name; applies full preset (provider, model, baseURL, apiKeyEnv). Falls back to setting only `config.model`. | -| `detector.ts` | `detectEnvironment()` — reads `package.json` deps, `composer.json` (PHP), `Gemfile` (Ruby), `Cargo.toml` (Rust), `*.csproj`/`*.sln` (C#), `build.gradle`/`pom.xml` (Java), `Package.swift` (Swift) to auto-detect runner. `envForRunner()` — returns `DetectedEnvironment` for a named runner. `fileTestCommand()` — builds per-runner command for a single test file. **`scopedCoverageCommand(env, relDir)`** — coverage command scoped to one directory (vitest: positional filter + `--coverage.include='/**'`; jest: `--testPathPattern` + `--collectCoverageFrom`); returns `null` for runners where both the executed tests AND instrumented files can't be reliably narrowed, so the caller falls back to the full command + report post-filter. Used by scoped `analyze`/`generate` to avoid instrumenting the whole repo. **`scopedTestCommand(env, relDir)`** — same idea without coverage (vitest `run `, jest `--testPathPattern=`); used by scoped `lacuna fix` to only run the tests under a directory. `multiFileTestCommand(env, files[])` — runs multiple files in one invocation with flags that force sequential single-thread execution and shared module registry (`--poolOptions.threads.singleThread=true --no-isolate` for vitest, `--runInBand` for jest) — used by the polluter bisect to reproduce state pollution between files. Supports: vitest, jest, mocha, pytest, go-test, phpunit, pest, rspec, cargo-test, dotnet-test, gradle-test, maven-test, swift-test. Languages: typescript, javascript, python, go, php, ruby, rust, csharp, java, swift. | +| `detector.ts` | `detectEnvironment()` — reads `package.json` deps, `composer.json` (PHP), `Gemfile` (Ruby), `Cargo.toml` (Rust), `*.csproj`/`*.sln` (C#), `build.gradle`/`pom.xml` (Java), `Package.swift` (Swift) to auto-detect runner. `envForRunner()` — returns `DetectedEnvironment` for a named runner. `fileTestCommand()` — builds per-runner command for a single test file. **`scopedCoverageCommand(env, relDir)`** — coverage command scoped to one directory (vitest: positional filter + `--coverage.include='/**'`; jest: `--testPathPattern` + `--collectCoverageFrom`); returns `null` for runners where both the executed tests AND instrumented files can't be reliably narrowed, so the caller falls back to the full command + report post-filter. Used by scoped `analyze`/`generate` to avoid instrumenting the whole repo. **`scopedTestCommand(env, relDir)`** — same idea without coverage (vitest `run `, jest `--testPathPattern=`); used by scoped `lacuna fix` to only run the tests under a directory. **`relatedCoverageCommand(env, relFile)`** — runs only the tests RELATED to one source file (vitest `related`/jest `--findRelatedTests`) with instrumentation narrowed to that file; used ONLY for the cheap patch-coverage AFTER measurement (see loop.ts), NOT the before-report. `multiFileTestCommand(env, files[])` — runs multiple files in one invocation with flags that force sequential single-thread execution and shared module registry (`--poolOptions.threads.singleThread=true --no-isolate` for vitest, `--runInBand` for jest) — used by the polluter bisect to reproduce state pollution between files. Supports: vitest, jest, mocha, pytest, go-test, phpunit, pest, rspec, cargo-test, dotnet-test, gradle-test, maven-test, swift-test, **playwright** (E2E). Languages: typescript, javascript, python, go, php, ruby, rust, csharp, java, swift. **Playwright is opt-in only** (`--e2e`), never auto-detected by `detectEnvironment` — most repos using Playwright also use Vitest/Jest for units, and that unit runner must keep winning auto-detection. `detectEnvironment` has type **overloads**: the no-config form narrows the return to `Exclude` (auto-detect never yields playwright); the config form returns the wide type. `fileTestCommand` routes the `playwright` case to `playwrightRunCommand` (from `playwright.ts`). | +| `playwright.ts` | Playwright (E2E) support module. `detectPlaywright(cwd)` — capability check for `@playwright/test`/`playwright` in deps (separate from `detectEnvironment`'s default-runner resolution). **Install helpers**: `installPlaywright(cwd, log)` — `npm install -D @playwright/test` then `npx playwright install` (browser binaries); returns success. `ensurePlaywrightForRun(cwd, { log, offerInstall })` — the guard at the top of the e2e/fix loops: returns true if already present, else prints the `PLAYWRIGHT_INSTALL_HINT` (`npm install -D @playwright/test && npx playwright install`) plus an `lacuna init` pointer, and — only when `offerInstall` (i.e. not `--dry-run`) AND interactive (`isInteractive()`: TTY + not `CI`) — offers a `confirm`+install inline. The check runs BEFORE any worker pool spawns, so the prompt never races workers; in CI/non-TTY it just prints the hint and returns false (never hangs). After the browser download, `warnIfHostDepsMissing` scans the captured output (`npx playwright install 2>&1`) for Playwright's Linux "missing dependencies" / `install-deps` warning and surfaces `sudo npx playwright install-deps` (needs sudo, can't be automated). **`ensurePlaywrightConfig(cwd, log)`** — scaffolds a framework-aware `playwright.config.ts` when the project has none (never overwrites): `testDir: ./tests`, `webServer.command` from package.json (`dev`→`npm run dev`, else `start`) + `url`/`baseURL` (`:3000` Next/CRA, `:5173` Vite). Emits a **3-project** layout: `setup` (testMatch `*.setup.ts`), `chromium` (default; `testIgnore` `*.setup.ts`+`*.auth.spec.ts` — the project lacuna's own snapshot/verify runs use), and `authenticated` (testMatch `*.auth.spec.ts`, `storageState` + `dependencies:['setup']`). **`ensureE2EAuthScaffolding(cwd, log)`** — writes `tests/test-config.ts` (`testUser {email,password}` from env + placeholders, `authRoutes`) and `tests/auth.setup.ts` (login → `storageState` at `playwright/.auth/user.json`), gitignores `playwright/.auth/`; never overwrites, idempotent. **Non-breaking invariant**: `storageState` lives ONLY on the `authenticated`/`*.auth.spec.ts` project — putting it on the default project would ENOENT-fail every `playwright test` until the user runs setup. lacuna runs specific spec files, so `setup` never fires for its runs. This is **Stage 1** of E2E auth coverage (scaffolding); Stage 2 (authenticated snapshots) and Stage 3 (flow exploration) are TODO — see the E2E-auth memory. Called by `ensurePlaywrightForRun` (both the already-installed and just-installed paths — a missing config is what forces the sequential fallback and stops the app booting) and by `init.ts`. `init.ts` reuses `installPlaywright` + `ensurePlaywrightConfig` for its opt-in E2E step. `PLAYWRIGHT_DEFAULTS` — the `DetectedEnvironment` row wired into `detector.ts`'s `RUNNER_DEFAULTS` (no meaningful coverage command for E2E). `loadPlaywrightConfig(cwd)` — best-effort regex read of `playwright.config.{ts,js,mjs}` for `baseURL`, `testDir` (default `tests`), and the `webServer` `command`/`url`; never evaluates the config, never throws (same posture as `typecheck.ts` with tsconfig). `playwrightRunCommand(file)` / `playwrightTestCommand()` — `npx playwright test [file] --reporter=json`. `parsePlaywrightResults(stdout)` — parses the JSON reporter tree (suites → specs → tests → results) into `{ passed, failed, flaky, failures[] }`; strips ANSI from messages, collects trace/screenshot attachments. **Resolves spec `file` paths against `config.rootDir` from the report** (Playwright reports them relative to rootDir = the testDir, NOT cwd) so the fix loop can actually read the failing file — resolving against cwd dropped the testDir segment. Returns null when output isn't parseable JSON (caller falls back to raw stderr). | | `runner.ts` | `runCommand(command, cwd, timeoutMs, onLine?)` — spawns shell command, streams stdout/stderr via `onLine` callback, kills process group with SIGKILL on timeout. Returns `RunResult { stdout, stderr, exitCode, success, timedOut? }`. Default timeout: 300s. | | `git-diff.ts` | Patch-coverage (`@diff`) git plumbing. `resolveDiffScope(cwd, explicitRef?)` → `DiffScope { baseRef, mergeBase, changed: Map> }`. Base-ref precedence: explicit `@diff:` → `origin/HEAD` → local `main`/`master` → `HEAD~1`; always diffs from `git merge-base HEAD` (Codecov's patch semantics — only what the branch changed since forking). Throws `GitDiffError` with an actionable hint (fetch the base / `git fetch --unshallow`) on unresolvable repo/base — callers exit 2. The diff is taken merge-base → **working tree** (identical to HEAD in CI; locally the working tree is what lcov instrumented, so its line numbers are the ones that match). `git diff --unified=0 --no-color --diff-filter=d` — zero context means hunk headers `@@ -a,b +c,d @@` ARE the changed new-side ranges (`c..c+d-1`, omitted `d`=1, `d`=0 = deletion-only → skip); no body parsing needed. `parseUnifiedDiff(output, cwd)` (exported for tests) keeps only source extensions, skips `/dev/null`, handles renames (new path) and quoted paths; refs are validated against a safe charset before shell interpolation. `countChangedLines(changed)` for CLI headers. `scopeDiffToDir(scope, absDir)` filters the changed-line map to files under a directory (via `isWithinDir`) — the `@diff ` case (patch coverage of just the changed lines inside a folder); base/merge-base untouched. A failed/empty diff run returns an empty map — never throws mid-run. | | `format.ts` | `formatFile(absPath, cwd, { enabled, env })` — runs the project's OWN formatter(s) on a freshly written/fixed test file so it matches repo style and clears the lint gate (the model often emits 2-space/no-semicolon blocks). Resolves `node_modules/.bin/eslint` + `.bin/prettier` **directly** (never `npx` → no surprise installs); if the project has NEITHER, it's a silent no-op (no canonical style to match — bundling one would fight the repo). Best-effort: every step timed out, failures swallowed. **Behavior-preserving**: prettier only touches whitespace, but `eslint --fix` can change code, so after eslint it RE-RUNS the file (via `fileTestCommand`, which has coverage disabled) and RESTORES the pre-eslint content if the fix broke the green tests. Gated by config `format` (default true). Wired into `processGap` (generate) + both fix loops, guarded by `!dryRun`. | @@ -97,6 +102,18 @@ Pure utilities. No AI, no CLI. | `index.ts` | `loadCoverage(config, cwd)` — delegates to `parseLcov` or `parseJsonSummary`. `coverageAgeSeconds(config, cwd)` — returns seconds since coverage file was last written (used by generate to skip re-running if < 10 min old). | | `gaps.ts` | Core gap detection. Key functions: `extractGaps(report, threshold)` — files below threshold from LCOV. `filterTestableGaps(gaps, userIgnore, opts?)` — filters by: user ignore list, `shouldIgnore()` (dirs + filename patterns), `testFileExists()` (skips if test already written — **unless `opts.includeExisting`**, which keeps below-threshold tested files so the loop can *extend* them), `hasTestableCode()` (content scan for functions/classes). **`hasTestableCode` detects: `function`, `class`, arrow with block body (`=> {`), AND exported arrow consts with EXPRESSION bodies (`export const f = (x) => x.map(…)`)** — the last was added because expression-body helpers (mappers/selectors) were judged untestable and skipped; keyed on `export … const … =>` so pure type files (`export type F = () => void`) stay skipped. **Index files are NOT name-ignored** (the old `/\/index\./` pattern was removed): a pure barrel has no testable unit so `hasTestableCode` skips it anyway, but logic-bearing `index.tsx` (component + its mappers) must be covered — blanket-ignoring by name was a main reason scoped generate "didn't cover all files in the folder". `findUncoveredFiles(report, sourceDir: string | string[], cwd, userIgnore, scopeDir?)` — accepts a single dir or an array; walks ALL dirs (or just `scopeDir` when given), skips files in LCOV report (normalises relative LCOV paths to absolute), skips if `testFileExists`, skips if no testable code. `findTestFiles(cwd, env, config, scopeDir?)` — walks all `config.sourceDir` dirs (or just `scopeDir`); used by `ProjectMemory` and the scope's has-tests check. **`isWithinDir(absPath, absDir)`** — prefix test used to restrict lcov-derived gaps to a scope subtree. **Patch-coverage (`@diff`) helpers**: `narrowGapsToDiff(gaps, changed, report, cwd)` — keeps only gaps in changed files, each narrowed to uncovered ∩ changed lines (report-present file whose changed lines are all covered → dropped; file with no report entry → every changed line is a target). `computePatchCoverage(report, changed, cwd, assumeUncovered?)` — Codecov math: covered changed executable lines / total changed executable lines ("executable" = has a `DA` record; files absent from the report count only when listed in `assumeUncovered`; empty denominator = 100%). `missingChangedFileGaps(changed, report, existingGaps, cwd, ignore)` — **hole-plug**: a changed file with an existing-but-unexecuted test file is in NEITHER the report NOR the gap set (`findUncoveredFiles` skips files that have tests), so without this its changed lines silently counted as covered → vacuous 100%; returns whole-file gaps for the testable subset of such files. | +### `src/lib/flows/` (E2E route discovery + DOM capture) + +| File | Purpose | +|------|---------| +| `discover.ts` | `discoverFlows(cwd, sourceDirs)` — finds user-reachable routes for E2E generation. Returns `{ framework, routeRoot, flows[] }` where each `Flow` is `{ route, title, sourceFile, dynamic, priority }`. **Dependency-gated**: Next.js filesystem routers are consulted only when `next` is a dep (or no `package.json`); React Router only when `react-router(-dom)` is a dep — prevents a Vite+React app with a `src/pages/` folder being misread as Next.js pages router. **Next.js app router** (`walkAppRouter`): a dir with a `page.{tsx,jsx,ts,js}` is a route; drops route groups `(grp)`, skips `@slot` parallel routes, `(.)`/`(..)` intercepting routes, `api/`, and `_private` dirs; maps `[id]`/`[...slug]` to dynamic. **Next.js pages router** (`walkPagesRouter`): each non-special file under `pages/`; excludes `_app`/`_document`/`api/`. **React Router** (`discoverReactRouter`): regex-parses both JSX (`}/>`) and object-config (`createBrowserRouter`/`useRoutes` `{ path, element }`) styles from source files; resolves the route's component to its source file by following its import (probing extensions); drops `*` splats; treats relative paths as top-level (best-effort). **Known limitation**: nested *relative* React Router child paths are captured as declared, not joined with the parent prefix (needs an AST walk) — degrades gracefully (a wrong path just fails its snapshot and the spec becomes a smoke test). `makeFlow` flags dynamic for both `[id]` and `:id`/`*` dialects. | +| `snapshot.ts` | `snapshotRoutes(routes, cwd, pwConfig, timeout, storageState?)` — captures each route's DOM to feed the generation prompt. **`storageState` (Stage 2)**: when set, `buildSnapshotSpec` emits `test.use({ storageState })` so the temp spec runs AUTHENTICATED (protected routes render their logged-in DOM instead of redirecting to login). Path resolves relative to cwd, matching where `auth.setup.ts` saves it. **Mechanism (Option B)**: writes a throwaway spec (`__lacuna_snapshot__.spec.ts`) into the project's testDir with routes baked in, runs it via the project's own `npx playwright test` (so the project's `webServer`/browser/version are used — no fragile cross-package resolution), reads back per-route JSON, deletes the temp spec. Each route record: `{ route, ok, url (redirect-aware), title, headings, interactives, testIds, aria, error }`. **Uses `locator('body').ariaSnapshot()`** — `page.accessibility.snapshot()` was removed in modern Playwright (1.61). `parseAriaSnapshot(aria)` (exported) parses the YAML-ish aria tree line-by-line into the interactive `{role,name}` palette + heading outline (deduped); `INTERACTIVE_ROLES` set gates which roles count. Also captures **`data-testid` values** via `page.locator('[data-testid]').evaluateAll(...)` → `{ testId, tag, text, role }[]` (deduped) — the aria tree carries no testids, so this is what lets the prompt offer `getByTestId` with real ids instead of invented ones. The `tag`/`role` are used by the injection verify to confirm a new testid landed on an actual interactive element. Per-route try/catch so one bad route doesn't lose the rest. **`snapshotInteractions(probes, cwd, pwConfig, timeout, storageState?)` (Stage 3 flow exploration)**: one extra browser run that, per probe `{route,role,name}`, re-navigates the route, clicks `getByRole(role,{name})`, waits for loaders, and captures the post-click DOM — so the prompt can describe UI revealed by an action (modals/forms/panels). Authenticated when storageState given; `--no-deps`; best-effort per probe (a un-clickable control is `ok:false`, skipped). **Loader-clear wait**: after `domcontentloaded` + `networkidle`, waits (best-effort, ≤8s) for any VISIBLE loading spinner/skeleton (`[data-testid*="load/spinner/skeleton"]`, `[aria-busy]`, `.spinner/.loading/.skeleton`, all `:visible`) to hide — so data-heavy authed dashboards (Firebase etc.) snapshot LOADED content, not the loader. No-op on pages without a visible loader; `:visible` is cross-version. The e2e system prompt also bans building the test around a spinner / using if-else/try-catch to force a pass. | +| `flowmap.ts` | **AST FlowMap — per-control → outcome mapping** (the principled fix for the toast/redirect misattribution regression, research priority #1). `buildFlowMap(sourceCode, cwd): FlowAction[] \| null` walks a page component's JSX for handler attributes (`onClick/onSubmit/onPress/onChange`), resolves each to its handler body (inline arrow / local `function`/`const` decl; a prop or imported handler → `external:true` with NO invented outcome), and extracts the OUTCOME that handler produces: a toast (`toast.*`/`showToast`/`enqueueSnackbar`/`notify`, kind inferred), a redirect (`router.push/replace`, `navigate/redirect`), or `opensModal` (`setShowX(true)`), plus notable `calls[]`. Each `FlowAction` is `{ control, by:'testid'\|'text'\|'label', handler, external, resolvedFrom?, outcomes, calls }`, control located by **data-testid > visible text > aria-label**. Resolves the **project's own** `typescript` at runtime via `createRequire(...).resolve('typescript',{paths:[cwd]})` (no bundled ~60MB dep, parse matches the project), returns null if unresolvable. **v2 ONE-HOP cross-file**: `buildFlowMap(src, cwd, sourceFileAbs?)` — when the entry file path is passed, a handler that is imported (`import {h} from './x'`) or destructured from a custom hook (`const {handleSave}=useMenuActions()`) is followed into its defining file and its outcomes extracted there (reported via `resolvedFrom`, `external:false`). A component PROP whose body lives in the caller stays `external`. Bounded to one hop, skips node_modules, sync fs; tsconfig-path+relative import resolution inlined so the standalone copy stays dependency-free. **Best-effort by design** — a shallow-but-correct mapping beats a confident-wrong one. **Why AST not regex**: regex finds every `toast()`/`router.push()` string but can't attribute them to the owning control, so a multi-flow page applied an *upsell* redirect as the success of *saving a record* (the "biggest regression"). Wired in `e2e-loop.ts` (computes `controlOutcomes` = actions with a concrete outcome) → `prompts/e2e.ts` "CONTROL → OUTCOME MAP" block, strictly per-control ("apply each outcome ONLY to its own control; never invent"). **Open-sourced** as a standalone repo: https://github.com/Octagon-simon/flowmap (canonical code stays here; the repo is an export, re-synced on release). | +| `resolve-libraries.ts` | **Import-chain-aware UI-library detection** for injection. `resolveComponentLibraries(sourceCode, fileAbs, cwd, depNames)` follows the component's local/barrel imports (reusing `context.ts`'s `readTsconfigAliases`/`resolveLocalImport` for alias + relative resolution) to the actual files, so a library re-exported through a barrel (`export { Button } from '@radix-ui/...'`) IS detected, while a barrel of purely **custom** components yields nothing — no false MUI/Radix guidance for a hand-rolled `