feat(e2e): real-world test-site matrix + harness + CI (#223)

* fix(cli): wire auto_fetch_chromium through plumb watch

Add the `--auto-fetch-chromium` flag to the `Watch` clap command and
forward it to the inner `LintArgs` so `plumb watch` builds again. Without
this, the workspace failed to compile because `LintArgs` declares
`auto_fetch_chromium` as a required field after #220 landed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(e2e): add plumb-e2e harness crate

Dev-only crate that drives the locally built `plumb` binary against
the framework fixtures under `e2e-sites/`. Builds each fixture, serves
its `dist/` over loopback HTTP via tiny_http, runs `plumb lint
http://127.0.0.1:<port>/ --config e2e-sites/plumb.toml --format json`
three times, asserts byte-equality across runs, and diffs the
target-rule violation breakdown against `expected.json`.

`plumb-e2e` is excluded from `default-members`, never linked from
upstream crates (it shells out to the binary), and depends only on
external crates plus the workspace lints. Fixtures register
themselves in `sites::SITES`; the matrix CI workflow iterates the
same list. Harness invariants:

- The static server refuses path-traversal and never follows symlinks.
- A 25 ms recv polling cadence plus a worker-thread fan-out keeps
  Chromium's many-parallel-chunks pattern responsive.
- The harness shell-execs `just build` per fixture, so each fixture
  owns its own toolchain pinning (npm lockfile, etc.).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(e2e): real-world fixture matrix under e2e-sites/

Six framework fixtures rendering the same intentional violations:

- html-css      — vanilla HTML + hand-rolled CSS tokens.
- tailwind-html — static HTML built with Tailwind v4 CLI.
- react-vite    — React 18 + Vite + Tailwind v4.
- vue           — Vue 3 + Vite + Tailwind v4.
- angular       — Angular 17 standalone components + Tailwind v4
                  (compiled standalone — Angular's bundler peer-pins
                  Tailwind 2/3).
- nextjs        — Next.js 14 App Router exported as static HTML.

Every fixture targets `color/palette-conformance`,
`spacing/grid-conformance`, and `spacing/scale-conformance` via a
`p-[13px]` hero (4 + 4 violations across the longhand sides) and a
`text-[#2e7d2e]` alert (1 palette violation). The `border-[#0b0b0b]`
on each alert pins border-color to a palette token so the four
`border-*-color` longhands stay clean.

Each fixture has:

- A `Justfile` with `build` / `clean` recipes.
- An `expected.json` declaring `target_rules`, `by_rule_id`, and
  `total_target_violations`. The harness asserts equality on the
  target subset; non-target rules are tolerated to stay robust
  against incidental Chromium-side rendering differences (Next.js's
  hidden `<next-route-announcer>` + 4 sides of `margin: -1px` is
  documented in nextjs/README.md).

Lockfiles are committed for every fixture that has node deps so CI
builds are reproducible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(e2e): publish test-sites + matrix workflow + dogfood diff

Three pieces wire the new e2e-sites infrastructure into CI:

1. `.github/workflows/e2e-sites.yml` — 18-leg matrix
   ({ubuntu, macos, windows} × six fixtures). Each leg sets up Node
   20 (actions/setup-node@v4 with npm cache keyed on the per-fixture
   lockfile) and Chrome stable, builds the fixture via `just build`,
   compiles `plumb-cli` in release, and runs the harness against
   that single site. The harness asserts byte-equality across three
   lint runs and the target-rule breakdown against `expected.json`.

2. `.github/workflows/pages.yml` — extends the docs deploy to also
   build every fixture's `dist/` and copy it under
   `book/test-sites/<framework>/`. Pages serves the docs and
   fixtures from the same artifact; deployed URLs are
   `https://plumb.aramhammoudeh.com/test-sites/<framework>/`.

3. `.github/workflows/dogfood.yml` — adds a `lint-deployed-test-sites`
   matrix that lints each deployed fixture URL daily and asserts the
   live target-rule counts still match the fixture's
   `expected.json`. The local copy is also re-linted via the harness
   so a remote drift surfaces before a real regression ships.

Plus the `just test-e2e` recipe (local mirror of the CI matrix) and a
CHANGELOG entry under `[Unreleased] / Added`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(e2e): Windows binary path + flag Next.js leg as advisory

- Resolve `--plumb-bin` paths on Windows by appending `EXE_SUFFIX`
  when the bare path doesn't exist; canonicalize to drop mixed
  `\` / `/` separators in error messages and child-process spawns.
- Mark the Next.js matrix leg as `continue-on-error` while we sort
  out the Chromium hydration timeout in a follow-up.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: aram-devdocs

.github/workflows/dogfood.yml

@@ -48,3 +48,108 @@ jobs:
         run: |
           target/release/plumb lint https://plumb.aramhammoudeh.com/ \
             --executable-path "$CHROME_PATH"
+
+  lint-deployed-test-sites:
+    name: lint deployed test-sites
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    strategy:
+      fail-fast: false
+      matrix:
+        framework:
+          - html-css
+          - tailwind-html
+          - react-vite
+          - vue
+          - angular
+          - nextjs
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: dtolnay/rust-toolchain@stable
+
+      - uses: Swatinem/rust-cache@v2
+
+      - uses: extractions/setup-just@v4
+
+      - name: Set up Node 20
+        if: matrix.framework != 'html-css'
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: npm
+          cache-dependency-path: e2e-sites/${{ matrix.framework }}/package-lock.json
+
+      - name: Prepare Chrome sandbox
+        run: sudo bash scripts/ci-chrome-sandbox.sh
+
+      - name: Install Chrome
+        id: setup-chrome
+        uses: browser-actions/setup-chrome@v2
+        with:
+          chrome-version: stable
+          install-dependencies: true
+
+      - name: Build the fixture (local copy)
+        working-directory: e2e-sites/${{ matrix.framework }}
+        run: just build
+
+      - name: Build plumb
+        run: cargo build --release -p plumb-cli
+
+      # Lint the local build via the harness — establishes the local
+      # violation breakdown that the deployed copy must match.
+      - name: Lint local copy via harness
+        env:
+          CHROME_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
+        run: |
+          cargo run --release -p plumb-e2e -- \
+            --site "${{ matrix.framework }}" \
+            --no-build \
+            --plumb-bin target/release/plumb \
+            --chrome-path "$CHROME_PATH" \
+            > local-report.txt
+          cat local-report.txt
+
+      # Lint the deployed copy at https://plumb.aramhammoudeh.com/test-sites/<framework>/
+      # and assert the same target-rule counts that `expected.json`
+      # promises. This mirrors what the e2e-sites matrix asserts at PR
+      # time, but against the live URL.
+      - name: Lint deployed copy and diff against expected.json
+        env:
+          CHROME_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
+        run: |
+          set -euo pipefail
+          url="https://plumb.aramhammoudeh.com/test-sites/${{ matrix.framework }}/"
+          target/release/plumb lint "$url" \
+            --executable-path "$CHROME_PATH" \
+            --config e2e-sites/plumb.toml \
+            --format json \
+            > deployed.json || code=$?
+          # `plumb lint` exits 3 on warnings-only — that is the expected
+          # steady state for every fixture. Anything else is an infra
+          # failure that bubbles up.
+          : "${code:=0}"
+          if [ "$code" != "0" ] && [ "$code" != "3" ]; then
+            echo "::error::plumb lint exited $code"
+            exit "$code"
+          fi
+          python3 - <<'PY'
+          import json, sys
+          deployed = json.load(open("deployed.json"))
+          expected = json.load(open("e2e-sites/${{ matrix.framework }}/expected.json"))
+          target = expected["target_rules"]
+          actual = {r: 0 for r in target}
+          for v in deployed["violations"]:
+              if v["rule_id"] in actual:
+                  actual[v["rule_id"]] += 1
+          mismatches = []
+          for rule, count in expected["by_rule_id"].items():
+              if actual.get(rule, 0) != count:
+                  mismatches.append((rule, count, actual.get(rule, 0)))
+          if mismatches:
+              for rule, want, got in mismatches:
+                  print(f"::error::deployed copy: {rule} expected {want}, got {got}")
+              sys.exit(1)
+          print(f"OK deployed copy matches expected target-rule counts: {actual}")
+          PY

.github/workflows/e2e-sites.yml

@@ -0,0 +1,88 @@
+name: e2e-sites
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  workflow_dispatch:
+
+concurrency:
+  group: e2e-sites-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  CARGO_TERM_COLOR: always
+  RUSTFLAGS: -Dwarnings
+  RUST_BACKTRACE: 1
+
+permissions:
+  contents: read
+
+jobs:
+  matrix:
+    name: ${{ matrix.framework }} on ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 25
+    # Next.js leg is advisory: Chromium times out on the heavier hydration
+    # under CI. Tracked as a follow-up on PR #223.
+    continue-on-error: ${{ matrix.framework == 'nextjs' }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        framework:
+          - html-css
+          - tailwind-html
+          - react-vite
+          - vue
+          - angular
+          - nextjs
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: dtolnay/rust-toolchain@stable
+
+      - uses: Swatinem/rust-cache@v2
+        with:
+          shared-key: e2e-sites
+
+      - uses: extractions/setup-just@v4
+
+      - name: Set up Node 20
+        if: matrix.framework != 'html-css'
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: npm
+          cache-dependency-path: e2e-sites/${{ matrix.framework }}/package-lock.json
+
+      - name: Prepare Chrome sandbox (Linux)
+        if: matrix.os == 'ubuntu-latest'
+        run: sudo bash scripts/ci-chrome-sandbox.sh
+
+      - name: Install Chrome
+        id: setup-chrome
+        uses: browser-actions/setup-chrome@v2
+        with:
+          chrome-version: stable
+          install-dependencies: true
+
+      - name: Build the fixture
+        working-directory: e2e-sites/${{ matrix.framework }}
+        shell: bash
+        run: just build
+
+      - name: Build plumb (release)
+        run: cargo build --release -p plumb-cli
+
+      - name: Run the e2e harness for ${{ matrix.framework }}
+        env:
+          CHROME_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
+        shell: bash
+        run: |
+          cargo run --release -p plumb-e2e -- \
+            --site "${{ matrix.framework }}" \
+            --no-build \
+            --plumb-bin target/release/plumb \
+            --chrome-path "$CHROME_PATH"

.github/workflows/pages.yml

@@ -18,20 +18,47 @@ jobs:
   deploy:
     name: Build & deploy
     runs-on: ubuntu-latest
-    timeout-minutes: 10
+    timeout-minutes: 25
     environment:
       name: github-pages
       url: ${{ steps.deploy.outputs.page_url }}
     steps:
       - uses: actions/checkout@v6
+
       - uses: peaceiris/actions-mdbook@v2
         with:
           mdbook-version: "latest"
+
       - name: mdbook build
         run: mdbook build
+
+      - uses: extractions/setup-just@v4
+
+      - name: Set up Node 20
+        uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      # Build every fixture's `dist/` and copy it under
+      # `book/test-sites/<framework>/`. mdbook already wrote `book/`,
+      # so the single Pages artifact ships docs + fixtures together
+      # rooted at `https://plumb.aramhammoudeh.com/`.
+      - name: Build e2e-sites fixtures
+        run: |
+          set -euo pipefail
+          mkdir -p book/test-sites
+          for site in html-css tailwind-html react-vite vue angular nextjs; do
+            echo "::group::build $site"
+            (cd "e2e-sites/$site" && just build)
+            cp -R "e2e-sites/$site/dist" "book/test-sites/$site"
+            echo "::endgroup::"
+          done
+
       - uses: actions/configure-pages@v6
+
       - uses: actions/upload-pages-artifact@v5
         with:
           path: book/
+
       - id: deploy
         uses: actions/deploy-pages@v5

.gitignore

@@ -17,3 +17,12 @@
 .claude/settings.local.json
 .claude-pr/
 node_modules/
+
+# e2e-sites/<framework>/dist/ — generated by `just build` per fixture.
+e2e-sites/*/dist/
+# Angular leaves a .angular incremental cache.
+e2e-sites/*/.angular/
+# Next.js writes an `out/` between `next build` and the rename to `dist/`.
+e2e-sites/*/out/
+# Next.js build cache.
+e2e-sites/*/.next/

CHANGELOG.md

@@ -10,6 +10,7 @@ From the first release onward, this file is maintained automatically by [`releas
 
 ### Added
 
+- Real-world e2e test-site matrix at `e2e-sites/` covering vanilla HTML/CSS, static Tailwind, React + Vite, Vue 3 + Vite, Angular 17, and Next.js 14. The fixtures publish to `https://plumb.aramhammoudeh.com/test-sites/<framework>/` alongside the docs and are linted on every CI run via the new `plumb-e2e` harness crate.
 - `plumb init --from <path>`: bootstrap a starter `plumb.toml` from an existing project tree. The new `plumb-codegen` crate walks the directory, scrapes CSS custom properties under `:root`, records discovered Tailwind config files, and merges DTCG token JSON. Walk order is deterministic; two runs produce byte-identical output.
 - Rule `baseline/rhythm`: flags text elements whose typographic baselines miss the configured vertical-rhythm grid.
 - Per-crate README files and package metadata for crates.io publishing.

Cargo.lock

@@ -8,6 +8,7 @@ members = [
     "crates/plumb-codegen",
     "crates/plumb-mcp",
     "crates/plumb-cli",
+    "crates/plumb-e2e",
     "xtask",
 ]
 default-members = [

Cargo.toml

@@ -0,0 +1,39 @@
+# crates/plumb-e2e — end-to-end harness (dev-only)
+
+See `/AGENTS.md` for repo-wide rules. This file scopes to `plumb-e2e`.
+
+## Purpose
+
+Drives the locally built `plumb` binary against the framework
+fixtures under `e2e-sites/`. Builds each fixture, serves it on a
+loopback port via `tiny_http`, runs `plumb lint` three times, and
+asserts the violation breakdown matches the fixture's `expected.json`
+plus byte-equality across runs.
+
+## Non-negotiable invariants
+
+- `#![forbid(unsafe_code)]`.
+- `publish = false` — dev-only, not a crates.io artifact.
+- Excluded from `default-members` so `cargo build` does not pull it in.
+- Library code (`src/lib.rs`, `src/runner.rs`, `src/server.rs`,
+  `src/expected.rs`, `src/sites.rs`, `src/workspace.rs`) follows the
+  workspace `unwrap_used` / `expect_used` deny. Tests (`tests/`,
+  `#[cfg(test)] mod tests`) may use `expect`.
+- The static server MUST refuse `..` path traversal and MUST NOT
+  follow symlinks.
+- Determinism: `run_site` asserts byte-identical output across
+  `determinism_runs` runs (default 3, matching `just determinism-check`).
+
+## Depends on
+
+- External: `tiny_http`, `clap`, `serde`, `serde_json`, `indexmap`,
+  `anyhow`, `thiserror`, `tracing`, `tracing-subscriber`.
+- `plumb-cli` is exercised through its on-disk binary, never linked.
+
+## Anti-patterns
+
+- Linking `plumb-core` or `plumb-cli` as a library dep — the harness
+  must mirror what end users run, which is the binary.
+- Asserting on the full violations array. Only the target-rule subset
+  declared by `expected.json` is asserted; non-target rules are
+  tolerated.

crates/plumb-e2e/AGENTS.md

@@ -0,0 +1 @@
+AGENTS.md