Automated Visual Regression Testing Pipeline with PixelMatch and Playwright Screenshots for CI

[elizabetheonoja-art]

Problem Statement / Feature Objective

UI changes across releases can introduce subtle regressions in component rendering, layout shifts, and data visualization accuracy that are difficult to detect with functional tests alone. An automated visual regression testing pipeline must capture pixel-perfect screenshots of every route, reusable component state, and responsive breakpoint using Playwright, then compare them against baseline images using PixelMatch with a configurable threshold. The pipeline must run in CI (GitHub Actions) on every PR, generate a visual diff report as a comment on the PR, and fail the check if the mismatch ratio exceeds a per-component threshold.

Technical Invariants & Bounds

• Viewports: three breakpoints (mobile: 375x812, tablet: 768x1024, desktop: 1920x1080). Each page is captured at all three.
• Threshold: global default maxDiffPixelRatio = 0.001 (0.1% of total pixels). Individual components can override via a test config (e.g., animated charts: 0.005).
• PixelMatch settings: threshold = 0.1 (ignore color differences below 10/255 per channel). Diff image output: PNG highlighting changed pixels in magenta.
• Baseline storage: screenshots stored in tests/visual/baselines/ organized by route/viewport. Updated manually via npm run visual:update-baselines.
• Artifact retention: diff images and report are uploaded as GitHub Actions artifacts, retained for 30 days.
• CI integration: the workflow runs on pull_request to main, installs dependencies, builds the app, starts the dev server, runs Playwright visual tests, generates the report, and posts results as a PR comment using the GitHub API.

Codebase Navigation Guide

• tests/visual/visual.spec.ts - Playwright test spec with test.describe blocks per route and test cases per viewport.
• tests/visual/config.ts - Configuration: viewports, thresholds per component/route, baseline directory path.
• tests/visual/utils/screenshot.ts - Helper: takeScreenshot(page, name, viewport) with mask selectors for dynamic content (animations, live timestamps).
• tests/visual/utils/compare.ts - Wraps PixelMatch: compare(baselinePath, actualPath, diffPath, threshold) returns { match, diffRatio, diffPixels }.
• tests/visual/reportGenerator.ts - Generates an HTML report with side-by-side baseline vs actual vs diff images.
• .github/workflows/visual-regression.yml - GitHub Actions workflow definition.
• scripts/update-baselines.ts - Script to copy current screenshots to baseline directory (run locally on main branch).

Implementation Blueprint

1. Install dependencies: @playwright/test, pixelmatch, pngjs. Configure Playwright in playwright.config.ts with projects for chromium (primary) and webkit (secondary).
2. Create tests/visual/config.ts: export viewports, thresholds, and a map of route -> component -> threshold overrides.
3. Build screenshot helper in tests/visual/utils/screenshot.ts: before screenshot, freeze animations via page.addStyleTag({ content: '*, *::before, *::after { animation: none !important; transition: none !important; }' }). Mask dynamic elements (date pickers, live timers) with Playwright's locator masking.
4. Implement compare function with pixelmatch: read baseline PNG and actual PNG, create diff PNG, compute mismatched pixels ratio against total pixels.
5. Write visual.spec.ts: for each route in the app (/, /dashboard, /map, /tariffs, /settings, /profile), take screenshots at all 3 viewports. For shared components (ResourceChart, DataGrid, Sidebar), test their individual visual states.
6. Build reportGenerator.ts: iterate over all test results, generate an HTML file with embedded images (base64) showing baseline, screenshot, and diff side by side with pass/fail badges.
7. Create .github/workflows/visual-regression.yml: trigger on pull_request, run on ubuntu-latest, cache Playwright browsers, build the app, start dev server, run npx playwright test tests/visual, upload diff images as artifacts, and post a PR comment using the gh CLI with a summary table (route, viewport, diff ratio, status).
8. Write update-baselines script that clears the baseline directory and copies the current screenshots from tests/visual/results/ to tests/visual/baselines/.

[Kami-no-san]

hi, i would like to work on this

[grantfox-oss]

🦊 GrantFox — @Kami-no-san has been assigned to this issue as part of the Official Campaign campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #67)
2. Your PR will be reviewed by the Utility-Protocol maintainers

Good luck! Track your progress on GrantFox.

[grantfox-oss]

🎉 This issue has been marked as completed on GrantFox as part of the Official Campaign campaign!

@Kami-no-san's PR #73 was approved and merged by @elizabetheonoja-art.

🏆 @Kami-no-san: You earned 35 FoxPoints for this contribution! Your current tier: Explorer (223 total points). Track your full progress on GrantFox.

👏 Great work, @Kami-no-san! Keep contributing to Utility-Protocol.