visual-automation-demo

A demo of visual automation using Playwright + Python.

This repo demonstrates a practical approach to visual testing without external SaaS tooling:

• Native Playwright screenshots
• Local baseline snapshot comparison in Python
• Stabilization controls to reduce flaky visual diffs
• A hybrid strategy combining visual checks with functional assertions

Why this project exists

Visual checks are great for catching layout and styling regressions quickly, but pixel-based comparisons can be noisy if tests are not controlled.

This project focuses on how to make visual testing credible in real teams:

• Stable browser context and viewport
• Animation/transition suppression
• Font-loading synchronization
• Targeted element snapshots, not only full-page snapshots
• Functional assertions where visuals are not enough

Tech stack

• Python 3.9+
• Playwright for Python
• pytest
• pytest-playwright
• Pillow (image diff for snapshot comparison)

Project structure

• tests/conftest.py: shared fixtures + visual stabilization utilities
• tests/visual_snapshots.py: baseline snapshot assertion helpers
• tests/test_visual_login.py: full-page, region-level, and hybrid tests

Setup

1. Create and activate a virtual environment:

python3 -m venv .venv
source .venv/bin/activate

2. Install dependencies:

pip install -e .

3. Install Playwright browser binaries:

python -m playwright install chromium

Run tests

Run everything:

pytest

Run a single file:

pytest tests/test_visual_login.py

First run creates baseline images in tests/snapshots/<browser>-<mode>/ (for example chromium-headless). Later runs compare current screenshots against those baselines.

If you also run headed mode, generate a separate headed baseline set:

pytest tests/test_visual_login.py --headed --update-snapshots

CI

GitHub Actions workflow: .github/workflows/visual-tests.yml

• Runs visual tests on push/PR
• Expects committed headless baselines in tests/snapshots/chromium-headless/
• Uses a macOS runner to match the baseline rendering environment
• Uploads test-results/visual as an artifact when the job fails

Update baselines intentionally

When UI changes are expected and approved:

pytest --update-snapshots

Optional: successful-login scenario credentials

One scenario uses real credentials from the-internet login page:

export VISUAL_USERNAME=<username>
export VISUAL_PASSWORD=<password>

If not set, that scenario is skipped.

Notes on visual strategy

This demo intentionally includes both:

• Visual assertions via snapshot comparison helpers
• Functional assertions (to_contain_text, to_have_url) for behavior correctness

That combination is typically stronger than relying on visual comparisons alone.