Skip to content

README.md

Latest commit

 

History

History
57 lines (40 loc) · 1.37 KB

README.md

File metadata and controls

57 lines (40 loc) · 1.37 KB
Raw

🪃 Dynamic Web Scraper with Selenium

Python Data Scraping Engineer roles. It demonstrates a real end-to-end workflow for a dynamic website: login, JavaScript-rendered content, infinite scroll, selector adaptation, validation, retries, and structured exports.

Selenium Playwright

What it shows

  • Selenium browser automation
  • Explicit waits and resilient extraction
  • Dynamic content / AJAX / infinite scroll
  • Selector self-healing Code Agent
  • JSON + CSV output
  • Pydantic validation
  • Local demo site for safe video recording
  • Unit tests and Playwright demo recording

Run locally

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
python -m app.cli serve-demo --host 127.0.0.1 --port 5000

In another terminal:

python -m app.cli scrape --base-url http://127.0.0.1:5000 --output-dir output

Record demo video

cd playwright
npm install
npx playwright install chromium
npm run demo:video

Built with Python + Selenium  ·  by Daniel Pizani  ·  No magic — just code