Amazon Product Scrapper Engine (Multi-Marketplace)

A professional-grade, full-stack Amazon scraping and data enrichment dashboard. Built to transform basic product keyword searches into high-fidelity datasets including Best Sellers Rank (BSR), Customer Reviews, Detailed Specifications, and Global Ratings.

---

Project Showcase

User Interface Dashboard

Dashboard for real-time monitoring of scraping progress, marketplace selection, and deep extraction toggles.

Enriched Data Export

Sample of the deep metadata captured, including BSR paths and review snippets exported cleanly to CSV.

---

Key Features

• Global Marketplace Support: Toggle between .com, .co.uk, .de, .ae, etc., with automatic localization.
• Professional Localization: Automatically sets regional zip codes (e.g., 10001 for US) to bypass IP-based currency conversion (USD vs PKR).
• Deep Data Enrichment:
  • BSR (Best Sellers Rank): Precise ranking path extraction.
  • Ratings: Star averages and global count validation.
  • Specifications: Full technical spec tables parsed into readable text.
  • Reviews: Snippets of the top featured customer reviews.
• Real-time Analytics: Built-in charts and average price distribution tracking.
• Clean CSV Export: Streamlined data structure with "Actual Price" and "Discounted Price" columns.

---

Architecture

• Backend: Python, FastAPI, Selenium (Headless Support), SQLAlchemy (Async), SQLite.
• Frontend: Next.js 15, TailwindCSS, Lucide React, Recharts.
• Database: Local SQLite with auto-healing schema generation.

---

Setup & Installation

1. Prerequisites

• Python 3.9+
• Node.js 18+
• Google Chrome (Latest)

2. Backend Setup

# Navigate to backend
cd backend

# Create virtual environment
python -m venv venv
source venv/bin/activate  # venv\Scripts\activate on Windows

# Install dependencies
pip install -r requirements.txt

# Run the API
uvicorn backend_api:app --reload

3. Frontend Setup

# Navigate to frontend
cd frontend

# Install dependencies
npm install

# Run the Dashboard
npm run dev

Open http://localhost:3000 to view the dashboard.

---

Repository Structure

├── backend/            # FastAPI Scraper & Logic
│   ├── scraper.py      # Core Selenium Logic (Location Setter, Deep Scraper)
│   ├── backend_api.py  # REST Endpoints
│   └── models.py       # DB Schema
├── frontend/           # Next.js Dashboard
│   ├── components/     # UI Components (SearchMode, PriceHistory)
│   └── app/            # Main Page Logic
└── scraped_csv/        # Local storage for exported datasets

---

Developer

Sameer Shah — AI & Full-Stack Developer
Portfolio

Tip

Pro Tip: Enable the "Deep Scrape" toggle in the dashboard to unlock BSR and Review data. Note that this visits each product page individually and may take 1-2 minutes for larger lists.