FRC Heatmap

Turn a wide-shot FRC match video into field-space robot heatmaps, alliance overlays, and exportable tracking data.

Upload an MP4 or paste a YouTube URL, calibrate the camera against the field image, seed all six robots, and render scouting-friendly movement heatmaps from a local web app.

Gallery

Broadcast frame	Alliance heatmap

Why This Exists

FRC scouting video is easy to watch and hard to quantify. This project projects robot motion from a broadcast-style field view onto a top-down field image so you can see where robots actually spend time, compare alliance flow, and export raw path data for deeper analysis.

What The App Does

• Accepts a local MP4 upload or downloads a YouTube source with yt-dlp
• Lets you choose a calibration frame and click matching anchors on the video and field image
• Seeds all six robots manually on the selected frame
• Tracks robot center points across the rest of the video with OpenCV optical flow
• Optionally proposes team numbers with OCR when tesseract is installed
• Generates:
  • one heatmap per robot/team
  • one combined red-vs-blue alliance heatmap
  • JSON and CSV tracking exports

Workflow

flowchart LR
    A[Upload MP4 or YouTube URL] --> B[Pick a frame]
    B --> C[Match video anchors to field anchors]
    C --> D[Seed 6 robot positions]
    D --> E[Run tracker]
    E --> F[Review OCR suggestions]
    F --> G[Generate heatmaps and exports]

Loading

Stack

• Frontend: React 19, TypeScript, Vite
• Backend: FastAPI, Pydantic, Uvicorn
• Vision pipeline: OpenCV, NumPy
• Optional OCR: pytesseract + local tesseract
• Optional ingest: yt-dlp

Quick Start

1. Start the backend

python3 -m venv .venv
.venv/bin/pip install -r backend/requirements.txt
.venv/bin/python -m uvicorn backend.app.main:app --reload

2. Start the frontend

cd frontend
npm install
npm run dev

Open http://localhost:5173.

How To Use It

1. Upload a match video or paste a YouTube link.
2. Scrub to a frame where all six robots are visible.
3. Click at least four matching landmarks on the video and on field-2026.png.
4. Switch to robot seeding mode and place all six robot markers.
5. Start analysis.
6. Review or edit the proposed team numbers.
7. Generate the final heatmaps and tracking files.

Output Artifacts

Each finished run can produce:

• heatmap-<team>.png: one heatmap per robot/team
• heatmap-alliances.png: combined red-vs-blue field occupancy
• tracking.json: full tracked coordinates in image and field space
• tracking.csv: spreadsheet-friendly version of the same data

Generated runtime assets are written under data/, while stress-test examples live under stress_runs/.

Project Layout

.
├── backend/
│   ├── app/
│   │   ├── main.py        # FastAPI routes
│   │   ├── pipeline.py    # ingestion, tracking, OCR, export rendering
│   │   ├── schemas.py     # API payload contracts
│   │   └── storage.py     # on-disk job/source management
│   └── scripts/
│       └── stress_test_youtube.py
├── frontend/
│   └── src/
│       ├── App.tsx        # calibration and export workflow UI
│       ├── api.ts         # API client
│       └── types.ts
├── field-2026.png         # top-down field asset used for projection
└── stress_runs/           # sample outputs from batch stress tests

Assumptions And Limits

• Best results come from a mostly fixed, full-field camera angle.
• Calibration is manual by design; the field image edges are not treated as automatic field bounds.
• Robot identity remains dependent on correct manual seeding and OCR review.
• OCR is optional and falls back cleanly when tesseract is not installed.
• The included stress_runs/ examples are pipeline stress tests, not scouting-grade ground truth.

Optional OCR Setup

brew install tesseract

Batch Stress Test

The repo includes a batch script for downloading and processing several YouTube match videos into sample output folders:

.venv/bin/python backend/scripts/stress_test_youtube.py \
  https://youtu.be/-hPRn0KyPQ0 \
  https://youtu.be/Bk3rAiqMjY8

Artifacts are written under stress_runs/<video_id>/.

Verification

cd frontend && npm run build
python3 -m compileall backend/app