Skip to content

tolu/lightning-heatmap

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Lightning Heatmap — Scandinavia

My dog is terrified of thunder, so this was a way of hypothetically figure out where we could move in Norway to give her the least amount of yearly scares 🐶⛈️😱

An aggregate map of historical lightning-strike density over Scandinavia, to find regions that are mostly or never hit. Data is scraped politely from lynkart.no and binned into Uber H3 hexagons, rendered as a magma/log choropleth.

This is a hobby project. Architecture is deliberately simple and free-tier Cloudflare only:

  • Local Node pipeline (packages/pipeline) scrapes lynkart and bakes H3 grids. H3 binning runs in Node (no Cloudflare Worker CPU limits).
  • Source of truth = raw weekly GeoJSON committed to git under data/ — so we never re-fetch.
  • Cloudflare R2 holds only the serving artifacts (grids, manifest, basemap), served to the frontend through the cached custom domain tiles.tolu.app.
  • Cloudflare Pages hosts the static frontend (packages/web, Vite + vanilla TS + MapLibre).

Layout

packages/
  shared/    pure domain logic (geojson, coverage, h3 binning, color domain) — used by both sides
  pipeline/  local Node CLI (scrape | bake | sync) + a local admin server with a Fetch button
  web/       Vite + MapLibre frontend → Cloudflare Pages
data/        committed raw weekly archives + coverage.json (source of truth)
ops/         one-off ops (basemap extract)

Map layer

The frontend (packages/web) is MapLibre GL rendering three stacked layers:

  • Basemap — a dark Protomaps vector basemap (Nordic extract, .pmtiles) served keyless from R2 through the pmtiles:// protocol. Place and road labels are brightened to near-white (the stock dark grays are hard to read over the heatmap); coastlines get a white line with a dark halo, plus dashed country borders, for orientation.
  • Heatmap — the H3 hex choropleth. Per-hexagon strike counts are colored on a magma + log scale with a global, pan-stable domain (top clamped near p99) that's baked into manifest.json, so a month-filtered view stays comparable to the all-time view. Only hit cells are drawn — never-hit areas are left unpainted, so the basemap shows through and the absence of a hex is itself the "rarely or never hit" signal (the whole point of the map).
  • Resolution / LOD — strikes are binned at H3 res 7 (~5 km²) and rolled up to res 6 and 5. The map serves them as one multi-resolution vector tileset (grid.pmtiles, built with tippecanoe) and swaps resolution by zoom; the finest level is viewport-limited so tiles stay small.

Interaction: a hover/tap tooltip (exact count + approx km²), a monthly date filter (sums the selected months client-side against the same global color domain), a magma legend, and a data-range / last-updated status bar read from status.json.

Prerequisites

  • Node ≥ 24 (nvm use)
  • A Cloudflare account with the R2 bucket already created.
  • tippecanoe for the bake's vector tileset (brew install tippecanoe), and pmtiles for regenerating the basemap (brew install pmtiles).

Cloudflare / R2 setup

The R2 bucket ligtning-heatmap is served publicly with CDN caching via the custom domain https://tiles.tolu.app (a subdomain of the tolu.app Cloudflare zone) — this replaces the rate-limited, uncached r2.dev URL. Three things are configured:

  1. R2 API token (for the pipeline's S3 writes) — R2 → Manage R2 API TokensCreate API tokenObject Read & Write (scope to ligtning-heatmap). Put the Access Key ID + Secret into a local .env (copy .env.example).
  2. CORS policy (so the browser can fetch JSON + range-request the basemap) — apply the committed ops/r2-cors.json with the Wrangler CLI. wrangler login is a one-time browser OAuth, separate from the S3 token above:
    npx wrangler login
    npx wrangler r2 bucket cors set ligtning-heatmap --file ops/r2-cors.json
    (Or paste the same JSON in the dash under R2 → bucket → SettingsCORS policy.)
  3. Custom domain (public reads, cached) — R2 → bucket → Settings → Custom Domains → Connect Domaintiles.tolu.app. Cloudflare auto-creates the proxied DNS record + edge cert; the CORS and Cache-Control settings carry over. (CLI: npx wrangler r2 bucket domain add ligtning-heatmap --domain tiles.tolu.app.)

Wrangler (v4) reference

Authoritative commands we use (from wrangler --help, v4.100):

npx wrangler r2 bucket domain add ligtning-heatmap --domain tiles.tolu.app  # public reads via CDN (cached)
npx wrangler r2 bucket cors set ligtning-heatmap --file ops/r2-cors.json

# Upload the basemap (Step 7) — note --remote writes to real R2, not local sim:
npx wrangler r2 object put ligtning-heatmap/basemap.pmtiles \
  --file <path.pmtiles> --content-type application/octet-stream --remote

# Deploy the frontend (Step 8):
npx wrangler pages project create lightning-heatmap --production-branch main
npx wrangler pages deploy packages/web/dist --project-name lightning-heatmap

The pipeline writes grids/manifest/coverage/status via the S3 API (@aws-sdk/client-s3, .env token) rather than wrangler r2 object put, since it uploads many objects per run. Wrangler is used for the one-off basemap upload, CORS, and Pages deploy.

Commands

npm install        # install all workspaces
npm run dev        # Vite dev server for the frontend
npm run build      # build the frontend (packages/web/dist)
npm run typecheck  # type-check all packages
npm test           # run unit spot-checks (vitest)

npm run scrape     # fetch the next missing week(s) into data/raw + coverage.json
npm run bake       # aggregate data/raw → H3 grids + manifest → R2
npm run sync       # scrape (to completion) then bake
npm run admin      # local control panel (localhost:5174): Fetch button + live progress

node packages/pipeline/src/cli.ts upload-basemap   # (re)upload ops/basemap/basemap.pmtiles to R2

npm run build && npx wrangler pages deploy packages/web/dist \
  --project-name lightning-heatmap --branch main   # deploy the frontend

See ops/basemap/extract.md for regenerating the basemap.

Live

https://lightning-heatmap.pages.dev — Cloudflare Pages frontend; grids + basemap served from R2 through the cached custom domain https://tiles.tolu.app.

Data so far: April–September 2025 plus the in-progress 2026 season (38 weeks, ~2.2M strikes). Only the lightning season is collected — SEASON_MONTHS (April–September) in packages/shared/src/config.ts filters which weeks the scraper fetches inside the BACKFILL_START/BACKFILL_END window, applied per year. To extend, bump those and npm run sync (or use the admin panel), then redeploy. For hands-off ongoing collection, schedule npm run sync locally (cron/launchd) — note it needs nvm's Node on PATH.

About

Lightning-strike density heatmap of Scandinavia — H3 hexbins + MapLibre on Cloudflare free-tier, to find the regions least hit by lightning 🐶⛈️

Topics

Resources

Stars

Watchers

Forks

Contributors