Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 14 additions & 167 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,10 @@
```bash
./start.command
```
On first run, `serve.py` downloads vendor dependencies (MapLibre, fonts, etc.) into `vendor/`.
On first run, `serve.py` downloads vendor dependencies into `vendor/`.
Your browser will open automatically at **http://localhost:8765**

3. **Add photos** — drag & drop JPEG/HEIC files (with GPS data) onto the upload zone,
or click it to browse your drive.
3. **Add photos** — drag & drop JPEG/HEIC files (with GPS data) onto the upload zone.

---

Expand All @@ -51,183 +50,31 @@
| Feature | Detail |
|---|---|
| 📍 Auto-pin | GPS EXIF read — no manual coordinates needed |
| 🔎 Destination search | Search any place and drop a pin |
| 🔎 Destination search | Search any place or paste GPS coordinates |
| 🖱 Right-click pin | Right-click the map to pin a location |
| 🏳 Countries visited | Flag emojis for every country you've visited |
| 📁 Albums | Named albums with optional date ranges |
| 🗓 Timeline | Chronological photo browser |
| 🖼 Lightbox | Full-size viewer with navigation and camera info |
| 📝 Notes | Add notes to any pin |
| 🛰 Map styles | Light, Dark, Terrain, 3D Terrain, Satellite, Globe |
| 🗺 Vector tiles | Smooth zoom, no flickering (OpenFreeMap) |
| 🛰 Map styles | Light, Bright, Dark, Terrain, 3D Terrain, Satellite, Globe |
| 🔄 Clustering | Pins cluster by zoom, expand on click |
| 💾 Auto-save | Background backup to disk via `serve.py` |
| 📦 Export / Import | Full dataset as compressed `.json.gz` |
| 🎬 Video export | Trip animation as WebM video (VP9) |
| 📡 Offline mode | Browse photos and cached tiles without internet |

## Auto-save & Persistence
## Documentation

- **IndexedDB** — all photos, albums, and metadata persist in the browser across sessions.
- **serve.py auto-save** — when running with the local server, data is also saved to `matrix-data.json` and photos to `matrix-photos/` on disk. This provides a durable backup that survives browser data clearing.
- **Export/Import** — use the settings menu to export your data as a gzip-compressed `.json.gz` file or import a backup. Importing supports both compressed (`.json.gz`) and plain (`.json`) files. Empty location pins are excluded from exports automatically. To migrate between machines, copy both the `.json.gz` export and the `matrix-photos/` directory to the new machine, then import.

## Tile Caching

Map tiles are cached in a multi-layer architecture designed for fast rendering and offline access:

### Storage Layers

| Layer | What it is | Speed | Persistence | Browser support |
|---|---|---|---|---|
| **L1 — Cache API** | Browser-side HTTP response cache, managed by the service worker | Instant (~0ms) | Cleared with browser data | Chrome, Firefox (not Safari) |
| **L2 — Disk cache** | Local filesystem at `matrix-tiles/`, served by `serve.py` | Fast (~5-10ms) | Persists until manually deleted or evicted | All browsers |
| **L3 — Origin** | Remote tile servers (OpenFreeMap, ArcGIS) | Network-dependent (~50-200ms) | N/A | All browsers |

### How it works

**Chrome / Firefox (with service worker):**

```
MapLibre requests tile
→ SW intercepts
→ L1 check (Cache API) — instant hit if previously fetched
→ L1 miss: race L2 (disk proxy) and L3 (origin) via Promise.any
→ Whichever responds first wins
→ Result stored in L1 for future requests
→ Origin fetches saved to L2 in background
```

**Safari (no service worker):**

Safari's service worker implementation has persistent issues (premature context termination, stale caches, failed tile loads). The SW is intentionally disabled in Safari. Tiles flow directly:

```
MapLibre requests tile
→ Browser fetches from origin (L3)
→ Proactive caching (data.js) prefetches tiles via serve.py
→ Disk cache (L2) available for offline use via serve.py proxy
```

### Data Storage (separate from tile caching)

| Storage | What it stores | Used by |
|---|---|---|
| **IndexedDB** | Photos, albums, metadata, thumbnails, geo caches | App data layer (`dbPut()` / `dbGetAll()`) |
| **Disk (matrix-data.json)** | Auto-save backup of all app data | `serve.py` auto-save endpoint |
| **Disk (matrix-photos/)** | Full-size images and thumbnails | `serve.py` photo storage |

IndexedDB and photo storage are unaffected by the service worker — app data persists identically in all browsers.

### Tile cache configuration

- **Disk cache limit:** 500 MB with LRU eviction (oldest tiles removed down to 80% when limit exceeded)
- **Eviction runs:** at startup and after each new tile is cached
- **Eviction logging:** written to `matrix-requests.log`
- **SW Cache API limit:** 10,000 entries with zoom-aware LRU (low-zoom tiles z≤8 protected from eviction)

### URL-based versioning

Tile URLs include a version segment (e.g., `20260415_001001_pt`) that changes when OpenFreeMap rebuilds their tile set. Cached tiles are never stale — when tiles update, the style JSON points to new URLs, the cache naturally misses, and fresh tiles are fetched. Old versioned tiles are eventually evicted by LRU.

### Proactive caching

After app load (10s delay), tiles for the world overview (z0–3) and pinned photo locations (z4–14) are prefetched in small batches. Already-cached tiles are skipped. This runs in the background without blocking interactive map use.

## Video Export

The **Play** button animates the map between your pinned locations in chronological order. **Export Video** records this animation using the browser's `MediaRecorder` API and saves it as a `.webm` file (VP9 codec, 40 Mbps). WebM plays natively in Chrome, Firefox, and VLC. For Apple ecosystem apps (QuickTime, iMovie, Photos), convert with `ffmpeg -i trip.webm trip.mp4`.

## Offline Support

The app works offline after your first visit:

- **Vendor libraries** are bundled locally in `vendor/` (auto-downloaded on first `serve.py` run)
- **Map tiles** are served from the L1/L2 cache (see above) — previously viewed areas render without internet
- **Photos, albums, and timeline** work fully offline (stored in IndexedDB)
- **Destination search and geocoding** require internet — they show a friendly message when offline
- An **orange banner** appears at the top when you're offline
- When you reconnect, everything resumes automatically — no action needed

## Keyboard Shortcuts

| Key | Action |
|---|---|
| `L` | Switch to Light Map |
| `B` | Switch to Bright Map |
| `D` | Switch to Dark Map |
| `T` | Switch to Terrain |
| `3` | Switch to 3D Terrain |
| `S` | Switch to Satellite |
| `G` | Switch to Globe |
| `F` | Fit all pins into view |

> Shortcuts are disabled when typing in any input field.

## Tips

- **JPEG photos from iPhones** almost always have GPS data embedded — they'll auto-pin perfectly.
- Photos **without GPS** still appear in the sidebar list and can be manually pinned via the metadata editor or destination search.
- **Right-click the map** to pin any location and add photos to it.
- **Countries visited** flags appear automatically as you click through your pins. Country codes are persisted so they load instantly on refresh.
- The app works in both **Chrome and Safari** on macOS.
- Nominatim (OpenStreetMap) is used for geocoding. Requests are rate-limited to 1 per second to comply with their usage policy.

## Testing

The app includes a Playwright integration test suite. Tests run against a temporary data directory so your real data is never touched.

**Prerequisites:** Node.js (for Playwright)

```bash
python3 serve.py --run-tests
```

Note: You may need to run `npx playwright install` first before executing the test suite

This will:
1. Create an isolated temp directory for test data
2. Generate test fixture images (with EXIF GPS data)
3. Install Playwright and Chromium (first run only)
4. Start the server and run all tests
5. Clean up and exit with the test result code

## Architecture: Map Stack

The app uses three separate services that work together to render interactive maps:

- **OpenStreetMap (OSM)** — the data source. A community-maintained database of geographic data (roads, buildings, boundaries, POIs). OSM provides the raw data but doesn't serve map tiles for app usage.
- **OpenFreeMap** — the tile server. Takes OSM's raw data, renders it into vector map tiles (`.pbf` files), and serves them alongside style definitions (JSON files that describe how to color roads, label cities, etc.). Free, no API key required. The app uses its `liberty` style (light) and `dark` style.
- **MapLibre GL JS** — the client-side rendering engine. A JavaScript library that takes tiles and style JSON from OpenFreeMap and renders an interactive, zoomable map on a `<canvas>` element in the browser. Handles panning, zooming, markers, clusters, and all map interaction.

The app offers six map styles:

| Style | Description |
|---|---|
| **Light Map** | Clean vector map with muted colors |
| **Dark Map** | Dark-themed vector map with normalized labels |
| **Terrain** | Light map with natural-earth shaded relief raster overlay |
| **3D Terrain** | True 3D elevation via AWS Terrain Tiles with hillshading. Pitch/bearing/exaggeration controls appear at bottom-left. Right-click shows elevation in meters. |
| **Satellite** | ArcGIS World Imagery raster tiles (Esri) |
| **Globe** | Spherical globe projection — pan to see the whole Earth |

The satellite and 3D Terrain views use separate raster tile sources unrelated to the OpenFreeMap/OSM ecosystem. 3D Terrain elevation data comes from **AWS Terrain Tiles** (free, no API key, terrarium encoding), capped at zoom 15 for maximum detail.

**Nominatim** (run by OpenStreetMap) is used for geocoding — converting place names to coordinates. Requests are rate-limited to 1 per second per their usage policy.

## Privacy

Everything stays **100% local**. No data is sent anywhere except OpenStreetMap/Nominatim for place lookups. No login required.

## Demo Mode

The app includes two automated demos you can trigger with keyboard shortcuts:

| Shortcut | Description |
|---|---|
| **Ctrl+Shift+D** | App walkthrough — automated demo with fake cursor that flies around the map, pins locations, switches tabs, and opens photos in lightbox. Demo pins are automatically cleaned up when the demo ends. |
| **Ctrl+Shift+G** | Globe rotation — switches to globe view and spins it 3 times along the equator. Uses GPU-rendered dots instead of DOM markers for smooth animation without jitter. Press again to stop early. |
- [Data persistence & migration](docs/persistence.md)
- [Data storage internals](docs/data-storage.md)
- [Tile caching architecture](docs/tile-caching.md)
- [Keyboard shortcuts](docs/keyboard-shortcuts.md)
- [Tips & features](docs/tips.md)
- [Architecture](docs/architecture.md)
- [Testing](docs/testing.md)
- [Demo modes](docs/demo-mode.md)

---

Built with [Claude Code](https://claude.ai/claude-code) using Claude Opus 4.6
Built with [Claude Code](https://claude.ai/claude-code) using Claude Opus 4.6
41 changes: 41 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Architecture

## Map stack

The app uses three services working together:

- **OpenStreetMap (OSM)** — the data source. A community-maintained database of roads, buildings, boundaries, and POIs.
- **OpenFreeMap** — the tile server. Renders OSM data into vector tiles (`.pbf`) and serves style JSON. Free, no API key required.
- **MapLibre GL JS** — the rendering engine. Takes tiles and style JSON and renders an interactive map on a `<canvas>` element in the browser.

## Map styles

| Style | Description |
|---|---|
| **Light Map** | Clean vector map with muted colors (Liberty style) |
| **Bright Map** | More vivid colors and greater landuse detail |
| **Dark Map** | Dark-themed vector map with normalized labels |
| **Terrain** | Light map with natural-earth shaded relief overlay |
| **3D Terrain** | True 3D elevation via AWS Terrain Tiles with hillshading |
| **Satellite** | ArcGIS World Imagery raster tiles (Esri) |
| **Globe** | Spherical globe projection |

3D Terrain elevation data comes from **AWS Terrain Tiles** (free, no API key, terrarium encoding).

## Geocoding

**Nominatim** (run by OpenStreetMap) handles geocoding — converting place names or GPS coordinates to map locations. Requests are rate-limited to 1 per second per their usage policy.

## Vendor dependencies

| Library | Version | Purpose |
|---|---|---|
| MapLibre GL JS | 5.24.0 | Map rendering |
| Supercluster | 8.0.1 | Point clustering |
| Josefin Sans | — | UI font |

Dependencies are managed in `dependencies.json` and downloaded on first run by `serve.py`.

## Privacy

Everything stays **100% local**. No data is sent anywhere except to OpenStreetMap/Nominatim for place lookups. No account, no login, no telemetry.
15 changes: 15 additions & 0 deletions docs/data-storage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Data Storage

All data storage is cross-browser (Chrome, Firefox, Safari) — none depends on the service worker.

| Storage | What it stores | Purpose | Cross-browser |
|---|---|---|---|
| **IndexedDB** | Photo objects (full-size base64 image + thumbnail + all metadata), album objects, geo caches | Primary data store — the app reads from here on every load | Yes |
| **Disk (matrix-data.json)** | JSON dump of all IndexedDB content | Backup that survives browser data clearing; auto-saved by `serve.py` | Yes |
| **Disk (matrix-photos/)** | Individual image files (`{id}.jpg`, `{id}_thumb.jpg`) extracted from base64 | Used by the export flow (avoids embedding huge base64 in JSON) and for lightbox display after import | Yes |

**IndexedDB** is the source of truth during normal use. `matrix-data.json` and `matrix-photos/` are redundant backups maintained by `serve.py`. If you clear browser data, the app offers to restore from `matrix-data.json` on next load.

## Scalability note

IndexedDB currently stores full-size images as base64 inside each photo record. At scale (thousands of large photos), this can cause high memory usage at startup since all records are loaded into memory. A future refactor will move full-size images to `matrix-photos/` only, keeping IndexedDB lean (metadata + thumbnails). See the [planned refactor](../plans/reactive-bubbling-creek.md) for details.
24 changes: 24 additions & 0 deletions docs/demo-mode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Demo Modes

The app includes two automated demos triggered by keyboard shortcuts.

## App walkthrough (Ctrl+Shift+D)

Launches an automated tour of the app with a fake animated cursor:

1. Flies to Paris and right-clicks to place a demo pin
2. Zooms out and flies to Sri Lanka and Turkey
3. Switches to Light Map and zooms into Saint Kitts
4. Hovers over country flags in the Countries Visited bar
5. Switches to the Albums tab and opens a photo in the lightbox

Demo pins are automatically cleaned up when the demo ends.

## Globe rotation (Ctrl+Shift+G)

Switches to Globe mode and spins the Earth along the equator:

- Starts over North America
- Rotates 3 full times (75 seconds total)
- Uses GPU-rendered circle dots instead of DOM markers for smooth jitter-free animation
- Press **Ctrl+Shift+G** again to stop early
30 changes: 30 additions & 0 deletions docs/keyboard-shortcuts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Keyboard Shortcuts

Shortcuts are disabled when typing in any input field.

## Map styles

| Key | Action |
|---|---|
| `L` | Switch to Light Map |
| `B` | Switch to Bright Map |
| `D` | Switch to Dark Map |
| `T` | Switch to Terrain |
| `3` | Switch to 3D Terrain |
| `S` | Switch to Satellite |
| `G` | Switch to Globe |

## Navigation

| Key | Action |
|---|---|
| `F` | Fit all pins into view |
| `←` / `→` | Navigate photos in lightbox |
| `Escape` | Close lightbox / modal / search |

## Demo modes

| Shortcut | Action |
|---|---|
| `Ctrl+Shift+D` | Start app walkthrough demo |
| `Ctrl+Shift+G` | Start / stop globe rotation demo |
27 changes: 27 additions & 0 deletions docs/persistence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Data Persistence

## How data is stored

- **IndexedDB** — all photos, albums, and metadata persist in the browser across sessions.
- **serve.py auto-save** — when running with the local server, data is also saved to `matrix-data.json` and photos to `matrix-photos/` on disk. This provides a durable backup that survives browser data clearing.

## Export / Import

Use the settings menu (⚙) to export your data as a gzip-compressed `.json.gz` file or import a backup. Importing supports both compressed (`.json.gz`) and plain (`.json`) files.

- Empty location pins are excluded from exports automatically.
- The export includes all photo metadata but not the full-size image files themselves.

## Migrating between machines

To move your data to another machine:

1. Export your data via Settings → **Export Data** → saves a `.json.gz` file
2. Copy both the `.json.gz` file and the `matrix-photos/` directory to the new machine
3. On the new machine, import via Settings → **Import Data**

The `matrix-photos/` directory contains full-size images and thumbnails. Without it, photos will appear in the list but won't display in the lightbox.

## Clearing the map tile cache

Settings → **Empty Map Cache** wipes the `matrix-tiles/` directory. Tiles are re-downloaded as you browse. This also flushes the browser's Cache API tile store.
Loading
Loading