Nowcasting: port, data validation, the two-source seam, and the centre-authority fix (tracking)

[kellylford]

Tracking issue for the nowcasting work on the nowcast-port branch (2026-07-03 through 2026-07-05): the feature port, the out-of-app data validation program, the discovery and measurement of the WeatherKit-vs-Open-Meteo seam, the harness-verified fix, and the design questions still open.

1. The port (branch nowcast-port)

Selectively re-ported the functional nowcasting features from docs/nowcasting-proposal on top of main's WeatherKit conditions work, deliberately excluding the radar map and all AI radar-description work. Four features, each behind a Developer Settings flag (all default on):

• Next Hour narration (nextHourNarrationEnabled) — one-sentence precipitation timing summary, first card on the precipitation screen
• Storm Approach (stormApproachEnabled) — ring-sampled directional precipitation card in Weather Around Me (direction, motion, arrival, nearby towns and saved cities); replaces the old surface-wind direction guess
• Storm Motion Accuracy (weatherAroundMeImprovementsEnabled) — mid-level steering winds with centroid cross-check and confidence-hedged wording, 16x3 sampling ring, rain/snow typing per town
• Next Hour layout (nowcastRefinementsEnabled) — renames Expected Precipitation to Next Hour, adds a tappable one-liner to City Detail

Port adaptations: the dry-convection reconciliation keys off the condition the app displays (post-WeatherKit-overlay) instead of the raw Open-Meteo code; the nowcast fetch is TTL-cached (3 min). Testing guide: iOS/NOWCAST_TESTING_GUIDE.md. Key commit: 043a00f.

2. The data validation program (tools/datatesting)

Everything is validated outside the app before Swift changes: nowcast_data_test.py runs line-for-line Python ports of the Swift algorithms against live APIs for 100 cities per run (interesting-weather finder: NWS active alerts + Open-Meteo precipitation scan), logging old-vs-new values per data point to CSV with cross-consistency checks. WeatherKit is sampled via its REST API (same backend as the app's framework). For US cities the nearest NWS station's observation acts as referee. Runs are one Finder double-click (Test Nowcast Data.command); results land in OneDrive RadarData.

Preserved findings (analysis only, raw data stays in RadarData): docs/nowcast-analyses/.

3. Findings

First validation run (2026-07-04): the ported features' data is sound. Narration and Storm Approach agreed 100/100 when reading the same source; the 5 direction mismatches against the old wind-inferred method were all the OLD method being wrong (e.g. 180 degrees off) — the measured justification for Storm Approach.

The seam (2026-07-04 live bug, then instrumented): the Next Hour narration reads WeatherKit (radar-informed) while Storm Approach read Open-Meteo (model). At east Madison the app claimed "rain for 48 minutes" while Storm Approach said not raining; the NWS station confirmed no rain fell — a WeatherKit low-intensity radar phantom (0.05-0.07 mm/h, Fog/Mist conditions). Instrumented measurement: the two cards contradicted each other for 19.4% of WeatherKit-covered cities during active weather, 11.6% on an ordinary day, 9.5% among truly quiet cities. Failure modes are characteristic: WeatherKit overdetects at trace intensity but nails real storms (3.6-10.4 mm/h); Open-Meteo lags storm spin-up and overpersists after decay. The NWS referee sided with WeatherKit 5-2 overall on "raining now" disputes.

4. The fix (implemented)

Spec: docs/NOWCAST_CENTRE_AUTHORITY_SPEC.md. One flag (nowcastCentreAuthorityEnabled, "Single Rain Authority" in Developer Settings) gating two inseparable parts:

• 0.2 mm/h intensity floor applied to every WeatherKit "is it precipitating" decision at the shared RadarService source (floor sits between measured phantoms and the lightest real storm)
• Storm Approach's centre state (raining here, intensity, arrival 0-60 min) adopts the same TTL-cached WeatherKit nowcast the narration uses; Open-Meteo answers only the 75-120 minute window and keeps all spatial work (ring, motion, towns, saved cities)

Verified by simulation BEFORE implementation across 102 WeatherKit-sampled cities: zero oversuppression (the floor never removed rain a station confirmed), 9/9 real-rain cases preserved, seam closed by construction. Implemented in 8240035; 125 unit tests passing (5 new IntensityFloorTests). Known residuals, accepted and documented in the spec: ~1.5% of refereed disputes where Open-Meteo had been right at the centre; ~5-7% transient cases where both sources are wrong together.

Status: installed on device 2026-07-05; early hands-on testing looks good.

5. Open design questions (deliberately not yet decided)

• Weather Around Me screen identity: Storm Approach samples a filled ~43-mile disk on a 2-hour clock; the directional tiles sample single points on the rim of the picker distance with no motion. Two instruments on one screen, one dial that only drives one of them. Options on the table: label the scopes honestly; scale net and clock together; split the screen's identity. The 43-mile scope is derived from the 2-hour horizon (storm travel distance), not data quality.
• Tile geometry bug: directional tiles use flat lat/lon degree math, so at the "150 mile" setting the east/west tiles sit at ~110 miles and diagonals at ~187. Great-circle destination math already exists in the codebase (GeoMath).
• The traveler's question ("pouring here, but dry where I'm heading 150 miles away") is a first-class use case that argues for keeping the two instruments distinct — and has an unserved time dimension (destination conditions AT ARRIVAL time, not current).
• Directional explorer: stepping through cities by direction is one of the most powerful parts of the feature set and should get investment regardless of the above.
• Guiding principle for all of it: no decoder-ring UX — restructuring must reduce concepts, not add settings.

References

• Spec: docs/NOWCAST_CENTRE_AUTHORITY_SPEC.md (nowcast-port)
• Analyses: docs/nowcast-analyses/ (nowcast-port)
• Testing guide: iOS/NOWCAST_TESTING_GUIDE.md (nowcast-port)
• Harness: tools/datatesting/ (all branches)
• Key commits: 043a00f (port), 4232a76 (harness), 5e7d2f5 (WeatherKit REST sampling), 2d7a15e (NWS referee + baseline), 9775935 (fix simulation), fa89d7f (spec), 8240035 (implementation), 536aebe (analyses)

🤖 Generated with Claude Code