rustwx

rustwx is a Rust-first weather workspace for model ingest, diagnostics, map rendering, sounding rendering, and thin optional Python bindings.

The design target is straightforward:

• keep the hot path in Rust
• support multiple gridded models behind one internal field model
• verify the science against existing reference implementations
• make high-volume plotting practical for operational use

This repo is early, but it is no longer a placeholder. The workspace compiles, tests, fetches live model data, renders real weather plots, and now carries a bounded weather-native HRRR map validation lane plus an initial real-data cross-section validation path.

Current scope

Built-in model/source coverage:

• HRRR
• HRRR-AK
• GFS
• GDAS
• GEFS
• AIGFS
• AIGEFS
• ECMWF open data
• AIFS via ECMWF AIFS-Single GRIB2 and local Earth2Archive NetCDF
• RAP
• NAM
• HIRESW
• SREF
• RTMA
• URMA
• NBM
• RRFS-A
• WRF/GDEX

Working validation lanes:

• selector-backed upper-air map plots
• cross-model derived batch rendering for HRRR, GFS, ECMWF open data, and RRFS-A
• cross-model ECAPE-derived map rendering for HRRR, GFS, ECMWF open data, and RRFS-A
• severe diagnostics exposed as regular derived maps plus bundled convenience runners
• bounded HRRR weather-native map validation through hrrr_native_proof
• separate real-data HRRR temperature cross-section validation through hrrr_temperature_xsection
• sounding rendering with a rustwx-owned parameter table and verified ecape-rs ECAPE/NCAPE values
• native composite reflectivity + UH validation for HRRR and RRFS-A

Current design constraints:

• Rust-first compute and render path
• Python optional, never the main execution path
• model adapters should feed common field/selector types
• generated validation artifacts should be reproducible from code in this repo

For the v0.5 model-addition contract, see docs/model_compatibility_v05.md. For the release summary, coverage snapshot, validation artifacts, and known boundaries, see docs/release_v0.5.0.md.

Weather-native plot engine direction

The renderer boundary is intentionally narrow:

• product builders own fetch, crop, units, and meteorological science
• rustwx-render owns projected geometry, request shaping, palettes, overlays, panel layout, and PNG/image output
• native projected contour-fill rendering is live for signature derived products such as CAPE, SRH, EHI, and STP
• rustwx-contour is the dedicated home for contour topology and filled-band extraction; direct synoptic products still keep the older projected overlay path where appropriate
• rustwx-cross-section is the dedicated home for section geometry, sampling, terrain masking, and section rendering
• hrrr_native_proof currently publishes bounded 2D map artifacts plus a cross-section hook manifest; the real cross-section preview is still generated by a separate proof binary
• render requests should carry explicit field grids, projected geometry, and product semantics instead of hidden Python-side state
• verification should stay layered: isolated render checks in crates/rustwx-render/verify, then end-to-end generated artifacts in a local output directory

See ARCHITECTURE.md for the current engine split and the next integration steps. For the current typed backend contracts around structured queries, comparison stats, artifact bundles, and named assets, see docs/weather_intelligence_backend.md.

Workspace layout

Crate	Role	Status
rustwx-core	shared domain types and field/selector contracts	usable
rustwx-models	model registry, source planning, URL and recipe fetch planning	usable
rustwx-io	probing, byte-range fetch, selector-backed GRIB extraction, field cache	usable
rustwx-prep	preprocessing helpers, currently WRF-style lake masking/interpolation	narrow but real
rustwx-calc	ECAPE, CAPE/CIN, severe wrapper APIs over metrust	usable, still growing
rustwx-render	projected weather-map renderer, layout, overlays, and PNG output	usable
rustwx-contour	contour topology and filled-band extraction primitives	foundational but real
rustwx-cross-section	cross-section geometry, sampling, terrain, and lightweight rendering	foundational but real
rustwx-sounding	sounding bridge and rendering over sharprs	usable
rustwx-wrf	WRF/GDEX NetCDF4 read adapter backed by vendored netcrust	narrow but usable
rustwx-cli	CLI and proof binaries	usable
rustwx-python	thin optional Python bindings, published to PyPI as rustwx	usable, still thin

Each crate has its own README in crates/<crate>/README.md.

Python Package

The optional Python bindings are packaged as rustwx on PyPI:

pip install rustwx

import rustwx

print(rustwx.list_models_json())

The Python package is intentionally thin. It exposes model/source metadata and projected-map rendering helpers while keeping fetch, decode, compute, and render hot paths in Rust.

What works today

Model and source plumbing

• model summaries and source priorities
• URL resolution for built-in operational and archive models
• latest-run probing for NOAA-style feeds
• forecast-hour availability probes
• full-family HRRR ingest on the main operator path, with structured extraction from local wrfsfc / wrfprs family files
• indexed-subset direct batching for NOAA/NCEP-style GRIB models, with one grouped structured-extraction pass feeding many direct plots per hour while fetching only the needed .idx message ranges when the source provides them
• generic full-file derived batching for HRRR, HRRR-AK, GFS, GDAS, GEFS, AIGFS, AIGEFS, ECMWF open data, AIFS, RAP, NAM, HIRESW, SREF, RRFS-A, and WRF/GDEX when the required selectors are available
• generic full-file ECAPE derived-map batching for the supported built-in models when the required thermodynamic fields are available
• generic member-GRIB ensemble reduction for direct map recipes, including mean/std/min/max/p10/p50/p90 and native-unit probability-of-exceedance maps
• local Earth2Archive NetCDF fetch/open support for AIFS-style inference output
• WRF/GDEX NetCDF4 reads through vendored netcrust; no native netcdf.dll is required for the current WRF read surface
• indexed byte-range fetch is the default for direct GRIB products that publish .idx sidecars; ECMWF open data and NetCDF archive paths remain whole-file until a source-specific subset mechanism exists
• on-disk artifact caching for fetched bytes and selector-backed field extraction

Diagnostics

• ECAPE-family grid wrappers, including SB/ML/MU triplet support
• failure-mask variants for ECAPE debug/verification work
• CAPE/CIN wrappers
• SRH, shear, STP, SCP, EHI, local-proxy SHIP, and BRI wrappers
• bundled "supported severe" proof fields

ECAPE map path

The old ECAPE panel runner has been retired. ECAPE maps are now ordinary derived products or focused research/display products:

• hrrr_derived_batch renders HRRR ECAPE fields, ECAPE/CAPE ratios, NCAPE, ECIN, and experimental ECAPE SCP/EHI/STP products as individual maps.
• derived_batch runs the same ECAPE-derived product family for the supported built-in models when the required thermodynamic fields are available.
• hrrr_ecape_ratio_display renders the creator-recommended ECAPE/CAPE ratio display with intuitive low-to-high ratio colors and ECAPE contour context.
• hrrr_ecape_profile_probe extracts one HRRR column for parcel/profile diagnostics.
• hrrr_ecape_grid_research computes full-grid ECAPE research summaries for a lat/lon swath, including connected-mask statistics for ECAPE and ECAPE-EHI thresholds.

The ECAPE solver is the vendored ecape-rs crate. Its validation target is direct parity with Python ecape-parcel parcel-path behavior, while rustwx uses it for grid-scale products and research catalogs.

Rendering

• Weather-backed filled maps
• projected-map framing that can consume native model projection metadata
• native projected contour-fill rendering for signature derived products
• wind barb overlays
• built-in fill-plus-isopleth recipes with contour labels where configured
• panel composition
• public projected-map helpers in the rustwx Python package
• contour topology extraction in rustwx-contour
• foundational cross-section rendering via rustwx-cross-section
• sounding PNG rendering with a rustwx-owned table/title layer
• verified ECAPE/NCAPE values in soundings, plus optional external ECAPE annotation blocks

What does not work yet

• full generic model-to-product orchestration
• true all-model severe suite parity
• generic 3D ingest/decode APIs
• production-grade ECMWF latest-run probing
• generic JSON-defined custom composite/grid-overlay rendering for agents; v0.5 exposes built-in composite/isopleth recipes and returns explicit errors for arbitrary composites / grid_overlays request fields
• full lake-aware preprocessing for non-WRF models
• GIF/animation orchestration in Rust
• typed Python API for render/fetch/calc workflows

Vendored dependencies

This workspace no longer requires sibling local repos to build. The former external crates are vendored under vendor/, and the checked-in basemap assets needed by the render stack now live under assets/basemap/.

Imported upstream provenance is recorded in vendor/VENDORED.md.

Quick start

Build and test

cargo test

For renderer-only verification without walking the full workspace:

cargo test --manifest-path crates/rustwx-render/verify/Cargo.toml

Basic CLI usage

cargo run -p rustwx-cli -- list
cargo run -p rustwx-cli -- show hrrr
cargo run -p rustwx-cli -- latest hrrr 20260414

Generate a selector-backed upper-air proof

cargo run -p rustwx-cli --bin plot_recipe_proof -- `
  --model gfs `
  --date 20260414 `
  --hour 18 `
  --forecast-hour 0 `
  --recipe 500mb_temperature_height_winds `
  --region conus

Generate HRRR ECAPE maps

cargo run -p rustwx-cli --release --bin hrrr_derived_batch -- `
  --recipe sbecape,mlecape,muecape,sb_ecape_derived_cape_ratio,ml_ecape_derived_cape_ratio,mu_ecape_derived_cape_ratio,sb_ecape_native_cape_ratio,ml_ecape_native_cape_ratio,mu_ecape_native_cape_ratio,sbncape,sbecin,mlecin,ecape_scp,ecape_ehi_0_1km,ecape_ehi_0_3km,ecape_stp `
  --date 20260414 `
  --cycle 23 `
  --forecast-hour 1 `
  --region midwest

Generate one generic ECAPE derived batch

cargo run -p rustwx-cli --release --bin derived_batch -- `
  --model rrfs-a `
  --recipe sbecape,mlecape,muecape,sb_ecape_derived_cape_ratio,ml_ecape_derived_cape_ratio,mu_ecape_derived_cape_ratio,sbncape,sbecin,mlecin,ecape_scp,ecape_ehi_0_1km,ecape_ehi_0_3km,ecape_stp `
  --date 20260414 `
  --cycle 20 `
  --forecast-hour 2 `
  --source aws `
  --region midwest

Generate ECAPE ratio display maps

cargo run -p rustwx-cli --release --bin hrrr_ecape_ratio_display -- `
  --date 20260425 `
  --cycle 0 `
  --forecast-hour 42 `
  --source nomads `
  --region southern-plains

Generate one generic derived batch

cargo run -p rustwx-cli --release --bin derived_batch -- `
  --model gfs `
  --all-supported `
  --date 20260414 `
  --cycle 18 `
  --forecast-hour 12 `
  --region midwest

Generate severe + ECAPE maps together from one shared heavy load

cargo run -p rustwx-cli --release --bin heavy_panel_hour -- `
  --model gfs `
  --date 20260414 `
  --cycle 18 `
  --forecast-hour 12 `
  --region midwest

Generate one full-file direct batch for GFS or RRFS-A

cargo run -p rustwx-cli --release --bin direct_batch -- `
  --model gfs `
  --all-supported `
  --date 20260414 `
  --cycle 18 `
  --forecast-hour 12 `
  --region midwest

cargo run -p rustwx-cli --release --bin direct_batch -- `
  --model rrfs-a `
  --all-supported `
  --date 20260414 `
  --cycle 20 `
  --forecast-hour 2 `
  --source aws `
  --region midwest

Generate one HRRR non-ECAPE hour pass

hrrr_non_ecape_hour is the main HRRR operator-facing batch entrypoint. It now defaults to NOMADS and the full-family HRRR ingest path.

cargo run -p rustwx-cli --release --bin hrrr_non_ecape_hour -- `
  --date 20260414 `
  --cycle 23 `
  --forecast-hour 1 `
  --region conus

Generate one GFS or ECMWF global fast-lane pass

non_ecape_hour is the shared global-model fast lane. For GFS and ECMWF, use --domain-set global-model to load one model hour once and fan it out across global, CONUS, North America, Europe, and the other continent-scale domains.

cargo run -p rustwx-cli --release --bin non_ecape_hour -- `
  --model gfs `
  --date 20260414 `
  --cycle 0 `
  --forecast-hour 24 `
  --domain-set global-model `
  --domain-jobs 4 `
  --all-supported

cargo run -p rustwx-cli --release --bin non_ecape_hour -- `
  --model ecmwf-open-data `
  --source ecmwf `
  --date 20260414 `
  --cycle 0 `
  --forecast-hour 24 `
  --domain-set global-model `
  --domain-jobs 4 `
  --all-supported

Generate the bounded HRRR native proof bundle

hrrr_native_proof is the weather-native 2D proof lane. It emits the curated map set plus a summary JSON and an explicit cross-section hook placeholder.

cargo run -p rustwx-cli --release --bin hrrr_native_proof -- `
  --date 20260414 `
  --cycle 23 `
  --forecast-hour 0 `
  --region midwest `
  --out-dir target/artifacts/hrrr_native

Generate the foundational HRRR temperature cross-section proof

hrrr_temperature_xsection is a separate proof lane today. It exercises real HRRR section extraction and lightweight section rendering, but it is not yet invoked by hrrr_native_proof.

cargo run -p rustwx-cli --release --bin hrrr_temperature_xsection -- `
  --date 20260414 `
  --cycle 23 `
  --forecast-hour 0 `
  --out-dir target/artifacts/hrrr_temperature_xsection

Generate a model sounding plot

cargo run -p rustwx-cli --release --bin sounding_plot -- `
  --model hrrr `
  --date 20260424 `
  --cycle 22 `
  --forecast-hour 1 `
  --source aws `
  --lat 35.222 `
  --lon -97.439 `
  --station-id "Norman OK" `
  --out-dir target/artifacts/soundings

For a box-averaged model sounding, add --sample-method box-mean --box-radius-km 25. The JSON manifest records the contributing grid points and effective box radius.

Generated artifacts

Generated image, manifest, and benchmark outputs are local artifacts and are not committed to the repository.

Quantity semantics are explicit where the implementation is narrower than a generic meteorological label:

• the current vorticity proof path is absolute vorticity, not unspecified "vorticity"
• isobaric dewpoint extraction is currently direct DPT-only where wired, not a generic derived-dewpoint path
• the current SHIP wrapper is the existing local wrf-rust hail-proxy formula, not yet a full SHARPpy-style canonical SHIP implementation

Timing JSON files are written beside generated PNGs when enabled.

For contour status specifically: current map contour overlays are still the existing rustwx-render path. rustwx-contour exists now as a standalone topology crate, but it is not yet the shared live map contour backend that drives every plotted map contour or filled band.

Performance notes

The fast path for upper-air plots is already heavily in Rust. On this machine, a cached Rust upper-air proof plot is sub-second, while a Python matplotlib/cartopy equivalent on the same cached GFS subset was about an order of magnitude slower.

For the native contour lane, the current benchmark/profiling harness lives in weather_native_bench. The latest southern-plains HRRR benchmark compares Rust native contour renders, forced legacy raster renders, and Python matplotlib/cartopy equivalents for stp_fixed, sbcape, and srh_0_1km.

The remaining expensive path is not upper-air rendering. It is full-grid severe/ECAPE computation, especially when solving SB/ML/MU parcel variants over large domains.

Review guidance

If you are reviewing the workspace crate by crate, start in this order:

1. rustwx-core
2. rustwx-models
3. rustwx-io
4. rustwx-calc
5. rustwx-render
6. rustwx-sounding
7. rustwx-prep
8. rustwx-cli
9. rustwx-python

That order matches the dependency flow.

Immediate next milestones

• broaden selector coverage and model adapters
• unify proof binaries into a real product CLI
• lift the generic direct/derived/ECAPE executor shape into windowed and severe products
• make heavy_panel_hour the preferred shared ECAPE+severe runner for one model hour
• finish severe-suite render plumbing
• make ECMWF probing and fetch planning more robust
• reduce remaining ECAPE wall time