Skip to content

Add soil module (SoilGrids 2.0) with NEH 630 Ch7 hydrologic soil group#12

Merged
rehsani merged 1 commit into
mainfrom
step-5-soil-and-hsg
May 4, 2026
Merged

Add soil module (SoilGrids 2.0) with NEH 630 Ch7 hydrologic soil group#12
rehsani merged 1 commit into
mainfrom
step-5-soil-and-hsg

Conversation

@rehsani
Copy link
Copy Markdown
Owner

@rehsani rehsani commented May 4, 2026

Summary

  • New `floodpath.soil` module: ISRIC SoilGrids 2.0 (250 m global, COG via `/vsicurl/` + `WarpedVRT` from native Goode Homolosine to WGS84). Fetches sand/silt/clay (% by mass) at user-selectable depths and derives the NEH Part 630 Chapter 7 hydrologic soil group (A/B/C/D).
  • HSG is the SOIL half of the SCS-CN runoff parameterisation. Combined with `floodpath.landuse` (already in place) it lets a future `hydrology.runoff` module compute Curve Numbers.

Module surface

Item What
`get_soilgrids_texture(lat, lon, buffer_deg, depths)` Fetch sand/silt/clay as a depth-weighted `TextureGrid`. Default depths = (0-5cm, 5-15cm, 15-30cm), the closest proxy for NEH 630 Ch7's 0-50 cm reference.
`TextureGrid` sand/silt/clay rasters + georef + `closure_residual()` diagnostic. Nodata (-32768) is masked to NaN.
`usda_texture_class(sand, silt, clay)` Full USDA Soil Survey Manual 1993 texture triangle, 12 canonical classes. Scalar or vectorised.
`texture_to_hsg(TextureGrid)` `USDA_TEXTURE_TO_HSG` lookup (NEH 630 Ch7 §630.0701 textural shortcut) -> `HSGGrid`. Custom mappings supported.
`HSGGrid` uint8 raster (1=A, 2=B, 3=C, 4=D, 0=nodata) + `class_counts()` / `fraction()` / `dominant_class()` / `as_letter_grid()`

Performance

SoilGrids reads were the bottleneck — 9 sequential `/vsicurl/` fetches for 3 vars × 3 depths took ~525 s. Parallelised via `ThreadPoolExecutor` (max 9 workers): ~73 s for the same workload (~7× speedup). ISRIC tolerates ~10 concurrent connections per client.

Test plan

  • `pytest -m "not integration"` — 211 passed (154 prior + 57 new soil tests; 16 deselected integration)
  • `pytest -m integration tests/soil/` — 3 passed (~169 s): live ISRIC fetch in Djibouti at default depths, single-depth fetch, fixture regression for the Robit Bata patch
  • All 12 USDA texture classes round-trip from textbook centroids; all four HSG groups (A/B/C/D) reachable
  • Robit Bata patch (Vertisol-rich Ethiopian highlands) classifies as 99.9% HSG D — geologically correct (cracking black clays, ~54% mean clay)
  • End-to-end smoke test on Robit Bata: 14 stages all succeed, including the new HSG plot at stage 12
  • Committed fixture is 8.4 KB (`tests/fixtures/robit_bata_soil.tif`, 33×33 multiband float32) — well under the per-fixture budget

Notes

  • Default texture-shortcut HSG assignment assumes "deep" soils with no shallow restrictive layer or high water table. Users with site-calibrated Ksat can supply a custom `mapping` to `texture_to_hsg` to bypass the textural shortcut.
  • `TextureGrid.closure_residual()` reports per-cell `|sand+silt+clay - 100|`. SoilGrids regressions are independent for each variable so closure is not exact (typical ~0.02% mean residual on the Robit Bata fixture).
  • Single white nodata pixel in the smoke-test HSG plot is the Goode → WGS84 reproject corner artefact and is expected.

…p derivation

floodpath.soil supplies the SOIL half of the SCS-CN runoff parameterisation.
Pair with floodpath.landuse to derive a Curve Number raster.

- floodpath/soil/soilgrids.py: get_soilgrids_texture(lat, lon, buffer_deg,
  depths) — fetches sand/silt/clay (% by mass) at user-selectable depth
  slices from the public ISRIC bucket via /vsicurl/ + WarpedVRT (native
  Goode Homolosine -> WGS84). Default depths=(0-5cm, 5-15cm, 15-30cm),
  depth-weighted to a 0-30 cm topsoil composition (closest available
  proxy for NEH 630 Ch7 Table 7-1's 0-50 cm reference depth). Fetches
  parallelised across (variable, depth) pairs — ~73 s for 9 reads vs
  ~525 s sequential. SoilGrids nodata (-32768) masked to NaN.
- floodpath/soil/utils.py: usda_texture_class(sand, silt, clay) — full
  USDA Soil Survey Manual 1993 / Soil Taxonomy 1999 texture triangle
  with all 12 canonical classes (sand, loamy sand, sandy loam, loam,
  silt loam, silt, sandy clay loam, clay loam, silty clay loam, sandy
  clay, silty clay, clay). Vectorised over numpy arrays.
- floodpath/soil/hsg.py: texture_to_hsg(TextureGrid) -> HSGGrid via the
  USDA_TEXTURE_TO_HSG mapping (NEH 630 Ch7 §630.0701 textural shortcut).
  Custom mappings supported for site-calibrated work.
- TextureGrid carries closure_residual() for sand+silt+clay~=100 sanity
  checks; HSGGrid carries class_counts(), fraction(), dominant_class(),
  as_letter_grid() helpers and a uint8 1-4 encoding (0=nodata).
- Tests: 60 total — 12 USDA centroid round-trips, 11 hsg unit cases,
  3 model property tests, plus a committed Robit Bata fixture (8.4 KB,
  33x33 multiband float32 GeoTIFF) with pinned per-cell stats and a
  100% HSG-D outcome (Vertisol-rich Ethiopian highlands). One live ISRIC
  fetch regression test guards the fixture; an integration test in
  Djibouti exercises the full /vsicurl/+WarpedVRT+parallel-fetch path.
@rehsani rehsani merged commit 3bacc7c into main May 4, 2026
1 check passed
@rehsani rehsani deleted the step-5-soil-and-hsg branch May 4, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant