Add compose grid endpoint for band math and grid composition#371
Merged
Conversation
Add POST /v2/domains/{domain_id}/grids/compose to create a new Grid by
selecting, computing, and conditionally composing bands from one or more
completed source grids. Supersedes the older blend/derive proposals
(#40, #41) with a single composition primitive.
API service:
- New api.resources.grids.compose module (schema, examples, router,
validation). Validates input existence/ownership/domain/completion,
alias/band references, output-key uniqueness, alignment, operator
arity, and unit compatibility. Multiply/divide derive output units;
additive ops require operands compatible with the output unit and
reject mixing unitless and unitful operands. Percent is preserved as
a unit. String-literal fallbacks and string categorical condition
values are rejected (categorical bands are integer-coded).
- Stays GDAL-free at runtime.
Griddle service:
- New handlers/compose.py plus the compose dispatch case. Re-validates
source-grid status/domain/owner and provenance checksum before
loading, enforces 2D-only alignment, converts operand units to the
output unit, propagates nodata, and fails on non-finite results
(e.g. divide-by-zero) instead of writing inf.
Tests: schema, router, and handler/dispatch coverage including unit
conversion, categorical-code validation, divide-by-zero, stringified
spatial-condition coordinates, checksum/status guards, and 3D rejection.
Categorical fbfm grid bands store integer Scott-Burgan codes, but users prefer the human-readable labels (GR1, NB1, ...). Resolve labels to codes at the API write boundary so stored rules are always integer codes and the processing services stay integer-only — no schema or worker changes. - lib.fuel_models: Scott-Burgan 40 label<->code table (45 codes, grounded in the griddle SB40 lookup) plus resolve_fuel_model_value(), which maps labels to codes, passes numbers through, handles lists, is case- and whitespace-insensitive, and raises on unknown labels. - Compose: categorical condition values and categorical `else` fallbacks accept labels; band references (alias.band) are left untouched. Unknown labels return 422. - Modifications: a shared resolver normalizes condition and action values; the compose endpoint reuses it for its post-compose modifications, so both endpoints behave consistently. Integer codes still work everywhere. - Restore the readable labels in the compose OpenAPI examples and note label support in both endpoints' docs.
…l resolution Address four review findings on the compose grid endpoint: - Conditional divide-by-zero: the non-finite check ran over the full computed array before the conditional fallback was applied, so a divide whose denominator was zero in cells the conditions exclude still failed the job. Compute the condition mask first and restrict the non-finite check to cells that keep the computed branch (and an inline-compute fallback to cells where it is actually used). - Cross-grid coordinate drift: alignment is validated only to a 1e-9 transform tolerance, but xarray arithmetic and where() align by exact coordinate label. Sub-nanometer drift between inputs built by different code paths could silently inner-join to NaN or an empty result. Normalize every input onto the first input's y/x coordinates after alignment validation. - Nodata + unit scaling: read the nodata sentinel from the raw band, before astype and the unit-conversion factor, so a sentinel like -9999 can't be scaled past == nodata recognition. - Band-aware label resolution: resolve_modification_fuel_model_labels now takes the grid's band types and resolves FBFM labels only on categorical bands, rejecting strings on continuous bands with a clear message instead of mis-resolving them. The modifications router runs it inside the transaction against the authoritative band metadata; compose passes its output band types. Adds regression tests for each fix.
The compose router hand-rolled request-only validation that Pydantic can express declaratively. Relocate the checks that depend solely on the request body into schema validators, leaving the router with only the validation that needs the loaded source-grid metadata. Schema (compose/schema.py): - InlineCompute gains an after-validator for operand arity (variadic vs binary), "at least one band operand", and "no string-literal operands"; ComposeCompute now inherits InlineCompute so the operator/operands fields and that validator exist once. - ComposeAttributeCondition gains an after-validator for value shape (`in` requires a list; ordering operators reject lists). - Scattered string-literal operator sets become named enum-based frozensets: VARIADIC_OPERATORS, CATEGORICAL_CONDITION_OPERATORS, _ORDERING_OPERATORS. Router (compose/router.py): - Drop _validate_compute_arity and _literal_is_numeric (now schema). - _validate_compute_units and _validate_conditions keep only the grid-metadata-dependent checks (band types, unit derivation, categorical operator restriction + label resolution); reuse the shared operator-set constants. These structural failures now surface as standard FastAPI 422 validation errors at parse time. Adds schema unit tests for the relocated checks and updates the in-condition router test for the list-shaped error body.
The compose request forced the caller to declare every output band's key, type, and unit, then the router just validated that those matched what it already derives from the operations and source grids. The key/type/unit were redundant (and getting the multiply/divide unit exactly right was pure busywork), so drop the `bands` field entirely. Output bands are now derived from the operations, in order (selects then computes): - key: the operation's `output` - type: the source band's type for `select`; always continuous for `compute` - unit: the source unit for `select`; the operand-derived unit for `compute` Optional `name`/`description` move onto `select`/`compute`. `compute` also gains an optional `unit` override (any dimensionally compatible unit; the worker converts to it) — `select` has none because it copies verbatim. Schema: - Remove ComposeOutputBand, build_compose_bands, and the `bands` field / its validators from CreateComposeRequest. - Add name/description to ComposeSelect; name/description/unit to ComposeCompute (unit canonical-validated). Router: - _validate_compute_units -> _derive_compute_unit: returns the output unit (derived, or a compatible override) rather than checking a supplied one. - _validate_compose_operations -> _build_output_bands: derives the ordered Band list while validating conditions and `else` fallbacks. Griddle is unchanged: it reads the resolved bands off the stored grid doc (which the API still computes) and ignores the new operation fields. Examples drop `bands`, gain a unit-override example, and the OpenAPI descriptions are rewritten. Tests updated for the derived shape.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
POST /v2/domains/{domain_id}/grids/composeto create a newGridby selecting, computing, and conditionally composing bands from one or more completed source grids. This is a single grid-composition primitive that supersedes the olderblend(#40) andderive(#41) proposals; it creates a normal asyncGridresource rather than a new resource type.What's included
API service (
api.resources.grids.compose)multiply/dividederive output units; additive ops (add/subtract/average/min/max) require operands compatible with the output unit and reject mixing unitless and unitful operands;%is preserved.Griddle service (
handlers/compose.py+ dispatch case)inf.FBFM40 fuel-model labels (
lib.fuel_models)fbfmconditions and fallbacks accept the human-readable Scott-Burgan labels (GR1,NB1, ...) interchangeably with the numeric codes. Labels resolve to codes at the API write boundary, so the processing services stay integer-only (no schema or worker changes).Testing
libresolver: 8 passedruffclean across all touched filesCloses #369
Supersedes #40, #41