Skip to content

fix(s1-rtc): consolidate every orbit group, not just the last-ingested one#203

Merged
lhoupert merged 1 commit into
feat/s1-rtc-stac-builderfrom
fix/s1-rtc-consolidate-all-orbits
Jun 23, 2026
Merged

fix(s1-rtc): consolidate every orbit group, not just the last-ingested one#203
lhoupert merged 1 commit into
feat/s1-rtc-stac-builderfrom
fix/s1-rtc-consolidate-all-orbits

Conversation

@lhoupert

Copy link
Copy Markdown
Contributor

Why

S1 RTC cubes end up with only one orbit consolidated on disk — staging shows 30TWM asc✓/desc✗, 32TNN & 30UVU asc✗/desc✓.

Root cause (traced + reproduced): each acquisition is ingested by a separate pod that strips all consolidated metadata (so time can resize), ingests its single orbit, then calls consolidate_s1_store(store, orbit_direction) — which only re-consolidated the one orbit passed in. So whichever orbit is ingested last is the only one left with on-disk consolidated metadata.

What

consolidate_s1_store now consolidates every orbit group present (root.groups()), then the root. Signature unchanged — orbit_direction is kept for logging / caller compatibility (no changes to the CLI or the data-pipeline callers).

root = zarr.open_group(str(store_path), mode="r", zarr_format=3)
for orbit_name, _ in root.groups():
    zarr.consolidate_metadata(str(store_path), path=orbit_name, zarr_format=3)
zarr.consolidate_metadata(str(store_path), zarr_format=3)

Safety: the pipeline's minimal append-fetch already pulls all zarr.json (ingest_v1_s1_rtc.py:336-337); only bulk chunks are skipped. So both orbits' metadata is local when this runs — the same reason the root consolidation already produces a correct both-orbit view.

Impact — cosmetic, not a correctness/deploy blocker

The root consolidated metadata is already complete for both orbits, and a consolidated root synthesizes a child orbit's view — so titiler (opens at root, selects via /orbit:vv) renders the unconsolidated orbit fine (verified: 30UVU ascending = HTTP 200 despite asc✗). The fix matters for clients that open a single orbit group standalone (e.g. following the STAC <cube>.zarr/<orbit> asset hrefs) with consolidated=True, which otherwise fall back to a listing. Existing cubes self-heal on their next re-ingest.

Tests (TDD, RED→GREEN)

tests/test_s1_rtc_ingest.py::TestConsolidation::test_consolidate_all_orbits_present — ingests both orbits, calls consolidate_s1_store(store, "descending") (one orbit passed), asserts both orbit groups are consolidated standalone.

⚠️ The assertion opens each orbit group standalone — asserting via root["ascending"].metadata.consolidated_metadata is a false-green (a consolidated root synthesizes the child view, so it's non-None even on the buggy code; verified empirically).

RED on ac3167f, GREEN after the fix. Full data-model suite green; ruff/mypy add zero new findings vs baseline.

Stacks on #202; targets #180.

🤖 Generated with Claude Code

…d one

S1 RTC cubes ended up with only one orbit consolidated on disk (staging:
30TWM asc✓/desc✗, 32TNN/30UVU asc✗/desc✓). Root cause: the pipeline
ingests acquisitions one orbit at a time in separate pods, each of which
strips all consolidated metadata (so `time` can resize), ingests its
orbit, then calls `consolidate_s1_store(store, orbit_direction)` — which
only re-consolidated the single orbit passed. So whichever orbit was
ingested last is the only one left with on-disk consolidated metadata.

Consolidate every orbit group present (iterate `root.groups()`), then the
root. The minimal append-fetch already pulls all `zarr.json`, so both
orbits' metadata is local — same reason the root consolidation already
works. Signature unchanged (`orbit_direction` kept for logging/callers).

Impact is cosmetic: the root consolidation is complete for both orbits and
readers opening at the root get a synthesized child view (titiler renders
the unconsolidated orbit fine). The fix matters for clients opening a
single orbit group standalone (the STAC `<cube>.zarr/<orbit>` hrefs) with
`consolidated=True`, which otherwise fall back to a listing.

Test asserts each orbit group is consolidated *standalone* — checking via
`root[orbit]` is a false-green because a consolidated root synthesizes the
child's view. Existing cubes self-heal on their next re-ingest.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant