Symptom
build_s1_rtc_stac_item() (src/eopf_geozarr/stac/s1_rtc.py) opens the cube with zarr.open_consolidated(zarr_store, zarr_format=3), which raises:
ValueError: Consolidated metadata requested with 'use_consolidated=True' but not found in ''.
In the live S1 GRD RTC pipeline this fails STAC registration (register_v1_s1_rtc.py / register_per_acquisition.py), so the cube item never registers / the previous item is left deleted.
Trigger (reproduced)
The per-tile cube (s1-rtc-{tile}.zarr) accumulates one time slice per acquisition. The failure occurs specifically when a slice is appended to an existing same-orbit group (e.g. the 2nd+ ascending acquisition for a tile):
| Scenario |
Result |
| Fresh single-acquisition store |
✅ builds |
| Append a new orbit group (descending into an ascending store) |
✅ builds |
| Append a slice to an existing same-orbit group (2nd ascending) |
❌ ValueError: Consolidated metadata ... not found |
Observed live on staging: 30TWM (descending appended into an ascending store) built fine; 31TEH (ascending appended into an existing ascending store) failed; a clean-slate 31TEH (fresh store) built fine.
Why it happens
The data-pipeline ingest does call consolidate_s1_store() after every acquisition (ingest_v1_s1_rtc.py:189,195 → s1_ingest.py:688-700, which consolidates the orbit group then the root). Yet after a same-orbit append on the S3 store, the root is read back without consolidated metadata, so open_consolidated raises. titiler reads the very same stores fine (its /info returns 200) because it does not require consolidated metadata.
Fix (done)
The builder must not require consolidated metadata — fall back to a direct zarr.open_group(zarr_store, mode="r", zarr_format=3) when it's absent (re-raising any other ValueError). Shipped in PR #180 (commit bbd18e3) with a regression test (tests/test_s1_stac.py::test_builds_from_non_consolidated_store: a 2-slice same-orbit store built with consolidate=False — fails before the fix, passes after).
Open sub-question (separate, not blocking)
Why does consolidate_s1_store() not leave the root consolidated after an S3 same-orbit append, despite being called? Likely S3 zarr-store consolidation behavior (note geozarr.py:1235 checks store.supports_consolidated_metadata). Worth tracking separately — the builder-tolerant fix unblocks production regardless, but losing consolidated metadata on appends forfeits its read optimization for all consumers that rely on it.
Symptom
build_s1_rtc_stac_item()(src/eopf_geozarr/stac/s1_rtc.py) opens the cube withzarr.open_consolidated(zarr_store, zarr_format=3), which raises:In the live S1 GRD RTC pipeline this fails STAC registration (
register_v1_s1_rtc.py/register_per_acquisition.py), so the cube item never registers / the previous item is left deleted.Trigger (reproduced)
The per-tile cube (
s1-rtc-{tile}.zarr) accumulates onetimeslice per acquisition. The failure occurs specifically when a slice is appended to an existing same-orbit group (e.g. the 2nd+ ascending acquisition for a tile):ValueError: Consolidated metadata ... not foundObserved live on staging:
30TWM(descending appended into an ascending store) built fine;31TEH(ascending appended into an existing ascending store) failed; a clean-slate31TEH(fresh store) built fine.Why it happens
The data-pipeline ingest does call
consolidate_s1_store()after every acquisition (ingest_v1_s1_rtc.py:189,195→s1_ingest.py:688-700, which consolidates the orbit group then the root). Yet after a same-orbit append on the S3 store, the root is read back without consolidated metadata, soopen_consolidatedraises.titilerreads the very same stores fine (its/inforeturns 200) because it does not require consolidated metadata.Fix (done)
The builder must not require consolidated metadata — fall back to a direct
zarr.open_group(zarr_store, mode="r", zarr_format=3)when it's absent (re-raising any otherValueError). Shipped in PR #180 (commitbbd18e3) with a regression test (tests/test_s1_stac.py::test_builds_from_non_consolidated_store: a 2-slice same-orbit store built withconsolidate=False— fails before the fix, passes after).Open sub-question (separate, not blocking)
Why does
consolidate_s1_store()not leave the root consolidated after an S3 same-orbit append, despite being called? Likely S3 zarr-store consolidation behavior (notegeozarr.py:1235checksstore.supports_consolidated_metadata). Worth tracking separately — the builder-tolerant fix unblocks production regardless, but losing consolidated metadata on appends forfeits its read optimization for all consumers that rely on it.