Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
059e065
docs: design for Sentinel-3 OLCI L1 EFR -> GeoZarr export
d-v-b Jun 21, 2026
ee43eba
docs: implementation plan for Sentinel-3 OLCI export
d-v-b Jun 21, 2026
c613e24
chore: regenerate uv.lock for released zarr-cm 0.4.1
d-v-b Jun 21, 2026
c49ab06
feat(s3-olci): add OLCI band mapping
d-v-b Jun 22, 2026
cbe586f
feat(s3-olci): add Sentinel3OlciRoot data-api model
d-v-b Jun 22, 2026
8debe8d
fix(s3-olci): remove Any from data-api model (use precise opaque-grou…
d-v-b Jun 22, 2026
5874f26
feat(s3-olci): add swath /2 decimation for overviews
d-v-b Jun 22, 2026
01abf31
fix(s3-olci): move xarray import to TYPE_CHECKING; strengthen decimat…
d-v-b Jun 22, 2026
f7998de
feat(s3-olci): add OLCI product detection
d-v-b Jun 22, 2026
c05a060
feat(s3-olci): swath spatial-convention attrs (no transform)
d-v-b Jun 22, 2026
67e86d4
feat(s3-olci): add convert_olci_optimized entry point
d-v-b Jun 22, 2026
26edc80
fix(s3-olci): correct overview min_dimension guard; document returned…
d-v-b Jun 22, 2026
b672759
feat(s3-olci): CLI auto-detect + convert-s3-olci-optimized command
d-v-b Jun 22, 2026
cec1e38
test(s3-olci): real OLCI product fixture + round-trip detection
d-v-b Jun 22, 2026
0b45392
test(s3-olci): golden-file snapshot of converted OLCI structure
d-v-b Jun 22, 2026
c673287
fix(s3-olci): strip inherited v2 encoding before v3 write; fix fixtur…
d-v-b Jun 22, 2026
ade00de
docs(s3-olci): document Sentinel-3 OLCI export support
d-v-b Jun 22, 2026
73a47b9
fix(s3-olci): fill-aware block-avg overviews, multiscales CMO, orphan…
d-v-b Jun 22, 2026
3849b24
fix(s3-olci): restore faithful 'columns' dim name in OLCI fixture (re…
d-v-b Jun 22, 2026
43d4bf0
fix(s3-olci): open with mask_and_scale=False; sanitize stale attrs; r…
d-v-b Jun 22, 2026
a0cf790
fix: revert shared sanitize_array_attrs to pre-OLCI semantics; add OL…
d-v-b Jun 22, 2026
a112179
fix(s3-olci): align coordinate decimation with coarsen trim on odd di…
d-v-b Jun 23, 2026
56b2b2e
docs(s3-olci): add Sentinel-3 OLCI GeoZarr demonstration notebook
d-v-b Jun 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,63 @@ Check if a variable is a grid_mapping variable by looking for references to it.

Validate that a specific band exists and is complete in the dataset.

## Supported Products

### Sentinel-2 MSI

Sentinel-2 MSI (MultiSpectral Instrument) L1C and L2A products are detected
automatically by `eopf-geozarr convert` and routed to the optimized multiscale
layout (`convert-s2-optimized`). The three native resolution groups (10 m, 20 m,
60 m) are reused as-is and coarser overviews (120 m, 360 m, 720 m) are computed
via /2 downsampling.

### Sentinel-3 OLCI L1 EFR

Sentinel-3 OLCI (Ocean and Land Colour Instrument) Level-1 EFR (Full Resolution)
products are detected automatically by `eopf-geozarr convert` and routed to the
dedicated OLCI converter. Unlike Sentinel-2, OLCI data uses **native swath geometry**:
measurements are stored on a per-pixel 2-D lat/lon grid with no reprojection to a
projected CRS. The exporter preserves this curvilinear geometry intact.

#### Auto-detection

```bash
eopf-geozarr convert S3A_OL_1_EFR.zarr output.zarr
```

#### Dedicated command

```bash
eopf-geozarr convert-s3-olci-optimized S3A_OL_1_EFR.zarr output.zarr \
--spatial-chunk 1024 \
--min-dimension 256 \
--compression-level 3
```

Key flags:

- `--spatial-chunk` — target spatial chunk size in pixels (default: 1024)
- `--compression-level` — Blosc/zstd compression level 1–9 (default: 3)
- `--min-dimension` — stop generating /2 overview levels once either spatial
dimension would drop below this value (default: 256)
- `--enable-sharding` — accepted but not yet wired into encoding (follow-up task)
- `--keep-scale-offset` — accepted but not yet wired into encoding (follow-up task)

#### What is converted

- **`/measurements`**: all 21 OLCI radiance bands at native full resolution, with
GeoZarr `spatial:` convention metadata and per-pixel 2-D `latitude`/`longitude`
coordinate arrays.
- **Overview subgroups** (`r2`, `r4`, …): /2-decimated copies of the measurements
stored as sibling Zarr groups under `measurements/`.
- **`/conditions` and `/quality`**: copied through unmodified.

> **Note:** OLCI support is initial/measurements-focused (v1). Tie-point grid
> groups (`conditions/geometry`, `meteorology`, `instrument`) are copied through but
> not converted to GeoZarr convention. Encoding wiring for `--enable-sharding`,
> `--spatial-chunk`, `--compression-level`, and `--keep-scale-offset` is accepted
> but scheduled as a follow-up task.

## Architecture

The library is organized into the following modules:
Expand Down
57 changes: 57 additions & 0 deletions docs/converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,63 @@ dt_optimized = convert_s2_optimized(

The result is a space-efficient multiscale pyramid: `/measurements/reflectance/{r10m, r20m, r60m, r120m, r360m, r720m}` where the native resolutions are preserved as-is and only the coarser levels are computed.

## Sentinel-3 OLCI L1 EFR Conversion

Sentinel-3 OLCI (Ocean and Land Colour Instrument) Level-1 EFR (Full Resolution)
products are supported. OLCI uses **native swath geometry**: measurements are stored
on a per-pixel 2-D lat/lon grid, with no reprojection to a projected CRS. The
exporter preserves this curvilinear geometry intact and generates /2-decimated
overview subgroups for multi-resolution access.

### Auto-detection

The generic `convert` command detects OLCI products automatically:

```bash
eopf-geozarr convert S3A_OL_1_EFR.zarr output.zarr
```

### Dedicated command

A dedicated command offers fine-grained control:

```bash
eopf-geozarr convert-s3-olci-optimized S3A_OL_1_EFR.zarr output.zarr \
--spatial-chunk 1024 \
--min-dimension 256 \
--compression-level 3
```

| Flag | Default | Description |
|------|---------|-------------|
| `--spatial-chunk` | 1024 | Target spatial chunk size in pixels |
| `--compression-level` | 3 | Blosc/zstd compression level (1–9) |
| `--min-dimension` | 256 | Minimum spatial dimension for overview levels |
| `--enable-sharding` | off | Accepted but not yet wired into encoding (follow-up task) |
| `--keep-scale-offset` | off | Accepted but not yet wired into encoding (follow-up task) |

### Output layout

```
output.zarr/
├── measurements/ # Native-resolution OLCI bands (oa01_radiance … oa21_radiance)
│ │ # with per-pixel latitude/longitude coordinates
│ ├── r2/ # 1/2-resolution overview
│ ├── r4/ # 1/4-resolution overview
│ └── ...
├── conditions/ # Copied through unmodified (tie-point geometry, meteorology)
└── quality/ # Copied through unmodified (quality flags)
```

Each measurement group carries GeoZarr `spatial:` convention metadata and
references the per-pixel 2-D coordinate arrays `latitude` and `longitude`.

> **Note:** OLCI support is initial/measurements-focused (v1). Tie-point grid
> groups in `conditions/geometry`, `meteorology`, and `instrument` are copied
> through but not converted to GeoZarr convention. Encoding wiring for
> `--enable-sharding`, `--spatial-chunk`, `--compression-level`, and
> `--keep-scale-offset` is accepted but scheduled as a follow-up task.

## Error Handling

The converter includes robust error handling and retry logic for network operations, ensuring reliable processing even in challenging environments.
Expand Down
Loading