Skip to content

Replace in-tree pcodec implementation with @eeholmes/zarrita-pcodec #112

Description

@eeholmes

Background

PR #106 added support for the pcodec Zarr codec directly into the GridLook codebase, including WASM assets, codec wrappers, registration logic, build configuration, and supporting files.

A standalone codec package now exists:

https://github.com/eeholmes/zarrita-pcodec

The goal of this issue is to remove all pcodec-specific implementation details from GridLook and use the external package instead.

Tasks

1. Add dependency

Add the external package dependency:

npm install github:eeholmes/zarrita-pcodec

Use the package import name:

@eeholmes/zarrita-pcodec

2. Register pcodec through the external package

Use the codec registration API provided by the package:

import { registry } from "zarrita";
import { registerPCodec } from "@eeholmes/zarrita-pcodec";

registerPCodec(registry);

Place this alongside any existing codec registration logic so it runs before Zarr arrays are opened.

3. Remove all pcodec implementation code added in PR #106

Review the full diff from PR #106 and remove any code that exists solely to support pcodec, including:

  • Local pcodec codec classes
  • Local codec registration logic
  • WASM loaders
  • WASM assets
  • Rust-generated artifacts
  • Build tooling added only for pcodec
  • Type definitions added only for pcodec
  • Tests that are specific to the in-tree implementation
  • Utility functions used only by the local implementation

Do not remove any generic codec infrastructure that is used by other codecs.

The intent is that GridLook no longer contains its own pcodec decoder implementation.

4. Preserve functionality

After cleanup:

  • pcodec datasets should continue to load through the external package.
  • Existing non-pcodec datasets should continue to work unchanged.
  • No functionality should be lost.

5. Validate with a real dataset

Verify that a pcodec-compressed Icechunk/Zarr store still opens successfully.

Test dataset:

https://earthmover-icechunk-era5.s3.amazonaws.com/icechunkV2

Known working variable:

/single/spatial/blh

The validation should confirm that GridLook can successfully open and read data from this variable without using any in-tree pcodec implementation.

Expected Result

GridLook should:

  • Depend on @eeholmes/zarrita-pcodec
  • Register the codec via:
import { registry } from "zarrita";
import { registerPCodec } from "@eeholmes/zarrita-pcodec";

registerPCodec(registry);
  • Contain no local pcodec decoder implementation.
  • Contain no pcodec-specific WASM assets.
  • Contain no pcodec-specific build machinery.
  • Continue to support pcodec-compressed datasets.

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions