Binder compatible docs / example notebooks

[espg]

All of our example notebooks should be updated-- a lot of development has occurred, and they are certainly out of date.

More fundamentally, our example notebooks should maximally accessible and usable (wherever possible). This means that they should be binder launch-able (with the exception of the cyrocloud example, which calls the lambda on cryocloud and can't be run on binder). However, other notebooks should be binder-ized.

This means that we need to ensure that we have the infrastructure at the repo level so that repo2docker can run cleanly and build the required deps to run our notebooks. Doing a pass over the [docs] metapack deps will also make sense here.

[espg]

🤖 from Claude

Implementation plan for binder-izing the example notebooks, grounded in the current repo state.

What's in the repo today

Notebooks (notebooks/, 4 total):

• custom_aggregations.ipynb — pure-Python tour of the config/aggregation API (zagg.config, zagg.processing.calculate_cell_statistics, zagg.schema.xdggs_spec). No AWS, no network. Fully binder-able.
• rasterized_zarr.ipynb — the README "Step 4" viz notebook; reads the public xagg S3 store anonymously (S3Store(..., skip_signature=True), ObjectStore(read_only=True)) and rasterizes HEALPix to polar stereo with cartopy/matplotlib. Binder-able (read-only, anonymous, PyPI deps only).
• jupyterhub_example.ipynb — reads the public xagg store anonymously (skip_signature=True) and also demonstrates agg(backend="lambda", ...). The read path is binder-able; the Lambda-dispatch path is not. Binder-able if the Lambda-dispatch cell is dropped or marked non-executing.
• cryocloud_example.ipynb — the explicit exception. Calls the Lambda (backend='lambda', arn:aws:lambda:us-west-2:429435741471:...), uses icechunk + IRSA credentials + boto3 sts get_caller_identity. Cannot run on binder — exclude.

Staleness found (all notebooks): legacy xagg naming throughout after the rename to zagg/englacial — %pip install 'zagg[analysis] @ git+https://github.com/englacial/xagg.git', bucket xagg, prefixes atl06/morton_aggregation_*. The public API names the notebooks import (agg, default_config, get_child_order, load_config, calculate_cell_statistics, xdggs_spec, config helpers) do match current src/zagg/__init__.py, so the API surface is largely intact; the rot is in install URLs, bucket/store paths, ARNs, and inline %pip of the old repo. Minor inconsistency: default_config is imported from zagg in some notebooks and zagg.config in others.

Binder infra: none. No .binder/, binder/, environment.yml, postBuild, apt.txt, requirements.txt, or repo2docker config anywhere; no "launch on binder" badge in README or notebooks. What exists is mkdocs-only: .readthedocs.yml, mkdocs.yml, docs/.

[docs] deps: the docs dependency-group in pyproject.toml is mkdocs-only (mkdocs, mkdocs-material[imaging], mkdocstrings, mkdocstrings-python) — it does not install jupyter or any viz deps, so it cannot run the notebooks as-is. The notebook/viz deps actually live in the analysis optional-extra (notebook, matplotlib, cartopy>=0.25, geopandas, xarray[io], cubed, cubed-xarray, xdggs). hvplot, icechunk, cubed-xarray are %pip install-ed inline — and only in the cryocloud notebook.

spherely fork caveat: none of the binder-able notebooks need it. They only read a prebuilt store (no catalog/SpatialIndex build), so the non-PyPI exact-S2 spherely fork is not on their import path. No exclusion needed on that account.

Phased steps

Phase 1 — repo2docker/binder infra. Add a .binder/ directory with a pip-based requirements.txt (or environment.yml) that installs zagg plus the notebook runtime (the analysis extra deps + jupyter), pinned and PyPI-resolvable only — no fork wheels, no git+ URLs. Add a postBuild if any non-pip setup is needed (likely none). Confirm a clean repo2docker . build succeeds.

Phase 2 — refresh the binder-able notebooks (custom_aggregations, rasterized_zarr, jupyterhub_example): update all xagg→zagg/englacial naming, fix the public store bucket/prefix to the current public location, drop/replace the legacy %pip install ... xagg.git cell (deps come from binder infra), normalize default_config imports to one source, re-run top-to-bottom against the anonymous public store, and clear outputs. For jupyterhub_example, gate or remove the agg(backend="lambda") cell so the notebook runs end-to-end on binder.

Phase 3 — dependency pass. Decide the binder dep mechanism and reconcile with pyproject.toml: either point binder at the analysis extra (+ add jupyter/notebook coverage) or add a dedicated notebook-runtime extra. Ensure everything a binder-able notebook imports (cartopy, matplotlib, xarray, obstore, zarr, mortie, pyproj, pandas, numpy) is declared and PyPI-installable. (See open question (1) on whether [docs] itself should gain these or stay mkdocs-only.)

Phase 4 — launch links. Add "launch on binder" badges to README.md and a short intro cell in each binder-able notebook linking it to binder, so the rendered notebooks satisfy the CLAUDE.md binder-link requirement. Explicitly exclude cryocloud_example.ipynb with an in-notebook note that it requires the cryocloud JupyterHub (IRSA creds + live Lambda) and is not binder-runnable.

Files likely touched

• notebooks/custom_aggregations.ipynb, notebooks/rasterized_zarr.ipynb, notebooks/jupyterhub_example.ipynb (refresh + clear outputs)
• notebooks/cryocloud_example.ipynb (add the "not binder-able" note only)
• pyproject.toml ([docs] group / analysis extra / new notebook extra — pending open question (1))
• New: .binder/requirements.txt (or .binder/environment.yml) + optional .binder/postBuild
• README.md (binder badges), possibly a docs/ notebooks index page

Acceptance criteria

• Every binder-able notebook (custom_aggregations, rasterized_zarr, jupyterhub_example) runs top-to-bottom on a clean repo2docker build using only PyPI-resolvable deps and anonymous/in-tree data.
• Each rendered binder-able notebook links to binder; README carries a launch badge.
• cryocloud_example.ipynb is excluded with a written rationale.
• All committed notebooks have outputs cleared.
• No git+/xagg-named installs; no spherely fork dependency on any binder path.

Open questions for review

1. Dep mechanism + [docs] scope: binder via pip requirements.txt reusing the analysis extra, or a conda environment.yml? And should the [docs] group stay mkdocs-only (binder reads analysis instead), or should a single extra cover both? Options: (a) pip + reuse analysis; (b) pip + new notebooks extra; (c) conda environment.yml.
2. In-scope set: confirm the three binder-able notebooks above are all still wanted, or is any of them deprecated/to be removed rather than refreshed?
3. Public data: confirm the current anonymous, publicly-reachable store path for the ATL06 aggregation the read notebooks should point at (the old s3://xagg/... bucket vs the source.coop mirror), so the data is anonymously reachable per the binder requirement. Should a small in-tree sample store be added as a fallback?
4. Lambda-dispatch cell in jupyterhub_example: (a) delete the agg(backend="lambda") cell, (b) keep it as a non-executing markdown/raw illustration, or (c) split that notebook so binder only ships the read/viz half?

---

Generated by Claude Code

[espg]

Dep mechanism + [docs] scope

Lean on this? I tend towards conda env since it allows better bundling of non-python deps, and is faster to build when using mamba.

For the [docs] group decision-- [docs] doesn't have much overlap so keep it mkdocs only. We should use/extend the [analysis] group instead of defining a new group.

In-scope set

jupyterhub_example shouldn't have the lambda invoke (section 6) -- only the cryocloud notebook should demo this (and shouldn't call to binder for that notebook). Yes, three binder notebooks, plus cryo notebook updated to be current with the api

Public data

References should all be to our source.coop data (the various zarrs live at https://source.coop/englacial/zagg/benchmarks )

Lambda-dispatch cell in jupyterhub_example

as mentioned above, remove it-- only lambda example is in cryo cloud. If there's functionality shown in jupyterhub_example that isn't in the cryo notebook, move it over to the cryo notebook