gridlook-jupyter: server extension + S3 range proxy (phase 4 of issue #1)

.github/workflows/jupyter.yml

@@ -0,0 +1,58 @@
+name: Jupyter
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - "jupyter/**"
+      - "src/**"
+      - "package.json"
+      - "package-lock.json"
+      - ".github/workflows/jupyter.yml"
+  pull_request:
+    paths:
+      - "jupyter/**"
+      - "src/**"
+      - "package.json"
+      - "package-lock.json"
+      - ".github/workflows/jupyter.yml"
+
+jobs:
+  build-and-test:
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Set up node
+        uses: actions/setup-node@v4
+        with:
+          node-version: "24"
+
+      - name: Set up uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          python-version: "3.12"
+
+      - name: Build the wheel (runs the vite build hook)
+        run: uv build --wheel jupyter -o jupyter/dist-py
+
+      - name: Assert the wheel carries the SPA
+        run: unzip -l jupyter/dist-py/*.whl | grep -q "gridlook_jupyter/static/index.html"
+
+      - name: Assert the wheel ships no source maps
+        run: "! unzip -l jupyter/dist-py/*.whl | grep -q '\\.map$'"
+
+      - name: Install the wheel with test deps
+        # setup-uv's python-version input already created and activated .venv
+        run: uv pip install "$(ls jupyter/dist-py/*.whl)[test]" ruff
+
+      - name: Ruff
+        run: |
+          ruff check jupyter/gridlook_jupyter jupyter/tests jupyter/hatch_build.py
+          ruff format --check jupyter/gridlook_jupyter jupyter/tests jupyter/hatch_build.py
+
+      - name: Pytest
+        run: pytest jupyter/tests -v

.gitignore

@@ -28,3 +28,6 @@ tsconfig.tsbuildinfo
 *.njsproj
 *.sln
 *.sw?
+
+# uv lockfile from jupyter test runs (not committed for this package)
+jupyter/uv.lock

jupyter/.gitignore

@@ -0,0 +1,9 @@
+# Built SPA copied in by the wheel build hook (hatch_build.py)
+gridlook_jupyter/static/
+__pycache__/
+*.egg-info/
+.venv/
+dist-py/
+build/
+.pytest_cache/
+.ruff_cache/

jupyter/README.md

@@ -0,0 +1,95 @@
+# gridlook-jupyter
+
+A jupyter-server extension that serves the built [gridlook](../README.md) viewer inside a
+JupyterHub/Jupyter environment, plus a **streaming S3 byte-range proxy** so the browser can read
+private buckets through the hub's credentials — credentials never reach the browser, and the
+buckets never need to be public. Phase 4 of the viewer plan
+([espg/gridlook#1](https://github.com/espg/gridlook/issues/1)).
+
+The primary deployment target is [CryoCloud](https://cryointhecloud.com/) (a 2i2c-operated
+JupyterHub): the hub role holds the S3 read credentials, and gridlook runs against buckets that
+are hub-viewable but not public (e.g. zagg output stores — see englacial/zagg#301 for the
+cost-visualization use case).
+
+## Install
+
+```bash
+pip install ./jupyter        # from a gridlook checkout — needs node >= 24 on PATH
+# or
+pip install gridlook_jupyter-<version>-py3-none-any.whl
+```
+
+The wheel embeds the built Vite app as package data. **Building the wheel requires node/npm**
+(the build hook runs `npm ci && npm run build` at the repo root and copies `dist/` into the
+wheel); installing a pre-built wheel does not. The extension auto-enables on install via
+`jupyter_server_config.d`.
+
+Editable installs (`pip install -e ./jupyter`) skip the frontend build — point
+`GridlookProxy.static_dir` at a locally built `dist/` instead (see below).
+
+## Launch
+
+There is no launcher card yet (that is phase 5). Open the app by URL:
+
+```
+<your-server-base>/gridlook/
+```
+
+e.g. on a hub: `https://hub.example.org/user/<you>/gridlook/`.
+
+## Routes
+
+| Route                         | What                                                                                                                                    |
+| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
+| `/gridlook/`                  | the static gridlook SPA                                                                                                                 |
+| `/gridlook/api/health`        | tiny JSON probe (`{"extension": "gridlook-jupyter", ...}`)                                                                              |
+| `/gridlook/s3/<bucket>/<key>` | streaming S3 proxy — GET/HEAD only, `Range` pass-through (206), no LIST                                                                 |
+| `/gridlook/hive/…`            | **reserved** for the phase-6 moczarr virtual-store endpoint (an `open_hive()` AOI served as one flat zarr store) — not implemented here |
+
+## Configuration
+
+Allowlist-only auth: **the proxy is disabled until you configure buckets.** Requests for
+non-allowlisted buckets get a 403 naming the bucket; with an empty allowlist every proxy request
+gets a 403 saying the proxy is disabled.
+
+Via traitlets (`jupyter_server_config.py`, or `--GridlookProxy.…` on the command line):
+
+```python
+c.GridlookProxy.allowed_buckets = ["my-zagg-outputs"]
+c.GridlookProxy.region = "us-west-2"          # optional; ambient AWS config otherwise
+c.GridlookProxy.static_dir = "/path/to/dist"  # optional; dev override for the SPA files
+```
+
+Or environment variables (used only when the trait is not configured):
+
+```bash
+export GRIDLOOK_ALLOWED_BUCKETS="my-zagg-outputs,another-bucket"
+export GRIDLOOK_S3_REGION="us-west-2"
+```
+
+S3 credentials come from the ambient chain (instance/pod role, `AWS_*` env, shared config) —
+the standard hub setup. The proxy streams responses chunk-by-chunk and never buffers whole
+objects; there are no presigned URLs, so nothing credential-shaped is ever exposed to the
+browser.
+
+## `s3://` inputs in the app
+
+When the SPA is served under `/gridlook/` (it detects this by probing `api/health`), pasting an
+`s3://bucket/prefix` dataset URL rewrites it to the proxy path (`…/gridlook/s3/bucket/prefix`)
+automatically. Served standalone (dev server, static hosting), `s3://` inputs are left
+untouched. Pasting the proxy URL directly always works.
+
+## Development
+
+```bash
+cd jupyter
+uv venv && uv pip install -e ".[test]" ruff
+.venv/bin/pytest tests -v
+.venv/bin/ruff check gridlook_jupyter tests hatch_build.py
+# run against a dev-built frontend:
+npm run build   # at the repo root
+jupyter server --GridlookProxy.static_dir="$(pwd)/../dist" --GridlookProxy.allowed_buckets='["my-bucket"]'
+```
+
+Tests exercise the proxy against an obstore `LocalStore` via the `GridlookProxy.store_factory`
+seam — no real S3 needed.

jupyter/gridlook_jupyter/__init__.py

@@ -0,0 +1,17 @@
+"""gridlook-jupyter: serve the gridlook viewer and an S3 byte-range proxy from jupyter-server."""
+
+__version__ = "0.1.0"
+
+
+def _jupyter_server_extension_points():
+    return [{"module": "gridlook_jupyter"}]
+
+
+def _load_jupyter_server_extension(serverapp):
+    from .extension import load_extension
+
+    load_extension(serverapp)
+
+
+# Alias kept for older jupyter-server enable paths.
+load_jupyter_server_extension = _load_jupyter_server_extension

jupyter/gridlook_jupyter/config.py

@@ -0,0 +1,80 @@
+"""Traitlets config surface for the gridlook extension."""
+
+import os
+from pathlib import Path
+
+from traitlets import List, Unicode, default
+from traitlets.config import Configurable
+
+#: Bounded per-process cache of bucket -> store (buckets come from the allowlist,
+#: so this stays tiny; the bound is belt-and-braces).
+_MAX_CACHED_STORES = 64
+
+
+def default_store_factory(bucket: str, region: str | None):
+    """Build an obstore store for *bucket* using the ambient AWS credential chain."""
+    from obstore.store import S3Store
+
+    kwargs = {"region": region} if region else {}
+    return S3Store(bucket, **kwargs)
+
+
+class GridlookProxy(Configurable):
+    """Config for the gridlook S3 proxy (jupyter_server_config / CLI / env)."""
+
+    allowed_buckets = List(
+        Unicode(),
+        help=(
+            "Buckets the proxy may read from. Empty (the default) disables the "
+            "proxy entirely. Env fallback: GRIDLOOK_ALLOWED_BUCKETS (comma-separated), "
+            "used only when this trait is not configured."
+        ),
+    ).tag(config=True)
+
+    region = Unicode(
+        "",
+        help=(
+            "AWS region for the S3 stores. Empty defers to the ambient AWS "
+            "configuration. Env fallback: GRIDLOOK_S3_REGION."
+        ),
+    ).tag(config=True)
+
+    static_dir = Unicode(
+        "",
+        help=(
+            "Directory holding the built gridlook SPA. Defaults to the static/ "
+            "directory packaged in the wheel; point it at a repo dist/ for development."
+        ),
+    ).tag(config=True)
+
+    @default("allowed_buckets")
+    def _default_allowed_buckets(self):
+        env = os.environ.get("GRIDLOOK_ALLOWED_BUCKETS", "")
+        return [b.strip() for b in env.split(",") if b.strip()]
+
+    @default("region")
+    def _default_region(self):
+        return os.environ.get("GRIDLOOK_S3_REGION", "")
+
+    @default("static_dir")
+    def _default_static_dir(self):
+        return str(Path(__file__).parent / "static")
+
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        # Seam for tests: swap in e.g. an obstore LocalStore factory so the
+        # proxy can be exercised without real S3. Plain attribute, not a trait.
+        self.store_factory = default_store_factory
+        self._stores: dict[str, object] = {}
+
+    @property
+    def enabled(self) -> bool:
+        return bool(self.allowed_buckets)
+
+    def get_store(self, bucket: str):
+        """Return (building if needed) the store for an allowlisted bucket."""
+        if bucket not in self._stores:
+            if len(self._stores) >= _MAX_CACHED_STORES:
+                self._stores.clear()
+            self._stores[bucket] = self.store_factory(bucket, self.region or None)
+        return self._stores[bucket]

jupyter/gridlook_jupyter/extension.py

@@ -0,0 +1,45 @@
+"""Wires the gridlook routes into a running jupyter-server."""
+
+import re
+from pathlib import Path
+
+from jupyter_server.base.handlers import AuthenticatedFileHandler
+from jupyter_server.utils import url_path_join
+from tornado import web
+
+from .config import GridlookProxy
+from .handlers import HealthHandler, S3ProxyHandler
+
+
+def load_extension(serverapp):
+    proxy = GridlookProxy(parent=serverapp)
+    static_dir = proxy.static_dir
+    if not (Path(static_dir) / "index.html").exists():
+        serverapp.log.warning(
+            "gridlook-jupyter: no built SPA at %s — /gridlook/ will 404. "
+            "Install from a wheel, or set GridlookProxy.static_dir to a gridlook dist/.",
+            static_dir,
+        )
+
+    base = url_path_join(serverapp.base_url, "gridlook")
+    escaped = re.escape(base)
+    handlers = [
+        (escaped + r"/api/health", HealthHandler),
+        (escaped + r"/s3/([^/]+)/(.+)", S3ProxyHandler),
+        # /gridlook/hive/... is RESERVED: phase 6 mounts the moczarr virtual-store
+        # endpoint there (an open_hive() AOI served as one flat zarr store).
+        # Do not claim that namespace with other routes.
+        (escaped + r"$", web.RedirectHandler, {"url": base + "/"}),
+        (
+            escaped + r"/(.*)",
+            AuthenticatedFileHandler,
+            {"path": static_dir, "default_filename": "index.html"},
+        ),
+    ]
+    serverapp.web_app.settings["gridlook_proxy"] = proxy
+    serverapp.web_app.add_handlers(".*$", handlers)
+    serverapp.log.info(
+        "gridlook-jupyter loaded: app at %s/, proxy %s",
+        base,
+        f"enabled for buckets {proxy.allowed_buckets}" if proxy.enabled else "disabled",
+    )