support ellipsoidal healpix and more indexing schemes through healpix-geo

[keewis]

This adds support for ellipsoidal healpix to the healpix view, by replacing @hcsmap/healpix with healpix-geo. This change in libraries also allows it to support different indexing schemes such as ring or zuniq, and means that we're not limited to the lossless range of number ($0 <= n < 2^{53}$)

There's a bunch of problems right now (my javascript / typescript knowledge is pretty limited):

• rendering is pretty slow, I suspect crossing the typescript / wasm boundary adds a lot of overhead (in addition to the more expensive operations on ellipsoids). This could probably be reduced by allocating arrays in wasm memory
• I don't appear to be able to store the parsed ellipsoid object, so every function call has to parse the ellipsoid object again (more overhead)
• typing of the functions in the javascript bindings of healpix-geo could be improved, right now I need a bunch of type overrides (but maybe that's just because I don't know as much about the type system)

Either way, hopefully this helps you unblock #191

[Karinon]

Hi keewis, thank you for the PR.

Since I am also at the KM-summit, I will not be able to check your PR in-depth this week, but we will probably meet at some point during the week.

Maybe we should first talk about healpix-geo. From what I could find you are the maintainer of this lib, right?

I think that the type-definitions you put into Healpix.vue should be rather be part of healpix-geo.

I also tried to store healpixGeo.parseEllipsoid(ellipsoid) and failed. With some help of AI it seems that the parseEllipsoid is not even necessary and that the code also worked simply with ellipsoid (if we ignore the Typescript-errors for now).

Regarding your TSchemeNamespace

type TSchemeNamespace = {
  lonLatToHealpix: (
    lon: number,
    lat: number,
    level: number,
    ellipsoid: healpixGeo.EllipsoidLike
  ) => bigint;
  healpixToLonLat: (
    ipix: bigint,
    level: number,
    ellipsoid: healpixGeo.EllipsoidLike
  ) => healpixGeo.Coordinate;
  vertex: (
    ipix: bigint,
    level: number,
    u: number,
    v: number,
    ellipsoid: healpixGeo.EllipsoidLike
  ) => healpixGeo.Coordinate;
};

Are those really always the right types? E.g. from what I could see in your healpix-geo, zuniq has the following signature for vertex

    static vertex(hash: bigint, u: number, v: number, ellipsoid?: EllipsoidLike | null): Coordinate;

So depth/level (why different names?) is missing. Is this on purpose?

I am not very familiar with WASM and typical library patterns for WASM, but I could maybe do a PR next week if you need some help.

[Karinon]

Also a trivial nitpick: I would like to have the JS console clean on the production-gridlook. So I would prefer to have no console.logs in the pushed code.

[keewis]

but we will probably meet at some point during the week.

yes, definitely

I think that the type-definitions you put into Healpix.vue should be rather be part of healpix-geo.

This for sure is a hack, I agree that this should not be needed in calling code (that was the intention behind problem 3 I mentioned in the initial post). The only reason it is there is to get the typing to resolve, and if you have any input for getting this right I'd be happy to take that.

With some help of AI it seems that the parseEllipsoid is not even necessary

That's interesting, I remember it failing when I tried it. I'd be happy to remove that call, as that removes some of the overhead I mentioned.

from what I could see in your healpix-geo, zuniq has the following signature for vertex

I didn't test zuniq, but that indexing scheme combines cell id and the refinement level into a single integer, so indeed the missing level is correct (once again, that's a hack). It does mean that I'll need to look at the code using nside, though: it shouldn't be passed for zuniq (or maybe healpix-geo.zuniq.* should take but ignore that parameter). Relatedly, I was going to replace nside with refinement level in the hover column (and in the code) but didn't get to it, yet.

depth/level (why different names?)

Mainly historical, depth is from the space-filling curve (z-order), whereas level is short for "refinement level", which is the official term used for DGGS. As the js bindings are new we shouldn't need to deal with that, so I'll just change that to level.

For the console.log I agree, it looks like I missed two instances when cleaning up.

[Karinon]

I had another talk with Codex on this topic. Especially about the healpix-geo interface, I will just paste the output I got:

I reviewed the WASM-facing API and the Gridlook call site, ignoring the math as requested.

Current Shape
The package exposes three static “scheme classes”: nested, ring, and zuniq. nested/ring have depth-taking methods like vertex(hash, depth, u, v, ellipsoid), while zuniq omits depth for reverse lookups because depth is encoded in the zuniq hash: nested.rs, ring.rs, zuniq.rs.

The most serious interface problem is that Gridlook casts every scheme to one shared signature with depth: Healpix.vue. That breaks for zuniq.vertex(...) and zuniq.healpixToLonLat(...); I confirmed the wrong call shape throws invalid dynamic union value at runtime.

Main Improvements

1. Add a unified high-level grid/context API.

const grid = healpixGeo.createGrid({
  scheme: "nested",
  depth,
  ellipsoid,
});

grid.vertex(cell, u, v);
grid.center(cell);
grid.cellAt(lon, lat);
grid.bitCombine(i, j);

This would remove dynamic casts in Gridlook, hide the zuniq signature difference, cache depth/layer state, and make call sites much harder to misuse.

2. Replace the current ellipsoid value API with a reusable handle.

Right now parseEllipsoid(...) returns a WASM-backed union, and passing it into a function consumes/moves it. Gridlook works around that by reparsing before every call: Healpix.vue. I confirmed reusing one parsed ellipsoid throws invalid dynamic union value.

Prefer:

const ellipsoid = healpixGeo.Ellipsoid.from({ semi_major_axis, inverse_flattening });
const grid = healpixGeo.createGrid({ scheme, depth, ellipsoid });

Rust-side, that means exported functions should borrow a handle, not consume an enum value. The current EllipsoidLike/parseEllipsoid surface is at ellipsoid.rs.

3. Add bulk APIs for hot paths.

Gridlook calls vertex inside nested loops for geometry generation: Healpix.vue. That crosses the JS/WASM boundary and allocates a WASM Coordinate object for every vertex.

A better public interface would expose something like:

grid.fillVertices(cell, steps, outLonLat);
grid.vertices(cell, steps); // returns Float64Array or Float32Array
grid.cellsAt(lonLatArray, outCells);

This keeps rendering in JS/Three, but moves repeated coordinate generation behind one WASM call.

4. Return plain values or typed arrays, not WASM-managed Coordinate objects, for high-frequency calls.

Coordinate is currently a generated class with free() and mutable properties: pkg/index.d.ts. That is awkward for value data and creates lifetime/GC pressure. For public ergonomic APIs, prefer [lon, lat], { lon, lat }, or typed-array output. Keep Coordinate only for low-level compatibility if needed.

5. Make depth/nside explicit and validated.

The API takes depth: number, while Gridlook stores nside and repeatedly computes Math.log2(nside): Healpix.vue. Add either fromNside(...), depthToNside(...), and nsideToDepth(...), or let createGrid accept either depth or nside and throw clear RangeErrors for non-integer/invalid values.

6. Separate plain input types from parsed WASM types.

The generated type says parseEllipsoid(obj: any): EllipsoidLike: pkg/index.d.ts. That hides the actual accepted JS shapes. Export something closer to:

type EllipsoidInput =
  | { radius: number }
  | { semi_major_axis: number; inverse_flattening: number }
  | { semi_major_axis: number; semi_minor_axis: number };

Then reserve Ellipsoid for the parsed reusable handle.

Priority
Highest-value path: add createGrid(...) plus reusable Ellipsoid handle first. Then add fillVertices(...)/batch APIs for the geometry path. Keep existing static functions as low-level compatibility wrappers, but stop making Gridlook consume them directly.

Verification: I ran npm run build:node to inspect the generated declarations and runtime wrappers. No source files were edited.

[espg]

Chiming in as an interested downstream user! I maintain mortie, a packed-morton HEALPix indexing spec/library — the packed words are close cousins of zuniq: left-aligned path bits, self-describing order, curve-ordered sorts. Additionally, I'm developing a writer/reader pair (zagg / moczarr) that produce sparse HEALPix zarr stores, and just started building a shard/coverage viewer on a gridlook fork.

This PR is directly enabling for that work — two of its three headline features are things I'd otherwise have to hand-roll: ids past 2^53 (my stores are order-24+ where number silently breaks), and the multi-scheme support (a morton word → nested/zuniq mapping is a few bit ops, so healpix-geo becomes a near-native backend for morton-encoded data). There's a write up / plan on how to build on this PR in espg/gridlook#8, so count me as an advocate for merging.

Happy to help get it there—

• Testing: I can exercise it against real high-order sparse datasets (the >2^53 regime, and the spherical mode specifically — my data pins the sphere convention, so I'd cover the non-ellipsoid path you presumably test least).
• Perf: on the issues you flagged — caching the parsed ellipsoid across calls seems like the first win; mortie's kernels are Rust, so I'm glad to look at the wasm-boundary/allocation side in healpix-geo itself if that's where the fix belongs.
• Review: happy to be a second pair of eyes on the TS typing overrides if useful.

Is there anything specific blocking this from your side that a downstream tester/collaborator could knock out?

[keewis]

thanks for the offer to help. I think if you have experience with WASM and rust it would be most helpful if you could check / review the js / ts bindings of healpix-geo to see if I'm doing anything wrong or inefficiently, and possibly we can try to replace the explicit loops in the gridlook code in this PR with building a region of data in the WASM memory and performing vectorized operations (provided that's more efficient).

I also am not as sure when it comes to the way I'm building the packages: am I using the right target (bundler), with a manual implementation of the js_namespaces feature since I couldn't get that to work with bundler specifically. This last point might become less important if we go with the grid object suggestion from above to abstract away the level parameter and the dispatching across the indexing scheme.

There's plenty of spherical datasets around (the example datasets, for example), so at least from that perspective this should be fine. That's not to say that testing with your data wouldn't be helpful either way.