morton index surface followup

[espg]

from #73

(4) from_latlon point-encode path not surfaced in the morton_index skin — PR #51

The pandas ExtensionArray and pyarrow ExtensionType skin only produces area cells (Kind::Area). The Rust kernel also supports Kind::Point encoding, but there is no points=True constructor parameter or any other public API surface for it. Users who want point-precision encoding (e.g. for observation-location indexing) have no public path. PR #51 flagged this explicitly.

(5) Arrow → pandas from_arrow hook not wired — PR #51

table.to_pandas() on a table with a MortonIndexDtype Arrow column yields a plain int64 Series, not a MortonIndexArray. The from_arrow hook on the dtype was deliberately skipped to stay in scope. This breaks round-tripping through Arrow-typed interchange (e.g. Parquet → pandas). PR #51 flagged it as a known gap.

[espg]

🤖 from Claude

Implementation plan for the two morton_index skin gaps from #51. Recommendation: ship both gaps as one PR — they touch disjoint code paths (point-encode is from_latlon/Rust; __from_arrow__ is dtype-only), share the same test module, and (5) is small enough that a separate PR isn't worth the overhead.

Current state (what exists / what's missing)

• Point encode is kernel-only, no binding. The kernel has encode_point(base_cell, tuples) (src_rust/src/decimal_morton.rs:206), which produces an order-29 Kind::Point word and requires all 29 stored tuples. But no PyO3 binding exposes it — lib.rs:1100-1108 registers only the area-producing rust_mi_from_nested, rust_mi_encode, etc. There is also no from_nested-to-point kernel helper (from_nested at decimal_morton.rs:490 is hardcoded Kind::Area, per its doc at :483).
• from_latlon hardcodes the area path. mortie/morton_index.py:131-145: rust_ang2pix(order) → nested → rust_mi_from_nested(nested, order), which always yields Kind::Area (kernel doc decimal_morton.rs:169-170, :483-485). There is no points parameter anywhere in the skin (grep finds only this one from_latlon).
• __from_arrow__ is simply absent. MortonIndexDtype (mortie/morton_index.py:74-91) has name/type/construct_array_type but no __from_arrow__, so table.to_pandas() falls back to a plain int64 Series. The Arrow side already has the pieces a hook would reuse: MortonIndexType.to_pandas_dtype() (mortie/arrow.py:97-101) and to_morton_index(array) (mortie/arrow.py:134-146).

Phase (a) — thread points=True → Kind::Point

1. Rust binding. Add a vectorized rust_mi_from_nested_point(nested_array, depth=29) (or extend rust_mi_from_nested with a point: bool) in src_rust/src/lib.rs mirroring rust_mi_from_nested (lib.rs:828-849), and register it (lib.rs:1100). Because encode_point needs all 29 tuples, the simplest kernel addition is a from_nested_point(nested) helper in decimal_morton.rs that decomposes an order-29 nested id into its 29 tuples and calls encode_point — i.e. the point twin of from_nested (decimal_morton.rs:490). cargo test + a kernel unit test that kind_of(...) == Kind::Point.
2. Python surface. Thread a points=False flag through MortonIndexArray.from_latlon (mortie/morton_index.py:131-145): when points=True, hash at order 29 (point encoding is order-29-only) and route through the new rust_mi_from_nested_point instead of rust_mi_from_nested. Rebuild with maturin develop --release.
3. Tests (in mortie/tests/test_morton_index.py, next to the existing from_latlon test at :80): assert points=True words decode to Kind::Point (via _rustie.rust_mi_decode kinds, which already returns 0=area/1=point, lib.rs:974-981), assert the point word's order is 29 and that coarsening / to_nested still works, and assert points=False is unchanged (area).

Phase (b) — wire __from_arrow__ for Arrow → pandas round-trip

1. Add the hook. Add __from_arrow__(self, array) to MortonIndexDtype (mortie/morton_index.py:74-91). It receives a pyarrow Array or ChunkedArray of the extension type and must return a MortonIndexArray; delegate to the existing mortie.arrow.to_morton_index (arrow.py:134-146), iterating chunks for ChunkedArray and concatenating via _concat_same_type (morton_index.py:238-240). Keep the pyarrow import lazy (the dtype module must stay numpy-only importable — see _require_pandas/_require_pyarrow).
2. Tests (extend mortie/tests/test_arrow.py, which already covers parquet round-trips at TestParquet:94-115): table.to_pandas() on a morton_index-typed column yields a Series whose .dtype.name == "morton_index" and whose values are a MortonIndexArray with bit-identical words; and a full Parquet → read_table → to_pandas() round-trip preserves the words and dtype.

Files likely touched

• src_rust/src/decimal_morton.rs — new from_nested_point helper (near :206/:490).
• src_rust/src/lib.rs — new rust_mi_from_nested_point binding + registration (:828, :1100).
• mortie/morton_index.py — from_latlon points param (:131); MortonIndexDtype.__from_arrow__ (:74).
• mortie/tests/test_morton_index.py (point-encode) and mortie/tests/test_arrow.py (__from_arrow__).

Acceptance criteria

• MortonIndexArray.from_latlon(lat, lon, points=True) produces words that decode as Kind::Point at order 29; points=False (default) is unchanged and produces Kind::Area.
• table.to_pandas() on a table with a morton_index Arrow column yields a MortonIndexArray-backed Series (not int64), with words preserved.
• Parquet → read_table → to_pandas() round-trips words and the morton_index dtype.
• Green: maturin develop --release, cargo test, flake8 mortie --select=E9,F63,F7,F82, pytest -v.

Open questions

1. Constructor surface for points — (1) a points=False flag on from_latlon, or (2) a separate from_points(lat, lon) classmethod? Leaning toward (1) for the smallest surface, but (2) reads more clearly given point encoding is order-29-only.
2. Order for point encoding — encode_point is order-29 by definition. Options: (1) ignore the order arg when points=True (always 29); (2) raise if order != 29 is passed with points=True; (3) accept order and coarsen-after (defeats the point semantics). Preference: (1) or (2).
3. Point bridge shape — add a dedicated rust_mi_from_nested_point binding, or extend rust_mi_from_nested with a point: bool param? A separate function keeps each binding single-purpose (matches the existing rust_mi_* style); a flag avoids a near-duplicate.
4. __from_arrow__ chunked input — confirm the hook should handle both pa.Array and pa.ChunkedArray (the latter is what table.to_pandas() passes per column). Plan assumes yes.
5. One PR or split? Recommending one PR (both gaps, two-phase checklist). Happy to split (5) out if you'd prefer the point-encode change reviewed in isolation.

---

Generated by Claude Code

[espg]

Constructor surface for points

(1) a points=False flag on from_latlon

Order for point encoding — encode_point is order-29 by definition. Options: (1) ignore the order arg when points=True (always 29); (2) raise if order != 29 is passed with points=True; (3) accept order and coarsen-after (defeats the point semantics). Preference: (1) or (2).

sorry, is there a seperate encode_point function? if so... why does it even take an order arg?? If this is asking about the flag option above (in that function, which does take order), then probably raise in the most concretely defined behavior

Point bridge shape — add a dedicated rust_mi_from_nested_point binding, or extend rust_mi_from_nested with a point: bool param? A separate function keeps each binding single-purpose (matches the existing rust_mi_* style); a flag avoids a near-duplicate.

lean on this? I don't have a strong opinion...

confirm the hook should handle both pa.Array and pa.ChunkedArray (the latter is what table.to_pandas() passes per column). Plan assumes yes.

yes.

One PR or split?

One PR