rename parent_morton event field to shard_key (#24)

[espg]

Closes #24

What this does

Hard-renames the grid-neutral shard identifier in the Lambda event payload from the HEALPix-flavored parent_morton to shard_key, and makes the HEALPix-specific child_order optional for non-HEALPix grids (still required/forwarded for HEALPix). No back-compat aliases — a clean break, per @espg.

Checklist

• Rename parent_morton → shard_key end-to-end (event field + the ProcessingMetadata contract + handler/runner logging & error responses)
• Make child_order optional for non-HEALPix (mirror existing parent_order handling); required for HEALPix
• Update the ATL templates if needed (determination below)
• Update all other parent_morton references (docs, auth example)
• Tests for the new event shape (HEALPix with child_order, rectilinear without; gate accepts shard_key, rejects missing / legacy field)

§1 by-name authorization (deployment/aws/)

CLAUDE.md §1 forbids editing deployment/aws/ unless an issue names it. @espg explicitly authorized editing deployment/aws/lambda_handler.py by name and asked that the handler and runner be kept in sync: #24 (comment) ("yes, edit the lambda_handler; I want the lambda_handler and runner kept in sync for this edit" and "hard rename").

Approach

Kept lambda_handler.py and runner.py in lockstep on the wire contract:

• deployment/aws/lambda_handler.py — event field parent_morton → shard_key (docstring, structured logging, the required-params gate, dispatch into process_shard, and both error/result payloads). child_order dropped from the unconditional required list and instead required only when the resolved config's grid type is healpix:

  grid_type = config.output.get("grid", {}).get("type", "healpix")
  if grid_type == "healpix" and "child_order" not in event:
      return {"statusCode": 400, ...}  # Missing required parameters: child_order

  parent_order stays in the base required list (the orchestrator always sends the key — None for non-HEALPix), and the dispatch now reads event.get("parent_order") (was a hard index), matching the setup path.

• src/zagg/runner.py — agg() now computes child_order only for HEALPix (get_child_order(config) if grid_type == "healpix" else None), so a rectilinear run no longer raises output.grid.child_order is required. _invoke_lambda_cell emits shard_key (was parent_morton) and only includes child_order in the event when it is set; its result dict key morton → shard_key (and the one consumer log line).

• src/zagg/schema.py + src/zagg/processing.py — the ProcessingMetadata TypedDict field and the process_shard metadata key are renamed parent_morton → shard_key (the process_shard docstring already advertised shard_key, so this resolves a latent mismatch the handler read across at metadata["parent_morton"]). The internal local/param in process_shard/_read_group is renamed too, since the function is already grid-agnostic (shard_key is its parameter). The deprecated, explicitly HEALPix-flavored process_morton_cell(parent_morton=...) keeps its parent_morton name intentionally.

• src/zagg/auth.py, docs/deployment/lambda.md — event-payload examples/tables updated; the lambda doc's "rename tracked in Reconcile Catalog API #24, not done here" note is rewritten to describe the landed grid-neutral schema.

ATL templates: no change needed

@espg asked whether the ATL config templates need updating. They do not:

• parent_morton never appears in src/zagg/configs/atl06.yaml or atl06_polar.yaml — it's purely an event field, never a config field.
• The child_order in atl06.yaml is the HEALPix grid-spec field (output.grid.child_order: 12), which is legitimate and stays. atl06_polar.yaml is a rectilinear grid and already has no child_order — which is exactly the case the optional-child_order change now supports end-to-end.

What was not touched (deliberately, out of scope): docs/design/architecture.md (internal morton algorithm prose), tests/conftest.py (a local morton variable in a fixture), docs/index.md (a process_morton_cell(parent_morton=...) call — the kept legacy API), and the morton DataFrame coordinate column (a real grid coord, unrelated to the event field).

How tested

• New tests/test_lambda_handler.py (loads the handler by path): missing shard_key → 400; legacy parent_morton no longer accepted → 400; HEALPix event missing child_order → 400; HEALPix event (with child_order) and rectilinear event (no child_order, parent_order=None) both dispatch and the shard key reaches process_shard.
• New TestInvokeLambdaCellEvent in tests/test_runner.py: the per-cell event built by _invoke_lambda_cell carries shard_key (no parent_morton); child_order present for HEALPix, omitted when None.
• Updated the _invoke_lambda_cell stub in tests/test_runner_concurrency.py to the new shard_key result key.
• Full suite: uv run pytest -v → 256 passed, 1 skipped (pre-existing skip). uv run ruff check src tests → clean. mypy on the changed source files → 0 new errors vs the main baseline (6 pre-existing, all missing-stub / unrelated).

Questions for review

• ruff format --check drift (pre-existing, not from this PR). On main HEAD (7cabbfc), ruff format --check already reports processing.py, runner.py, schema.py, lambda_handler.py, and test_runner.py as "would reformat" — true with both the env's ruff and the CI-pinned v0.14.10. I verified this against a clean checkout before my edits, so it's standing drift, not something I introduced. Per the "don't fix unrelated pre-existing CI failures" rule I did not mass-reformat those files; my added lines follow the surrounding style and my new test_lambda_handler.py is already ruff-format-clean. Flagging in case you'd like a separate format-only sweep.
• child_order still sent in setup-mode events. I scoped the optional/omit behavior to the per-cell process event (_invoke_lambda_cell). The setup event still includes child_order (value None for non-HEALPix); the setup handler only reads it inside the grid_type == "healpix" branch, so None is harmless there. Say the word if you'd like setup made symmetric (omit-when-None) too.

---

Generated by Claude Code

[espg]

🤖 from Claude (review)

Fresh-context adversarial self-review of cb35564. Traced the data flow through from_config → process_shard and both the runner and handler gates, and ran the changed tests.

No blocking defects. The hard rename is complete and the child_order-optional logic is correct and symmetric.

What I verified:

• No stray parent_morton event-field references. Remaining hits are all intentional: processing.py (the deprecated process_morton_cell param), tests/conftest.py (local morton var), docs/index.md:84 (a process_morton_cell(parent_morton=...) call — the kept legacy API, not an event), and docs/design/architecture.md:125 (internal morton-algorithm prose). None are event-field usages.
• child_order gate is symmetric. Runner (agg) and handler (_handle_process) both key off config.output.get("grid", {}).get("type", "healpix"). A HEALPix run cannot slip through without child_order (runner's get_child_order raises; handler independently re-gates). Rectilinear is not wrongly rejected (runner sets child_order=None and omits the field; handler gate only fires for healpix). Confirmed atl06→healpix, atl06_polar→rectilinear.
• event.get("parent_order") introduces no None bug. from_config prefers grid_cfg["parent_order"] from config and only falls back to the kwarg for HEALPix; rectilinear ignores it, so None is harmless. parent_order stays required (key always sent by the orchestrator).
• Result-key rename morton → shard_key has exactly one consumer (the per-cell error-warning log), correctly updated; the concurrency-test stub was updated to match. metadata["shard_key"] matches the renamed ProcessingMetadata field.
• Tests genuinely assert the rename — legacy parent_morton → 400, missing shard_key → 400, HEALPix child_order gate, and both dispatch shapes.

Standing (non-blocking, already disclosed in the PR body):

• The setup-mode event still sends child_order unconditionally (None for rectilinear). Out of scope for this process-mode rename and harmless (the setup handler reads it only inside the healpix branch). Left for a follow-up if/when setup mode is generalized to non-HEALPix.
• The pre-existing ruff format drift on the touched modules (present on main before this PR) is left untouched per "don't fix unrelated CI failures."

---

Generated by Claude Code

[espg]

🤖 from Claude

Status: complete and green — single-phase small-fix, all phases done. CI: ruff, test (3.12), test (3.13), and build all passing (arm64 layer build still finishing; deploy/pr-preview skipped on the draft). Self-review found no blocking defects. Marking waiting for your review/merge.

Two non-blocking items are flagged under "Questions for review" in the PR body: (1) pre-existing ruff format drift on the touched modules (present on main before this PR — left untouched per the "don't fix unrelated CI failures" rule; the ruff check gate is green), and (2) whether you want the setup-mode event made symmetric (omit child_order when None) as a follow-up. And to answer your explicit question: the ATL templates did not need changes — parent_morton never appears in them (it's an event field only), and child_order in atl06.yaml is the legitimate HEALPix grid-spec field while the rectilinear atl06_polar.yaml already omits it.

---

Generated by Claude Code

[espg]

🤖 from Claude

Resolved the merge conflict — the branch is no longer dirty.

Cause: PR #33 (the #30 sort/hash grouping + Arrow handoff + experimental kernel reducer) merged to main after this branch was cut, and it rewrote the same process_shard/_read_group region of src/zagg/processing.py that this PR renames. processing.py was the only conflicting file.

How resolved: merged origin/main into the branch (not rebased — §1 forbids force-push) in commit 48c13dc. Took main's version of processing.py wholesale (so all of #33's grouping/arrow/kernel/nan-contract work is preserved intact) and re-applied only this PR's rename onto it: the internal parent_morton local/param + the "parent_morton" metadata key → shard_key, across _read_group (kept its arrow= param from #33) and process_shard (log strings "morton"→"shard" too). The deprecated process_morton_cell(parent_morton=…) keeps its name, as before. No other file conflicted.

Verified after merge: full suite 272 passed, 1 skipped (the spherely-fork skip), ruff check src tests clean. The metadata key now written by process_shard ("shard_key") lines up with the renamed ProcessingMetadata field and the handler's metadata["shard_key"] read.

Still waiting on your merge. (Note: the pre-existing ruff format --check whole-file drift flagged in the PR body is unchanged — not introduced here.)

---

Generated by Claude Code