v1.0: packaging & extras audit (pyproject extras + wheel matrix)

[espg]

🤖 from Claude

Sub-issue of #48 (v1.0 sweep), Tier-2.

Scope

Confirm the build/packaging surface is correct and minimal before the 1.x contract:

• Verify the runtime dependency set stays numpy-only; that pandas/pyarrow/cdshealpix and the notebook stack live only in the pandas/pyarrow/test/bench/docs extras (pyproject.toml).
• Audit the build-wheels.yml matrix (abi3-py310; the 3.10/3.11/3.12 + linux/macos/windows/arm coverage) and the import smoke test on each wheel.
• Confirm the maturin-only build path (no setuptools resurrection; stale setup.py/setup.cfg/*~ removed) and that the dynamic, tag-driven version is single-sourced from Cargo.toml.

Acceptance criteria

A short audit note + any minimal pyproject.toml/workflow fixes; no change to the tag-driven publish pipeline without explicit sign-off.

Refs #48.

[espg]

Verify the runtime dependency set stays numpy-only; that pandas/pyarrow/cdshealpix and the notebook stack live only in the pandas/pyarrow/test/bench/docs extras (pyproject.toml).

yes. We can add subpackage deps like [notebook] , [tests], etc... but runtime deps stay numpy.

Confirm the maturin-only build path (no setuptools resurrection; stale setup.py/setup.cfg/*~ removed) and that the dynamic, tag-driven version is single-sourced from Cargo.toml.

I strongly dislike having to manually bump cargo.toml everytime that I want to cut a new release version. The python single source versioning is great, and works well-- tag a branch with the new version to release, and everything updates correctly. This should be the case for the cargo.toml version update to. I understand that this is out of the python single source versioning realm, but surely we can define a workflow that updates the cargo.toml file in a similar fashion for what we do with updating the change log (i.e., before we move towards building the release wheels)???

[espg]

🤖 from Claude

Recommendation: mirror the existing changelog-update job — on a *.*.* tag push, derive the version from the tag, write it into Cargo.toml's version = line, and commit it before the wheel build runs. This replaces today's "fail if out of sync" guard with "sync automatically", so Cargo.toml is never hand-edited and the wheel always matches the tag.

Point 1 — confirmed. Runtime deps stay numpy-only: pyproject.toml:21-23 lists only numpy>=1.20 under [project].dependencies, and dynamic = ["version"] (pyproject.toml:7) keeps the version single-sourced via the maturin backend (pyproject.toml:1-3). pandas/pyarrow/cdshealpix and the notebook stack all live under [project.optional-dependencies] (pyproject.toml:29-72). Adding further sub-package extras like [notebook]/[tests] is purely additive and doesn't touch the runtime set — fine to do.

Point 2 — the manual Cargo.toml bump. You're right that this is the one spot breaking the tag-driven flow. The mechanism that forces it today is explicit: build-wheels.yml:216-226 (and the duplicated copies at :257-267 and :301-311) is a Verify release tag matches Cargo.toml version step that hard-fails the release if the tag and Cargo.toml's version = differ. That's the friction you're describing — maturin reads Cargo.toml at build time as the dynamic version source, so today the only way through is to bump Cargo.toml by hand before tagging.

The clean fix is to make the workflow do the bump instead of asserting it, mirroring the changelog job (build-wheels.yml:14-192), which already: parses the tag (:28-31, tag=${GITHUB_REF#refs/tags/}), edits a tracked file in place (:127-171), and commits it back with a bot identity + git push origin main (:173-192). A version sync reuses that exact pattern. Two variants:

• (1) In-place edit in the existing release flow (simplest). Add a sync-version step that runs needs- before the build jobs (the build jobs already gate on needs: [update-changelog], :196/:244/:286). It does TAG=${GITHUB_REF#refs/tags/} then either:
  • sed -E -i 's/^version = ".*"/version = "'"$TAG"'"/' Cargo.toml — zero new tooling, same sed/grep idiom already used in the verify step (:221); or
  • cargo set-version "$TAG" (from the cargo-edit crate) — semver-aware and won't mismatch the version = regex, but adds a toolchain install + a new dev tool (a §4 dependency-discussion item, so flagging rather than assuming).
    Then commit/push back exactly like :173-192, and replace the three verify-and-fail steps with this. Tradeoff: smallest diff, but the bump logic would either be folded into update-changelog or duplicated alongside it.
• (2) A dedicated sync-version job that the build jobs needs:. Cleaner separation and a single source of truth for the bump; the changelog and version commits become two ordered steps in one pre-build job (or two jobs with an explicit needs: order). Tradeoff: slightly more YAML and one more ordering edge to get right, but no duplicated logic and easy to reason about.

I'd lean toward (1) with sed folded into (or sequenced right after) update-changelog: it's the least new surface, matches the existing sed idiom, and avoids a new dependency.

Risks / constraints to flag:

• This is a change to the tag-driven publish pipeline and .github/workflows/build-wheels.yml, which CLAUDE.md §1 and §7 say must not be touched without explicit sign-off. So this stays a discuss; the actual edit would be a follow-up implement only after you greenlight.
• Ordering vs. the build. maturin reads Cargo.toml at build time, so the bump must land before the maturin-action build steps (:228-234, :269-276, :313-317). If we commit-and-push the bump but the build jobs actions/checkout@v4 (:206 etc.) check out the tag ref, they'll get the pre-bump tree — so either the build jobs must checkout the post-bump commit, or the bump must be applied ephemerally in each build job's checkout (modify-but-don't-commit). This is the main correctness detail to nail down.
• Keeping main in sync between releases. The changelog job pushes back to main (:181-191); if we likewise commit the bump to main, Cargo.toml's committed version drifts to the released value between releases (matching how the Python side already behaves implicitly). If instead we only modify it ephemerally for the build, Cargo.toml on main stays at its dev value and never needs a hand-edit — but then the published wheel's version comes purely from the tag, not from the committed file. Both are viable; they're just different contracts.

Questions (answerable in one pass):

1. Once greenlit, OK to add a sync-version step to build-wheels.yml and remove the three verify-and-fail steps (:216-226, :257-267, :301-311)?
2. Prefer plain sed (no new tooling) or cargo set-version (semver-aware, but a new cargo-edit dev dependency per §4)?
3. Should the bumped Cargo.toml be committed back to the branch (drifts the committed version, like the changelog push to main), or only modified ephemerally in the wheel-build checkout (committed file stays at its dev value)?

---

Generated by Claude Code

[espg]

🤖 from Claude

Point 1 — confirmed, runtime stays numpy-only. [project].dependencies lists only numpy>=1.20 (pyproject.toml:21-23); the version is dynamic = ["version"] (pyproject.toml:7) sourced through the maturin backend (pyproject.toml:1-3). Everything heavier is under [project.optional-dependencies] (pyproject.toml:29-72): pandas, pyarrow, test (pulls cdshealpix/pandas/pyarrow), bench, docs (the notebook stack). Adding further sub-package extras like [notebook]/[tests] is purely additive and never touches the runtime set — fine to do.

Point 2 — making the Cargo.toml bump tag-driven instead of hand-bumped. You're right that this is the single spot breaking the tag-driven flow, and the mechanism forcing the manual bump is explicit: a Verify release tag matches Cargo.toml version step that hard-fails the release when the tag differs from the version = line, duplicated in all three build jobs at build-wheels.yml:216-226, :257-267, and :301-311 (it greps Cargo.toml:3). maturin reads Cargo.toml as the dynamic version source at build time, so today the only way past that guard is to edit Cargo.toml by hand before tagging.

The clean fix is to make the workflow do the bump rather than assert it — exactly mirroring the changelog job (build-wheels.yml:14-192), which already parses the tag (:30-31, tag=${GITHUB_REF#refs/tags/}), edits a tracked file in place (:127-171), and commits it back with a bot identity + git push origin main (:173-192).

Three concrete approaches:

• (1) sed the version = line, mirroring the changelog job (recommended). Add a pre-build step TAG=${GITHUB_REF#refs/tags/}; sed -E -i 's/^version = ".*"/version = "'"$TAG"'"/' Cargo.toml, then replace the three verify-and-fail steps with it. Zero new tooling — it reuses the exact sed/grep idiom already in the verify step (:221) and the changelog job. Tradeoff: a regex bump rather than a semver-aware one (a malformed tag would write a malformed version), and the logic either folds into update-changelog or sits right beside it.
• (2) cargo set-version <tag> (from cargo-edit). Semver-aware, won't mismatch the version = regex. Tradeoff: adds a cargo-edit install to the release job — a new tool, so a §4 dependency-discussion item I'm flagging rather than assuming.
• (3) maturin dynamic version from the tag, dropping the Cargo.toml edit entirely. Source the wheel version from the tag at build time (env/maturin version override) so Cargo.toml is never rewritten. Tradeoff: most "single-source" in spirit, but Cargo.toml's committed version then permanently diverges from releases and cargo build/crates metadata stops matching the published wheel — a larger behavioral change than (1)/(2).

Recommendation: (1) — least new surface, matches the existing sed idiom, no new dependency, and it turns the friction step into the fix.

Guardrails (why this stays discuss). This is a change to .github/workflows/build-wheels.yml and the tag-driven publish pipeline, which §1/§7 say must not be touched without explicit sign-off — and the *.*.* tag push itself triggers the TestPyPI→PyPI publish. So the edit is a follow-up implement only after you greenlight. One correctness detail to nail down regardless of approach: the bump must land before the maturin-action build steps (:228-234, :269-276, :313-317), and the build jobs actions/checkout@v4 (:206/:250/:291) check out the tag ref — so either they must check out the post-bump commit, or the bump is applied ephemerally inside each build checkout (modify-but-don't-commit).

Questions (answerable in one pass):

1. Once greenlit, OK to add the bump step to build-wheels.yml and remove the three verify-and-fail steps (:216-226, :257-267, :301-311)?
2. Prefer (1) plain sed (no new tooling), (2) cargo set-version (semver-aware, new cargo-edit dev tool per §4), or (3) maturin dynamic-from-tag (no Cargo.toml edit, but committed version diverges)?
3. Should the bumped Cargo.toml be committed back (drifts the committed version, like the changelog push to main), or applied ephemerally only in the wheel-build checkout (committed file stays at its dev value)?

---

Generated by Claude Code

[espg]

Once greenlit, OK to add a sync-version step to build-wheels.yml and remove the three verify-and-fail steps

yes.

Prefer plain sed (no new tooling) or cargo set-version (semver-aware, but a new cargo-edit dev dependency per §4)?

plain sed (no new tooling)

Should the bumped Cargo.toml be committed back to the branch (drifts the committed version, like the changelog push to main)

Yes, Cargo.toml should be committed back to main

[espg]

See also comment on #66 about bumping binder reqs when cutting a new release