[Feature]: Make multi-crate releases fast and forbid publish cycles

[shinaoka]

Type

Feature request / release-workflow design

What are you trying to do?

Make multi-crate tenferro releases fast, deterministic, and safely resumable without discovering package-order failures during irreversible publication or repeatedly running unrelated workspace tests.

The release skill should prepare the complete human handoff, including a guarded executable shell script, after the tagged release has passed its required validation.

What makes it hard today?

The v0.3.0 publication exposed two versioned workspace dev-dependencies that pointed forward in the publication order:

• tenferro-runtime -> (dev) tenferro-cpu, while tenferro-cpu -> tenferro-runtime is a normal dependency.
• tenferro-xla -> (dev) tenferro-einsum, while tenferro-einsum follows XLA in the publication order.

Cargo resolves versioned dev-dependencies during cargo package, even with --no-verify. The first edge therefore formed a publish-time cycle, and the second caused another forward-reference failure. Publication needed release-helper-only bootstrap patches after the immutable tag had already been partially published.

Repeated helper corrections also risk triggering expensive CPU/GPU workspace validation even though tagged Rust source and manifests have not changed. Human handoff currently requires manually assembling the publication command or wrapper instead of having the release skill generate a reviewed, resumable script.

Proposed direction

1. Forbid publish cycles and forward dev-dependencies

Add an explicit rule to REPOSITORY_RULES.md:

• A publishable workspace crate must not have a versioned normal, build, or dev-dependency on a publishable crate that appears later in the canonical publication order.
• The complete graph including dev-dependencies must not contain a publication cycle.
• Tests that require a higher layer or both sides of a dependency boundary must live in a publish = false workspace test crate, unless the canonical publication order can be changed without violating architecture.
• Published crates must remain packageable from their manifests without temporary local [patch.crates-io] bootstrap configuration.

Resolve the existing runtime/CPU and XLA/einsum violations. Prefer moving cross-layer tests into the minimum number of non-publishable test crates rather than retaining inverted test dependencies in published manifests.

Enforce the rule in scripts/check-publish-layout.py and repository contract tests so a future release fails before tagging.

2. Generate the human publication script from the release skill

Extend the existing tenferro-release-publish skill and canonical release workflow to generate a guarded shell script that:

• pins the selected helper and clean immutable release checkout;
• includes only explicitly approved new-package arguments;
• is restart-safe and uses the helper's registry/provenance checks;
• requires one exact lowercase y confirmation before irreversible publication;
• is written with mode 0700 and passes bash -n;
• never publishes when run by an agent; it is only a human-maintainer handoff artifact.

Do not create a second overlapping release skill.

3. Make release validation change-aware

Document and enforce a minimal validation matrix:

• Run full workspace, CPU/GPU, coverage, docs, and CI-parity validation once on the exact release commit before tagging.
• Treat successful required CI for that exact commit SHA as reusable canonical evidence. Do not rerun identical local or publication-time workspace tests when neither the commit nor required job configuration changed.
• Classify manifest-only changes before selecting validation:
  • package/workspace version, internal version requirement, description, repository, documentation, keywords, categories, or other publication metadata changes require metadata, publish-layout, package/archive, and non-uploading dry-run checks, but not unrelated CPU/GPU or full workspace tests when exact-SHA CI already passed;
  • dependency source/version, default feature, feature wiring, target configuration, build script, native library, or compiler-profile changes may alter compiled behavior and require affected tests plus the applicable CI tier;
  • ambiguous manifest changes take the safer affected-test path, not an automatic full-workspace rerun.
• For post-tag helper/workflow-only corrections that do not modify tagged package source or manifests, run focused helper tests, Python compilation, publish-layout checks, and real non-uploading cargo package / cargo publish --dry-run reproductions relevant to the failure.
• Do not rerun unrelated CPU/GPU or full workspace tests solely because the release helper, workflow documentation, generated shell handoff, or publication metadata changed.
• Any Rust source, dependency semantics, feature, backend, build configuration, or tagged-source change still requires the applicable normal validation and a new release commit/version as appropriate.

4. Move reusable policy into shared agent rules

Separate reusable release policy from tenferro-specific details:

• Move cross-repository rules for publication DAGs, forward versioned dev-dependencies, publish-cycle prevention, human-only publication, generated handoff scripts, restart safety, and change-aware validation into the appropriate common or Rust rule files in tensor4all-agent-rules.
• Keep tenferro-specific crate names, canonical order, approval package names, commands, and test-crate placement in REPOSITORY_RULES.md and the tenferro release workflow.
• Update tenferro references to consume the shared rules without vendoring or duplicating their normative text.
• Check sibling tensor4all repositories for compatible release guidance and avoid imposing tenferro-specific topology on them.

Relevant areas

• REPOSITORY_RULES.md
• scripts/check-publish-layout.py
• scripts/release-publish.py
• scripts/test-release-publish.py
• ai/contribution-workflows/release-publish.md
• Existing tenferro-release-publish skill and thin adapters
• tensor4all-agent-rules common/Rust release and repository policy
• crates/tenferro-runtime/Cargo.toml
• crates/tenferro-xla/Cargo.toml
• New or existing publish = false integration-test workspace package(s)

Dependency, backend, and API impact

• No public tensor API, backend behavior, AD semantics, feature flags, or external dependencies should change.
• Crate-boundary changes are limited to test ownership and non-publishable test crates.
• The safety boundary remains unchanged: agents validate and prepare; only a human maintainer may execute cargo publish.

Non-goals

• Do not alter or replace immutable published v0.3.0 archives or tags.
• Do not weaken new-package approval, archive-content, crates.io visibility, provenance, or unyanked-state checks.
• Do not skip the full pre-tag release validation.
• Do not add permanent release-time manifest rewrites or compatibility shims.

Acceptance criteria

• REPOSITORY_RULES.md explicitly forbids publication cycles and forward versioned dev-dependencies among publishable crates.
• scripts/check-publish-layout.py rejects both a normal/dev cycle and an acyclic forward dev-dependency, with focused RED/GREEN contract tests.
• Existing tenferro-runtime -> (dev) tenferro-cpu and tenferro-xla -> (dev) tenferro-einsum violations are removed from publishable manifests.
• Cross-layer tests retain equivalent coverage in one or more justified publish = false workspace test crates, or the issue records why a safe publication-order change is preferable.
• Every publishable crate can run cargo package --locked in canonical order without temporary local crates.io patches.
• The existing release skill generates a mode-0700, bash -n-clean, restart-safe human publication script with one exact lowercase y confirmation.
• The generated script carries the exact approved new-package arguments and cannot bypass helper provenance/registry gates.
• The workflow defines a deterministic change-aware validation matrix and focused tests prove helper-only and publication-metadata-only changes do not request unrelated full workspace or CPU/GPU validation.
• Successful required CI is reused only when it belongs to the exact release commit SHA and relevant CI configuration.
• Manifest classification distinguishes publication metadata from dependency, feature, build, target, native-library, and compiler-profile semantics.
• Applicable full or affected validation remains mandatory when tagged Rust source or compiled dependency/build/backend behavior changes.
• Cross-repository policy is added to the appropriate tensor4all-agent-rules common/Rust rules with focused validation.
• REPOSITORY_RULES.md retains only tenferro-specific constraints and references the shared policy without normative duplication.
• Relevant release workflow and curated worklog documentation are updated.

Verification

• Focused publish-layout fixtures for forward dev edges and mixed normal/dev cycles.
• Package all publishable crates in canonical order against an empty/local registry simulation or equivalent fixture that exposes unpublished forward dependencies.
• Exercise generated script cancellation and exact-y continuation without upload.
• Exercise initial and resumed helper runs with non-uploading dry-runs.
• Run repository-rules review and the validation tier selected by the final diff.
• Validate shared agent rules against tenferro and at least one compatible sibling-repository fixture or documented review case.

Provenance and related context

This issue is based on observed tenferro v0.3.0 publication failures, not external prototype code. Related release work is recorded in:

• fix: correct release target to v0.3.0 #1647 — corrected tenferro v0.3.0 release preparation
• fix: discover Cargo dry-run release archives #1648 — Cargo 1.97 release archive handling
• docs/worklogs/2026-07-04-release-publish-workflow.md

No external code or licensing material is proposed for reuse.

[shinaoka]

Implemented and merged via #1651 (merged 2026-08-09).

• Part 1: publish-safety checks in check-publish-layout.py (forward versioned normal/build/dev edge checks + publication-cycle detection); tenferro-runtime→tenferro-cpu and tenferro-xla→tenferro-einsum forward dev-deps moved to path-only (unversioned) — accepted deviation from the issue's publish = false test-crate preference (documented in the PR; cross-layer tests cannot relocate).
• Part 2: release-publish.py --generate-script emits the mode-0700, restart-safe, exact-y human publication script.
• Part 3: release-validation-policy.py change-aware validation lanes + fail-closed exact-SHA CI reuse (verify_release_ci).
• Part 4: REPOSITORY_RULES.md Publication Order section, skill mirrors, worklog docs/worklogs/2026-08-10-release-publish-safety.md, and shared policy in tensor4all-agent-rules (sibling PR rules: add publication and release safety to common repository rules tensor4all-agent-rules#10 merged).

Verification: 60+ tests (script execution under PTY, cancellation, exact-y continuation), publish-layout fixtures, package --locked green, check-pr-fast green, reviewer-gpt 3 rounds verdict READY.