Skip to content

Converge the release instead of executing it once - #209

Merged
shanev merged 2 commits into
mainfrom
fix/release-convergence
Aug 4, 2026
Merged

Converge the release instead of executing it once#209
shanev merged 2 commits into
mainfrom
fix/release-convergence

Conversation

@shanev

@shanev shanev commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Why

A release is retried by re-running the workflow or re-pushing the tag — and both assume every step is safe to run twice. Publishing isn't, so the recovery an operator reaches for first is guaranteed to fail.

That's how 0.24.0 ended up published to PyPI, missing from its GitHub release, and impossible to re-tag: the relay-bundling step failed after a successful upload, and the natural fix-and-re-tag hit 400 File already exists.

Reordering alone wouldn't fix this. There is always a last irreversible step, and uv publish uploads two files non-atomically — a wheel-succeeds/sdist-fails split is reachable wherever you put it. The retry semantics are the problem, not the ordering.

Four changes, ordered by what actually caused the incident

1. The corpus is enumerated, not listed. trace-v8-*.ndjson was spelled out in two workflows, the packaging check, the release tarball, and suites in another repo. The v8 → v9 rename was missed in release.yml — the one place no PR exercises. conformance_fixture_names() answers the question instead, so a rename is invisible to consumers.

2. The release path runs on every PR. Bundling moved from inline YAML into scripts/bundle-relay.sh, so a new release-dryrun job runs the identical code with the single irreversible action stubbed out. A step that only executes on a tag is discoverable only by breaking a release.

3. Publishing is last. Build and bundle both complete before the first irreversible byte leaves the runner.

4. Publishing converges. --check-url was documented as making re-runs idempotent and doesn't — it predicts from a CDN-cached index and compares against freshly built artifacts, but sdists aren't byte-reproducible, so a rebuild is a different file under the same name and the upload is attempted anyway. scripts/publish-pypi.sh asks the authority — is this version on PyPI — both before uploading and after a failed upload.

The dry run paid for itself immediately

Running it locally surfaced eight stale artifacts (0.6.0 → 0.20.0) sitting in dist/, which uv publish uploads wholesale. The publish script now refuses anything that isn't the version being cut:

::error::dist/ holds artifacts that are not droste 0.24.0: droste-0.20.0-py3-none-any.whl …
::error::clear dist/ and rebuild — publishing would upload them too

And the case that made 0.24.0 unrecoverable is now a clean no-op:

$ scripts/publish-pypi.sh 0.24.0
droste 0.24.0 is already on PyPI — nothing to publish   (exit 0)

Verification

1263 passed, 3 skipped. Both workflows parse; both scripts exercised locally end to end.

Separately: v0.24.0's artifacts (sdist, wheel, relay tarball) are now attached to its GitHub release — the sdist and wheel pulled from PyPI so they're byte-identical to what's published.

Unrelated flake seen once and not reproducible: test_native_and_pyodide_deliver_the_same_ordered_terminal_event_lifecycle.

🤖 Generated with Claude Code

shanev and others added 2 commits August 4, 2026 12:12
A release is retried by re-running the workflow or re-pushing the tag, and
both assume every step is safe to run twice. Publishing is not, so the
recovery an operator reaches for first was guaranteed to fail. That is how
0.24.0 ended up published to PyPI, missing from its GitHub release, and
impossible to re-tag.

Four changes, in order of what actually caused it.

The conformance corpus is enumerated, not listed. Its filenames were spelled
out in two workflows, the packaging check, the release tarball, and suites in
another repository; the v8 -> v9 rename was missed in release.yml, the one
place no pull request exercises. conformance_fixture_names() answers the
question instead, so a rename is invisible to every consumer.

The release path runs on every pull request. Bundling moved out of inline YAML
into scripts/bundle-relay.sh so CI can run the identical code with the single
irreversible action stubbed out. A step that only ever executes on a tag is
discoverable only by breaking a release.

Publishing is last. Every fallible local step -- build, bundle -- now finishes
before the first irreversible byte leaves the runner, so a later failure can
no longer strand a completed upload.

Publishing converges. --check-url was documented as making re-runs idempotent
and does not: it predicts from a CDN-cached index and compares against freshly
built artifacts, but sdists are not byte-reproducible, so a rebuild is a
different file under the same name and the upload is attempted regardless. Ask
the authority instead -- is this version on PyPI -- both before uploading and
after a failed upload. An already-published version is now a no-op.

The dry run earned itself immediately by catching eight stale artifacts in a
local dist/, which uv publish would have uploaded alongside the release; the
publish script now refuses any artifact that is not the version being cut.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The packaging job builds a wheel and never syncs an environment, so importing
droste.testing to list the corpus failed there. The directory is the single
source either way, so glob it: no import, no dependency on job ordering, and
a rename still reaches every consumer. conformance_fixture_names() stays for
library callers.

Pass the built version to the dry-run publish too. It was handed a
placeholder, which the stale-artifact guard correctly rejected against a
dist/ holding the real build -- the guard working, exercised wrongly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@shanev
shanev merged commit 4bb9d40 into main Aug 4, 2026
11 checks passed
@shanev
shanev deleted the fix/release-convergence branch August 4, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant