Skip to content

promote-apps: reused promotion branch + --force-with-lease breaks after each merge #262

Description

@Svaag

Problem

promote-apps.yml pushes to a long-lived, reused branch promotion/app-sha-pins and commits with git push --force-with-lease (step "Commit promotion branch"). When a prior promotion PR is merged, GitHub auto-deletes that branch — and the next dispatch then fails:

[promotion/app-sha-pins <sha>] Promote ...
 1 file changed, 1 insertion(+), 1 deletion(-)
 ! [rejected]        promotion/app-sha-pins -> promotion/app-sha-pins (stale info)
error: failed to push some refs

--force-with-lease (no explicit lease value) verifies against the remote-tracking ref. The runner only checks out github.sha (main), so when the branch doesn't exist remotely there's no lease to verify and the push is refused as "stale info." It only succeeds when the branch already exists remotely (so the implicit fetch establishes the lease).

A secondary symptom: because the branch is reused and never reset, it can drift behind main (observed once at ahead-12/behind-60), producing a conflicting promotion PR full of already-merged "Promote …" commits.

Impact

Every promotion that follows a merged promotion fails until someone manually recreates the branch at main, e.g.:

gh api -X POST repos/AS215932/network-operations/git/refs \
  -f ref=refs/heads/promotion/app-sha-pins -f sha=<current main sha>

This bit us twice in one session (failed run 27645837403, plus a conflicting PR that had to be rebuilt). It's confusing and blocks the deploy path.

Proposed fix (pick one)

  • A — fresh branch per run (preferred): open each promotion on a unique branch (e.g. promotion/app-sha-pins-${{ github.run_id }} or …-<short-sha>). No force-push, no lease, no stale drift; each PR is a clean diff vs main. Optionally close/supersede older open promotion PRs.
  • B — reset the reusable branch to main at the start of each run: recreate/--force promotion/app-sha-pins from the checked-out main before committing, so it always exists and is based on current main.
  • C — make the push robust: fetch the branch if present and create-if-missing, or drop the lease (--force). Least preferred — keeps the drift smell.

Acceptance

  • Dispatching promote-apps immediately after a prior promotion PR merged succeeds without manual branch recreation.
  • The resulting promotion PR is a clean, single-purpose diff against current main (no superseded "Promote …" commits, never CONFLICTING).

Refs: .github/workflows/promote-apps.yml (step "Commit promotion branch"). Surfaced while promoting noc-agent for the proactive NOC loop (#260, #261).

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions