Skip to content

[BUG] Slicer branch names collide across re-runs of the same issue (NO-GO on push) #1054

Description

@Defilan

Bug Description

Re-running llmkube foreman slice <issue> on an issue that was sliced before can fail a slice with NO-GO at the push step, because the slicer's branch names are a pure function of (issue, slice-name) and collide with branches left on the fork by the earlier run.

Steps to Reproduce

  1. Run llmkube foreman slice 700 --planner-url ... --planner-model ... — the planner emits slices, each pushed to foreman/slicer-700/<slice-name>.
  2. Run the same command again on issue 700. The planner is non-deterministic but tends to reuse slice names (observed: rocm-smi-daemonset produced by two independent planner calls).
  3. The new coder branches from current main, so its history diverges from the stale remote branch tip.

Expected Behavior

A re-run of the slicer on the same issue completes without colliding with a prior run's artifacts.

Actual Behavior

The coder's git push origin HEAD:foreman/slicer-<issue>/<slice-name> is a non-fast-forward against the pre-existing branch and is rejected. The coder correctly refuses to force-push, so the slice ends ExecutorSucceeded but verdict NO-GO ("push to fork failed"). integrate then cascade-fails (UpstreamFailed) and reconcile cascades off it. The whole Workload fails on a stale branch, not on any code problem.

Controlled evidence from a live run: two slices on the same node with the same push credentials, one with a fresh branch name (pushed fine, GO) and one whose branch already existed from a 20h-old run (push rejected, NO-GO). The only difference was the pre-existing branch.

Proposed Fix

Make slicer branch names run-scoped so re-runs can never collide. In buildSliceWorkload (pkg/cli/slice.go), replace foreman/slicer-<issue>/<name> with a per-Workload unique segment, e.g. foreman/slicer-<issue>-<shortid>/<name>, where <shortid> is stable across the slices of one Workload (so integrate/reconcile still resolve the same branches) but unique per invocation. Thread the same id into the integrate/reconcile payload slice refs.

Alternatives considered:

  • Teach the coder to force-push slicer branches. They are ephemeral, machine-owned run artifacts, so force-push is defensible; but it is unsafe under two concurrent runs of the same slice, whereas run-scoped names are collision-free by construction.
  • Delete foreman/slicer-<issue>/* at the start of a run. Racy under concurrency and destroys an in-flight run's branches.

Run-scoped names are the smallest safe fix.

Environment

Additional Context

Surfaced during the first live lab run of the graduated slicer. The pipeline's dependsOn gating and cascade-fail behavior worked correctly; this is purely branch-name hygiene across runs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/foremanForeman: the agentic fleet orchestrator add-onbugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions