AgentPlaybookRun: failing stage crashloops instead of failing the run (sandbox restartPolicy: OnFailure)

[ibolton336]

Summary

A playbook stage whose container exits non-zero never fails the AgentPlaybookRun. The sandbox pod is created with restartPolicy: OnFailure, so the container restarts indefinitely — the pod never reaches a terminal phase, the Sandbox never reports a terminal reason, the stage AgentRun stays Running, and the playbook run hangs with subsequent stages Pending forever. The fail-fast semantics from #36 ("Fails the entire playbook run if any stage fails; subsequent stages are never started") are unreachable for exit-code failures.

The success path works correctly (verified: PodSucceeded → stage AgentRun Succeeded → next stage starts).

Environment

• Controller built from main @ fff23af (merge of ✨ Add AgentPlaybook and AgentPlaybookRun controllers #36), deployed on minikube (docker driver, cri-dockerd, Kubernetes v1.35.1)
• Agent Sandbox controller v0.5.x

Reproduction

Two-stage playbook where stage 1's Agent image is a stub that fails:

FROM busybox:1.36
CMD ["sh","-c","echo failing; sleep 3; exit 1"]

apiVersion: konveyor.io/v1alpha1
kind: AgentPlaybook
metadata: {name: stub-playbook-fail}
spec:
  stages:
    - {name: doomed,     agentRef: stub-fail}
    - {name: never-runs, agentRef: stub-ok}

Observed: stage pod restarts repeatedly (restartPolicy=OnFailure, 4+ restarts observed, container state terminated exitCode=1 between restarts); Sandbox Ready=False reason=DependenciesNotReady ("Pod is Running but not Ready"); playbook run stuck phase=Running | doomed=Running never-runs=Pending for 2.5+ minutes until manually deleted. A control playbook with exit 0 stages completed in ~20s.

Expected: stage doomed → Failed, run phase=Failed, never-runs never started.

Possible directions

1. Run-to-completion stage pods use restartPolicy: Never (or a bounded retry, backoffLimit-style) so failure becomes observable.
2. Alternatively/additionally, the AgentRun controller treats sustained CrashLoopBackOff / repeated terminated-with-error states as failure.
3. At minimum, document the constraint for stage image authors.

Happy to provide the full repro manifests or test against a patch.