Describe the bug
CycleNodeStatusTransitioner.timedOut() dereferences status.timeoutTimestamp without checking for nil. If a CycleNodeStatus reaches DrainingPods with status.timeoutTimestamp unset, the controller panics, controller-runtime recovers/requeues, and the node cycle never progresses.
Affected code:
pkg/controller/cyclenodestatus/transitioner/util.go
CycleNodeStatusTransitioner.timedOut()
Current behavior:
return time.Now().After(t.cycleNodeStatus.Status.TimeoutTimestamp.Time)
TimeoutTimestamp is defined as *metav1.Time and is optional, so this can be nil.
To Reproduce
- Have a
CycleNodeStatus in phase DrainingPods.
- Ensure
.status.timeoutTimestamp is absent/null.
- Let the
cyclenodestatus.controller reconcile the object.
- Observe a nil pointer panic from
timedOut().
Expected behavior
The controller should never panic on a persisted CR state. If timeoutTimestamp is nil, Cyclops should either initialize it, treat the CNS as not timed out yet, or fail the CNS with a clear error.
Screenshots or Logs
Redacted production stack trace:
ERROR Observed a panic {"controller": "cyclenodestatus.controller",
"object": {"name":"<cycle-node-status-name>","namespace":"<operator-namespace>"},
"panic": "runtime error: invalid memory address or nil pointer dereference",
"stacktrace": "... cyclenodestatus/transitioner.(*CycleNodeStatusTransitioner).timedOut()
.../pkg/controller/cyclenodestatus/transitioner/util.go:42
.../transitioner.(*CycleNodeStatusTransitioner).transitionDraining()
.../pkg/controller/cyclenodestatus/transitioner/transitions.go:164 ..."}
Kubernetes Cluster Version
AWS EKS 1.33 (v1.33.11-eks-40737a8).
Cyclops Version
v1.10.5
Additional context
This can leave node cycles stuck in DrainingPods / WaitingTermination. In AWS environments, that can strand detached EC2 worker instances that are no longer ASG-managed but continue billing.
Suggested fix:
func (t *CycleNodeStatusTransitioner) timedOut() bool {
if t.cycleNodeStatus.Status.TimeoutTimestamp == nil {
return false
}
return time.Now().After(t.cycleNodeStatus.Status.TimeoutTimestamp.Time)
}
It would be helpful to add a regression test covering a CNS in DrainingPods with nil status.timeoutTimestamp.
Describe the bug
CycleNodeStatusTransitioner.timedOut()dereferencesstatus.timeoutTimestampwithout checking for nil. If aCycleNodeStatusreachesDrainingPodswithstatus.timeoutTimestampunset, the controller panics, controller-runtime recovers/requeues, and the node cycle never progresses.Affected code:
pkg/controller/cyclenodestatus/transitioner/util.goCycleNodeStatusTransitioner.timedOut()Current behavior:
TimeoutTimestampis defined as*metav1.Timeand is optional, so this can be nil.To Reproduce
CycleNodeStatusin phaseDrainingPods..status.timeoutTimestampis absent/null.cyclenodestatus.controllerreconcile the object.timedOut().Expected behavior
The controller should never panic on a persisted CR state. If
timeoutTimestampis nil, Cyclops should either initialize it, treat the CNS as not timed out yet, or fail the CNS with a clear error.Screenshots or Logs
Redacted production stack trace:
Kubernetes Cluster Version
AWS EKS
1.33 (v1.33.11-eks-40737a8).Cyclops Version
v1.10.5Additional context
This can leave node cycles stuck in
DrainingPods/WaitingTermination. In AWS environments, that can strand detached EC2 worker instances that are no longer ASG-managed but continue billing.Suggested fix:
It would be helpful to add a regression test covering a CNS in
DrainingPodswith nilstatus.timeoutTimestamp.