fix: surface reconcile errors as Degraded condition on EtcdCluster status#395
fix: surface reconcile errors as Degraded condition on EtcdCluster status#395xrl wants to merge 1 commit into
Conversation
…atus An EtcdCluster stuck on a persistent reconcile error was invisible in .status: fetchAndValidateState returned a nil state on the ownership and upgrade-path error paths so the deferred status update never ran, leaving the status completely empty. Errors from later phases (e.g. an invalid TLS validityDuration failing with "failed to parse ValidityDuration" during certificate creation) did reach the status update, but updateConditions derived Degraded solely from member health and reported ClusterHealthy while every reconcile failed; the cluster-state phase error was never even seen by the deferred update. Return a populated reconcileState on the early error paths (omitting a StatefulSet the operator does not own so its replica counts cannot leak into status), capture the cluster-state phase error for the deferred update, and thread the reconcile error into updateStatus/updateConditions, setting Degraded=True with reason ReconcileFailed and the error message. Signed-off-by: Xavier Lange <xrlange@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: xrl The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @xrl. Thanks for your PR. I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
1 participant
An EtcdCluster failing every reconcile could be invisible in .status: the ownership and unsupported-upgrade-path errors returned a nil state so the deferred status update never ran (completely empty status), while errors from later phases — e.g. an invalid TLS validityDuration failing during certificate creation — did update status but reported a misleading Degraded=False/ClusterHealthy because conditions were derived from member health alone. This returns a populated state on those early error paths (omitting a StatefulSet the operator does not own so its replica counts cannot leak into status), captures the cluster-state phase error for the deferred update, and threads the reconcile error into the status update so Degraded=True with reason ReconcileFailed and the error message. Tested with go test ./internal/... including new Reconcile-level tests through a fake client with the status subresource.
PR series — operability fixes & TLS
Small single-purpose PRs from live kind-cluster testing of the operator. Each stands alone unless an After is listed. → = this PR.
events.k8s.ioRBAC so operator Events are actually recordedmembers[]/leaderIDfrom one health snapshot (consistent leader)altNames.ipAddressesinto certificatesvalidityDuration(365d,100d12h) as documentedDegradedcondition (was empty status)--max-concurrent-reconciles)spec.tls.{peer,client}surfaces (breaking alpha API)TLSReadycondition + TLS lifecycle EventsPeerCANotShared/metricsendpointEtcdBackupCR → object storage (S3/GCS)🟢 ready · ⚪ draft