Skip to content

fix(cert-manager): resolve reconciliation issues with enableCertificateOwnerRef#352

Open
valen-mascarenhas14 wants to merge 4 commits into
etcd-io:mainfrom
valen-mascarenhas14:cert-manager-fix
Open

fix(cert-manager): resolve reconciliation issues with enableCertificateOwnerRef#352
valen-mascarenhas14 wants to merge 4 commits into
etcd-io:mainfrom
valen-mascarenhas14:cert-manager-fix

Conversation

@valen-mascarenhas14

Copy link
Copy Markdown
Contributor

Fixes #331

PR Description

Problem

The operator cannot reconcile ETCD clusters when using cert-manager configured with enableCertificateOwnerRef=true. This manifests in two ways:

  1. Certificate creation fails due to invalid duration parsing:

    ERROR Failed to create Client Certificate. error: "failed to parse ValidityDuration: time: unknown unit \"d\" in duration \"365d\""
    
  2. Owner reference conflicts when cert-manager's enableCertificateOwnerRef=true sets Certificate as the controller owner of TLS secrets, preventing the operator from managing them.

Root Causes

Duration Parsing Issue:
Go's time.ParseDuration doesn't support day units (d), but the sample configuration and API documentation use day-based durations like "365d" and "100d12h".

Owner Reference Issue:
When enableCertificateOwnerRef=true, cert-manager sets itself as the controller owner of TLS secrets. The operator was attempting to overwrite this ownership, causing reconciliation failures.

Solution

1. Duration Parsing Fix:
Enhanced parseValidityDuration to automatically convert day units to hours:

  • "365d""8760h" (365 × 24 hours)
  • "100d12h""2412h" (100 × 24 + 12 hours)
  • Maintains backward compatibility with hour-based durations

2. Owner Reference Fix:
Modified secret ownership logic to:

  • Skip ownership patching when a controller owner already exists
  • Preserve cert-manager's ownership of TLS secrets
  • Log informational messages instead of errors

@k8s-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: valen-mascarenhas14
Once this PR has been reviewed and has the lgtm label, please assign jmhbnz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot

Copy link
Copy Markdown

Hi @valen-mascarenhas14. Thanks for your PR.

I'm waiting for a etcd-io member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Signed-off-by: Valen Mascarenhas <valen.mascarenhas@ibm.com>
Comment thread internal/controller/utils.go Outdated
}
hours := days * 24
result.WriteString(fmt.Sprintf("%gh", hours))
numStr.Reset()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this chunk of code can be made very simple. Just trimming the last character and converting the remaining to Integer, should help.

Please try and check!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GunaKKIBM The current logic was kept generic to support duration strings like 23d4h too

@GunaKKIBM

Copy link
Copy Markdown
Contributor

@valen-mascarenhas14, could you please add the test results here? Pod logs should be good.

@ahrtr ahrtr requested a review from ArkaSaha30 June 9, 2026 18:22

@ArkaSaha30 ArkaSaha30 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we validate if the cascading cleanup happens on EtcdCluster deletion when the certificate secret is having cert-manager ownerRef?

Manual validation should be good for now.

Comment thread internal/controller/utils.go Outdated
days, err := strconv.ParseFloat(numStr.String(), 64)
if err != nil {
return 0, fmt.Errorf("failed to parse day value in ValidityDuration: %w", err)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use regex to simplify the parsing logic?
Something like regexp.MustCompile(([0-9.]+)(d)) to get the no of days and then calculate the hours?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, have used regex to simplify the parsing logic

Comment thread internal/controller/utils.go Outdated
result.WriteByte(ch)
}
}
// Append any remaining number

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block can also be omitted if we use regex instead of index parsing.

@valen-mascarenhas14

Copy link
Copy Markdown
Contributor Author

Can we validate if the cascading cleanup happens on EtcdCluster deletion when the certificate secret is having cert-manager ownerRef?
Manual validation should be good for now.

@ArkaSaha30 I noticed that cascading cleanup would fail when the certificate secret had cert-manager ownerRef. I've made the code changes for the same . Below is how it looks after the code changes. PTAL.

valen@valens-MacBook-Air etcd-operator % kubectl apply -f config/samples/sample_certmanager_etcdcluster.yaml                                                 

clusterissuer.cert-manager.io/selfsigned created
etcdcluster.operator.etcd.io/etcdcluster-sample-cert created
valen@valens-MacBook-Air etcd-operator % kubectl get certificates -A 
NAMESPACE   NAME                                 READY   SECRET                               AGE
default     etcdcluster-sample-cert-client-tls   True    etcdcluster-sample-cert-client-tls   14s
default     etcdcluster-sample-cert-peer-tls     True    etcdcluster-sample-cert-peer-tls     14s
default     etcdcluster-sample-cert-server-tls   True    etcdcluster-sample-cert-server-tls   14s
valen@valens-MacBook-Air etcd-operator % kubectl delete -f config/samples/sample_certmanager_etcdcluster.yaml

clusterissuer.cert-manager.io "selfsigned" deleted
etcdcluster.operator.etcd.io "etcdcluster-sample-cert" deleted from default namespace
valen@valens-MacBook-Air etcd-operator % kubectl get certificates -A                                         
No resources found

Signed-off-by: Valen Mascarenhas <valen.mascarenhas@ibm.com>
Signed-off-by: Valen Mascarenhas <valen.mascarenhas@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot reconcile ETCD cluster when using cert-manager configured with enableCertificateOwnerRef=true

4 participants