[Issue]: sops-operator helm chart broken when enabling metrics

[manuel-koch]

Description

I tried to enable metrics and Service Monitor in helm chart of sops-operator ( chart version 0.25.3 ) to see metrics in Grafana/Prometheus.

I exhibit at least three problem with that chart version:

• service monitor using https port, but workload doesn't seem to expose https
• no k8s Service deployed, but ServiceMonitor requires one
• workload not using the metrics port correctly

AI-generated code or design

Yes – code

Additional context

Using the values.yaml to configure the helm chart:

"metrics": {
   "enabled": True
}

Suggested solution

• creating a k8s Service resource
• add new config values to configure metrics port and additional volume mounts for custom secrets
• adapting command line to use configured port for metrics
• adjust ServiceMonitor to use the configured metrics port
• allow additional volume mounts ( needed to get Azure Managed Identity running, to fetch private age key from a key-vault )

diff --git a/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/monitor.yaml b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/monitor.yaml
index 4853489..5cd9f04 100644
--- a/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/monitor.yaml
+++ b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/monitor.yaml
@@ -11,11 +11,8 @@ metadata:
 spec:
   endpoints:
     - path: /metrics
-      port: https
-      scheme: https
-      bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
-      tlsConfig:
-        insecureSkipVerify: true
+      port: metrics
+      scheme: http
   selector:
     matchLabels:
       app.kubernetes.io/name: {{ include "sops-secrets-operator.name" . }}
diff --git a/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/operator.yaml b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/operator.yaml
index 427a1a0..60c60e1 100644
--- a/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/operator.yaml
+++ b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/operator.yaml
@@ -60,7 +60,7 @@ spec:
               drop: {{ .Values.securityContext.container.capabilities.drop }}
               add: {{ .Values.securityContext.container.capabilities.add }}
           {{- end }}
-          {{- if or .Values.gcp.enabled .Values.gpg.enabled .Values.secretsAsFiles }}
+          {{- if or .Values.gcp.enabled .Values.gpg.enabled .Values.secretsAsFiles .Values.extraVolumeMounts }}
           volumeMounts:
           {{- end }}
           {{- if .Values.gcp.enabled }}
@@ -80,11 +80,17 @@ spec:
             mountPath: {{ .mountPath }}
             readOnly: true
           {{- end }}
+          {{- if .Values.extraVolumeMounts }}
+          {{- toYaml .Values.extraVolumeMounts | nindent 10 }}
+          {{- end }}
           command:
           - /usr/local/bin/manager
           args:
-          # The address the metric endpoint binds to. (default ":8080")
-          #- "-metrics-bind-address=127.0.0.1:8080"
+          {{- if .Values.metrics.enabled }}
+          - "-metrics-bind-address=:{{ .Values.metrics.port }}"
+          {{- else }}
+          - "-metrics-bind-address=127.0.0.1:{{ .Values.metrics.port }}"
+          {{- end }}
           - "-health-probe-bind-address=:{{ .Values.healthProbes.port }}"
           # Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
           - "-leader-elect"
@@ -162,7 +168,7 @@ spec:
             {{- end }}
           resources:
             {{- toYaml .Values.resources | nindent 12 }}
-      {{- if or .Values.gcp.enabled .Values.gpg.enabled .Values.secretsAsFiles }}
+      {{- if or .Values.gcp.enabled .Values.gpg.enabled .Values.secretsAsFiles .Values.extraVolumes }}
       volumes:
       {{- end }}
       {{- if .Values.gcp.enabled }}
@@ -191,6 +197,9 @@ spec:
         secret:
           secretName: {{ .secretName }}
       {{- end }}
+      {{- if .Values.extraVolumes }}
+      {{- toYaml .Values.extraVolumes | nindent 6 }}
+      {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}
diff --git a/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/service.yaml b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/service.yaml
new file mode 100644
index 0000000..dca266a
--- /dev/null
+++ b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/templates/service.yaml
@@ -0,0 +1,17 @@
+{{- if .Values.metrics.enabled }}
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ include "sops-secrets-operator.fullname" . }}-metrics
+  labels:
+    {{- include "sops-secrets-operator.labels" . | nindent 4 }}
+spec:
+  ports:
+    - name: metrics
+      port: {{ .Values.metrics.port }}
+      targetPort: {{ .Values.metrics.port }}
+      protocol: TCP
+  selector:
+    app.kubernetes.io/name: {{ include "sops-secrets-operator.name" . }}
+    app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
diff --git a/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/values.yaml b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/values.yaml
index ac54d2d..67cda97 100644
--- a/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/values.yaml
+++ b/k8s/lib/helm/charts-vendored/sops-secrets-operator/sops-secrets-operator/0.25.3/sops-secrets-operator/values.yaml
@@ -144,6 +144,12 @@ secretsAsFiles: []
 #  mountPath: "/etc/foo"
 #  secretName: mysecret
 
+# -- Additional volume mounts for the operator container
+extraVolumeMounts: []
+
+# -- Additional volumes for the operator pod
+extraVolumes: []
+
 # -- Operator container resources
 resources: {}
 # We usually recommend not to specify default resources and to leave this as a conscious
@@ -199,5 +205,7 @@ rbac:
 metrics:
   # -- Enable prometheus metrics
   enabled: false
+  # -- Port the metrics endpoint binds to
+  port: 8080
   # -- Additional labels for ServiceMonitor
   additionalLabels: {}

Alternative solutions considered

• metrics don't work with the current chart content - no alternative
• can't use Azure Managed Identity to use Key-Vault because the secret can't be mounted as a volume - no alternative

[isindir]

The bug confirmed, I remember about this issue, just need to find time for it