issues upgrading to latest

[linkandzelda91]

Describe the bug
I pulled my existing values and tried to upgrade, got an error:

gdewitt@k3s:~$ !872
helm upgrade --install --namespace "ess" ess oci://ghcr.io/element-hq/ess-helm/matrix-stack -f ./essvalues.yaml --wait
Pulled: ghcr.io/element-hq/ess-helm/matrix-stack:26.6.2
Digest: sha256:752ddc775dc7cd5663d114968a37e647c6fa29c3d26501a3a0b00b714404a712
Error: UPGRADE FAILED: pre-upgrade hooks failed: 1 error occurred:
	* job ess-synapse-check-config failed: BackoffLimitExceeded

gdewitt@k3s:~$ !872
helm upgrade --install --namespace "ess" ess oci://ghcr.io/element-hq/ess-helm/matrix-stack -f ./essvalues.yaml --wait
Pulled: ghcr.io/element-hq/ess-helm/matrix-stack:26.6.2
Digest: sha256:752ddc775dc7cd5663d114968a37e647c6fa29c3d26501a3a0b00b714404a712
Error: UPGRADE FAILED: pre-upgrade hooks failed: 1 error occurred:
	* job ess-synapse-check-config failed: BackoffLimitExceeded

gdewitt@k3s:~$ kubectl logs -n ess job/ess-synapse-check-config -c render-config
failed to open file: open /secrets/ess-generated/SYNAPSE_EXTRA: no such file or directory

gdewitt@k3s:~$ kubectl get secret ess-generated -n ess
NAME            TYPE     DATA   AGE
ess-generated   Opaque   12     306d
gdewitt@k3s:~$ kubectl get jobs -n ess
NAME                          STATUS     COMPLETIONS   DURATION   AGE
ess-deployment-markers-post   Complete   1/1           4s         293d
ess-deployment-markers-pre    Complete   1/1           7s         5m1s
ess-init-secrets              Complete   1/1           9s         4m53s
ess-synapse-check-config      Failed     0/1           4m39s      4m39s
gdewitt@k3s:~$ kubectl describe secret ess-generated -n ess
Name:         ess-generated
Namespace:    ess
Labels:       app.kubernetes.io/component=matrix-tools
              app.kubernetes.io/instance=ess-init-secrets
              app.kubernetes.io/managed-by=matrix-tools-init-secrets
              app.kubernetes.io/name=init-secrets
              app.kubernetes.io/part-of=matrix-stack
              app.kubernetes.io/version=0.8.0
Annotations:  <none>

Type:  Opaque

Data
====
SYNAPSE_REGISTRATION_SHARED_SECRET:               32 bytes
MAS_ENCRYPTION_SECRET:                            64 bytes
MAS_RSA_PRIVATE_KEY:                              2374 bytes
POSTGRES_ADMIN_PASSWORD:                          32 bytes
POSTGRES_MATRIX_AUTHENTICATION_SERVICE_PASSWORD:  32 bytes
POSTGRES_SYNAPSE_PASSWORD:                        32 bytes
SYNAPSE_EXTRA:                                    114 bytes
SYNAPSE_MACAROON:                                 32 bytes
SYNAPSE_SIGNING_KEY:                              54 bytes
ELEMENT_CALL_LIVEKIT_SECRET:                      32 bytes
MAS_ECDSA_PRIME256V1_PRIVATE_KEY:                 138 bytes
MAS_SYNAPSE_SHARED_SECRET:                        32 bytes

a lot of this troubleshooting is from AI guidance, I do not use k8s frequently so if it's gone wrong sorry.

I'm on ubuntu if it matters

gdewitt@k3s:~$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=25.04
DISTRIB_CODENAME=plucky
DISTRIB_DESCRIPTION="Ubuntu 25.04"
gdewitt@k3s:~$ 

[linkandzelda91]

here's my essvalues.yaml, removed my oauth2 secret

gdewitt@k3s:~$ cat essvalues.yaml 
elementWeb:
  ingress:
    host: chat.mayogirl.com
ingress:
  tlsEnabled: false
matrixAuthenticationService:
  additional:
    1-custom-config:
      config: |
        upstream_oauth2:
          providers:
            - id: "01K4DY4T64MFS95MQBQPQC87DH"
              human_name: Authentik
              issuer: "https://authentik.mayogirl.com/application/o/matrix/"
              client_id: "removed"
              client_secret: "removed"
              token_endpoint_auth_method: client_secret_basic
              scope: "openid profile email"
              claims_imports:
                localpart:
                  action: require
                  template: "{{ user.preferred_username }}"
                displayname:
                  action: suggest
                  template: "{{ user.name }}"
                email:
                  action: suggest
                  template: "{{ user.email }}"
                  set_email_verification: always
  ingress:
    host: account.mayogirl.com
matrixRTC:
  ingress:
    host: mrtc.mayogirl.com
  sfu:
    additional:
      10-external-ip.yml:
        config: |
          rtc:
            use_external_ip: false
            # To workaround https://github.com/livekit/livekit/issues/2088
            # Any IP address is acceptable, it doesn't need to be a correct one,
            # it just needs to be present to get LiveKit to skip checking all local interfaces
            # We assign here a TEST-NET IP which is
            # overridden by the NODE_IP env var at runtime
            node_ip: 198.51.100.1
serverName: mayogirl.com
synapse:
  ingress:
    host: mayogirl.com
elementAdmin:
  enabled: true
  ingress:
    host: essadmin.mayogirl.com
gdewitt@k3s:~$