feat: Add Valkey Sentinel & HAProxy support in High Availability setup. by khtee · Pull Request #137 · valkey-io/valkey-helm

khtee · 2026-02-06T08:56:32Z

Feat #22

dmaes · 2026-02-11T13:03:11Z

Nice work!

I have some notes/questions:

You'll want to disable the default statefulset when sentinel is enabled.
Do you plan on adding a HAProxy deployment, so clients don't have to be aware of sentinel? ( See for example https://arystech.com/blog/setting-up-haproxy-with-redis-sentinel-for-high-availability-on-microk8s-kubernetes )

khtee · 2026-02-11T13:07:44Z

Nice work!

I have some notes/questions:

You'll want to disable the default statefulset when sentinel is enabled.

Do you plan on adding a HAProxy deployment, so clients don't have to be aware of sentinel? ( See for example https://arystech.com/blog/setting-up-haproxy-with-redis-sentinel-for-high-availability-on-microk8s-kubernetes )

Good points! Will work on both improvements.

yoannrt · 2026-02-11T15:17:39Z

Good job,

I have some questions too,
I assume to discover who is master, you have to ask sentinel SENTINEL GET-MASTER-ADDR-BY-NAME mymaster ?
(Currently, the replica service is sticked to pod-0 in the STS)

Also, would that make sense to run sentinel as a side container in the replica STS pods ?

amontalban · 2026-02-13T22:47:56Z

Thank you @khtee can't wait to get this merged in 🙏

dmaes · 2026-02-14T06:50:58Z

Also, would that make sense to run sentinel as a side container in the replica STS pods ?

You'll want to disable the default statefulset when sentinel is enabled

I got confused, thinking valkey-sentinel also runs the server process, it's been a while since I used redis/valkey. But yes, @khtee you'll want to run sentinel and the server side-to-side in the same pod, don't split them out in different statefulsets.

yoannrt · 2026-02-17T15:38:51Z

Do you plan on adding a HAProxy deployment, so clients don't have to be aware of sentinel?

@dmaes I guess when you suggest to implement HAproxy, it's for sentinel incompatible clients workloads ?
So this would be optional and the default implementation would be a valkey/sentinel HA setup ?

khtee · 2026-02-19T06:14:04Z

Also, would that make sense to run sentinel as a side container in the replica STS pods ?

You'll want to disable the default statefulset when sentinel is enabled

I got confused, thinking valkey-sentinel also runs the server process, it's been a while since I used redis/valkey. But yes, @khtee you'll want to run sentinel and the server side-to-side in the same pod, don't split them out in different statefulsets.

Added following enhancements.

Sentinel now runs as a sidecar in valkey pod.
HAProxy with sidecar watcher to track failover.

dmaes · 2026-02-19T07:58:51Z

Do you plan on adding a HAProxy deployment, so clients don't have to be aware of sentinel?

@dmaes I guess when you suggest to implement HAproxy, it's for sentinel incompatible clients workloads ? So this would be optional and the default implementation would be a valkey/sentinel HA setup ?

That's correct.

The truly kubernetes-native way would probably be to have a sentinel-master Service, using a sentinel.valkey.io/master: "true" label selector, and then some watcher that updates that label on the correct Pod,
but a HAproxy deployment is the easier option to implement, and is how most other Redis charts do it.

Signed-off-by: KHTee <teekahhui@hotmail.com>

Co-authored-by: Dieter Maes <dieter.maes@dmaes.be> Signed-off-by: khtee <75174583+khtee@users.noreply.github.com>

Signed-off-by: KHTee <teekahhui@hotmail.com>

Allow HAProxy to retry DNS resolution during startup when pending for Valkey node to start. Essentially it does the following - Try to use the last known IP. - If none, query the libc resolver (DNS). - If that fails, resolve to none (meaning the server has no IP address yet, but HAProxy won't crash) and wait for the runtime resolver health-checks to pick up the DNS correctly. Signed-off-by: KHTee <teekahhui@hotmail.com>

lazariv · 2026-03-02T12:50:57Z

Waiting for this to be merged. Thanks for your work!

Munken · 2026-03-31T19:50:12Z

    #   labels:
    #     severity: error
+
+haproxy:


Suggested change

haproxy:

# Enable haproxy in front of sentinel

# This is allows running redis in sentinel mode even when the client is not compatible

# https://arystech.com/blog/setting-up-haproxy-with-redis-sentinel-for-high-availability-on-microk8s-kubernetes

haproxy:

fix(chart): resolve schema validation and template errors

jose-10000 · 2026-04-03T08:31:59Z

Hi @khtee

I’ve been testing this with ACLs enabled + TLS active and found some discovery gaps when the default user is restricted. I've sent a PR to @khtee's fork with the following hardening fixes:

Inter-Sentinel Auth: Added sentinel-user/pass to fix quorum discovery ("empty array" issue).
Role Segregation: Support for a dedicated monitorUser (separate from replication).
Security: Fixed credential leaks in logs (redirected to stderr) and added native SHA256 hashing.
Schema: Fixed values.schema.json and HAProxy tag types to pass helm lint.

I've verified that ACL + TLS now work seamlessly. I'm currently finishing validation for HAProxy with credentials and will update soon.

Thanks for your contribution!

- Updating HAProxy watcher for near-instant IP-based failover. - Refactoring init scripts to support dynamic topology and universal auth/TLS injection. - Adding smart L7 health checks in HAProxy to handle ACL-protected nodes. - Fully parameterizing Service and ConfigMap ports for end-to-end flexibility.

…notations in haproxy-deployment

…schema Valkey Sentinel: Fixing Auth/ACL gaps in TLS environments and security hardening (Enhancements for PR valkey-io#137)

yoannrt · 2026-04-09T22:49:50Z

@@ -188,3 +188,34 @@ Validate replica authentication configuration
 {{- end }}
 {{- end -}}



Suggested change

{{/*

Validate haproxy is used in replica mode

*/}}

{{- define "valkey.validateHaproxyRequirements" -}}

{{- if and .Values.haproxy.enabled (not .Values.replica.enabled) }}

{{- fail "Haproxy is only relevant in replica mode with clients incompatible with Sentinel." }}

{{- end }}

{{- end -}}

update of deploy_valkey.yaml to fail if haproxy is enabled in standalone mode.

Suggested change

{{- include "valkey.validateHaproxyRequirements" . }}

lazariv

Redis/Valkey pub/sub (SUBSCRIBE) connections are long-lived and idle by design — they block waiting for messages. HAProxy's timeout client/timeout server treats them as stale and drops them, causing clients to see ConnectionError: Connection closed by server and forcing reconnect loops.

Adding timeout tunnel to the HAProxy defaults section solves this cleanly. In HAProxy, timeout tunnel governs bidirectional connections after the initial handshake — exactly the pattern pub/sub uses. Setting it to 0 keeps SUBSCRIBE connections alive indefinitely while preserving normal client/server timeouts as a safety net for regular command connections.

Without this, users running any pub/sub workload (Socket.IO, Celery, Sidekiq, etc.) through HAProxy must either set client/server timeouts to 0 (triggering HAProxy warnings) or accept periodic disconnects.

lazariv · 2026-04-13T11:21:38Z

+      log global
+      timeout connect {{ .Values.haproxy.config.timeout.connect }}
+      timeout client {{ .Values.haproxy.config.timeout.client }}
+      timeout server {{ .Values.haproxy.config.timeout.server }}


Suggested change

timeout server {{ .Values.haproxy.config.timeout.server }}

timeout server {{ .Values.haproxy.config.timeout.server }}

timeout tunnel {{ .Values.haproxy.config.timeout.tunnel }}

option clitcpka

option srvtcpka

lazariv · 2026-04-13T11:23:05Z

+                    },
+                    "server": {
+                      "type": "string"
+                    }


Suggested change

}

},

"tunnel": {

"type": "string"

}

lazariv · 2026-04-13T11:24:30Z

+    timeout:
+      connect: 5s
+      client: 1m
+      server: 1m


Suggested change

server: 1m

server: 1m

# Timeout for long-lived bidirectional connections (e.g. pub/sub).

# Set to 0 to keep pub/sub SUBSCRIBE connections alive indefinitely.

tunnel: 0s

jose-10000 · 2026-04-27T06:56:55Z

I've implemented a preStop hook. I found that improper shutdowns during updates were tripping up Sentinel and risked data loss, so this change ensures a more graceful exit. I made a new pull request @khtee

jose-10000 · 2026-04-30T11:16:04Z

I've added the workloadAnnotations block to haproxy-deployment.yaml. I noticed that the upstream main branch recently introduced workloadAnnotations for other workloads, so adding it here ensures HAProxy is aligned with main once this PR is eventually merged.

Do you have a rough ETA for this merge? Please let me know if there is anything else I can help review or test to get this PR over the finish line. I'm planning to adopt these changes soon and would like to have an idea of when it might be merged. Thanks!

jose-10000 · 2026-05-18T07:02:40Z

I have some reservations about the approach of running Sentinel as a sidecar within the same pod as the Valkey instance.

The main concern is related to Helm upgrade workflows: when a pod is taken down during a rolling update or a Helm upgrade, the entire pod is terminated — including both the Valkey container and the Sentinel sidecar. This means we lose a Sentinel node during the upgrade process.

Losing a Sentinel during an upgrade can temporarily break the quorum. For example, in a typical 3-Sentinel setup, losing one during an upgrade leaves only 2 Sentinels, which may still meet quorum — but in smaller or less redundant setups, this can prevent failover from working correctly during the exact moment it may be needed most.

A more resilient architecture would deploy Sentinels as independent pods (e.g., via a separate StatefulSet), decoupled from the Valkey data pods. This ensures that Sentinel availability is not tied to the lifecycle of the data nodes, preserving quorum integrity throughout rolling updates and upgrades.

dmaes · 2026-05-18T08:18:06Z

@jose-10000 across all other best known redis sentinel helm charts, running in the same pod is the most common setup, and has been working just fine for most people. So I (and I think that goes for others too), never really considered using different statefulsets?

jose-10000 · 2026-05-18T08:43:45Z

@jose-10000 across all other best known redis sentinel helm charts, running in the same pod is the most common setup, and has been working just fine for most people. So I (and I think that goes for others too), never really considered using different statefulsets?

Thanks for the context, @dmaes! Totally makes sense, I know most charts (like Bitnami) do it this way to keep things simplier. I just wanted to point out that coupling their lifecycles can be tricky for quorum during upgrades, but I'm fine keeping the current approach. Just wanted to share my two cents

trabelsieyal · 2026-05-20T14:47:18Z

Any idea when will this be released? can be amazing

mlimardo1984 · 2026-06-04T14:41:21Z

Any idea when will this be released? can be amazing

Hello Team, just want to reiterate here that implementation is great and we are very much looking forward to have it released. Is there any news about it?

Thanks!

r4d1f · 2026-06-09T12:04:57Z

            - name: {{ $key }}
              value: "{{ $val }}"


I suggest adding the ability to pass extraEnvs for metrics from secrets as follows, for cases where a separate ACL has been created:

extraEnvs: REDIS_USER: metrics REDIS_PASSWORD: valueFrom: secretKeyRef: name: valkey-users key: metrics-pwd

Suggested change

- name: {{ $key }}

value: "{{ $val }}"

- name: {{ $key }}

{{- if kindIs "map" $val }}

{{- toYaml $val | nindent 14 }}

{{- else }}

value: {{ $val | quote }}

{{- end }}

Isn't it even cleaner to do this instead? #133

jsoref · 2026-06-09T17:42:12Z

@khtee this repository appears to expect DCO sign-offs, and unfortunately it looks like your fork didn't enforce that when you merged @jose-10000's Valkey Sentinel: Fixing Auth/ACL gaps in TLS environments and security hardening (Enhancements for PR #137) khtee/valkey-helm#1 -- I'm not sure how to fix this...

Nils98Ar · 2026-06-10T21:35:26Z

@khtee this repository appears to expect DCO sign-offs, and unfortunately it looks like your fork didn't enforce that when you merged @jose-10000's Valkey Sentinel: Fixing Auth/ACL gaps in TLS environments and security hardening (Enhancements for PR #137) khtee/valkey-helm#1 -- I'm not sure how to fix this...

@khtee You could add missing sign-offs in the last e.g. 10 commits via commit amend like this and force push:

git rebase HEAD~10 --exec 'git log -1 --format=%B | grep -qi "^Signed-off-by:" || git commit --amend -s --no-edit'
git push --force-with-lease

kaisen-san · 2026-06-18T16:07:49Z

Just FYI, besides the valid review comments people left on this PR, which I think should be taken into consideration, it also needs to be rebased, and some tests are failing. I also ran some local analysis and noted a few things that could be improved – I'll post the findings in another comment. I'm saying all this because I've noticed some people are expecting this to be merged, but I'm not sure if that will happen anytime soon (hope I'm wrong! 🙂)

yoannrt · 2026-06-18T19:20:16Z

across all other best known redis sentinel helm charts, running in the same pod is the most common setup, and has been working just fine for most people. So I (and I think that goes for others too), never really considered using different statefulsets?

Hi @dmaes , I thought too running sentinel as a sidecar was the simplest/cleaner option but I'm reconsidering it since having those components in 2 different sts has some advantages.
@khtee I think you were in the right way from the beginning so I sent you a PR, see khtee#4 . Thanks again

pat-s · 2026-07-28T07:33:06Z

@daanvinken @Bloodraven21
Pinging you here as you've been most active lately in committing to main.

It would really be great to finalize this feature (in here or a separate clean branch) so this can finally find it's way in. I am also happy to clean up if there's a commitment to review and get this merged then, avoiding additional future conflicts.

Looking forwarding hearing back from you and getting HA support into the chart!

kaisen-san · 2026-07-28T14:01:08Z

@daanvinken @Bloodraven21 Pinging you here as you've been most active lately in committing to main.

It would really be great to finalize this feature (in here or a separate clean branch) so this can finally find it's way in. I am also happy to clean up if there's a commitment to review and get this merged then, avoiding additional future conflicts.

Looking forwarding hearing back from you and getting HA support into the chart!

Same. I have some improvements I'd like to submit, but I'm not sure if it's worth trying to include them in this PR or creating a separate one.

Bloodraven21 · 2026-07-28T14:24:01Z

first can you rebase and make it up to date

second give me a exact values files to test it out on local

Bloodraven21 · 2026-07-28T14:25:06Z

@daanvinken @Bloodraven21 Pinging you here as you've been most active lately in committing to main.
It would really be great to finalize this feature (in here or a separate clean branch) so this can finally find it's way in. I am also happy to clean up if there's a commitment to review and get this merged then, avoiding additional future conflicts.
Looking forwarding hearing back from you and getting HA support into the chart!

Same. I have some improvements I'd like to submit, but I'm not sure if it's worth trying to include them in this PR or creating a separate one.

please make a spearate pr and raise it will look into it

Bloodraven21 · 2026-07-28T14:27:10Z

@daanvinken @Bloodraven21 Pinging you here as you've been most active lately in committing to main.

It would really be great to finalize this feature (in here or a separate clean branch) so this can finally find it's way in. I am also happy to clean up if there's a commitment to review and get this merged then, avoiding additional future conflicts.

Looking forwarding hearing back from you and getting HA support into the chart!

please create a spearate one

khtee force-pushed the main branch from a9d23ba to c9ef751 Compare February 9, 2026 00:12

dmaes mentioned this pull request Feb 11, 2026

Implement HA with Sentinel #22

Open

khtee marked this pull request as draft February 18, 2026 23:57

khtee changed the title ~~feat: Add Valkey Sentinel support in High Availability setup.~~ feat: Add Valkey Sentinel & HAProxy support in High Availability setup. Feb 19, 2026

khtee marked this pull request as ready for review February 19, 2026 01:19

khtee marked this pull request as draft February 19, 2026 01:30

khtee marked this pull request as ready for review February 19, 2026 06:11

dmaes suggested changes Feb 19, 2026

View reviewed changes

Comment thread valkey/values.yaml Outdated

Comment thread valkey/templates/haproxy-deployment.yaml Outdated

Comment thread valkey/templates/haproxy-deployment.yaml

Comment thread valkey/values.yaml Outdated

Comment thread valkey/values.yaml

khtee force-pushed the main branch from 35e72a3 to 26d5117 Compare February 20, 2026 01:47

khtee and others added 5 commits February 20, 2026 09:53

feat: Add Valkey Sentinel support in High Availability setup.

4c7c981

Signed-off-by: KHTee <teekahhui@hotmail.com>

feat: Add Valkey Sentinel & HAProxy support in High Availability setup.

32de638

Signed-off-by: KHTee <teekahhui@hotmail.com>

Add securityContext to HAProxy & Sentinel watcher

708c707

Co-authored-by: Dieter Maes <dieter.maes@dmaes.be> Signed-off-by: khtee <75174583+khtee@users.noreply.github.com>

Use .Chart.AppVersion for sentinel watcher

adc429d

Signed-off-by: KHTee <teekahhui@hotmail.com>

khtee force-pushed the main branch from 26d5117 to 7184810 Compare February 20, 2026 01:55

khtee requested a review from dmaes February 20, 2026 01:56

LouisVallat mentioned this pull request Feb 25, 2026

Replace Bitnami Redis dependency and provide Helm chart support for Valkey passbolt/charts-passbolt#122

Open

tkarger reviewed Mar 11, 2026

View reviewed changes

Comment thread valkey/templates/haproxy-configmap.yaml Outdated

tkarger reviewed Mar 12, 2026

View reviewed changes

Comment thread valkey/templates/haproxy-deployment.yaml

tkarger reviewed Mar 12, 2026

View reviewed changes

Comment thread valkey/values.yaml Outdated

tkarger reviewed Mar 12, 2026

View reviewed changes

Comment thread valkey/values.schema.json

Munken reviewed Mar 31, 2026

View reviewed changes

jose added 3 commits April 2, 2026 16:45

feat(sentinel): add native ACL support with secure password handling

2421963

fix(chart): resolve schema validation and template errors

add Inter-Sentinel and Sentinel-Monitor authentication configuration

fccfbbc

updated

0e9d6f3

jose-10000 mentioned this pull request Apr 3, 2026

Valkey Sentinel: Fixing Auth/ACL gaps in TLS environments and security hardening (Enhancements for PR #137) khtee/valkey-helm#1

Merged

jose and others added 3 commits April 7, 2026 14:29

Clenning haproxy-service and added support for commonLabels and podAn…

d6b55d5

…notations in haproxy-deployment

Merge pull request #1 from jose-10000/fix/sentinel-acl-hardening-and-…

d8f854d

…schema Valkey Sentinel: Fixing Auth/ACL gaps in TLS environments and security hardening (Enhancements for PR valkey-io#137)

yoannrt reviewed Apr 9, 2026

View reviewed changes

lazariv suggested changes Apr 13, 2026

View reviewed changes

r4d1f suggested changes Jun 9, 2026

View reviewed changes

This was referenced Aug 1, 2026

feat(valkey): add Valkey Sentinel for automatic failover #234

Open

feat(valkey): add an optional HAProxy front-end for Sentinel #235

Open

-haproxy:
+# Enable haproxy in front of sentinel
+# This is allows running redis in sentinel mode even when the client is not compatible
+# https://arystech.com/blog/setting-up-haproxy-with-redis-sentinel-for-high-availability-on-microk8s-kubernetes
+haproxy:

		@@ -188,3 +188,34 @@ Validate replica authentication configuration
		{{- end }}
		{{- end -}}

+{{/*
+Validate haproxy is used in replica mode
+*/}}
+{{- define "valkey.validateHaproxyRequirements" -}}
+{{- if and .Values.haproxy.enabled (not .Values.replica.enabled) }}
+  {{- fail "Haproxy is only relevant in replica mode with clients incompatible with Sentinel." }}
+{{- end }}
+{{- end -}}

-      timeout server {{ .Values.haproxy.config.timeout.server }}
+      timeout server {{ .Values.haproxy.config.timeout.server }}
+      timeout tunnel {{ .Values.haproxy.config.timeout.tunnel }}
+      option clitcpka
+      option srvtcpka

-      server: 1m
+      server: 1m
+      # Timeout for long-lived bidirectional connections (e.g. pub/sub).
+      # Set to 0 to keep pub/sub SUBSCRIBE connections alive indefinitely.
+      tunnel: 0s

Uh oh!

Conversation

khtee commented Feb 6, 2026

Uh oh!

dmaes commented Feb 11, 2026

Uh oh!

khtee commented Feb 11, 2026

Uh oh!

yoannrt commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amontalban commented Feb 13, 2026

Uh oh!

dmaes commented Feb 14, 2026

Uh oh!

yoannrt commented Feb 17, 2026

Uh oh!

khtee commented Feb 19, 2026

Uh oh!

dmaes commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lazariv commented Mar 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Munken Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

jose-10000 commented Apr 3, 2026

Uh oh!

yoannrt Apr 9, 2026

Choose a reason for hiding this comment

Uh oh!

lazariv left a comment

Choose a reason for hiding this comment

Uh oh!

lazariv Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lazariv Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

lazariv Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

jose-10000 commented Apr 27, 2026

Uh oh!

jose-10000 commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jose-10000 commented May 18, 2026

Uh oh!

dmaes commented May 18, 2026

Uh oh!

jose-10000 commented May 18, 2026

Uh oh!

trabelsieyal commented May 20, 2026

Uh oh!

mlimardo1984 commented Jun 4, 2026

Uh oh!

r4d1f Jun 9, 2026

Choose a reason for hiding this comment

Uh oh!

geodimm Jun 11, 2026

Choose a reason for hiding this comment

Uh oh!

jsoref commented Jun 9, 2026

Uh oh!

Nils98Ar commented Jun 10, 2026

Uh oh!

kaisen-san commented Jun 18, 2026

yoannrt commented Feb 11, 2026 •

edited

Loading

dmaes commented Feb 19, 2026 •

edited

Loading

lazariv Apr 13, 2026 •

edited

Loading

jose-10000 commented Apr 30, 2026 •

edited

Loading

yoannrt commented Jun 18, 2026 •

edited

Loading

kaisen-san commented Jul 28, 2026 •

edited

Loading

Bloodraven21 commented Jul 28, 2026 •

edited

Loading