Skip to content

feat(cluster): CLUSTER REPLICATE + role propagation via gossip - #186

Merged
kacy merged 1 commit into
mainfrom
feat/cluster-replicate
Feb 19, 2026
Merged

feat(cluster): CLUSTER REPLICATE + role propagation via gossip#186
kacy merged 1 commit into
mainfrom
feat/cluster-replicate

Conversation

@kacy

@kacy kacy commented Feb 19, 2026

Copy link
Copy Markdown
Owner

summary

implements CLUSTER REPLICATE and the gossip machinery needed to propagate role changes across the cluster.

gossip layer (ember-cluster)

  • new NodeUpdate::RoleChanged variant (binary tag 6) — carries node id, incarnation, is_primary flag, and optional primary id for replicas. fully serialized/deserialized alongside the existing update types.
  • GossipEvent::RoleChanged(NodeId, bool, Option<NodeId>) emitted when a peer's role changes and the incarnation is newer than what we've seen.
  • GossipEngine::queue_role_update() queues the update for epidemic dissemination via piggybacking on Ping/Ack messages.
  • replicates: Option<NodeId> added to MemberState.

server cluster layer (ember-server)

  • ClusterCoordinator::cluster_replicate(&str) — validates target exists and is a primary, updates local node role + replicates field, registers self in primary's replica list, queues gossip announcement, saves nodes.conf.
  • ClusterCoordinator::is_replica() and primary_addr_for_slot(u16) for replica-aware routing.
  • GossipEvent::RoleChanged handled in the event consumer to keep ClusterState in sync with gossip.
  • CLUSTER REPLICATE stub replaced with the real implementation; CLUSTER FAILOVER stub updated to a clearer "not yet implemented" message.

protocol layer (ember-protocol)

  • Command::is_write() — predicate covering all mutation commands (SET, DEL, INCR, LPUSH, ZADD, HSET, SADD, FLUSHDB, BGSAVE, BGREWRITEAOF, RESTORE, vector and protobuf writes, etc.)
  • Command::primary_key() — returns the first key argument for slot calculation.

replica read/write policy

  • writes on replicas return MOVED <slot> <primary_addr> or READONLY … if no primary found.
  • reads on replicas are served locally (slot routing bypassed via cluster_slot_check).

what was tested

  • all unit tests pass (cargo test -p ember-cluster -p ember-protocol -p ember-server)
  • cargo clippy --workspace -- -D warnings clean
  • new integration tests for CLUSTER REPLICATE: unknown node, self-replicate rejection, invalid id
  • gossip unit tests: apply_role_changed_updates_member, stale_role_changed_ignored
  • command unit tests: is_write_returns_true_for_mutations, is_write_returns_false_for_reads, primary_key_returns_first_key
  • cluster unit tests: replicate state update, is_replica, primary_addr_for_slot

design notes

role changes use incarnation numbers (same mechanism as suspicion refutation) to prevent stale gossip from overwriting newer state. replicas serve reads locally because in most deployments read-scaling is the primary motivation for replication; write redirection via MOVED maintains protocol compatibility with redis cluster clients.

…gossip

- add `NodeUpdate::RoleChanged` gossip message variant (tag 6) with
  binary encode/decode; carries node id, incarnation, is_primary, and
  optional replicates field
- add `GossipEvent::RoleChanged(NodeId, bool, Option<NodeId>)` emitted
  by apply_updates() when a peer's role changes
- add `GossipEngine::queue_role_update()` to piggyback role changes on
  outgoing pings and acks
- add `replicates: Option<NodeId>` to `MemberState` to track which
  primary each replica mirrors
- add `ClusterCoordinator::cluster_replicate()` — validates target,
  updates local state, queues gossip announcement, saves nodes.conf
- add `ClusterCoordinator::is_replica()` and `primary_addr_for_slot()`
  for replica-aware routing
- handle `GossipEvent::RoleChanged` in spawn_gossip to update ClusterState
- add `Command::is_write()` to identify mutation commands
- add `Command::primary_key()` to extract the first key for slot routing
- replace REPLICATE and FAILOVER stubs in connection.rs; REPLICATE now
  dispatches to cluster_replicate(), FAILOVER returns a clear message
- add replica write rejection in execute(): writes on replicas are
  redirected to the primary via MOVED; reads are served locally
- update cluster_slot_check() to skip slot routing on replicas
- update integration tests: replicate_stub → targeted error-path tests;
  failover_stub updated to check new message text
@kacy
kacy merged commit 25bd676 into main Feb 19, 2026
4 of 7 checks passed
@kacy
kacy deleted the feat/cluster-replicate branch February 19, 2026 00:30
kacy added a commit that referenced this pull request Feb 19, 2026
…gossip (#186)

- add `NodeUpdate::RoleChanged` gossip message variant (tag 6) with
  binary encode/decode; carries node id, incarnation, is_primary, and
  optional replicates field
- add `GossipEvent::RoleChanged(NodeId, bool, Option<NodeId>)` emitted
  by apply_updates() when a peer's role changes
- add `GossipEngine::queue_role_update()` to piggyback role changes on
  outgoing pings and acks
- add `replicates: Option<NodeId>` to `MemberState` to track which
  primary each replica mirrors
- add `ClusterCoordinator::cluster_replicate()` — validates target,
  updates local state, queues gossip announcement, saves nodes.conf
- add `ClusterCoordinator::is_replica()` and `primary_addr_for_slot()`
  for replica-aware routing
- handle `GossipEvent::RoleChanged` in spawn_gossip to update ClusterState
- add `Command::is_write()` to identify mutation commands
- add `Command::primary_key()` to extract the first key for slot routing
- replace REPLICATE and FAILOVER stubs in connection.rs; REPLICATE now
  dispatches to cluster_replicate(), FAILOVER returns a clear message
- add replica write rejection in execute(): writes on replicas are
  redirected to the primary via MOVED; reads are served locally
- update cluster_slot_check() to skip slot routing on replicas
- update integration tests: replicate_stub → targeted error-path tests;
  failover_stub updated to check new message text
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant