feat(replication): primary → replica data sync stream - #187
Merged
Conversation
…hard - make AofRecord::to_bytes/from_bytes pub for wire protocol use - add write_snapshot_bytes/read_snapshot_from_bytes to snapshot.rs for in-memory snapshot serialization without filesystem I/O - add ReplicationEvent struct: shard_id, offset, record - add ShardRequest::SerializeSnapshot and ShardResponse::SnapshotData - thread optional broadcast::Sender<ReplicationEvent> through spawn_shard, run_shard, and process_message; publishes after each mutation - add Engine::subscribe_replication() to create broadcast receivers - thread replication_tx through EngineConfig for wiring in main.rs
adds a dedicated TCP replication channel between primary and replica nodes: - replication.rs: new module with ReplicationServer and ReplicationClient - primary side: per-shard snapshot handshake + incremental AofRecord stream - replica side: snapshot load + record replay with exponential backoff reconnect - wire protocol: little-endian framing with version, shard sync, and resync msgs - aof_record_to_shard_request: maps all 20+ record types to ShardRequest - cluster.rs: set_engine, start_replication_server, start_replication_client - replication port = data_port + gossip_offset + 2 (default: 16381) - cluster_replicate now starts the client after updating topology - replication_info for INFO replication output - server.rs: wire engine into coordinator + start replication server at startup - main.rs: create broadcast channel, inject into EngineConfig when cluster enabled - config.rs: include replication_tx in EngineConfig initializer - connection.rs: INFO replication section (role, replicas, master addr)
kacy
added a commit
that referenced
this pull request
Feb 19, 2026
* feat(core): add ReplicationEvent broadcast and SerializeSnapshot to shard - make AofRecord::to_bytes/from_bytes pub for wire protocol use - add write_snapshot_bytes/read_snapshot_from_bytes to snapshot.rs for in-memory snapshot serialization without filesystem I/O - add ReplicationEvent struct: shard_id, offset, record - add ShardRequest::SerializeSnapshot and ShardResponse::SnapshotData - thread optional broadcast::Sender<ReplicationEvent> through spawn_shard, run_shard, and process_message; publishes after each mutation - add Engine::subscribe_replication() to create broadcast receivers - thread replication_tx through EngineConfig for wiring in main.rs * fix(core): update shard test calls for replication_tx param * feat(server): implement replication stream (primary → replica sync) adds a dedicated TCP replication channel between primary and replica nodes: - replication.rs: new module with ReplicationServer and ReplicationClient - primary side: per-shard snapshot handshake + incremental AofRecord stream - replica side: snapshot load + record replay with exponential backoff reconnect - wire protocol: little-endian framing with version, shard sync, and resync msgs - aof_record_to_shard_request: maps all 20+ record types to ShardRequest - cluster.rs: set_engine, start_replication_server, start_replication_client - replication port = data_port + gossip_offset + 2 (default: 16381) - cluster_replicate now starts the client after updating topology - replication_info for INFO replication output - server.rs: wire engine into coordinator + start replication server at startup - main.rs: create broadcast channel, inject into EngineConfig when cluster enabled - config.rs: include replication_tx in EngineConfig initializer - connection.rs: INFO replication section (role, replicas, master addr)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
summary
implements the replication stream between primary and replica nodes (phase 6, pr 2). when a node runs
CLUSTER REPLICATE <id>, it connects to the primary's replication port and loads a full-state snapshot, then receives all subsequent mutations as a live stream.replication.rs — new module with two types:
ReplicationServer: accepts TCP connections from replicas. sends a per-shard binary snapshot (using the existing snapshot format in-memory) followed by an unbounded stream ofAofRecordframes. on broadcast channel lag, sendsMSG_RESYNCand closes; the replica reconnects and resync'sReplicationClient: connects to the primary, loads the snapshot into the local engine via key-based routing, replays incremental records, and reconnects with exponential backoff (500ms → 30s) on any disconnectwire protocol (little-endian framing):
cluster.rs — adds
set_engine,start_replication_server,start_replication_client, andreplication_info.cluster_replicatenow connects to the primary after topology update. replication port =data_port + gossip_offset + 2.server.rs / main.rs — broadcast channel (65536 capacity) is created when cluster mode is enabled, injected into
EngineConfig, and the engine is set on the coordinator after startup so the replication server can use it.connection.rs —
INFO replicationsection withrole,connected_replicas(primary) ormaster_host/port/link_status(replica).what was tested
cargo test --workspace: all unit tests pass. cluster integration tests show the same pre-existing failures as main (12, down from 14 — two tests now pass that didn't before)cargo clippy --workspace -- -D warnings: cleancargo build --workspace: cleandesign considerations
the broadcast channel is the bridge between shard mutations and the replication stream. each shard publishes
ReplicationEvent { shard_id, offset, record }after every successful write. the server subscribes a new receiver per replica connection, so each replica gets an independent stream starting from the current position. snapshots are serialized in-memory usingio::Cursor<Vec<u8>>— avoids disk I/O for full-sync, which matters on fast reconnects.replica routing uses the same FNV-1a key-hash as the primary (via
engine.route(key, request)), so records always land on the correct shard without a separate shard-ID lookup.