Full-sync streaming compression - review vs replication base - #22
Open
roshkhatri wants to merge 1 commit into
Open
Full-sync streaming compression - review vs replication base#22roshkhatri wants to merge 1 commit into
roshkhatri wants to merge 1 commit into
Conversation
roshkhatri
force-pushed
the
repl-streaming-compression-fullsync-v2
branch
from
July 30, 2026 17:07
eb23371 to
d8abd39
Compare
Signed-off-by: Roshan Khatri <rvkhatri@amazon.com>
roshkhatri
force-pushed
the
repl-streaming-compression-fullsync-v2
branch
from
July 30, 2026 17:42
d8abd39 to
11a2fc6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-sync RDB payload compression (disk-based, diskless, dual-channel) as one commit over the squashed replication branch (
dbe758135), for a clean review diff. Ported to the new inline streamWriter/streamReader API (compression_rio is gone on this base).Compression is capability-negotiated: a replica advertises
REPLCONF capa compressionwhen itsrepl-compressionislz4-stream; the primary compresses a full sync only when every replica in the attaching cohort advertised it (cohort-AND). If any attaching replica is not capable, the payload is sent plaintext to all of them. $EOF framing stays plaintext; the frame checksum replaces the RDB CRC64. A link dropped mid-frame is recoverable truncation (resync), not corruption.Verified: build clean, unit tests green (truncation + in-frame corruption),
repl-fullsync-compression14/14,repl-compressiongreen, compressed CI matrix green.Counterpart of PR #20 / upstream valkey-io#4075; sibling of PR #21.