docs: defer compressed on-disk RDB (S3.1/S3.2) to v2

.agents/planning/realtime-data-compression/design/detailed-design.md

@@ -31,7 +31,7 @@ Valkey keeps values in memory as SDS strings. For workloads dominated by `OBJ_ST
 
 This feature adds **opt-in, transparent, server-side, in-memory compression** for `OBJ_STRING` values. Compressed values are automatically decompressed on every **client-facing read path** (client commands, scripts, transactions, the replication feed, AOF writer, module API), so no client or operational tooling changes are required. The feature is disabled by default; enabled, it is fully observable via `INFO`, controllable via `CONFIG SET compression-*` and a new `COMPRESSION` subcommand container, and bounded in both CPU and memory overhead.
 
-Note: the **on-disk RDB file may contain compressed values** (to preserve the memory win across restarts and shrink snapshot size/save time). This is the one intentional exception to "decompressed at every boundary" — it is an internal persistence format, not a client-facing read path. See §2.6 for the full persistence-boundary specification; full-sync replication RDB streams are always uncompressed per R2.6.8.
+Note: **in v2** the on-disk RDB file may contain compressed values (to preserve the memory win across restarts and shrink snapshot size/save time) — the one intentional exception to "decompressed at every boundary," an internal persistence format rather than a client-facing read path. **In v1 this is deferred:** disk RDB is saved uncompressed and the sweeper re-compresses after load (see the §2.6 scope note). See §2.6 for the full persistence-boundary specification; full-sync replication RDB streams are always uncompressed per R2.6.8 in both versions.
 
 ### 1.2 Headline scope statement
 
@@ -281,7 +281,9 @@ This is a deliberate simplification over an earlier design (PR #10) which propos
 
 ### 2.6 Persistence
 
-- **R2.6.1** **RDB on-disk format** (Q12, research `persistence-and-replication.md`):
+> **v1/v2 scope (decided 2026-06-24):** Compressed **on-disk RDB** (R2.6.1–R2.6.4 — the `RDB_ENC_COMPRESSED` encode/decode path) is **deferred to v2**. It is a local-snapshot optimization (smaller/faster RDB, faster warm-up to the compressed state after restart), not a correctness requirement and not a sync-speed improvement. In **v1, every RDB save target — disk, full-sync replication, AOF preamble, DUMP/MIGRATE — emits uncompressed** (each compressed value routed through `objectGetUncompressedView` at save time, per PR #24); on load there is nothing compressed to read, and the sweeper re-compresses in memory afterward. Consequently in v1 the `RDB_ENC_COMPRESSED` byte stays **reserved-but-unused** and `RDB_VERSION` is **not** bumped. R2.6.1–R2.6.4 below describe the **v2** on-disk format. R2.6.5–R2.6.8 (AOF / replication / DUMP / full-sync all uncompressed) hold in **both** v1 and v2.
+
+- **R2.6.1** **RDB on-disk format** (Q12, research `persistence-and-replication.md`) — **v2**:
   - New string encoding marker `RDB_ENC_COMPRESSED` (value `4`). The marker is generic — it says "a compressed payload follows" without naming the algorithm, so future backends (LZ4, snappy, hardware) can reuse the same encoding byte.
   - Layout per compressed value: `[RDB_ENCVAL | alg_magic (len-encoded) | alg_meta (len-encoded) | uncompressed_len (len-encoded) | compressed_len (len-encoded) | compressed frame bytes]`. `alg_magic` is the four-byte algorithm tag (ASCII `ZSTD`, reserved `LZ4 ` etc.); `alg_meta` is interpreted per algorithm — for ZSTD it is the `dict_id` of the referenced dictionary, for a hypothetical LZ4 backend it would be `0` or mode bits. Unknown `alg_magic` → reject as corrupt.
   - Dictionary bytes (ZSTD-specific) are written as `RDB_OPCODE_AUX` entries (key `"compression-dict-<dict_id>"`, value = raw bytes) **before** any compressed value that references them. Non-dict-based algorithms may omit AUX.
@@ -292,7 +294,7 @@ This is a deliberate simplification over an earlier design (PR #10) which propos
 - **R2.6.5** **AOF**: always uncompressed RESP. Writer routes every `robj` through `objectGetUncompressedView` before emitting. (Q12)
 - **R2.6.6** **Replication feed**: always uncompressed RESP. `feedReplicationBufferWithObject` routes through `objectGetUncompressedView`. Cross-version replication unaffected. (Q12)
 - **R2.6.7** **`DUMP` / `RESTORE` / `MIGRATE`**: v1 decompresses before emitting the RDB chunk. Compressed-in-place migration is v2. (Q12)
-- **R2.6.8** **Full-sync replication RDB** (primary → replica during `SYNC`/`PSYNC` full resync): emitted **uncompressed** regardless of `compression-master-switch` state on the primary. When the RDB writer is invoked with a replication sink, every compressed value is routed through `objectGetUncompressedView` before serialization — same helper used by `feedReplicationBufferWithObject`. Disk RDB (local save / `BGSAVE` target) continues to use the `RDB_ENC_COMPRESSED` path from R2.6.1. This keeps cross-version replication working without replica-side awareness and preserves the "wire stays uncompressed RESP/RDB, disk may be compressed" property. Opt-in compressed full-sync (via `REPLCONF` negotiation) is a v2 extension point. (Q12)
+- **R2.6.8** **Full-sync replication RDB** (primary → replica during `SYNC`/`PSYNC` full resync): emitted **uncompressed** regardless of `compression-master-switch` state on the primary. When the RDB writer is invoked with a replication sink, every compressed value is routed through `objectGetUncompressedView` before serialization — same helper used by `feedReplicationBufferWithObject`. In **v2**, disk RDB (local save / `BGSAVE` target) uses the `RDB_ENC_COMPRESSED` path from R2.6.1; in **v1** disk RDB is also uncompressed (the compressed disk path is deferred — see the §2.6 scope note), so the disk-vs-wire distinction only becomes observable in v2. This keeps cross-version replication working without replica-side awareness and preserves the "wire stays uncompressed RESP/RDB, disk may be compressed" property. Opt-in compressed full-sync (via `REPLCONF` negotiation) is a v2 extension point. (Q12)
 
 ### 2.7 Introspection surfaces
 

.agents/planning/realtime-data-compression/implementation/plan.md

@@ -204,10 +204,9 @@ Every subcommand calls into S2 public API; S4 owns reply schema, command JSON, a
 - [ ] **S4.2 — `COMPRESSION` command tree full implementation**: `STATUS`, `DICT LIST`, `DICT DROP`, `SWEEP`, `DEBUG`. Reply-schema tests per §7.4.
 - [ ] **S4.3 — Latency monitor events** per R2.10 (train start/finish, worker stall, etc.)
 - [ ] **S7.2 — CI — long-running perf regression**: nightly job running §7.3 under AddressSanitizer + ThreadSanitizer.
-- [ ] **S3.1 — RDB encode path**: new `RDB_ENC_COMPRESSED` marker, varint-encoded alg_magic/alg_meta/sizes, AUX entries for dictionaries (ZSTD). Implements R2.6.1–R2.6.4.
-- [ ] **S3.2 — RDB decode path**: lookup dict by id, rehydrate `OBJ_ENCODING_COMPRESSED` robj. Unknown-dict error path per R2.6.5.
-- [ ] **S3.3 — Full-sync RDB uncompressed flag**: implements R2.6.8 (new this walkthrough) — full-sync RDB always emits uncompressed regardless of `compression-master-switch` state. Disk RDB still compressed.
-- [ ] **S3.4 — AOF**: compressed values serialized as their uncompressed command forms (`SET key value`). R2.7.
+- [x] **S3.4 — AOF uncompressed (v1)**: AOF on-disk format stays uncompressed RESP. This is unconditional and version-independent — AOF is never compressed in this design. Two AOF write paths exist; only the second can meet a compressed value: (1) **command propagation** (`feedAppendOnlyFile`) writes the command from `argv` at execution time ("on arrival"), before the sweeper ever runs, so it never sees compression; (2) **AOF rewrite** (`BGREWRITEAOF` / auto-rewrite) regenerates the file from the live keyspace and reads *stored* values, which the sweeper may have compressed since arrival — the rewrite helpers (`rioWriteBulkObject`, and `rdbSaveStringObject` when the RDB preamble is enabled) decompress every compressed value before emit (R2.6.5). **Covered by PR #24.**
+- [~] **S3.1 / S3.2 — Compressed on-disk RDB (encode + decode) → deferred to v2.** `RDB_ENC_COMPRESSED` on-disk encoding + AUX dict entries + decode (R2.6.1–R2.6.4). This is a *local-snapshot optimization* (smaller/faster RDB file, faster warm-up back to the compressed state after a restart) — **not** a correctness requirement and **not** a sync-speed improvement (full-sync is uncompressed in v1 regardless, R2.6.8). v1 is correct without it: disk RDB stores uncompressed and the sweeper re-compresses after load. In v1 the `RDB_ENC_COMPRESSED` (=4) encoding byte stays reserved-but-unused and `RDB_VERSION` is **not** bumped. See §8. *(deferral decision 2026-06-24, @GilboaAWS lead)*
+  - **Note — what was "S3.3" (full-sync uncompressed flag):** dropped as a standalone task. The flag that forces the wire uncompressed only has meaning once the disk RDB can be compressed (i.e., in v2). At that point, per-target selection — disk save (compressed) vs. a `REPLCONF compression yes`-negotiated replica (compressed sync) vs. a legacy replica (uncompressed) — is just an implementation detail of the v2 compressed-RDB + `REPLCONF` feature, not a separate deliverable. In v1 there is nothing to do: every target saves uncompressed already (PR #24), so full-sync is uncompressed by construction.
 
 **End of Phase 1 gate:**
 - Feature works end-to-end on a single replica set with `compression-master-switch compression` + `compression-automatic-sweeper enabled` + manual dict training.
@@ -291,6 +290,7 @@ Total calendar: ~11 weeks assuming no significant delays. Series work was estima
 From §1.4 (non-goals) and the walkthrough decisions — do **not** implement in this plan:
 
 - Wire-level compression negotiation (`REPLCONF compression yes`) — deferred to v2 (Threads #2, #31)
+- Compressed on-disk RDB (`RDB_ENC_COMPRESSED` encode/decode — S3.1/S3.2) — deferred to v2 (decision 2026-06-24). A local-snapshot-only optimization (smaller/faster RDB, faster post-restart warm-up); v1 saves uncompressed and the sweeper re-compresses after load. No correctness or sync-speed impact. `RDB_ENC_COMPRESSED`=4 stays reserved-but-unused; `RDB_VERSION` not bumped in v1.
 - IO threads for decompression — rejected in v1 per Appendix §C.7 (Thread #25)
 - Async decompression path for long reads — v2 opt-in (Thread #26 context)
 - LIST/HASH/ZSET per-element compression — v1 is STRING-only (§1.4)