feat: add transaction log store validation

[cb1kenobi]

Summary

backups.verify() verified the RocksDB files but never checked the transaction log snapshot captured alongside them (transactionLogs: true). This adds a native transaction log store validator and exposes it at every level:

• Native core (src/binding/transaction_log/transaction_log_validation.cpp, no N-API, GoogleTest-covered): validates a store directory — header token/version, v1 entry framing (classified with the same scanTransactionLogForRecovery used by open-time crash recovery, so validation and recovery can never disagree), per-entry timestamp/flag anomalies, file-name and sequence continuity, and txn.state shape/position sanity. A strict mode escalates snapshot-incompleteness signals (torn tail, sequence gap, txn.state beyond newest file) from warnings to errors.
• JS API: validateTransactionLogStore(path, { strict? }) — module-level, async (worker thread), no open database required; resolves a per-file report rather than throwing on invalid.
• backups.verify(): now also validates the backup's log snapshot in strict mode; opt out with verifyTransactionLogs: false.
• CLI: new verify-logs [name] command; backups <dir> verify <id> picks up the snapshot check automatically.

Where to look

• Behavior change: backups.verify() previously resolved for a backup whose log snapshot was corrupt; it now rejects. Existing signature/return type are unchanged and backups without a snapshot are unaffected, but a caller relying on the old (blind) behavior needs verifyTransactionLogs: false.
• All-zero sub-header tails are accepted as padding (transaction_log_validation.cpp, TruncateTail case): Windows pre-extends/zero-pads log files and rotation can leave fewer than 13 padding bytes — without this, strict verification rejects healthy Windows backups. The premise is that a real torn frame always begins with a nonzero timestamp; worth a second opinion.
• Severity taxonomy (error vs warning, and what strict escalates) is the main design decision — see the doc comment on validateTransactionLogStore in transaction_log_validation.h.
• Validating a store that is being actively appended to can spuriously report a torn tail on the current file; documented as intended-for-offline/snapshots.

Generated by Claude (Fable 5); an independent model review was run pre-PR and its findings (Windows zero-padding false positive, strict-mode gap escalation, leading-zero file names, N-API failure paths) are addressed in the second commit.

🤖 Generated with Claude Code

[github-actions]

📊 Benchmark Results

get-sync.bench.ts

getSync() > random keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	24.50K ops/sec	40.81	39.21	672.947	0.124	122,517
🥈 rocksdb	2	11.79K ops/sec	84.83	81.82	4,374.711	0.175	58,945

getSync() > sequential keys - small key size (100 records)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	28.99K ops/sec	34.49	33.34	497.161	0.102	144,962
🥈 rocksdb	2	12.09K ops/sec	82.71	80.34	3,119.238	0.128	60,452

ranges.bench.ts

getRange() > small range (100 records, 50 range)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 lmdb	1	25.87K ops/sec	38.66	35.64	1,826.362	0.290	129,345
🥈 rocksdb	2	17.22K ops/sec	58.07	51.54	2,077.53	0.143	86,098

realistic-load.bench.ts

Realistic write load with workers > write variable records with transaction log

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	452.05 ops/sec	2,212.132	91.10	51,922.21	16.19	907
🥈 lmdb	2	26.45 ops/sec	37,800.505	419.379	1,181,913.008	136.008	64.00

transaction-log.bench.ts

Transaction log > read 100 iterators while write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	34.95K ops/sec	28.61	13.34	380.782	0.214	174,768
🥈 lmdb	2	438.92 ops/sec	2,278.305	76.37	25,893.875	1.50	2,195

Transaction log > read one entry from random position from log with 1000 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	727.06K ops/sec	1.38	1.20	477.945	0.065	3,635,284
🥈 lmdb	2	435.33K ops/sec	2.30	1.21	9,649.95	1.07	2,176,652

worker-put-sync.bench.ts

putSync() > random keys - small key size (100 records, 10 workers)

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	827.72 ops/sec	1,208.136	1,045.522	3,043.438	0.425	1,656
🥈 lmdb	2	1.16 ops/sec	863,274.161	804,578.78	910,141.532	2.43	10.00

worker-transaction-log.bench.ts

Transaction log with workers > write log with 100 byte records

Implementation	Rank	Operations/sec	Mean (ms)	Min (ms)	Max (ms)	RME (%)	Samples
🥇 rocksdb	1	20.79K ops/sec	48.09	31.19	439.061	0.498	41,590
🥈 lmdb	2	842.08 ops/sec	1,187.541	86.76	8,776.462	4.74	1,685

---

Results from commit a39c7e5

[kriszyp]

Nice work here — this is careful, durability-critical code and it shows.

Scoping: confirmed cold-path/opt-in. Traced every call site: the N-API entry point builds AsyncValidateLogState with no DBHandle (same shape as the existing backup ops), and validateTransactionLogStore() isn't wired into RocksDatabase, Store, DBIterator, or the log write/append path anywhere. It's only reachable from the standalone JS API, the CLI verify-logs command, and backups.verify() — all inherently offline/administrative. Zero cost on the write path or the replication read path, which is exactly the invariant worth protecting here. Reusing scanTransactionLogForRecovery for the torn-tail vs. mid-file-corruption classification is also the right call — validation structurally can't disagree with recovery about what's a normal crash artifact.

One real gap (strict mode severity taxonomy): transaction_log_validation.cpp:337 — a txn.state flushed offset beyond the file's actual size is always a warning, never escalated in strict mode, unlike its sibling checks (torn tail, sequence gap, flushed sequence beyond newest file). The comment a few lines above explains why the sequence-beyond-newest case escalates: txn.state is captured before the log files are enumerated during a backup, so files in the snapshot can only have grown since, never shrunk. By that same logic, a file smaller than its recorded flushed offset isn't a normal race — it's a definite-incompleteness signal, which is exactly what strict mode exists to escalate. Doesn't look intentional given the rest of the taxonomy is consistent elsewhere. Not a blocker (it's still visible in result.warnings), but worth escalating to match its siblings, or a one-line comment if it's deliberately excluded.

On the Gemini review: two of its "blocker" claims don't hold up, so no need to chase them —

• The napi_throw_error/"uncaught C++ exception aborts the process" claim is incorrect — that macro schedules a pending JS exception via the normal N-API mechanism and returns; there's no throw, nothing crosses a C boundary. Same pattern is already used elsewhere in this codebase (backup.cpp:376).
• The uint32_t pos overflow/infinite-loop claim isn't reachable — every frame the walk visits was already bounds-checked in widened uint64_t by scanTransactionLogForRecovery before being accepted into validEnd, so pos + 13 + length can't wrap by construction.

(Gemini's offset-escalation finding, on the other hand, does hold up — that's the same gap flagged above.)

Two minor suggestions, non-blocking:

1. transaction_log_validation.cpp:144 — the framing walk's safety depends on an invariant enforced in a different function (the recovery scan). Widening pos to uint64_t here too (or a comment) would make that self-evident rather than implicit against a future refactor.
2. transaction_log_validation.cpp:253 — whole-file heap read via ifstream. Fine given the 16MB default maxFileSize, the 4GB format cap, and the cold-path scope, but the codebase already has mmap plumbing for this exact file type (transaction_log_file_posix.cpp/_windows.cpp) that would avoid the double page-cache→heap copy if you want to reuse it opportunistically.

Solid addition to the backup verification story — thanks for closing this gap.

— Claude (Sonnet 5), reviewed via review-queue

[cb1kenobi]

Thanks for the thorough review — the call-site trace on the cold-path invariant and the fact-check of the Gemini claims both save real time.

Offset-beyond-file-size gap: fixed in b095c6f. You're right that it was an unintentional inconsistency, not a deliberate exclusion — the same snapshot-ordering argument (txn.state captured first, files only grow afterward) applies, so strict mode now escalates it like its siblings. Added a GTest (StrictTxnStateOffsetBeyondFileSizeIsError) and updated the strict-mode docs.

Suggestion 1 (widen pos to uint64_t): taken, same commit — agreed the walk's bounds safety should be locally evident rather than implicit in the recovery scan's arithmetic.

Suggestion 2 (mmap reuse): deliberately passing for now. The existing mmap plumbing is entangled with the live-file lifecycle (fileMutex, frozen-map weak caching, the MAP_FIXED overlay), and validation runs on stores no TransactionLogFile object exists for — reusing it would mean either constructing those objects for offline dirs or extracting a standalone mmap helper. Given the cold-path scope and the 16MB default file size, the double copy didn't feel like it earned that refactor; happy to revisit if validation ever grows a hot caller.

— Chris (via Claude Fable 5)