Add FileSystem SyncFile API (#14739)#14739
Conversation
|
@xingbowang has exported this pull request. If you are a Meta employee, you can view the originating Diff in D104918547. |
✅ clang-tidy: No findings on changed linesCompleted in 389.2s. |
06ce8d5 to
55fcf14
Compare
Codex Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.🟡 Codex Code ReviewAuto-triggered after CI passed — reviewing commit 55fcf14 ❌ Codex review failed before producing findings. ℹ️ About this responseGenerated by Codex CLI. Limitations:
Commands:
|
55fcf14 to
9533e1b
Compare
Claude Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit 55fcf14 SummaryWell-designed API addition that cleanly replaces the manual reopen+sync+close pattern. The implementation follows existing RocksDB conventions and preserves correct error semantics. Two missing instrumentation overrides are the primary concern. High-severity findings (2):
Full review (click to expand)Findings🔴 HIGHH1. Missing
|
| Context | SyncFile affected? | Behavior correct? | Action needed? |
|---|---|---|---|
| WritePreparedTxnDB | No direct interaction | N/A | None |
| ReadOnly DB | ReadOnlyFileSystem::SyncFile → FailReadOnly() | Yes | None |
| CompactionService | SyncFile not serialized for remote compaction | N/A | None |
| BlobDB | Uses standard FileSystem | Yes | None |
| EncryptedFileSystem | Falls back to default reopen+sync+close | Correct but suboptimal | Optimization opportunity (L1) |
| FaultInjectionTestFS | PR adds override delegating to FileSystem::SyncFile |
Yes — fault injection on sub-ops preserved | None |
| RemapFileSystem | PR adds override with path remapping | Yes | None |
| MockFileSystem | PR adds no-op override returning OK | Yes | None |
| TimedFileSystem | No override — sync timing lost | No | Fix required (H1) |
| CountedFileSystem | No override — sync counting lost | No | Fix required (H2) |
Dismissed findings from debate
- TEST_SYNC_POINT_CALLBACK file handle leak (initially HIGH): Dismissed. When a test callback injects failure after successful
ReopenWritableFile, theunique_ptrdestructor still callsClose()per theFSWritableFilecontract (file_system.hlines 1126-1127). No leak occurs. - Hard link race condition (initially HIGH): Dismissed. The external SST ingestion API contract already requires that the application does not modify the file after calling
IngestExternalFile. This is a pre-existing constraint, not a new issue. - FaultInjectionTestFS missing override (initially MEDIUM): Dismissed. The PR DOES add
SyncFileoverrides to bothFaultInjectionTestFSandFaultInjectionTestEnv, delegating to the base class to preserve fault injection through sub-operations.
Positive Observations
- Clean TODO resolution: Directly addresses the TODO at
db/external_sst_file_ingestion_job.ccand GitHub issue Avoid using ReopenWritableFile for sync file to disk #13741. - Excellent API documentation: The header comments thoroughly document thread safety, error semantics, default behavior, and override guidance.
- Consistent API design: Follows the established pattern of
LinkFile,RenameFile, etc. with proper parameter ordering and naming. - Comprehensive unit tests: 8 new tests covering sync/fsync selection, reopen error propagation, close error surfacing, and error precedence for both
EnvandFileSystemlayers. - Proper wrapper coverage:
EnvWrapper,FileSystemWrapper,CompositeEnv,LegacyFileSystemWrapper,ReadOnlyFileSystem,RemapFileSystem,MockFileSystem, and fault injection wrappers are all handled. - Error precedence is correct: Sync errors take priority over close errors, preserving critical durability information.
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/code_review.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Codex Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.🟡 Codex Code ReviewAuto-triggered after CI passed — reviewing commit 9533e1b ❌ Codex review failed before producing findings. ℹ️ About this responseGenerated by Codex CLI. Limitations:
Commands:
|
Claude Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit 9533e1b SummaryWell-scoped PR that extracts a common reopen+sync+close pattern into a clean High-severity findings (0): No high-severity findings. Full review (click to expand)Findings🔴 HIGHNone. 🟡 MEDIUMM1. Missing
|
| Context | Executes SyncFile? | Assumptions hold? | Action needed? |
|---|---|---|---|
| ReadOnly DB | No (ingestion blocked) | N/A | ReadOnlyFileSystem::SyncFile correctly returns FailReadOnly |
| WritePreparedTxnDB | Yes (standard ingestion) | Yes | Safe |
| EncryptedFileSystem | Delegates to underlying FS | Yes (sync doesn't need encryption) | Document intent |
| MockFileSystem | No-op (returns OK) | Yes | Safe for tests |
| FaultInjectionTestFS | Calls FileSystem::SyncFile base |
Yes — routes through its own ReopenWritableFile |
Correct design |
| Remote/distributed FS | ReopenWritableFile returns NotSupported |
Yes — propagated correctly | Safe |
| Secondary DB | Possible via try_catch_up |
Yes | Safe |
FaultInjection call routing verification:
FaultInjectionTestFS::SyncFile()callsFileSystem::SyncFile()(base class, notFileSystemWrapper)FileSystem::SyncFile()callsthis->ReopenWritableFile()— polymorphic dispatch hitsFaultInjectionTestFS::ReopenWritableFile()- This wraps the file in
TestFSWritableFile, which has fault-injectableSync(),Fsync(), andClose()hooks - If it instead called
FileSystemWrapper::SyncFile(), it would forward totarget_->SyncFile(), completely bypassing fault injection - Verdict: The design is correct and the comment in the code explains the rationale well
EncryptedFileSystem bypass verification:
EncryptedFileSysteminheritsFileSystemWrapper::SyncFile()→ forwards totarget_->SyncFile()- The underlying FS's
SyncFiledefault calls its ownReopenWritableFile→Sync→Close - This syncs the raw encrypted bytes on disk without needing encryption context
- The encryption layer's
ReopenWritableFile(which sets up cipher streams) is correctly NOT needed here - Verdict: Safe — sync is a durability operation, not a data transformation
Positive Observations
- Clean API design: The
SyncFileAPI follows existing patterns (LinkFile,RenameFile) and integrates naturally into the FileSystem hierarchy. - Comprehensive error handling: The sync-versus-close error precedence is correctly implemented in both
Env::SyncFileandFileSystem::SyncFile, with Close always being called even when Sync fails. - Thorough unit tests: 8 new tests covering success, reopen failure, close-error-after-sync, and sync-error-precedence for both Env and FileSystem layers, using well-designed test helper classes.
- Correct fault injection design: The FaultInjectionTestFS/Env overrides correctly call the base class method (not the wrapper) to ensure fault injection hooks are exercised. The comments explain the design rationale.
- Addresses tech debt: Resolves the TODO comment at issue Avoid using ReopenWritableFile for sync file to disk #13741 and simplifies the ingestion code.
- Test sync point migration: Existing tests (
SyncFailure,SyncFileNotSupported) are correctly updated to use new sync point names.
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/code_review.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Summary: - Adds `FileSystem::SyncFile` and `Env::SyncFile` so sync-only callers do not need to use `ReopenWritableFile` directly. - Implements the default path by reopening the file, calling `Sync` or `Fsync`, closing it, and preserving sync-before-close error precedence. - Adds direct default-path tests for success, reopen `NotSupported`, close failure, and sync-versus-close failure precedence for both `Env` and `FileSystem`; the helpers avoid clang-tidy NRVO diagnostics. - Removes the no-op encrypted file-system override and documents why fault-injection overrides intentionally call the base default implementation rather than wrapper forwarding. - Documents the new public API in the unreleased release notes; Sally and Warm Storage integration stay in child diffs. Differential Revision: D104918547
9533e1b to
119aee4
Compare
Codex Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.🟡 Codex Code ReviewAuto-triggered after CI passed — reviewing commit 119aee4 ❌ Codex review failed before producing findings. ℹ️ About this responseGenerated by Codex CLI. Limitations:
Commands:
|
Claude Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit 119aee4 SummaryClean, well-designed API addition that extracts the reopen+sync+close pattern into a reusable High-severity findings (0): No high-severity findings. Medium-severity findings (2):
Full review (click to expand)Findings🟡 MEDIUMM1. Missing
|
| Context | SyncFile behavior | Correct? |
|---|---|---|
| PosixFileSystem | Base default: reopen (O_APPEND, no truncation) + sync + close | Yes |
| FaultInjectionTestFS | Calls FileSystem::SyncFile base → virtual dispatch to own ReopenWritableFile → file tracking preserved |
Yes |
| FaultInjectionTestEnv | Calls Env::SyncFile base → virtual dispatch to own ReopenWritableFile → file tracking preserved |
Yes |
| EncryptedFileSystem | FileSystemWrapper::SyncFile forwards to underlying FS → no encryption needed for sync-only |
Yes |
| ReadOnlyFileSystem | Returns FailReadOnly() |
Yes |
| RemapFileSystem | Remaps path, then forwards to FileSystemWrapper::SyncFile |
Yes |
| MockFileSystem | No-op, returns OK | Yes |
| FileSystemTracingWrapper | Forwards to target, bypasses tracing | Gap (M1) |
| CountedFileSystem | Forwards to target, bypasses counting | Gap (M1) |
| CompositeEnv | Bridges Env→FileSystem correctly | Yes |
| LegacyFileSystemWrapper | Bridges FileSystem→Env correctly | Yes |
Positive Observations
- Improved error handling: The new implementation explicitly calls
Close()and handles close errors, whereas the old code relied on the destructor (which silently ignores close failures). This is a genuine improvement in data integrity. - Clean API extraction: The reopen+sync+close pattern was duplicated inline. Extracting it into a virtual method enables easy overriding by custom file systems (e.g., remote FS can implement sync natively without reopen).
- Correct fault injection routing: The
FaultInjectionTestFS::SyncFileandFaultInjectionTestEnv::SyncFileimplementations correctly call the base classSyncFile(not the wrapper forwarding), ensuring virtual dispatch routes through their ownReopenWritableFileoverrides. This is well-documented in code comments. - Comprehensive test coverage: 8 new tests cover success paths, reopen failure, close failure, and error precedence for both Env and FileSystem layers.
- Proper handling of
NotSupported: TheExternalSstFileIngestionJob::Preparecaller correctly ignoresNotSupportedfromSyncFile, maintaining compatibility with remote file systems.
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/code_review.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Summary: Pull Request resolved: facebook#14739 - Adds `FileSystem::SyncFile` and `Env::SyncFile` so sync-only callers do not need to use `ReopenWritableFile` directly. - Implements the default path by reopening the file, calling `Sync` or `Fsync`, closing it, and preserving sync-before-close error precedence. - Adds direct default-path tests for success, reopen `NotSupported`, close failure, and sync-versus-close failure precedence for both `Env` and `FileSystem`; the helpers avoid clang-tidy NRVO diagnostics. - Removes the no-op encrypted file-system override and documents why fault-injection overrides intentionally call the base default implementation rather than wrapper forwarding. - Documents the new public API in the unreleased release notes; Sally and Warm Storage integration stay in child diffs. Differential Revision: D104918547
Summary: This diff: - Adds public `Env::SyncFile` and `FileSystem::SyncFile` APIs for syncing or fsyncing a file by name without requiring callers to reopen it directly. - Implements the default `Env` and `FileSystem` behavior by reopening the file as writable, calling `Sync` or `Fsync`, closing the handle, and returning the sync/fsync error ahead of a close error when both fail. - Wires `SyncFile` through the Env/FileSystem bridge and wrapper layers, including `EnvWrapper`, `FileSystemWrapper`, `CompositeEnvWrapper`, `RemapFileSystem`, read-only file systems, mock file systems, and fault-injection wrappers. - Keeps fault-injection `SyncFile` overrides on the base default implementation so the operation still exercises the wrapper's `ReopenWritableFile`, wrapped-file `Sync`/`Fsync`, and `Close` hooks instead of bypassing them through target forwarding. - Updates external SST ingestion to call `FileSystem::SyncFile` instead of manually reopening an ingested file as writable, while preserving the `NotSupported` skip behavior and failure logging. - Adds release-note coverage for the new public API and unit coverage for default `Env`/`FileSystem` success, reopen failure, close failure, sync-versus-close failure precedence, and external SST sync behavior. Differential Revision: D104918547
119aee4 to
a02c818
Compare
Summary: This diff: - Adds public `Env::SyncFile` and `FileSystem::SyncFile` APIs for syncing or fsyncing a file by name without requiring callers to reopen it directly. - Implements the default `Env` and `FileSystem` behavior by reopening the file as writable, calling `Sync` or `Fsync`, closing the handle, and returning the sync/fsync error ahead of a close error when both fail. - Wires `SyncFile` through the Env/FileSystem bridge and wrapper layers, including `EnvWrapper`, `FileSystemWrapper`, `CompositeEnvWrapper`, `RemapFileSystem`, read-only file systems, mock file systems, and fault-injection wrappers. - Keeps fault-injection `SyncFile` overrides on the base default implementation so the operation still exercises the wrapper's `ReopenWritableFile`, wrapped-file `Sync`/`Fsync`, and `Close` hooks instead of bypassing them through target forwarding. - Updates external SST ingestion to call `FileSystem::SyncFile` instead of manually reopening an ingested file as writable, while preserving the `NotSupported` skip behavior and failure logging. - Adds release-note coverage for the new public API and unit coverage for default `Env`/`FileSystem` success, reopen failure, close failure, sync-versus-close failure precedence, and external SST sync behavior. Differential Revision: D104918547
a02c818 to
bdedb71
Compare
Summary: Pull Request resolved: facebook#14739 This diff: - Adds public `Env::SyncFile` and `FileSystem::SyncFile` APIs for syncing or fsyncing a file by name without requiring callers to reopen it directly. - Implements the default `Env` and `FileSystem` behavior by reopening the file as writable, calling `Sync` or `Fsync`, closing the handle, and returning the sync/fsync error ahead of a close error when both fail. - Wires `SyncFile` through the Env/FileSystem bridge and wrapper layers, including `EnvWrapper`, `FileSystemWrapper`, `CompositeEnvWrapper`, `RemapFileSystem`, read-only file systems, mock file systems, and fault-injection wrappers. - Keeps fault-injection `SyncFile` overrides on the base default implementation so the operation still exercises the wrapper's `ReopenWritableFile`, wrapped-file `Sync`/`Fsync`, and `Close` hooks instead of bypassing them through target forwarding. - Updates external SST ingestion to call `FileSystem::SyncFile` instead of manually reopening an ingested file as writable, while preserving the `NotSupported` skip behavior and failure logging. - Adds release-note coverage for the new public API and unit coverage for default `Env`/`FileSystem` success, reopen failure, close failure, sync-versus-close failure precedence, and external SST sync behavior. Differential Revision: D104918547
Codex Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.🟡 Codex Code ReviewAuto-triggered after CI passed — reviewing commit bdedb71 ❌ Codex review failed before producing findings. ℹ️ About this responseGenerated by Codex CLI. Limitations:
Commands:
|
Claude Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit bdedb71 SummaryClean, well-structured PR that adds High-severity findings (0): Full review (click to expand)Findings🔴 HIGHNo high-severity findings. 🟡 MEDIUMM1. Behavioral change: Close() status now checked —
|
| Context | Status | Notes |
|---|---|---|
| PosixFileSystem | Safe | Overrides ReopenWritableFile, default SyncFile works correctly |
| EncryptedFileSystem | Safe | Inherits FileSystemWrapper::SyncFile which delegates to target; target's default calls virtual ReopenWritableFile which EncryptedFileSystem overrides |
| FaultInjectionTestFS | Safe | Calls FileSystem::SyncFile base which uses virtual dispatch to this->ReopenWritableFile(), exercising fault injection hooks |
| ReadOnlyFileSystem | Safe | Overrides SyncFile to return FailReadOnly() |
| RemapFileSystem | Safe | Overrides SyncFile to remap path then delegate |
| MockFileSystem | Safe | Overrides SyncFile to return OK |
| TimedFileSystem | Minor gap | Delegates via FileSystemWrapper; SyncFile not individually timed |
| CountedFileSystem | Minor gap | Delegates via FileSystemWrapper; SyncFile not individually counted |
| CompositeEnv bridge | Safe | Correctly bridges Env::SyncFile to FileSystem::SyncFile |
Positive Observations
- Addresses longstanding TODO (issue Avoid using ReopenWritableFile for sync file to disk #13741) cleanly
- Error precedence logic (sync error > close error) is correct and well-tested
- Fault injection design is clever — calling base class default ensures wrapper hooks are exercised via virtual dispatch
- Comprehensive unit tests covering success, reopen failure, close failure, and error precedence for both Env and FileSystem layers
- Proper handling of
NotSupportedpreserved at the ingestion call site - All wrapper layers properly wired (9 different FS/Env implementations updated)
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/code_review.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Summary: Adds public Env::SyncFile and FileSystem::SyncFile APIs and switches external SST ingestion to use the FileSystem-level sync hook. This diff: - Adds public `Env::SyncFile` and `FileSystem::SyncFile` APIs for syncing or fsyncing a file by name without requiring callers to reopen it directly. - Implements the default `Env` and `FileSystem` behavior by reopening the file as writable, calling `Sync` or `Fsync`, closing the handle, and returning the sync/fsync error ahead of a close error when both fail. - Wires `SyncFile` through the Env/FileSystem bridge and wrapper layers, including `EnvWrapper`, `FileSystemWrapper`, `CompositeEnvWrapper`, `RemapFileSystem`, read-only file systems, mock file systems, and fault-injection wrappers. - Keeps fault-injection `SyncFile` overrides on the base default implementation so the operation still exercises the wrapper's `ReopenWritableFile`, wrapped-file `Sync`/`Fsync`, and `Close` hooks instead of bypassing them through target forwarding. - Updates external SST ingestion to call `FileSystem::SyncFile` instead of manually reopening an ingested file as writable, while preserving the `NotSupported` skip behavior and failure logging. - Adds release-note coverage for the new public API and unit coverage for default `Env`/`FileSystem` success, reopen failure, close failure, sync-versus-close failure precedence, and external SST sync behavior. Differential Revision: D104918547
bdedb71 to
dead360
Compare
Codex Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.🟡 Codex Code ReviewAuto-triggered after CI passed — reviewing commit dead360 ❌ Codex review failed before producing findings. ℹ️ About this responseGenerated by Codex CLI. Limitations:
Commands:
|
dead360 to
99e3a62
Compare
Claude Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit dead360 SummaryPR #14739 adds High-severity findings (0): No high-severity findings. Full review (click to expand)Findings🟡 MEDIUMM1. Missing
|
| Context | Does code execute? | Assumptions hold? | Action needed? |
|---|---|---|---|
| ReadOnly DB | YES — returns FailReadOnly() |
YES | None — correctly blocked |
| EncryptedFileSystem | YES — forwards to target | YES — sync doesn't need encryption | None |
| Remote/distributed FS | YES — may return NotSupported |
YES — handled in ingestion | None |
| FaultInjectionTestFS | YES — calls base class default | YES — virtual dispatch exercises hooks | None |
| WritePreparedTxnDB | NO — SyncFile is only in ingestion path | N/A | None |
| User-defined timestamps | NO — SyncFile is only in ingestion path | N/A | None |
| Concurrent ingestion | POSSIBLE — each job has unique file paths | YES — no path conflicts | None |
Positive Observations
- Correct fault injection design:
FaultInjectionTestFS::SyncFilecallsFileSystem::SyncFile(base class) instead of forwarding to target. This ensures virtual dispatch routesReopenWritableFilethrough the fault injection wrapper, exercising all hooks. This is a thoughtful and deliberate design choice, well-documented in the code comment. - Proper error precedence: The sync-before-close error handling correctly prioritizes sync failures over close failures, matching the existing RocksDB convention.
- Clean file handle lifecycle: Explicit
Close()followed by unique_ptr destruction is safe —PosixWritableFile::~PosixWritableFile()checksfd_ >= 0and the base destructors are no-ops. - Good test coverage of the default implementation: All four error scenarios (success, reopen failure, close failure, sync-vs-close precedence) are tested for both
EnvandFileSystemlayers. - Addresses long-standing TODO: This resolves issue Avoid using ReopenWritableFile for sync file to disk #13741 and the TODO comment that has been in the codebase since the original ingestion implementation.
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/code_review.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Codex Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.🟡 Codex Code ReviewAuto-triggered after CI passed — reviewing commit 99e3a62 ❌ Codex review failed before producing findings. ℹ️ About this responseGenerated by Codex CLI. Limitations:
Commands:
|
Claude Code Review - OBSOLETESuperseded by a newer AI review. Expand to see the original review.✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit 99e3a62 SummaryWell-designed API that addresses a real need (issue #13741). The implementation is correct, follows existing conventions, and has good test coverage. No high-severity findings. No high-severity findings. Full review (click to expand)Findings🔴 HIGHNone. 🟡 MEDIUMM1. Close Error Propagation — Intentional Behavioral Change —
|
| Context | SyncFile path | Correct? |
|---|---|---|
| PosixFileSystem | Inherits base FileSystem::SyncFile → calls own ReopenWritableFile |
✅ |
| WinFileSystem | Inherits base FileSystem::SyncFile → calls own ReopenWritableFile |
✅ |
| EncryptedFileSystem | FileSystemWrapper::SyncFile → delegates to target |
✅ (sync doesn't need encryption) |
| FaultInjectionTestFS | Calls FileSystem::SyncFile base → virtual dispatch to own ReopenWritableFile |
✅ |
| FaultInjectionTestEnv | Calls Env::SyncFile base → virtual dispatch to own ReopenWritableFile |
✅ |
| ReadOnlyFileSystem | Returns FailReadOnly() |
✅ (added in PR) |
| RemapFileSystem | Remaps path via EncodePathWithNewBasename → delegates |
✅ (added in PR) |
| MockFileSystem | Returns IOStatus::OK() (no-op) |
✅ |
| CompositeEnv | Bridges Env::SyncFile → FileSystem::SyncFile |
✅ (added in PR) |
| FileSystemWrapper | Delegates to target_->SyncFile |
✅ (added in PR) |
| EnvWrapper | Delegates to target_.env->SyncFile |
✅ (added in PR) |
| LegacyFileSystemWrapper | Bridges FileSystem::SyncFile → Env::SyncFile |
✅ (added in PR) |
| FileSystemTracingWrapper | Inherits FileSystemWrapper delegation (not traced) |
|
| CountedFileSystem | Inherits FileSystemWrapper delegation (counters bypassed) |
Assumption Stress Test
Claim: "SyncFile default is equivalent to old manual pattern"
- Precondition 1: ReopenWritableFile behavior unchanged → ✅ Same call
- Precondition 2: Sync/Fsync behavior unchanged → ✅ Same call with same IOOptions
- Precondition 3: Close behavior equivalent →
⚠️ Old: implicit (destructor, errors lost). New: explicit (errors captured). See M1. - Precondition 4: Error precedence correct → ✅ Sync error returned over close error
Claim: "Fault injection calls base class to exercise wrapper hooks"
- Verified via virtual dispatch:
FaultInjectionTestFS::SyncFile→FileSystem::SyncFile→this->ReopenWritableFile()→ virtual dispatch toFaultInjectionTestFS::ReopenWritableFile. ✅ Correct.
Claim: "EncryptedFileSystem delegation is safe"
EncryptedFileSysteminheritsFileSystemWrapper::SyncFile→target_->SyncFile. The target (e.g., PosixFS) reopens the raw file and fsyncs. Since fsync only flushes OS buffers and doesn't read/write data content, encryption is irrelevant. ✅ Correct.
Double-close safety:
PosixWritableFile::Close()setsfd_ = -1. Destructor checksif (fd_ >= 0). No double-close. ✅ Safe.
Positive Observations
- Clean API design: Follows the established
FileSystempattern perfectly. Default implementation provided, can be overridden by custom filesystems. - Fault injection strategy: Calling base class default instead of forwarding is a clever approach that preserves hook exercising. Well-commented.
- Comprehensive test coverage: Unit tests cover sync/fsync selection, reopen failure, close failure, and sync-vs-close error precedence — both for
EnvandFileSystemlayers. - Error precedence: The sync-before-close error priority is a good design choice — the sync error is more actionable than the close error.
- Code simplification: The ingestion job code is significantly simpler with the new API — reduced from ~15 lines of manual logic to a single API call.
- Release note present: Public API change is documented.
- Resolves technical debt: Addresses the TODO comment and issue Avoid using ReopenWritableFile for sync file to disk #13741 that explicitly requested this API.
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/code_review.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Summary: Adds public path-level file sync APIs to RocksDB so callers can ask `Env` or `FileSystem` to sync a named file without hand-rolled reopen logic. This diff: - Adds public `Env::SyncFile` and `FileSystem::SyncFile` APIs for syncing or fsyncing a file by name without requiring callers to reopen it directly. - Documents the `SyncFile` contract: filesystems may override it as a no-op when flush/close already provides durability, and RocksDB callers should use it instead of hand-rolled `ReopenWritableFile()+Sync()/Fsync()` so filesystems can reject post-close data-write reopen paths. - Implements the default `Env` and `FileSystem` behavior by reopening the file as writable, calling `Sync` or `Fsync`, closing the handle, returning the sync/fsync error ahead of a close error when both fail, and explicitly consuming the close status on the sync-failure path. - Wires `SyncFile` through the Env/FileSystem bridge and wrapper layers, including `EnvWrapper`, `FileSystemWrapper`, `CompositeEnvWrapper`, `EncryptedFileSystem`, `RemapFileSystem`, read-only file systems, mock file systems, and fault-injection wrappers. - Keeps fault-injection `SyncFile` overrides on the base default implementation so the operation still exercises the wrapper's `ReopenWritableFile`, wrapped-file `Sync`/`Fsync`, and `Close` hooks instead of bypassing them through target forwarding. - Updates external SST ingestion to call `FileSystem::SyncFile` instead of manually reopening an ingested file as writable, while preserving the `NotSupported` skip behavior and failure logging. - Adds release-note coverage for the new public API and unit coverage for default `Env`/`FileSystem` success, reopen failure, close failure, sync-versus-close failure precedence, and external SST sync behavior. There was an earlier version of this API which got reverted (PR facebook#13987) due to internal change broken. Re-apply the change. The internal issue will be resolved in next release. Reviewed By: pdillinger Differential Revision: D104918547
99e3a62 to
4c8887c
Compare
🟡 Codex Code ReviewAuto-triggered after CI passed — reviewing commit 4c8887c ❌ Codex review failed before producing findings. ℹ️ About this responseGenerated by Codex CLI. Limitations:
Commands:
|
✅ Claude Code ReviewAuto-triggered after CI passed — reviewing commit 4c8887c SummaryWell-designed API addition that follows established RocksDB patterns. The High-severity findings (0): No high-severity findings. Full review (click to expand)Findings🟡 MEDIUMM1. Missing SyncFile in observability wrappers —
|
| Context | Implementation | Correct? |
|---|---|---|
| ReadOnlyFileSystem | Returns FailReadOnly() | Yes |
| EncryptedFileSystem | Delegates to FileSystemWrapper (target forwarding) | Yes — sync doesn't touch content |
| FaultInjectionTestFS | Calls FileSystem::SyncFile base default | Yes — virtual dispatch exercises wrapped ReopenWritableFile |
| RemapFileSystem | Remaps path then forwards | Yes |
| MockFileSystem | Returns IOStatus::OK() | Yes |
| CompositeEnv | Bridges Env→FileSystem | Yes |
| CountedFS/TimedFS/TracingFS | Inherits forwarding | Works but not instrumented |
Positive Observations
- Clean API design following the established dual-layer (Env + FileSystem) pattern
- Correct error handling with sync-error-over-close-error precedence
- Smart fault injection strategy using base class virtual dispatch
- Good unit test coverage (8 new tests covering success, failure, and error precedence)
- Addresses the longstanding TODO in
external_sst_file_ingestion_job.cc:169-175
ℹ️ About this response
Generated by Claude Code.
Review methodology: claude_md/code_review.md
Limitations:
- Claude may miss context from files not in the diff
- Large PRs may be truncated
- Always apply human judgment to AI suggestions
Commands:
/claude-review [context]— Request a code review/claude-query <question>— Ask about the PR or codebase
Summary:
Adds public path-level file sync APIs to RocksDB so callers can ask
EnvorFileSystemto sync a named file without hand-rolled reopen logic.This diff:
Env::SyncFileandFileSystem::SyncFileAPIs for syncing or fsyncing a file by name without requiring callers to reopen it directly.SyncFilecontract: filesystems may override it as a no-op when flush/close already provides durability, and RocksDB callers should use it instead of hand-rolledReopenWritableFile()+Sync()/Fsync()so filesystems can reject post-close data-write reopen paths.EnvandFileSystembehavior by reopening the file as writable, callingSyncorFsync, closing the handle, returning the sync/fsync error ahead of a close error when both fail, and explicitly consuming the close status on the sync-failure path.SyncFilethrough the Env/FileSystem bridge and wrapper layers, includingEnvWrapper,FileSystemWrapper,CompositeEnvWrapper,EncryptedFileSystem,RemapFileSystem, read-only file systems, mock file systems, and fault-injection wrappers.SyncFileoverrides on the base default implementation so the operation still exercises the wrapper'sReopenWritableFile, wrapped-fileSync/Fsync, andClosehooks instead of bypassing them through target forwarding.FileSystem::SyncFileinstead of manually reopening an ingested file as writable, while preserving theNotSupportedskip behavior and failure logging.Env/FileSystemsuccess, reopen failure, close failure, sync-versus-close failure precedence, and external SST sync behavior.There was an earlier version of this API which got reverted (PR #13987) due to internal change broken. Re-apply the change. The internal issue will be resolved in next release.
Reviewed By: pdillinger
Differential Revision: D104918547