Feature/macos nsfp fileprovider#848
Conversation
Implement a new VFS backend (mode: MacOSNSFileProvider/nsfp) that uses Apple's NSFileProvider framework to provide native Files on Demand support on macOS 12+. Architecture: - VfsNSFP plugin: manages NSFileProvider domain lifecycle, syncs file metadata and access tokens to the App Group shared container, and triggers periodic sync cycles via a 30-second poll timer. - FileProvider extension (appex): runs as a separate sandboxed process, enumerates items from a shared plist, downloads files via direct WebDAV requests, and uploads new files via HTTP PUT/MKCOL. - NsfpDomainManager: handles async domain registration/removal with fileproviderd, including stale domain cleanup and fallback recovery. - NsfpXpcHandler: anonymous NSXPCListener for future bidirectional communication between the main app and the extension. Key features: - Transparent file hydration with real download progress in Finder - Direct WebDAV upload for files dragged into the virtual folder - Automatic metadata refresh after each sync cycle - Server-side deletion detection for NSFP-managed files in discovery - Stale item cleanup on HTTP 404 during fetch - Change tracking with didDeleteItemsWithIdentifiers for Finder updates - User-friendly localized error messages - Token refresh handling with race condition prevention Changes to existing code: - vfs.h/cpp: add MacOSNSFileProvider mode enum and VfsSetupParams constructor with spaceId/displayName/syncEngine - discovery.cpp: handle NSFP files during remote deletion detection - syncjournaldb.cpp: add getFileRecordsByFileId helper - httpcredentials.h: expose accessToken for shared container config - CMakeLists.txt: wire up extension and plugin build targets
Replace hardcoded Team ID (P4D766R5ZA) with the CMake cache variable
APPLE_DEVELOPMENT_TEAM so any developer can build and sign the app
with their own Apple Developer account.
The App Group identifier is now derived at configure time as
${APPLE_DEVELOPMENT_TEAM}.${APPLICATION_REV_DOMAIN} and passed to:
- Entitlements (via configure_file)
- Info.plist (via CMake variable expansion)
- ObjC sources (via compile definition APP_GROUP_IDENTIFIER)
- Xcode build attributes (DEVELOPMENT_TEAM)
Usage: cmake -DAPPLE_DEVELOPMENT_TEAM=XXXXXXXXXX ..
|
That's a very cool feature, but I'm not able to try it out Am I missing something? |
Implement direct WebDAV operations (upload, delete, move) in the FileProvider extension to eliminate XPC dependency. Fix multi-account and multi-space support by using per-domain config and metadata plists. Fix deletion detection by using per-domain prevFileIds caches to prevent race conditions between concurrent extension processes. Add content staging for uploads, configurable App Group entitlements for both main app and extension, and automatic legacy file cleanup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
These files are referenced in src/cmd/CMakeLists.txt but were not tracked in git, causing build failures on other machines. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Alternative VFS plugin using extended attributes for placeholder tracking. Not enabled by default (not in VIRTUAL_FILE_SYSTEM_PLUGINS) but available for builds that prefer xattr-based dehydration over the NSFileProvider approach. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
@sh4tteredd I forgot to push some files. Everything should now be in Git. I have also made some improvements. |
|
Hey, thanks for this PR. Obviously this is a big one, and it will take time to land. Maybe there is a chance to split it down to smaller PRs that are easier to grasp? I added the analysis of what Copilot thinks of it to the description, maybe that helps others to understand what is the intention. |
Make the macOS FileProvider extension authoritative instead of relying on the plist snapshot written by the classic sync engine. The enumerator now talks to the server directly via PROPFIND, with a cache-first probe for instant repeat-opens and a background refresh. - FileProviderConfig: read server URL / credentials from the App Group - FileProviderWebDAV: shared NSURLSession, PROPFIND / GET / PUT / MKCOL / MOVE / DELETE - FileProviderItemCache: file-id<->path map and per-container etag with child lists - FileProviderWorkingSetDelta: etag-based change/deletion detection for enumerateChanges - tests: standalone harness covering cache, WebDAV parsing and the delta computation
- Hold the domain manager's private state in a shared_ptr so async NSFileProviderManager completion handlers can no longer touch freed memory when a space is removed mid-flight (use-after-free on the XPC queue). - Add an optional forceRecreate path to addDomain() and trigger it once per domain (gated by a marker in the App Group container) to clear a corrupted fileproviderd replica that jammed new Finder operations.
In NSFileProvider mode the spaces are reached through Finder Locations, so the on-disk sync root only backs the journal and holds no user-visible files. - Don't add the sync root to Finder Favorites. - Place the backing sync root in a hidden Application Support location instead of an ~/OpenCloud folder full of empty placeholders. - Record created directories in the journal directly instead of writing an empty local directory skeleton to disk.
|
Thanks a lot, and totally fair it is a big one. Happy to break it up to make review tractable. A couple of notes first: Part of the size is just that this branch is behind main, so the diff currently includes unrelated upstream churn (the proxy/networksettings removal, clientproxy, the cfapi API changes, the Transifex updates). I'll rebase onto current main first, which should already shrink it noticeably. Generic VFS plumbing — the mode-agnostic changes in src/libsync/vfs/, the plugin loader, path helpers, journal and discovery hooks. No macOS-specific behavior; reviewable on its own. the quota "undefined" fix in folderstatusmodel (a few lines), |
mj-obs
force-pushed
the
feature/macos-nsfp-fileprovider
branch
from
127c605 to
e02543f
Compare
fileproviderd only re-enumerates when the sync anchor changes and ignores didUpdateItems whose itemVersion is unchanged. Our anchor (count+max modtime) and itemVersion (modtime only) are both invariant under a rename on OCIS, so server-side renames never reached Finder. - Anchor: replace count+max(modtime) with FPItemSetSignature, an order-independent FNV-1a hash of each item's fileId|path|etag, so a rename or move always moves the anchor. - Working-set delta: compare fingerprints (etag+path) instead of etag only, so a path-only change counts as changed. - itemVersion: split into contentVersion (modtime) and metadataVersion (hash of filename|parentId|size|modtime) so a rename bumps the metadata version.
3 participants
Summary
This PR adds a native macOS File Provider extension as a new Virtual File System (VFS) backend.
Apple's NSFileProvider framework is the
modern, sandboxed approach to on-demand sync (similar to iCloud Drive) available on macOS 12+.
Key Components
src/extensions/fileprovider/OpenCloudFileProviderExtension, enumerator, item model, thumbnail provider, and XPC servicesrc/plugins/vfs/fileprovider/vfs_fileprovider) integrating the extension into OpenCloud's VFS abstraction layersrc/cmd/nsfpdiagnostic.mmopencloudcmdto inspect File Provider domain statesrc/OpenCloud.entitlementsCMakeLists.txt)OBJCXXon Apple, wires up the extension target and newfileproviderVFS pluginSyncJournalDB,discovery.cpp, andvfs.h/cppto support the new backendArchitecture
The File Provider extension runs out-of-process as a macOS app extension, communicating with
the main app via XPC (
FileProviderXPCService). It implements Apple'sNSFileProviderExtensionprotocol with three sub-components:
FileProviderEnumerator— tells macOS which files/folders exist (drives on-demand hydration)FileProviderItem— maps OpenCloudSyncFileItemmetadata toNSFileProviderItemFileProviderThumbnails— generates thumbnails on demandNotable Changes to Existing Code
src/libsync/vfs/vfs.h/.cpp— new hook points for the file provider backendsrc/libsync/discovery.cpp— discovery adjustments for file provider semanticssrc/libsync/common/syncjournaldb.cpp— journal DB changes to track provider statesrc/libsync/creds/httpcredentials.h— minor credential surfacing for XPC use