Add S3 storage sync

.github/workflows/dotnet.yml

@@ -44,6 +44,20 @@ jobs:
           PASV_MIN_PORT: 21000
           PASV_MAX_PORT: 21010
 
+      localstack:
+        image: localstack/localstack:latest
+        ports:
+          - 4566:4566
+        options: >-
+          --health-cmd "curl -f http://localhost:4566/_localstack/health || exit 1"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+        env:
+          SERVICES: s3
+          DEBUG: 0
+          EDGE_PORT: 4566
+
     steps:
     - uses: actions/checkout@v5
     - name: Setup .NET
@@ -56,6 +70,9 @@ jobs:
       run: dotnet format --verify-no-changes
     - name: Build
       run: dotnet build --no-restore
+    - name: Create S3 test bucket
+      run: |
+        docker exec $(docker ps -q -f ancestor=localstack/localstack:latest) awslocal s3 mb s3://test-bucket
     - name: Test
       run: dotnet test --no-build --verbosity normal
       env:
@@ -69,3 +86,8 @@ jobs:
         FTP_TEST_USER: testuser
         FTP_TEST_PASS: testpass
         FTP_TEST_ROOT: ""
+        S3_TEST_BUCKET: test-bucket
+        S3_TEST_ACCESS_KEY: test
+        S3_TEST_SECRET_KEY: test
+        S3_TEST_ENDPOINT: http://localhost:4566
+        S3_TEST_PREFIX: sharpsync-tests

CLAUDE.md

@@ -33,18 +33,20 @@ dotnet test --logger trx --results-directory TestResults
 ```
 
 #### Running Integration Tests
-Integration tests require external services (SFTP server). Use the provided scripts:
+Integration tests require external services (SFTP, FTP, S3 via LocalStack). Use the provided scripts:
 
 ```bash
-# Linux/macOS - automatically starts Docker SFTP server
+# Linux/macOS - automatically starts Docker test servers
 ./scripts/run-integration-tests.sh
 
-# Windows - automatically starts Docker SFTP server
+# Windows - automatically starts Docker test servers
 .\scripts\run-integration-tests.ps1
 
 # Or manually with Docker Compose
 docker-compose -f docker-compose.test.yml up -d
 export SFTP_TEST_HOST=localhost SFTP_TEST_PORT=2222 SFTP_TEST_USER=testuser SFTP_TEST_PASS=testpass SFTP_TEST_ROOT=/home/testuser/upload
+export FTP_TEST_HOST=localhost FTP_TEST_PORT=21 FTP_TEST_USER=testuser FTP_TEST_PASS=testpass FTP_TEST_ROOT=/
+export S3_TEST_BUCKET=test-bucket S3_TEST_ACCESS_KEY=test S3_TEST_SECRET_KEY=test S3_TEST_ENDPOINT=http://localhost:4566
 dotnet test --verbosity normal
 docker-compose -f docker-compose.test.yml down
 ```
@@ -67,7 +69,7 @@ dotnet pack --configuration Release --version-suffix preview
 The project uses GitHub Actions for CI/CD. The pipeline currently:
 - Builds on Ubuntu only (multi-platform testing planned)
 - Runs tests with format checking
-- Includes SFTP and FTP integration tests using Docker-based servers
+- Includes SFTP, FTP, and S3 integration tests using Docker-based servers (LocalStack for S3)
 - Automatically configures test environment variables for integration tests
 
 ## High-Level Architecture
@@ -89,7 +91,7 @@ SharpSync is a **pure .NET file synchronization library** with no native depende
    - `WebDavStorage` - WebDAV with OAuth2, chunking, and platform-specific optimizations (implemented, needs tests)
    - `SftpStorage` - SFTP with password and key-based authentication (fully implemented and tested)
    - `FtpStorage` - FTP/FTPS with secure connections support (fully implemented and tested)
-   - `StorageType` enum includes: S3 (planned for future versions)
+   - `S3Storage` - Amazon S3 and S3-compatible storage (MinIO, LocalStack) with multipart uploads (fully implemented and tested)
 
 3. **Authentication** (`src/SharpSync/Auth/`)
    - `IOAuth2Provider` - OAuth2 authentication abstraction (no UI dependencies)
@@ -109,14 +111,15 @@ SharpSync is a **pure .NET file synchronization library** with no native depende
 
 ### Key Features
 
-- **Multi-Protocol Support**: Local, WebDAV, SFTP, and FTP/FTPS storage (extensible to S3)
+- **Multi-Protocol Support**: Local, WebDAV, SFTP, FTP/FTPS, and S3-compatible storage
 - **OAuth2 Authentication**: Full OAuth2 flow support without UI dependencies (WebDAV)
 - **SSH Key & Password Auth**: Secure SFTP authentication with private keys or passwords
 - **FTP/FTPS Support**: Plain FTP, explicit FTPS, and implicit FTPS with password authentication
+- **S3 Compatibility**: Full AWS S3 support plus S3-compatible services (MinIO, LocalStack, etc.)
 - **Smart Conflict Resolution**: Rich conflict analysis for UI integration
 - **Selective Sync**: Include/exclude patterns for files and folders
 - **Progress Reporting**: Real-time progress events for UI binding
-- **Large File Support**: Chunked uploads with platform-specific optimizations
+- **Large File Support**: Chunked/multipart uploads with platform-specific optimizations
 - **Network Resilience**: Retry logic and error handling with automatic reconnection
 - **Parallel Processing**: Configurable parallelism with intelligent prioritization
 
@@ -127,6 +130,7 @@ SharpSync is a **pure .NET file synchronization library** with no native depende
 - `WebDav.Client` (2.9.0) - WebDAV protocol
 - `SSH.NET` (2025.1.0) - SFTP protocol implementation
 - `FluentFTP` (52.0.2) - FTP/FTPS protocol implementation
+- `AWSSDK.S3` (3.7.*) - Amazon S3 and S3-compatible storage
 - Target Framework: .NET 8.0
 
 ### Platform-Specific Optimizations
@@ -285,11 +289,11 @@ The core library is production-ready, but several critical items must be address
 
 ### 🔄 CAN DEFER TO v1.1+
 
-10. **~~SFTP~~/~~FTP~~/S3 Implementations** (SFTP ✅ DONE, FTP ✅ DONE)
+10. **~~SFTP~~/~~FTP~~/~~S3~~ Implementations** ✅ **ALL DONE!**
     - ✅ SFTP now fully implemented with comprehensive tests
     - ✅ FTP/FTPS now fully implemented with comprehensive tests
-    - S3 remains future feature for v1.1+
-    - StorageType enum prepared for future S3 implementation
+    - ✅ S3 now fully implemented with comprehensive tests and LocalStack integration
+    - All major storage backends are now complete!
 
 11. **Performance Benchmarks**
     - BenchmarkDotNet suite for sync operations
@@ -325,30 +329,28 @@ The core library is production-ready, but several critical items must be address
 
 **Minimum Acceptance Criteria:**
 - ✅ Core sync engine tested (achieved)
-- ⚠️ All storage implementations tested (LocalFileStorage ✅, SftpStorage ✅, FtpStorage ✅, WebDavStorage ❌)
+- ⚠️ All storage implementations tested (LocalFileStorage ✅, SftpStorage ✅, FtpStorage ✅, S3Storage ✅, WebDavStorage ❌)
 - ❌ README matches actual API (completely wrong)
 - ✅ No TODOs/FIXMEs in code (achieved)
 - ❌ Examples directory exists (missing)
-- ✅ Package metadata accurate (SFTP and FTP now implemented!)
-- ✅ Integration test infrastructure (Docker-based CI testing for SFTP and FTP)
+- ✅ Package metadata accurate (SFTP, FTP, and S3 now implemented!)
+- ✅ Integration test infrastructure (Docker-based CI testing for SFTP, FTP, and S3)
 
-**Current Score: 5/9 (56%)** - Improved from 33%! (FTP implementation complete)
+**Current Score: 5/9 (56%)** - S3 implementation complete!
 
 ### 🎯 Post-v1.0 Roadmap (Future Versions)
 
-**v1.0** ✅ SFTP and FTP Implemented!
+**v1.0** ✅ All Major Storage Backends Implemented!
 - ✅ SFTP storage implementation (DONE!)
 - ✅ FTP/FTPS storage implementation (DONE!)
-- ✅ Integration test infrastructure with Docker for SFTP and FTP (DONE!)
+- ✅ S3 storage implementation with AWS S3 and S3-compatible services (DONE!)
+- ✅ Integration test infrastructure with Docker for SFTP, FTP, and S3/LocalStack (DONE!)
 
 **v1.1**
 - Code coverage reporting
 - Performance benchmarks
 - Multi-platform CI testing (Windows, macOS)
 - Additional conflict resolution strategies
-
-**v1.2**
-- S3-compatible storage implementation
 - Advanced filtering (regex support)
 
 **v2.0**

docker-compose.test.yml

@@ -31,6 +31,24 @@ services:
       timeout: 5s
       retries: 5
 
+  localstack:
+    image: localstack/localstack:latest
+    ports:
+      - "4566:4566"
+    environment:
+      SERVICES: s3
+      DEBUG: 0
+      DATA_DIR: /tmp/localstack/data
+      EDGE_PORT: 4566
+    volumes:
+      - localstack-data:/tmp/localstack
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:4566/_localstack/health"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
 volumes:
   sftp-data:
   ftp-data:
+  localstack-data:

scripts/run-integration-tests.ps1

@@ -1,4 +1,4 @@
-# Script to run SharpSync integration tests with Docker-based SFTP server
+# Script to run SharpSync integration tests with Docker-based test servers (SFTP, FTP, S3/LocalStack)
 # Usage: .\scripts\run-integration-tests.ps1 [-TestFilter "filter"]
 
 param(
@@ -10,7 +10,7 @@ $ErrorActionPreference = "Stop"
 $ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
 $ProjectRoot = Split-Path -Parent $ScriptDir
 
-Write-Host "🚀 Starting SFTP test server..." -ForegroundColor Cyan
+Write-Host "🚀 Starting test servers (SFTP, FTP, S3/LocalStack)..." -ForegroundColor Cyan
 Set-Location $ProjectRoot
 docker-compose -f docker-compose.test.yml up -d
 
@@ -32,7 +32,7 @@ while ($elapsed -lt $timeout) {
     Write-Host "   Waiting... ($elapsed`s/$timeout`s)" -ForegroundColor Gray
 }
 
-# Final check
+# Final check for SFTP
 if (-not $isHealthy) {
     Write-Host "❌ SFTP server failed to become healthy within $timeout`s" -ForegroundColor Red
     Write-Host "📋 Server logs:" -ForegroundColor Yellow
@@ -41,13 +41,61 @@ if (-not $isHealthy) {
     exit 1
 }
 
+Write-Host "⏳ Waiting for FTP server to be ready..." -ForegroundColor Yellow
+$elapsed = 0
+$isHealthy = $false
+
+while ($elapsed -lt $timeout) {
+    $containerStatus = docker-compose -f docker-compose.test.yml ps ftp
+    if ($containerStatus -match "healthy") {
+        Write-Host "✅ FTP server is ready" -ForegroundColor Green
+        $isHealthy = $true
+        break
+    }
+    Start-Sleep -Seconds 2
+    $elapsed += 2
+    Write-Host "   Waiting... ($elapsed`s/$timeout`s)" -ForegroundColor Gray
+}
+
+Write-Host "⏳ Waiting for LocalStack (S3) to be ready..." -ForegroundColor Yellow
+$elapsed = 0
+$isHealthy = $false
+
+while ($elapsed -lt $timeout) {
+    $containerStatus = docker-compose -f docker-compose.test.yml ps localstack
+    if ($containerStatus -match "healthy") {
+        Write-Host "✅ LocalStack is ready" -ForegroundColor Green
+        $isHealthy = $true
+        break
+    }
+    Start-Sleep -Seconds 2
+    $elapsed += 2
+    Write-Host "   Waiting... ($elapsed`s/$timeout`s)" -ForegroundColor Gray
+}
+
+# Create S3 test bucket in LocalStack
+Write-Host "📦 Creating S3 test bucket..." -ForegroundColor Cyan
+docker-compose -f docker-compose.test.yml exec -T localstack awslocal s3 mb s3://test-bucket 2>$null
+
 # Set environment variables for tests
 $env:SFTP_TEST_HOST = "localhost"
 $env:SFTP_TEST_PORT = "2222"
 $env:SFTP_TEST_USER = "testuser"
 $env:SFTP_TEST_PASS = "testpass"
 $env:SFTP_TEST_ROOT = "/home/testuser/upload"
 
+$env:FTP_TEST_HOST = "localhost"
+$env:FTP_TEST_PORT = "21"
+$env:FTP_TEST_USER = "testuser"
+$env:FTP_TEST_PASS = "testpass"
+$env:FTP_TEST_ROOT = "/"
+
+$env:S3_TEST_BUCKET = "test-bucket"
+$env:S3_TEST_ACCESS_KEY = "test"
+$env:S3_TEST_SECRET_KEY = "test"
+$env:S3_TEST_ENDPOINT = "http://localhost:4566"
+$env:S3_TEST_PREFIX = "sharpsync-tests"
+
 Write-Host "🧪 Running tests..." -ForegroundColor Cyan
 
 # Run tests with optional filter

scripts/run-integration-tests.sh

@@ -1,14 +1,14 @@
 #!/bin/bash
 
-# Script to run SharpSync integration tests with Docker-based SFTP server
+# Script to run SharpSync integration tests with Docker-based test servers (SFTP, FTP, S3/LocalStack)
 # Usage: ./scripts/run-integration-tests.sh [test-filter]
 
 set -e
 
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
 
-echo "🚀 Starting SFTP test server..."
+echo "🚀 Starting test servers (SFTP, FTP, S3/LocalStack)..."
 docker-compose -f "$PROJECT_ROOT/docker-compose.test.yml" up -d
 
 echo "⏳ Waiting for SFTP server to be ready..."
@@ -25,7 +25,7 @@ while [ $ELAPSED -lt $TIMEOUT ]; do
     echo "   Waiting... (${ELAPSED}s/${TIMEOUT}s)"
 done
 
-# Final check
+# Final check for SFTP
 if ! docker-compose -f "$PROJECT_ROOT/docker-compose.test.yml" ps sftp | grep -q "healthy"; then
     echo "❌ SFTP server failed to become healthy within ${TIMEOUT}s"
     echo "📋 Server logs:"
@@ -34,13 +34,53 @@ if ! docker-compose -f "$PROJECT_ROOT/docker-compose.test.yml" ps sftp | grep -q
     exit 1
 fi
 
+echo "⏳ Waiting for FTP server to be ready..."
+ELAPSED=0
+while [ $ELAPSED -lt $TIMEOUT ]; do
+    if docker-compose -f "$PROJECT_ROOT/docker-compose.test.yml" ps ftp | grep -q "healthy"; then
+        echo "✅ FTP server is ready"
+        break
+    fi
+    sleep 2
+    ELAPSED=$((ELAPSED + 2))
+    echo "   Waiting... (${ELAPSED}s/${TIMEOUT}s)"
+done
+
+echo "⏳ Waiting for LocalStack (S3) to be ready..."
+ELAPSED=0
+while [ $ELAPSED -lt $TIMEOUT ]; do
+    if docker-compose -f "$PROJECT_ROOT/docker-compose.test.yml" ps localstack | grep -q "healthy"; then
+        echo "✅ LocalStack is ready"
+        break
+    fi
+    sleep 2
+    ELAPSED=$((ELAPSED + 2))
+    echo "   Waiting... (${ELAPSED}s/${TIMEOUT}s)"
+done
+
+# Create S3 test bucket in LocalStack
+echo "📦 Creating S3 test bucket..."
+docker-compose -f "$PROJECT_ROOT/docker-compose.test.yml" exec -T localstack awslocal s3 mb s3://test-bucket 2>/dev/null || true
+
 # Set environment variables for tests
 export SFTP_TEST_HOST=localhost
 export SFTP_TEST_PORT=2222
 export SFTP_TEST_USER=testuser
 export SFTP_TEST_PASS=testpass
 export SFTP_TEST_ROOT=/home/testuser/upload
 
+export FTP_TEST_HOST=localhost
+export FTP_TEST_PORT=21
+export FTP_TEST_USER=testuser
+export FTP_TEST_PASS=testpass
+export FTP_TEST_ROOT=/
+
+export S3_TEST_BUCKET=test-bucket
+export S3_TEST_ACCESS_KEY=test
+export S3_TEST_SECRET_KEY=test
+export S3_TEST_ENDPOINT=http://localhost:4566
+export S3_TEST_PREFIX=sharpsync-tests
+
 echo "🧪 Running tests..."
 cd "$PROJECT_ROOT"
 

src/SharpSync/SharpSync.csproj

@@ -15,14 +15,14 @@
     <Authors>André Polykanine</Authors>
     <Company>Oire Software</Company>
     <Product>SharpSync</Product>
-    <Description>A pure .NET file synchronization library supporting WebDAV, SFTP, FTP/FTPS, and local storage with conflict resolution, selective sync, and progress reporting.</Description>
+    <Description>A pure .NET file synchronization library supporting WebDAV, SFTP, FTP/FTPS, S3, and local storage with conflict resolution, selective sync, and progress reporting.</Description>
     <Copyright>Copyright © 2025 André Polykanine, Oire Software</Copyright>
     <PackageLicenseExpression>Apache-2.0</PackageLicenseExpression>
     <PackageRequireLicenseAcceptance>false</PackageRequireLicenseAcceptance>
     <PackageProjectUrl>https://github.com/Oire/sharp-sync</PackageProjectUrl>
     <RepositoryUrl>https://github.com/Oire/sharp-sync</RepositoryUrl>
     <RepositoryType>git</RepositoryType>
-    <PackageTags>file-sync;synchronization;webdav;sftp;ftp;ftps;nextcloud;owncloud;backup;sync</PackageTags>
+    <PackageTags>file-sync;synchronization;webdav;sftp;ftp;ftps;s3;aws;nextcloud;owncloud;backup;sync</PackageTags>
     <Version>1.0.0</Version>
     <IncludeSourceRevisionInInformationalVersion>false</IncludeSourceRevisionInInformationalVersion>
     <!-- Suppress warnings about version-specific RIDs in SQLitePCLRaw v3.x -->
@@ -39,6 +39,7 @@
     <PackageReference Include="WebDav.Client" Version="2.9.0" />
     <PackageReference Include="SSH.NET" Version="2025.1.0" />
     <PackageReference Include="FluentFTP" Version="52.1.0" />
+    <PackageReference Include="AWSSDK.S3" Version="3.7.*" />
     <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.11.0" PrivateAssets="all" />
     <PackageReference Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="9.0.0" PrivateAssets="all" />
   </ItemGroup>