🚀 Rust File Backend (RFB)

Rust File Backend (RFB) is a high-performance, enterprise-grade file management system combining the memory safety and speed of Rust with a modern React frontend. Built for cost-efficiency through content-addressable storage (deduplication) and scalability via parallel multipart uploads.

💎 Core Features

⚡ Blazing Performance

Built on Axum web framework and Tokio async runtime
Handles thousands of concurrent requests with minimal overhead
10× less memory usage compared to Node.js/Python equivalents

💰 Intelligent Storage Deduplication

SHA-256 Content Hashing eliminates duplicate storage
Instant "uploads" for previously stored content
Drastically reduced storage costs across all users

🛡️ Multi-Layered Security

ClamAV Integration: Real-time virus scanning
Magic Byte Verification: File type validation beyond extensions
Path Traversal Protection: Aggressive filename sanitization
JWT Authentication: Secure token-based access control
CAPTCHA Registration: Bot-resistant account creation
S3-Compatible Storage: Encrypted transport layer

🧩 Resilient Parallel Uploads

Custom chunked upload engine with parallel workers
Exponential backoff retry mechanism
Multi-GB file support on unstable connections
Configurable chunk sizes (default: 10MB)

🔗 File Sharing System

Public Share Links: Time-limited, token-based sharing with unique URLs
Password Protection: Argon2-hashed passwords for sensitive shares
Granular Permissions: view (inline preview) or download (attachment) modes
Folder Sharing: Share entire folders with browsable file listings
Access Logging: Track views, downloads, and password attempts with IP/User-Agent
Public Share Page: Beautiful, responsive frontend for recipients
Media Preview: Inline image, video, audio, and PDF preview on shared links

🖼️ Automatic Thumbnail Generation

WebP Format: Optimized thumbnails (256px) for minimal bandwidth
Multi-Format Support: Images, PDFs (via pdftocairo), and Videos (via ffmpeg)
Encrypted File Detection: Skips password-protected PDFs gracefully
Dedicated Worker: Separate thumbnail-worker process for asynchronous generation
Lazy Loading: Frontend loads thumbnails asynchronously with smooth animations

📋 Advanced File Operations

Copy/Paste: Recursive folder duplication with deduplication
Bulk Actions: Move, delete, and copy multiple items
Archive Preview: Inspect ZIP, 7z, RAR, TAR without extraction
Download Tickets: Time-limited shareable links
PDF Preview: Inline document viewing
Favorites: Star/unstar files and folders for quick access

🔗 File Sharing

Share Links: Public or user-specific sharing with configurable permissions
Password Protection: Argon2id-hashed passwords for secure access
Expiration Control: Configurable expiry up to 1 year
View/Download Modes: Inline display or attachment download
Access Logging: Track views, downloads, and password attempts
Public Share Page: Standalone viewer with media preview

🔍 Advanced Search & Filtering

Full-Text Search: Real-time filename search with debouncing
Fuzzy/Similarity Search: Find files even with typos
Regex & Wildcard: Power-user search patterns
Tag Filtering: Filter by user-defined tags
Category Filtering: Filter by auto-detected file categories
Date Range: Filter files by creation date
Size Range: Filter files by size boundaries
Favorites Filter: Show only starred items

🏗️ Architecture

System Overview

┌─────────────┐      ┌──────────────┐      ┌─────────────┐
│   React     │─────▶│  Axum API    │─────▶│  PostgreSQL  │
│  Frontend   │      │  (Rust)      │      │  Database    │
└─────────────┘      └──────────────┘      └─────────────┘
                            │
                            ├─────▶ S3/RustFS (File Storage)
                            └─────▶ ClamAV (Virus Scanning)

┌──────────────────────────────────────────────────────────┐
│  Background Workers (separate processes)                 │
│  ├── Worker: Virus scanning, cleanup, facts updates      │
│  └── Thumbnail Worker: WebP thumbnail generation         │
└──────────────────────────────────────────────────────────┘

Backend (`api/`)

Technology Stack:

Language: Rust 2024 Edition
Web Framework: Axum 0.7
ORM: SeaORM (PostgreSQL & SQLite)
Storage: AWS SDK for Rust (S3-compatible)
Security: JWT, Argon2, ClamAV, CAPTCHA
Runtime: Tokio async

Key Modules:

api/handlers/ — HTTP request handlers (auth, files, upload, captcha, users, settings, shares, health)
services/ — Business logic (file, upload, metadata, scanner, audit, facts, share, thumbnail, worker)
entities/ — Database models (SeaORM)
infrastructure/ — Storage, database, scanner adapters
utils/ — Validation, auth, encryption helpers

Features:

Chunked multipart uploads with resume capability
Content-based deduplication (SHA-256)
Background virus scanning queue
Metadata extraction (EXIF, ID3, PDF, Office)
Recursive folder operations
Download ticket generation
File sharing with password protection and access logs
WebP thumbnail generation (images, PDFs, videos)
Favorites and folder tree navigation

Frontend (`web/`)

Technology Stack:

Framework: React 18 + TypeScript
Build Tool: Vite 5
Runtime: Bun
Styling: Tailwind CSS + Glassmorphism
Icons: Lucide React
Tables: TanStack Table v8
State: React Context + Hooks

Key Components:

features/dashboard/ — Main file manager interface
features/auth/ — Login, register, OIDC
features/share/ — Public share page (password gate, media preview, folder browsing)
lib/ — Reusable file manager library
services/ — API client (upload, file operations, sharing)
components/ — Modals, toasts, context menus, sidebar with shares

Features:

Drag-and-drop file upload
Real-time upload progress with parallel chunks
Copy/Cut/Paste with keyboard shortcuts
Bulk selection and operations
File preview modals (images, PDF, archives)
Archive content inspection
Responsive grid/list views
Thumbnail previews with lazy loading
Share management (create, revoke, view logs)
Public share page with media viewer
Sidebar with active shares navigation
Favorites toggle and filtering

🚀 Quick Start

Prerequisites

Rust 1.84+
Bun 1.1+
PostgreSQL 14+ (or SQLite for development)
RustFS, MinIO, or AWS S3
Redis (optional, for caching)
ClamAV (optional, for scanning)
pdftocairo (optional, for PDF thumbnails)
ffmpeg (optional, for video thumbnails)

Local Development

Clone Repository

git clone https://github.com/appsjuragan/rust-file-backend.git
cd rust-file-backend

Backend Setup

cd api
cp ../.env.sample .env
# Edit .env with your database and S3 credentials

# Run migrations
cargo run --bin rust-file-backend -- --mode migrate

# Start API server
cargo run --bin rust-file-backend -- --mode api

# Start background worker (separate terminal)
cargo run --bin rust-file-backend -- --mode worker

# Start thumbnail worker (separate terminal)
cargo run --bin rust-file-backend -- --mode thumbnail-worker

Frontend Setup

cd web
cp .env.example .env
# Edit .env to set VITE_API_URL=http://localhost:3000

bun install
bun run dev

Access Application

Frontend: http://localhost:5173
API Docs: http://localhost:3000/swagger-ui
Health Check: http://localhost:3000/health

Windows Quick Start

./run.bat

🐳 Docker Deployment

Build Images

# Backend
docker build -t rfb-api:latest ./api

# Frontend
docker build --build-arg VITE_API_URL=https://your-api-domain.com -t rfb-web:latest ./web

Pre-built Images (GHCR)

Official images are available on GitHub Container Registry:

# Pull Backend (v9)
docker pull ghcr.io/appsjuragan/rust-file-backend-api:v9

# Pull Frontend (v9)
docker pull ghcr.io/appsjuragan/rust-file-backend-web:v9

Production Compose Stack

The compose stack includes:

API server — Axum HTTP service
Background worker — Virus scanning, cleanup, facts updates
Thumbnail worker — WebP thumbnail generation (images, PDFs, videos)
PostgreSQL — Primary database
RustFS — S3-compatible object storage
Web — Nginx-served React frontend

docker compose up -d

📡 API Reference

Authentication

POST /register — Create new user (CAPTCHA-protected)
POST /login — Authenticate and receive JWT
POST /captcha — Generate CAPTCHA challenge
GET /auth/oidc/login — OIDC authentication flow
GET /auth/oidc/callback — OIDC callback handler

File Operations

POST /upload — Single file upload
POST /files/upload/init — Initialize chunked upload
GET /files/upload/sessions — List pending upload sessions
PUT /files/upload/:id/chunk/:num — Upload chunk
POST /files/upload/:id/complete — Finalize upload
DELETE /files/upload/:id — Abort chunked upload
GET /files — List files (with pagination, search, filters)
GET /files/:id — Download file
POST /files/:id/ticket — Generate download ticket
GET /download/:ticket — Download via ticket
DELETE /files/:id — Delete file/folder
PUT /files/:id/rename — Rename or move item
POST /files/:id/favorite — Toggle favorite status
GET /files/:id/thumbnail — Get WebP thumbnail

Bulk Operations

POST /files/bulk-delete — Delete multiple items
POST /files/bulk-move — Move multiple items
POST /files/bulk-copy — Copy multiple items (with recursion)

Folders

POST /folders — Create new folder
GET /folders/tree — Get full folder tree for navigation
GET /files/:id/path — Get folder breadcrumb path

Sharing

POST /shares — Create a share link (public/user, password, permissions)
GET /shares — List user's shares (optionally filter by file)
DELETE /shares/:id — Revoke a share link
GET /shares/:id/logs — Get share access logs

Public Share (No Auth Required)

GET /share/:token — Get shared item info (filename, type, permissions)
POST /share/:token/verify — Verify share password
GET /share/:token/download — Download shared file (with optional file_id for folder items)
GET /share/:token/list — List shared folder contents

Advanced

POST /pre-check — Check if file exists (deduplication)
POST /files/link — Link existing storage file
GET /files/:id/zip-contents — Preview archive contents

User & Settings

GET /users/me — Get user profile
PUT /users/me — Update profile
GET /users/avatar/:user_id — Get public avatar image
POST /users/me/avatar — Upload personal avatar
GET /users/me/facts — Get storage statistics
GET /settings — Get user preferences
PUT /settings — Update preferences

System

GET /health — Health check (DB, storage, version)
GET /system/validation-rules — Get file validation rules

Full API documentation available at /swagger-ui endpoint.

📦 Postman Collection

Import api/postman_collection.json for ready-to-use API requests with:

Pre-configured authentication
Example payloads for all endpoints including sharing
Environment variables
Postman test scripts for auto-setting tokens

🧪 Testing

Backend Tests

cd api
cargo test

Frontend Tests

cd web
bun test

Security Scanning

# Dependency audit
cargo audit

🧹 Code Quality

Formatting

# Backend
cd api && cargo fmt

# Frontend
cd web && bun run format

Linting

# Backend
cd api && cargo clippy -- -D warnings

# Frontend
cd web && bun run lint

All code follows:

Rust 2024 edition conventions
ESLint + Prettier for TypeScript
No unused imports or dead code

🔧 Configuration

Backend Environment Variables

DATABASE_URL=postgresql://user:pass@localhost/rfb
JWT_SECRET=your-secret-key
# OIDC (optional)
OIDC_ISSUER_URL=https://accounts.google.com
OIDC_CLIENT_ID=your-client-id
OIDC_CLIENT_SECRET=your-client-secret
OIDC_REDIRECT_URL=http://localhost:3000/auth/oidc/callback
OIDC_SKIP_DISCOVERY=false
MINIO_ENDPOINT=http://localhost:9000
MINIO_BUCKET=file-storage
MINIO_ACCESS_KEY=rustfsadmin
MINIO_SECRET_KEY=rustfsadmin
MINIO_REGION=us-east-1
CHUNK_SIZE=10485760
MAX_FILE_SIZE=1073741824
CLAMAV_HOST=localhost
CLAMAV_PORT=3310
ENABLE_VIRUS_SCAN=true
ALLOWED_ORIGINS=http://localhost:3000,http://localhost:5173

See .env.sample in the project root for a complete reference.

Frontend Environment Variables

VITE_API_URL=http://localhost:3000
VITE_CHUNK_SIZE=10485760

📊 Performance Benchmarks

Upload Speed: 500MB/s on local network
Concurrent Users: 10,000+ simultaneous connections
Memory Usage: ~50MB base (API server)
Deduplication Savings: Up to 80% storage reduction
Chunk Upload Parallelism: 4 workers default

🤝 Contributing

Fork the repository
Create feature branch (git checkout -b feature/amazing-feature)
Commit changes (git commit -m 'Add amazing feature')
Push to branch (git push origin feature/amazing-feature)
Open Pull Request

📜 License

Licensed under the MIT License. See LICENSE file for details.

Created with ❤️ by the AppsJuragan team.

Name		Name	Last commit message	Last commit date
Latest commit History 170 Commits
.agent/workflows		.agent/workflows
.github/workflows		.github/workflows
api		api
desktop-sync		desktop-sync
web		web
.env.sample		.env.sample
.gitignore		.gitignore
README.md		README.md
docker-compose.yml		docker-compose.yml
podman-up.bat		podman-up.bat
run.bat		run.bat

Folders and files

Latest commit

History

Repository files navigation