Skip to content

feat: add safe SQLite backup and restore - #8

Merged
1074016045 merged 3 commits into
mainfrom
feature/v0.6-database-backup-restore
Jul 26, 2026
Merged

feat: add safe SQLite backup and restore#8
1074016045 merged 3 commits into
mainfrom
feature/v0.6-database-backup-restore

Conversation

@1074016045

@1074016045 1074016045 commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Summary

Adds safe local SQLite database backup, strict validation, and explicit offline restore workflows.

Backup

  • uses the installed better-sqlite3 online backup API
  • includes committed WAL-mode data without copying live WAL/SHM files
  • publishes a versioned .sqlite and .manifest.json pair
  • verifies byte size, SHA-256, SQLite integrity, foreign keys, schema, and migration metadata
  • prevents overwrite and concurrent-name races
  • uses inode-aware cleanup and restrictive local file handling

Validation

  • strictly validates the versioned manifest
  • rejects unknown fields, unsupported versions, unsafe filenames, symlinks, special files, malformed metadata, and path traversal
  • verifies file size, SHA-256, SQLite integrity, foreign keys, core application schema, and Drizzle migration metadata
  • opens backup candidates read-only
  • returns bounded, content-free CLI errors
  • supports machine-readable JSON validation output

Offline restore

  • supports non-mutating dry runs
  • requires --confirm-offline for write restores
  • requires --replace for existing destinations
  • creates and retains a validated pre-restore safety backup
  • validates a same-directory candidate before installation
  • prevents destination overwrite races
  • restores exact original database/WAL/SHM bytes after handled failures
  • fails closed and retains recovery files if rollback cannot complete
  • detects persistent locks and recovery residue from interrupted operations
  • never runs migrations automatically

Commands

Create a backup:

pnpm --silent db:backup -- \
  --database <path> \
  --output-dir <directory> \
  --name <safe-name>

Validate a backup:

pnpm --silent db:backup:validate -- \
  --manifest <backup.manifest.json> \
  --json

Perform a non-mutating restore dry run:

pnpm --silent db:restore -- \
  --manifest <backup.manifest.json> \
  --database <destination> \
  --dry-run

Restore to a new destination:

pnpm --silent db:restore -- \
  --manifest <backup.manifest.json> \
  --database <destination> \
  --confirm-offline

Replace an existing destination:

pnpm --silent db:restore -- \
  --manifest <backup.manifest.json> \
  --database <destination> \
  --replace \
  --confirm-offline

Safety boundaries

  • backup files contain sensitive local database data
  • backups remain local unless the operator deliberately moves them
  • backup encryption and compression are not implemented
  • recordings and Uploaded Audio bytes are not included
  • this is not a complete application or media backup
  • restore is an offline administrative operation
  • --confirm-offline is an operator assertion, not automatic process detection
  • restore does not run migrations
  • pre-restore safety backups are retained
  • backup retention and deletion remain manual
  • crash-proof atomicity across sudden termination, host failure, or power loss is not guaranteed
  • persistent operation locks and recovery residue make interrupted retries fail closed
  • manual recovery may be required after an uncleanly interrupted multi-file restore

Tests

Added 144 focused tests covering:

  • committed WAL backup consistency
  • exclusion of uncommitted and rolled-back data
  • strict manifest and SQLite validation
  • traversal, symlink, hard-link, inode, and special-file cases
  • concurrent backup attempts
  • partial publication failures
  • inode-owned cleanup
  • dry-run non-mutation
  • create-new and replacement restore
  • retained pre-restore backups
  • exact database/WAL/SHM rollback
  • injected replacement and post-validation failures
  • interrupted-operation residue
  • bounded path-free text and JSON CLI output
  • temporary-directory and real-data isolation

Verification

  • focused database-maintenance tests: 144/144 passed
  • Linux-sensitive ownership regression: 5/5 repeated runs passed
  • complete database backup integration tests: 54/54 passed in three consecutive runs
  • full Vitest suite: 457/457 across 33 files
  • lint: passed
  • typecheck: passed
  • boundary evaluation: passed, 37 fixtures
  • understanding evaluation: passed, 56 fixtures
  • Playwright Chromium: 6/6
  • production build: passed
  • git diff --check: passed
  • no React warnings
  • no unhandled rejections or open handles observed
  • isolated CLI backup, validation, dry-run, create-new restore, and replacement restore demonstrations passed

Scope integrity

  • no dependency or lockfile change
  • no package version change
  • migrations 0000–0006 unchanged
  • no migration 0007
  • no database schema change
  • no API route change
  • no web restore UI
  • no cloud backup
  • no compression or encryption
  • no recording or Uploaded Audio media backup
  • no retention or observability implementation

Known limitations

  • sudden process death and power-loss crash atomicity are not guaranteed
  • theoretical pathname-swap races cannot be completely eliminated through the available SQLite runtime API
  • backups are unencrypted
  • backups exclude media files
  • restore requires the operator to stop the application and worker
  • this workflow is not formal disaster-recovery certification

Remaining v0.6 work

  • configurable retention
  • content-free operational observability

@1074016045
1074016045 merged commit 5b382e5 into main Jul 26, 2026
2 of 3 checks passed
@1074016045
1074016045 deleted the feature/v0.6-database-backup-restore branch July 26, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant