Embedded document database for Swift. No server, no schema, no ceremony.
Store, query, and stream any Codable type — directly on device, with indexed queries, optional AES-256-GCM encryption, crash recovery, and real backpressure streaming. No Core Data stack, no migrations, no SQL.
What's genuinely different: a Codable-native, schemaless API with built-in per-record AES-256-GCM encryption, actor-based concurrency, and automatic CRC-32 + dirty-flag crash recovery — in one dependency-light package, no SQL and no migrations.
A good fit when:
- Your data is document-shaped and changes schema between app versions — there is no schema to migrate.
- You want per-record encryption without a third-party dependency.
- You are already writing async/await Swift and want a storage layer built on actors, not main-thread contexts or SQL.
- You want to stream large datasets without materializing them in memory.
Probably not the right choice when:
- You need the maturity, tooling, and query planner of SQLite or Core Data.
- Minimal disk footprint or raw single-metric speed is the only priority — NyaruDB2 is competitive with Core Data and Realm on those (see Performance), not categorically ahead.
// Package.swift
dependencies: [
.package(url: "https://github.com/galileostudio/NyaruDB2.git", from: "0.3.0")
],
targets: [
.target(name: "YourApp", dependencies: ["NyaruDB2"])
]Requirements: Swift 5.9+ · iOS 15+ · macOS 13+
import NyaruDB2
struct Article: Codable, Sendable {
let id: Int
let title: String
let author: String
let publishedAt: Date
}
// Open the database (creates the directory if needed)
let db = try NyaruDB(
path: "/path/to/db",
options: DatabaseOptions(compression: .gzip, format: .msgpack)
)
// Open a typed collection with secondary indexes
let articles = try await db.collection(
"articles",
of: Article.self,
options: CollectionOptions(
idField: "id",
indexedFields: ["author", "publishedAt"]
)
)
// Insert
try await articles.insert(Article(id: 1, title: "Hello", author: "Ana", publishedAt: .now))
// Bulk insert — validates all documents before writing any
try await articles.insert(contentsOf: moreArticles)
// Write batch — accumulate mixed operations (synchronously, no await),
// validated up front and applied all-or-nothing against errors
try await articles.writeBatch { batch in
batch.insert(newArticle)
batch.update(editedArticle)
batch.upsert(maybeNewArticle)
batch.delete(id: 42)
}
// Query
let recent = try await articles.find()
.where("publishedAt", isGreaterThan: Date().addingTimeInterval(-86400 * 7))
.sort(by: "publishedAt", ascending: false)
.limit(20)
.execute()
// Partial update — only the changed fields; result is validated against Article before writing
try await articles.patch(id: 1, changes: ["title": "Updated title"])
// Delete by predicate
let removed = try await articles.find()
.where("author", isEqualTo: "spam-bot")
.delete()
// Delete many by id in a single batched pass
try await articles.delete(ids: [3, 5, 8])
// Pull-based stream — memory stays bounded regardless of collection size
for try await article in articles.stream(batchSize: 64) {
process(article)
}
// Reclaim space left by deletions
try await articles.compact()
try await db.close()A collection is a typed, actor-isolated handle to a set of documents stored under a shared directory. Opening the same collection twice returns a cached handle; there is no risk of concurrent writers corrupting it.
let users = try await db.collection("users", of: User.self, options: options)CollectionOptions configures the primary key field, secondary indexes, and the partition key. Compression, serialization format, and the encryption key are database-wide (DatabaseOptions) and are frozen into each collection's manifest when it is first created.
The idField is always indexed. Declare additional fields at open time:
CollectionOptions(
idField: "id",
indexedFields: ["email", "region", "score"]
)You can add or remove indexed fields between opens — NyaruDB2 builds missing indexes with a single scan and removes dropped ones. The idField, partitionKey, and compression are frozen after the first open.
Indexed operations run in O(log n). Unindexed fields fall back to a full scan with the predicate applied in memory.
// Equality, comparisons, ranges
users.find()
.where("score", isGreaterThanOrEqualTo: 100)
.where("score", isLessThan: 500)
// Set membership
users.find().where("tier", isIn: ["gold", "platinum"])
// Text predicates
users.find().where("email", endsWith: "@example.com")
users.find().where("username", like: "ana%") // SQL-style wildcards
users.find().where("slug", glob: "2026-*-post") // glob wildcards
// Logical composition — chained wheres are AND; use Predicate for OR/NOT
users.find()
.where("age", isGreaterThanOrEqualTo: 18)
.where(.or([
.equal("country", "BR"),
.equal("country", "PT"),
]))
// Existence
users.find().whereExists("phoneNumber")
// Sort, page
users.find()
.sort(by: "name")
.offset(40)
.limit(20)
.execute()
// Inspect the plan before running
let plan = await users.find().where("score", isGreaterThan: 50).explain()writeBatch accumulates mixed operations and applies them as one all-or-nothing unit:
try await users.writeBatch { batch in
batch.insert(newUser)
batch.update(changedUser)
batch.delete(id: retiredUserID)
}Every operation is validated before anything is written: a duplicate id, a missing update target, a thrown body, or two operations on the same id abort the batch with zero side effects, and a failed write is rolled back. This is atomicity against errors, not crashes — NyaruDB2 has no write-ahead log by design, so a process crash mid-flush can persist part of the batch (per-record CRC + dirty-flag recovery still guarantees integrity of what was written). At most one operation may target a given document id per batch.
There are exactly three ways to write, each with a distinct purpose and the same all-or-nothing validation:
| Use when | |
|---|---|
insert(_:) / update(_:) / upsert(_:) / delete(id:) |
One document at a time |
insert(contentsOf:) |
You already hold an array of new documents |
writeBatch(_:) |
Accumulating from multiple sources, or mixing inserts with updates/deletes |
An insert-only writeBatch takes the same fast path as insert(contentsOf:) — pick by ergonomics, not performance.
When documents share a partition key (e.g. region, category), route them to dedicated shard files:
CollectionOptions(idField: "id", partitionKey: "region")Queries that filter on the partition key touch only the matching shard. Full scans read shards concurrently.
// Recommended: random key stored in the Keychain
let key = NyaruCrypto.generateRandomKey()
// Password-derived key (PBKDF2-HMAC-SHA256, 210k iterations)
let salt = NyaruCrypto.generateSalt() // persist alongside the database
let key = try NyaruCrypto.deriveKey(fromPassword: "passphrase", salt: salt)
let db = try NyaruDB(path: path, options: DatabaseOptions(encryptionKey: key))Encryption covers every record payload, every index snapshot, and the collection manifest. Shard filenames are HMAC-derived so partition values are not visible in the filesystem. Opening with the wrong key fails immediately at the manifest with NyaruError.decryptionFailed.
Use
generateRandomKey()+ Keychain for new integrations.deriveKey(fromPassword:salt:)is for cases where the key must be re-derived from user input; its PBKDF2 cost resists offline brute force, but a hardware-bound key is always stronger.
Every record carries a CRC-32 checksum. The shard file header has a dirty flag that is set (and fsynced) before the first write and cleared on clean close. On the next open:
- Any shard whose flag is still set is fully scanned.
- Records with bad checksums are tombstoned.
- Torn trailing writes are truncated.
- All indexes are rebuilt from the recovered data.
The operation is automatic and requires no intervention.
Clean opens are O(1). On every clean sync, each shard persists its scan-derived state (live count, free slots) to a small .state sidecar. A clean open adopts that state instead of reading the entire data region — the database opens instantly regardless of file size. The sidecar is trusted only when the dirty flag is clear and the recorded file size matches; anything suspicious falls back to the full scan.
Deleted documents are tombstoned in-place — space is not immediately reclaimed. Call compact() to rewrite shards and rebuild indexes:
// Compact a specific collection
try await articles.compact()
// Only compact if fragmentation is worth it (threshold: DatabaseOptions.maxFragmentation)
if try await articles.needsCompaction() {
try await articles.compact()
}Compaction preserves encryption and compression and never re-reads documents: each shard reports its old→new offset map and index pointers are remapped in place, with payload checksums reused so nothing is re-hashed. It runs incrementally, one shard at a time — concurrent reads and writes interleave between shard cycles, so the worst-case stall is a single shard's rewrite, not the whole compaction (on an 8-shard benchmark, get() p99 during compaction is ~4 µs).
Records are durable as they are written; recovery-free opens additionally require a sync(), which persists index snapshots, flushes shard state, and clears the dirty flags:
// Explicit — e.g. on scenePhase == .background
try await articles.sync() // one collection
try await db.sync() // every open collection
// Or automatic, via DatabaseOptions:
DatabaseOptions(autoSync: .afterWrites(500)) // every 500 written documents
DatabaseOptions(autoSync: .interval(30)) // first write ≥30 s after the last syncThere are no background timers — an idle database never wakes up. The expensive part of a sync (encoding and compressing index snapshots) runs off the collection's actor over copy-on-write captures, so reads and writes keep flowing while it happens. Without any sync, a crash simply pays the recovery scan on the next open.
collection.metrics() returns cumulative counters — which access paths queries actually took (index lookups, covered queries, full/partition scans), shard I/O bytes, compaction activity, and whether any shard needed crash recovery at open. Use it to verify that the queries you expect to be index-served really are.
NyaruDB2 uses swift-log to emit structured log events from the engine internals. By default, logs go to stdout at .info level.
// Control the log level
NyaruLogger.logLevel = .debug // see detailed compaction stats, sidecar negotiation
NyaruLogger.logLevel = .trace // every index probe and read
// Route to OSLog, a file, or a telemetry backend
import Logging
import OSLog
LoggingSystem.bootstrap { label in
var handler = StreamLogHandler.standardOutput(label: label)
handler.logLevel = .info
return handler
}| Level | Emitted when |
|---|---|
info |
Database/collection open/close, compaction start/finish with duration |
warning |
Index rebuild (crash recovery), shard recovered from dirty state |
debug |
Collection closed, sidecar-scan fallback, shard compaction stats (before/after) |
trace |
Individual index probes and shard reads (off by default) |
| Option | Notes |
|---|---|
.gzip |
Portable, ~10× size reduction on typical document payloads |
.lzfse |
Apple platforms only; faster decompression, moderate ratio |
.lz4 |
Apple platforms only; recommended only for large documents |
.none |
No compression |
format: .msgpack |
Binary serialization; use .json for human-readable storage |
Default recommendation: gzip + msgpack for production; none + json for debugging.
100,000 documents, 3 iterations, release build, macOS on Apple Silicon. Every
database runs the same operations and materializes results to dictionaries;
NyaruDB2 uses its recommended msgpack format. Insert is throughput (higher is
better); everything else is seconds (lower is better).
| Database | Insert docs/s | Get(s) | Query(s) | Update(s) | Delete(s) | BatchDel(s) | Disk(MB) | Peak mem(MB) |
|---|---|---|---|---|---|---|---|---|
| NyaruDB2 | 127,751 | 0.0062 | 0.0037 | 0.0248 | 0.1395 | 0.1231 | 17.6 | 219.7 |
| SQLite | 63,851 | 0.0211 | 0.0050 | 0.0261 | 0.2810 | 0.0037 | 31.5 | 440.0 |
| Realm | 62,748 | 0.0114 | 0.0062 | 1.0603 | 8.1617 | 0.0031 | —¹ | 340.3 |
| Couchbase | 66,938 | 0.0112 | 0.0218 | 0.0768 | 0.6193 | 1.1245 | 47.2 | 27.1 |
| CoreData | 95,231 | 0.0556 | 0.0051 | 0.1191 | 1.8841 | 0.1405 | 17.3 | 399.9 |
NyaruDB2 leads insert, query, update, and per-document delete. SQLite and Realm
win bulk delete (a truncate/deleteAll); Couchbase wins point reads and peak
memory. Cross-database numbers come from a separate harness (Realm and Couchbase
require their own runtimes) and are provided for orientation, not as a
run-it-in-this-repo claim.
¹ Realm's on-disk size is not reliably captured by the current harness; it is ~16 MB in practice.
Measured with the bundled benchmark (10,000 documents, batch size 1,000, Apple Silicon, release build). SQLite runs in WAL mode with prepared statements and the same secondary indexes. Times are milliseconds — lower is better.
| InsertMany | InsertBatch | Delete (1k) | Compact | File size | |
|---|---|---|---|---|---|
| NyaruDB2 (gzip + msgpack) | 97 | 91 | 5.3 | 19.0 | 1.1 MB |
| NyaruDB2 (none + msgpack) | 41 | 39 | 5.3 | 31.6 | 11.4 MB |
| SQLite (WAL) | 36 | 31 | 0.8 | 24.4 | 11.8 MB |
Highlights:
- Uncompressed inserts run within ~15% of SQLite — with gzip enabled, the extra time buys a file ~10× smaller.
- Compaction beats SQLite's VACUUM thanks to offset-remapped indexes (no document is re-read, re-parsed, or re-hashed).
- Fully index-covered queries answer
count()with zero disk I/O and skip all redundant parsing.
Reproduce it yourself:
swift run -c release NyaruDB2Benchmark -q -d 10000 # gzip + msgpack vs SQLite
swift run -c release NyaruDB2Benchmark --compression none --format msgpack -d 10000- Positioned I/O — all storage goes through
pread/pwrite(one syscall per access); point reads speculatively fetch header + payload in a single call. - Parallel CPU pipeline — compression, encryption, encoding, decoding, checksums, and index merges all fan out across cores for batch operations.
- Skip-scan extraction — MsgPack index keys are extracted by skipping over unwanted fields via length prefixes; large content fields are never decoded just to index a document.
- Binary index snapshots — hand-rolled snapshot format with interned shard IDs, an order of magnitude faster than Codable to persist and load.
- Streaming incremental compaction — live records flow into the compacted file in bounded zero-copy chunks (peak memory near the shard size instead of ~3× it), one shard at a time so reads never queue behind the whole compaction.
- Chunked scans — whole-file walks (scans, rebuilds, recovery) use a 4 MiB sliding window: a 764 MB shard rebuild peaks at +139 MB instead of +780 MB.
- Parallel multi-shard reads — pointer fetches and full scans read independent shards concurrently, so latency tracks the slowest shard, not the sum.
- Parallel residual queries — non-indexed predicates are evaluated across all cores, and
sort + limitselects through a bounded top-K heap instead of sorting every match. - Coalesced pointer reads — a query that resolves many index pointers reads each shard's records in coalesced, offset-sorted passes over a sliding window rather than one
preadper record. - Sort pushdown — a limited query that sorts on an indexed field walks the sort index in order, filtered by the predicate's pointer set, so the page comes out already sorted with no in-memory sort.
- O(1) clean opens — see Crash Recovery & Fast Opens.
Extended scenarios (unit-insert scaling curve, read latency under compaction, memory peaks, residual predicates) and their recorded baselines live in BENCHMARKS.md.
All operations throw NyaruError. Common cases:
do {
try await users.insert(user)
} catch NyaruError.duplicateID(let id) {
// Document with this id already exists
} catch NyaruError.decryptionFailed {
// Wrong key or corrupted record
} catch NyaruError.collectionTypeMismatch(let name) {
// idField / partitionKey / compression changed between opens
}Full API reference is generated with jazzy and published on GitHub Pages:
gem install jazzy
jazzy # reads .jazzy.yaml, outputs to docs/Every public symbol carries doc comments, so the generated reference covers the complete API surface. See CHANGELOG.md for release notes.
Apache 2.0 © 2026 galileostudio. See LICENSE.
Inspired by the original NyaruDB by kelp404.