diff --git a/README.md b/README.md index 37156925..c6c603d5 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,7 @@ a low-latency, memory-efficient, distributed cache written in Rust. designed to - **memory limits** — per-shard byte-level accounting with configurable limits - **lru eviction** — approximate LRU via random sampling when memory pressure hits - **persistence** — append-only file (AOF) and point-in-time snapshots +- **encryption at rest** — optional AES-256-GCM encryption for AOF and snapshot files (compile with `--features encryption`) - **pipelining** — multiple commands per read for high throughput - **interactive CLI** — `ember-cli` with REPL, syntax highlighting, tab-completion, inline hints, cluster subcommands, and built-in benchmark - **graceful shutdown** — drains active connections on SIGINT/SIGTERM before exiting @@ -53,6 +54,10 @@ cargo build --release # with persistence ./target/release/ember-server --data-dir ./data --appendonly +# with encryption at rest (requires --features encryption) +./target/release/ember-server --data-dir ./data --appendonly \ + --encryption-key-file /path/to/keyfile + # concurrent mode (experimental, 2x faster for GET/SET) ./target/release/ember-server --concurrent @@ -135,6 +140,7 @@ redis-cli -p 6380 --tls --insecure PING | `--tls-key-file` | — | path to server private key (PEM) | | `--tls-ca-cert-file` | — | path to CA certificate for client verification | | `--tls-auth-clients` | no | require client certificates (`yes` or `no`) | +| `--encryption-key-file` | — | path to 32-byte key file for AES-256-GCM encryption at rest (requires `--features encryption`) | ## build & development @@ -235,7 +241,7 @@ contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). | 4 | clustering (raft, gossip, slots, migration) | ✅ complete | | 5 | developer experience (observability, CLI, clients) | 🚧 in progress | -**current**: 85 commands, 886 tests, ~18k lines of code (excluding tests) +**current**: 85 commands, 906 tests, ~18k lines of code (excluding tests) ## security @@ -244,7 +250,7 @@ see [SECURITY.md](SECURITY.md) for: - security considerations for deployment - recommended configuration -**note**: use `--requirepass` to enable authentication. protected mode is active by default when no password is set, rejecting non-loopback connections on public binds. +**note**: use `--requirepass` to enable authentication. protected mode is active by default when no password is set, rejecting non-loopback connections on public binds. for encryption at rest, see `--encryption-key-file` — key loss means data loss, so back up your key file separately from your data. ## license diff --git a/SECURITY.md b/SECURITY.md index 7861789f..93714134 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -52,6 +52,26 @@ if using AOF or snapshots: - aof files contain all write commands in binary format - snapshots contain the full keyspace state +### encryption at rest + +ember supports optional AES-256-GCM encryption for AOF and snapshot files. enable it by building with `--features encryption` and passing `--encryption-key-file`: + +```bash +# generate a 32-byte random key +dd if=/dev/urandom bs=32 count=1 > /path/to/ember.key +chmod 600 /path/to/ember.key + +ember-server --data-dir ./data --appendonly --encryption-key-file /path/to/ember.key +``` + +key management considerations: + +- **key loss = data loss** — there is no recovery mechanism if the key file is lost. back it up separately from your data directory +- **use a secrets manager** in production (e.g., HashiCorp Vault, AWS Secrets Manager) rather than storing the key file on the same disk as the data +- **key file permissions** — restrict to `600` (owner read/write only) +- **key rotation** — run `BGREWRITEAOF` and `BGSAVE` after swapping the key file to re-encrypt all persistence files with the new key. the old key is no longer needed once rewriting completes +- **transparent migration** — existing plaintext files are read normally even after enabling encryption. they are migrated to the encrypted format on the next `BGREWRITEAOF` or `BGSAVE` + ## security updates security fixes are released as patch versions. we recommend staying up to date with the latest release. diff --git a/bench/README.md b/bench/README.md index 713b11de..d5467a22 100644 --- a/bench/README.md +++ b/bench/README.md @@ -85,6 +85,19 @@ AOF with `appendfsync everysec` (default): persistence overhead is minimal with everysec fsync. `appendfsync always` has significant impact (~50% reduction). +### with encryption enabled + +AES-256-GCM encryption at rest (AOF and snapshots). requires building with `--features encryption`: + +| mode | SET throughput | overhead vs plaintext | +|------|----------------|----------------------| +| ember concurrent | — | — | +| ember sharded | — | — | + +*results pending — run `bench/bench-encryption.sh` on a dedicated VM to populate.* + +note: encryption only affects persistence writes. GET throughput should be unchanged since reads come from the in-memory keyspace. + ### scaling efficiency | cores | ember sharded SET | scaling factor | @@ -118,6 +131,9 @@ ember offers two modes with different tradeoffs: # build with jemalloc for best performance cargo build --release -p ember-server --features jemalloc +# for encryption benchmarks, also enable the encryption feature +cargo build --release -p ember-server --features jemalloc,encryption + # quick sanity check (ember only) ./bench/bench-quick.sh @@ -127,6 +143,9 @@ cargo build --release -p ember-server --features jemalloc # memory usage test ./bench/bench-memory.sh +# encryption overhead (requires --features encryption) +./bench/bench-encryption.sh + # comprehensive comparison using redis-benchmark (redis + dragonfly) ./bench/compare-redis.sh @@ -169,6 +188,7 @@ gcloud compute instances delete ember-bench --zone=us-central1-a | `bench-memory.sh` | memory usage with 1M keys | | `compare-redis.sh` | comprehensive comparison using redis-benchmark | | `bench-memtier.sh` | comprehensive comparison using memtier_benchmark | +| `bench-encryption.sh` | encryption at rest overhead (plaintext vs AES-256-GCM) | | `setup-vm.sh` | bootstrap dependencies on fresh ubuntu VM | ## configuration diff --git a/bench/bench-encryption.sh b/bench/bench-encryption.sh new file mode 100755 index 00000000..1b239f7d --- /dev/null +++ b/bench/bench-encryption.sh @@ -0,0 +1,165 @@ +#!/usr/bin/env bash +# +# encryption overhead benchmark — compares persistence throughput with and +# without AES-256-GCM encryption at rest. +# +# usage: +# bash bench/bench-encryption.sh +# +# requirements: +# - ember built with: cargo build --release -p ember-server --features jemalloc,encryption +# - redis-benchmark installed (redis 6+ for --threads) + +set -euo pipefail + +REQUESTS="${BENCH_REQUESTS:-200000}" +CLIENTS="${BENCH_CLIENTS:-50}" +PIPELINE="${BENCH_PIPELINE:-16}" +VALUE_SIZE="${BENCH_VALUE_SIZE:-64}" +CPU_CORES=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 1) +THREADS="${BENCH_THREADS:-$CPU_CORES}" + +PLAINTEXT_PORT="${PLAINTEXT_PORT:-6390}" +ENCRYPTED_PORT="${ENCRYPTED_PORT:-6391}" + +EMBER_BIN="${EMBER_BIN:-./target/release/ember-server}" + +PLAINTEXT_PID="" +ENCRYPTED_PID="" +TMPDIR_PLAIN="" +TMPDIR_ENC="" +KEY_FILE="" + +cleanup() { + [[ -n "$PLAINTEXT_PID" ]] && kill "$PLAINTEXT_PID" 2>/dev/null && wait "$PLAINTEXT_PID" 2>/dev/null || true + [[ -n "$ENCRYPTED_PID" ]] && kill "$ENCRYPTED_PID" 2>/dev/null && wait "$ENCRYPTED_PID" 2>/dev/null || true + [[ -n "$TMPDIR_PLAIN" ]] && rm -rf "$TMPDIR_PLAIN" + [[ -n "$TMPDIR_ENC" ]] && rm -rf "$TMPDIR_ENC" + [[ -n "$KEY_FILE" ]] && rm -f "$KEY_FILE" +} +trap cleanup EXIT + +wait_for_server() { + local port=$1 + local name=$2 + local retries=50 + while ! redis-cli -p "$port" ping > /dev/null 2>&1; do + retries=$((retries - 1)) + if [[ $retries -le 0 ]]; then + echo "error: $name did not start on port $port" >&2 + exit 1 + fi + sleep 0.1 + done +} + +extract_rps() { + local csv_output=$1 + local test_name=$2 + echo "$csv_output" | grep "\"$test_name\"" | cut -d',' -f2 | tr -d '"' | cut -d'.' -f1 +} + +format_number() { + printf "%'d" "$1" +} + +calc_overhead() { + local baseline=$1 + local encrypted=$2 + if [[ "$baseline" -gt 0 ]]; then + local pct=$(( (baseline - encrypted) * 100 / baseline )) + echo "${pct}%" + else + echo "n/a" + fi +} + +# --- checks --- + +if [[ ! -x "$EMBER_BIN" ]]; then + echo "error: ember-server not found at $EMBER_BIN" >&2 + echo "build with: cargo build --release -p ember-server --features jemalloc,encryption" >&2 + exit 1 +fi + +if ! command -v redis-benchmark &> /dev/null; then + echo "error: redis-benchmark not found. install redis tools first." >&2 + exit 1 +fi + +# --- setup --- + +echo "=============================================" +echo "EMBER ENCRYPTION OVERHEAD BENCHMARK" +echo "=============================================" +echo "requests: $REQUESTS" +echo "clients: $CLIENTS" +echo "pipeline: $PIPELINE" +echo "value size: ${VALUE_SIZE}B" +echo "threads: $THREADS" +echo "=============================================" +echo "" + +# temp directories for persistence +TMPDIR_PLAIN=$(mktemp -d) +TMPDIR_ENC=$(mktemp -d) + +# generate a 32-byte random key file +KEY_FILE=$(mktemp) +dd if=/dev/urandom bs=32 count=1 2>/dev/null > "$KEY_FILE" + +echo "starting plaintext server on port $PLAINTEXT_PORT..." +$EMBER_BIN --port "$PLAINTEXT_PORT" --data-dir "$TMPDIR_PLAIN" --appendonly > /dev/null 2>&1 & +PLAINTEXT_PID=$! +wait_for_server "$PLAINTEXT_PORT" "ember (plaintext)" + +echo "starting encrypted server on port $ENCRYPTED_PORT..." +$EMBER_BIN --port "$ENCRYPTED_PORT" --data-dir "$TMPDIR_ENC" --appendonly --encryption-key-file "$KEY_FILE" > /dev/null 2>&1 & +ENCRYPTED_PID=$! +wait_for_server "$ENCRYPTED_PORT" "ember (encrypted)" + +echo "" + +# --- pipelined throughput --- + +echo "=== PIPELINED THROUGHPUT (P=$PIPELINE, $THREADS threads) ===" +echo "" + +PLAIN_CSV=$(redis-benchmark -p "$PLAINTEXT_PORT" -t set,get -n "$REQUESTS" -c "$CLIENTS" -P "$PIPELINE" -d "$VALUE_SIZE" --threads "$THREADS" --csv -q 2>/dev/null) +ENC_CSV=$(redis-benchmark -p "$ENCRYPTED_PORT" -t set,get -n "$REQUESTS" -c "$CLIENTS" -P "$PIPELINE" -d "$VALUE_SIZE" --threads "$THREADS" --csv -q 2>/dev/null) + +PLAIN_SET=$(extract_rps "$PLAIN_CSV" "SET") +PLAIN_GET=$(extract_rps "$PLAIN_CSV" "GET") +ENC_SET=$(extract_rps "$ENC_CSV" "SET") +ENC_GET=$(extract_rps "$ENC_CSV" "GET") + +printf "%-25s %15s %15s %10s\n" "" "plaintext" "encrypted" "overhead" +printf "%-25s %15s %15s %10s\n" "---" "---" "---" "---" +printf "%-25s %15s %15s %10s\n" "SET (P=$PIPELINE)" "$(format_number "$PLAIN_SET")/s" "$(format_number "$ENC_SET")/s" "$(calc_overhead "$PLAIN_SET" "$ENC_SET")" +printf "%-25s %15s %15s %10s\n" "GET (P=$PIPELINE)" "$(format_number "$PLAIN_GET")/s" "$(format_number "$ENC_GET")/s" "$(calc_overhead "$PLAIN_GET" "$ENC_GET")" + +echo "" + +# --- single-request latency --- + +echo "=== SINGLE REQUEST THROUGHPUT (P=1) ===" +echo "" + +PLAIN_CSV_1=$(redis-benchmark -p "$PLAINTEXT_PORT" -t set,get -n "$REQUESTS" -c "$CLIENTS" -P 1 -d "$VALUE_SIZE" --threads "$THREADS" --csv -q 2>/dev/null) +ENC_CSV_1=$(redis-benchmark -p "$ENCRYPTED_PORT" -t set,get -n "$REQUESTS" -c "$CLIENTS" -P 1 -d "$VALUE_SIZE" --threads "$THREADS" --csv -q 2>/dev/null) + +PLAIN_SET_1=$(extract_rps "$PLAIN_CSV_1" "SET") +PLAIN_GET_1=$(extract_rps "$PLAIN_CSV_1" "GET") +ENC_SET_1=$(extract_rps "$ENC_CSV_1" "SET") +ENC_GET_1=$(extract_rps "$ENC_CSV_1" "GET") + +printf "%-25s %15s %15s %10s\n" "" "plaintext" "encrypted" "overhead" +printf "%-25s %15s %15s %10s\n" "---" "---" "---" "---" +printf "%-25s %15s %15s %10s\n" "SET (P=1)" "$(format_number "$PLAIN_SET_1")/s" "$(format_number "$ENC_SET_1")/s" "$(calc_overhead "$PLAIN_SET_1" "$ENC_SET_1")" +printf "%-25s %15s %15s %10s\n" "GET (P=1)" "$(format_number "$PLAIN_GET_1")/s" "$(format_number "$ENC_GET_1")/s" "$(calc_overhead "$PLAIN_GET_1" "$ENC_GET_1")" + +echo "" +echo "=== DONE ===" +echo "" +echo "note: GET overhead should be near zero since encryption only affects" +echo "persistence writes (AOF). GET reads from in-memory keyspace." diff --git a/crates/ember-cli/src/benchmark.rs b/crates/ember-cli/src/benchmark.rs index 0e77f4d5..843fe780 100644 --- a/crates/ember-cli/src/benchmark.rs +++ b/crates/ember-cli/src/benchmark.rs @@ -492,7 +492,7 @@ mod tests { assert_eq!(v.len(), 64); // all bytes should be lowercase ascii for &b in &v { - assert!(b >= b'a' && b <= b'z'); + assert!(b.is_ascii_lowercase()); } } diff --git a/crates/ember-persistence/README.md b/crates/ember-persistence/README.md index d9706ad5..0b5ecf1e 100644 --- a/crates/ember-persistence/README.md +++ b/crates/ember-persistence/README.md @@ -9,6 +9,7 @@ each shard gets its own persistence files (`shard-{id}.aof` and `shard-{id}.snap - **aof** — append-only file writer/reader with CRC32 integrity checks. binary TLV format, configurable fsync (always, every-second, OS-managed). gracefully handles truncated records from mid-write crashes - **snapshot** — point-in-time serialization of an entire shard's keyspace. writes to a `.tmp` file first, then atomic rename to prevent partial snapshots from corrupting existing data - **recovery** — startup sequence: load snapshot, replay AOF tail, skip expired entries. handles corrupt files gracefully (logs warning, starts empty) +- **encryption** — optional AES-256-GCM encryption for AOF records and snapshot files (enabled with `--features encryption`). accepts a 32-byte key file (raw bytes or hex). existing plaintext files are read transparently and migrated on the next rewrite/snapshot - **format** — low-level binary serialization helpers: length-prefixed bytes, integers, floats, checksums, header validation ## file formats @@ -19,6 +20,8 @@ supported record types: SET, DEL, EXPIRE, LPUSH, RPUSH, LPOP, RPOP, ZADD, ZREM, **snapshot (v2)** — `[ESNP magic][version][shard_id][entry_count][entries...][footer_crc32]` where entries are type-tagged (string=0, list=1, sorted set=2, hash=3, set=4). v1 snapshots (no type tags) are still readable. +**encrypted formats (v3)** — when an encryption key is configured, AOF records and snapshot entries are wrapped with AES-256-GCM: `[nonce (12 bytes)][ciphertext][auth tag (16 bytes)]`. the file headers and version bytes remain plaintext so the reader can detect encrypted files and select the right decoding path. plaintext v2 files are still readable even with a key configured. + ## usage ```rust diff --git a/crates/ember-server/README.md b/crates/ember-server/README.md index e4a84b98..de62ce3d 100644 --- a/crates/ember-server/README.md +++ b/crates/ember-server/README.md @@ -25,6 +25,10 @@ cargo run --release -p ember-server -- --data-dir ./data --appendonly --appendfs # with prometheus metrics on port 9100 cargo run --release -p ember-server -- --metrics-port 9100 + +# with encryption at rest (requires --features encryption) +cargo run --release -p ember-server --features encryption -- \ + --data-dir ./data --appendonly --encryption-key-file /path/to/keyfile ``` compatible with `redis-cli` and any RESP3 client. diff --git a/tests/integration/src/cluster.rs b/tests/integration/src/cluster.rs index c028ab61..04232a48 100644 --- a/tests/integration/src/cluster.rs +++ b/tests/integration/src/cluster.rs @@ -105,7 +105,7 @@ async fn cluster_keyslot() { let mut c = server.connect().await; let slot = c.get_int(&["CLUSTER", "KEYSLOT", "foo"]).await; - assert!(slot >= 0 && slot < 16384, "slot out of range: {slot}"); + assert!((0..16384).contains(&slot), "slot out of range: {slot}"); // same key should always hash to the same slot let slot2 = c.get_int(&["CLUSTER", "KEYSLOT", "foo"]).await;