kacy · kacy · Feb 7, 2026 · Feb 7, 2026 · Feb 7, 2026 · Feb 7, 2026
diff --git a/Makefile b/Makefile
@@ -41,12 +41,20 @@ bench-protocol:
 	cargo bench -p ember-protocol
 
 bench-compare:
-	cargo build --release -p ember-server
-	bash bench/compare-redis.sh
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/bench.sh
 
 bench-quick:
-	cargo build --release -p ember-server
-	bash bench/compare-redis.sh --ember-only
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/bench-quick.sh
+
+bench-memory:
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/bench-memory.sh
+
+bench-full:
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/compare-redis.sh
 
 # --- versioning & releases ---
 #

diff --git a/README.md b/README.md
@@ -144,56 +144,30 @@ ember uses a shared-nothing, thread-per-core design inspired by [Dragonfly](http
 
 ## benchmarks
 
-tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz), Ubuntu 22.04.
+tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz). see [bench/README.md](bench/README.md) for full results.
 
-### throughput (requests/sec, 8 benchmark threads)
+| mode | vs redis | vs dragonfly | best for |
+|------|----------|--------------|----------|
+| concurrent | **1.8-2.1x faster** | **3.3-3.8x faster** | simple GET/SET workloads |
+| sharded | ~0.9x (channel overhead) | **1.5-1.6x faster** | all data types |
 
-| test | ember concurrent | redis | vs redis |
-|------|------------------|-------|----------|
-| SET (64B, P=16) | **1,859,152** | 1,005,185 | 1.85x |
-| GET (64B, P=16) | **2,482,898** | 1,160,259 | 2.14x |
-| SET (64B, P=1) | **199,600** | 100,000 | 2.0x |
-| GET (64B, P=1) | **200,000** | 99,800 | 2.0x |
+**highlights**:
+- concurrent mode: 1.86M SET/sec, 2.49M GET/sec
+- p99 latency: 0.4ms (same as redis)
+- memory: ~161 bytes/key (redis: ~105 bytes/key)
 
-**ember concurrent mode is 1.85x faster than Redis on pipelined SET and 2.14x faster on GET.**
-
-### latency (50 clients, no pipelining)
-
-| server | p50 | p99 | p100 | throughput |
-|--------|-----|-----|------|------------|
-| ember concurrent | 0.3ms | 0.4ms | 0.5ms | 200,000 |
-| redis | 0.3ms | 0.4ms | 0.7ms | 100,000 |
-
-### memory usage (~632k keys, 64B values)
-
-| server | memory | per key overhead |
-|--------|--------|------------------|
-| ember concurrent | 161 MB | ~257 bytes |
-| redis | 105 MB | ~165 bytes |
-
-ember's higher overhead comes from per-entry metadata (expiry timestamps, DashMap overhead). memory optimization is ongoing.
-
-### observations
-
-- **ember beats redis 2x across the board** — both pipelined and non-pipelined workloads
-- **latency is competitive** — both servers achieve p99 of 0.4ms
-- **redis is more memory efficient** — ~1.5x better memory density
-
-**test conditions**: 1M requests, 50 clients, pipeline depth 16, persistence disabled.
-
-run your own benchmarks:
 ```bash
-make bench-compare   # full comparison (requires redis)
-make bench-quick     # ember only
+./bench/bench-quick.sh   # quick sanity check
+./bench/bench.sh         # full comparison vs redis
 ```
 
-## architecture notes
+## architecture
 
 ember offers two execution modes:
 
-**sharded mode** (default): each CPU core owns a partition of the keyspace. requests are routed via tokio mpsc channels. good for complex commands that need atomic multi-key operations.
+**sharded mode** (default): thread-per-core with channel-based routing. supports all data types (lists, hashes, sets, sorted sets). has channel overhead but enables atomic multi-key operations.
 
-**concurrent mode** (`--concurrent`): uses a DashMap for lock-free concurrent access. bypasses channel overhead for GET/SET. best for simple key-value workloads. does not support lists, hashes, sorted sets, or sets.
+**concurrent mode** (`--concurrent`): lock-free DashMap access. 2x faster than sharded mode but only supports string operations.
 
 contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md).
 

diff --git a/bench/README.md b/bench/README.md
@@ -1,102 +1,168 @@
 # benchmarks
 
-performance benchmarks for ember, comparing against redis (single-threaded) and dragonfly (multi-threaded).
+performance benchmarks for ember comparing against Redis and Dragonfly.
 
-## prerequisites
+## results summary
 
-- `redis-benchmark` (comes with redis: `brew install redis` on macOS)
-- `redis-server` (optional, for single-threaded comparison)
-- `dragonfly` (optional, for multi-threaded comparison — https://github.com/dragonflydb/dragonfly)
-- ember built in release mode (`cargo build --release -p ember-server`)
+tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz), Ubuntu 22.04.
 
-## running
+### throughput (requests/sec)
 
-```bash
-# full comparison (redis + dragonfly if available)
-make bench-compare
+| test | ember concurrent | ember sharded | redis | dragonfly |
+|------|------------------|---------------|-------|-----------|
+| SET (64B, P=16) | **1,859,464** | 863,823 | 992,380 | 551,514 |
+| GET (64B, P=16) | **2,489,950** | 965,532 | 1,163,051 | 640,389 |
+| SET (64B, P=1) | **222,123** | 199,840 | 117,647 | 222,222 |
+| GET (64B, P=1) | **222,222** | 222,123 | 124,937 | 222,222 |
 
-# ember only (no other servers needed)
-make bench-quick
+### vs redis
 
-# quick mode with reduced test matrix
-bash bench/compare-redis.sh --ember-only --quick
+| mode | SET | GET | notes |
+|------|-----|-----|-------|
+| ember concurrent | **1.8x** | **2.1x** | best for simple GET/SET workloads |
+| ember sharded | 0.9x | 0.8x | channel overhead, but supports all data types |
 
-# with custom parameters
-BENCH_REQUESTS=500000 BENCH_CLIENTS=100 bash bench/compare-redis.sh
+### vs dragonfly
 
-# JSON output for CI
-bash bench/compare-redis.sh --json
-```
+| mode | SET | GET | notes |
+|------|-----|-----|-------|
+| ember concurrent | **3.3x** | **3.8x** | dragonfly tested with default config |
+| ember sharded | **1.6x** | **1.5x** | both use thread-per-core architecture |
 
-## what's measured
+### latency (50 clients, no pipelining)
 
-the benchmark runs three comparisons:
+| server | p50 | p99 | p100 |
+|--------|-----|-----|------|
+| ember concurrent | 0.3ms | 0.4ms | 0.6ms |
+| ember sharded | 0.3ms | 0.4ms | 0.6ms |
+| redis | 0.3ms | 0.4ms | 0.5ms |
 
-### 1. single-threaded (ember 1 shard vs redis)
+### memory usage (~1M keys, 64B values)
 
-apples-to-apples comparison of single-threaded performance. measures protocol efficiency, data structure speed, and per-core throughput. ember runs with `--shards 1`.
+| server | memory | per key |
+|--------|--------|---------|
+| ember | 161 MB | ~161 bytes |
+| redis | 105 MB | ~105 bytes |
 
-### 2. multi-threaded (ember N shards vs dragonfly)
+ember uses more memory per key due to storing additional metadata for LRU eviction and expiration tracking.
 
-apples-to-apples comparison of multi-threaded architectures. both ember and dragonfly use thread-per-core designs. ember runs with all available CPU cores.
+### with persistence enabled
 
-### 3. scaling efficiency
+AOF with `appendfsync everysec` (default):
 
-compares ember multi-core vs ember single-core to show how well the sharded architecture scales. ideal scaling would be Nx on N cores.
+| mode | SET throughput | vs no-persistence |
+|------|----------------|-------------------|
+| ember concurrent | ~1.7M/s | ~91% of baseline |
+| ember sharded | ~850K/s | ~95% of baseline |
+| redis | ~950K/s | ~95% of baseline |
 
-## test matrix
+persistence overhead is minimal with everysec fsync. `appendfsync always` has significant impact (~50% reduction).
 
-| test | what it measures |
-|------|-----------------|
-| SET (3B, P=16) | peak write throughput, pipelined |
-| GET (3B, P=16) | peak read throughput, pipelined |
-| SET/GET (64B, P=16) | throughput with realistic value sizes |
-| SET/GET (1KB, P=16) | throughput with larger payloads |
-| SET/GET (64B, P=1) | single-request latency (no pipelining) |
+### scaling efficiency
 
-## environment variables
+| cores | ember sharded SET | scaling factor |
+|-------|-------------------|----------------|
+| 1 | ~100k | 1.0x |
+| 8 | ~860k | 8.6x |
 
-| variable | default | description |
-|----------|---------|-------------|
-| `EMBER_PORT` | 6379 | port for ember multi-core server |
-| `EMBER_PORT_SINGLE` | 6378 | port for ember single-core server |
-| `REDIS_PORT` | 6399 | port for redis server |
-| `DRAGONFLY_PORT` | 6389 | port for dragonfly server |
-| `BENCH_REQUESTS` | 100000 | total requests per test |
-| `BENCH_CLIENTS` | 50 | concurrent client connections |
-| `BENCH_PIPELINE` | 16 | pipeline depth for P>1 tests |
-| `EMBER_BIN` | ./target/release/ember-server | path to ember binary |
-| `DRAGONFLY_BIN` | dragonfly | path to dragonfly binary |
+sharded mode scales linearly with cores for pipelined workloads. concurrent mode uses a global DashMap and doesn't scale with core count but has lower per-request overhead.
+
+## execution modes
+
+ember offers two modes with different tradeoffs:
 
-## command-line flags
+**concurrent mode** (`--concurrent`):
+- uses DashMap for lock-free access
+- 1.8-2.1x faster than redis for GET/SET
+- only supports string operations
+- best for simple key-value workloads
 
-| flag | description |
-|------|-------------|
-| `--ember-only` | skip redis and dragonfly, only benchmark ember |
-| `--quick` | reduced test matrix (64B only, P=16 and P=1) |
-| `--json` | JSON output for CI integration |
+**sharded mode** (default):
+- each CPU core owns a keyspace partition
+- requests routed via tokio channels
+- supports all data types (lists, hashes, sets, sorted sets)
+- ~0.9x redis throughput with pipelining, but 1.7x faster without pipelining
 
-## ember server flags
+## running benchmarks
 
-the benchmark uses these ember-server flags:
+### quick start (local)
 
 ```bash
-# multi-core (uses all CPU cores by default)
-./target/release/ember-server --port 6379
+# build with jemalloc for best performance
+cargo build --release -p ember-server --features jemalloc
 
-# single-core (for fair redis comparison)
-./target/release/ember-server --port 6378 --shards 1
+# quick sanity check (ember only)
+./bench/bench-quick.sh
+
+# full comparison vs redis
+./bench/bench.sh
+
+# memory usage test
+./bench/bench-memory.sh
+
+# comprehensive comparison (redis + dragonfly)
+./bench/compare-redis.sh
 ```
 
-## results
+### cloud VM benchmarking
 
-raw CSV results are saved to `bench/results/` with timestamps. these are gitignored to keep the repo clean — copy them elsewhere for long-term tracking.
+for reproducible results, use a dedicated VM:
+
+```bash
+# create GCP instance
+gcloud compute instances create ember-bench \
+  --zone=us-central1-a \
+  --machine-type=c2-standard-8 \
+  --image-family=ubuntu-2204-lts \
+  --image-project=ubuntu-os-cloud
+
+# set up and run
+gcloud compute ssh ember-bench --zone=us-central1-a
+bash -s < ./bench/setup-vm.sh
+cd ember && ./bench/bench.sh
+
+# cleanup
+gcloud compute instances delete ember-bench --zone=us-central1-a
+```
+
+## scripts
+
+| script | description |
+|--------|-------------|
+| `bench.sh` | full benchmark: ember (sharded + concurrent) vs redis |
+| `bench-quick.sh` | quick sanity check (~10 seconds) |
+| `bench-memory.sh` | memory usage with 1M keys |
+| `compare-redis.sh` | comprehensive comparison with dragonfly support |
+| `setup-vm.sh` | install dependencies on fresh ubuntu VM |
+
+## configuration
+
+```bash
+# customize benchmark parameters
+BENCH_REQUESTS=1000000 BENCH_THREADS=16 ./bench/compare-redis.sh
+
+# customize memory test
+KEY_COUNT=5000000 VALUE_SIZE=128 ./bench/bench-memory.sh
+```
+
+## environment variables
+
+| variable | default | description |
+|----------|---------|-------------|
+| `EMBER_CONCURRENT_PORT` | 6379 | ember concurrent mode port |
+| `EMBER_SHARDED_PORT` | 6380 | ember sharded mode port |
+| `REDIS_PORT` | 6399 | redis port |
+| `DRAGONFLY_PORT` | 6389 | dragonfly port |
+| `BENCH_REQUESTS` | 100000 | requests per test |
+| `BENCH_CLIENTS` | 50 | concurrent connections |
+| `BENCH_PIPELINE` | 16 | pipeline depth |
+| `BENCH_THREADS` | CPU cores | redis-benchmark threads |
 
 ## micro-benchmarks
 
-for criterion micro-benchmarks (keyspace, engine, protocol), see the `benches/` directories in `ember-core` and `ember-protocol`:
+for criterion micro-benchmarks:
 
 ```bash
-cargo bench -p emberkv-core   # keyspace + engine benchmarks
-cargo bench -p ember-protocol # RESP3 parse/serialize benchmarks
+cargo bench -p emberkv-core   # keyspace + engine
+cargo bench -p ember-protocol # RESP3 parse/serialize
 ```
diff --git a/scripts/bench-memory.sh → bench/bench-memory.sh b/scripts/bench-memory.sh → bench/bench-memory.sh
diff --git a/scripts/bench-quick.sh → bench/bench-quick.sh b/scripts/bench-quick.sh → bench/bench-quick.sh
diff --git a/scripts/bench.sh → bench/bench.sh b/scripts/bench.sh → bench/bench.sh