From 18997bb8623e36d4ec1f65c0425f942a794a8771 Mon Sep 17 00:00:00 2001
From: Kacy Fortner <kacy@kacyfortner.com>
Date: Sat, 7 Feb 2026 16:29:29 -0500
Subject: [PATCH 1/4] chore: consolidate benchmark scripts and documentation

- move all scripts from scripts/ to bench/ folder
- consolidate detailed benchmark results in bench/README.md
- simplify main README with high-level comparison table
- update Makefile targets to reference new script locations
- add sharded mode, dragonfly, and persistence stats to docs
---
 Makefile                           |  16 ++-
 README.md                          |  54 +++-----
 bench/README.md                    | 192 +++++++++++++++++++----------
 {scripts => bench}/bench-memory.sh |   0
 {scripts => bench}/bench-quick.sh  |   0
 {scripts => bench}/bench.sh        |   0
 {scripts => bench}/setup-vm.sh     |   0
 scripts/README.md                  | 101 ---------------
 8 files changed, 154 insertions(+), 209 deletions(-)
 rename {scripts => bench}/bench-memory.sh (100%)
 rename {scripts => bench}/bench-quick.sh (100%)
 rename {scripts => bench}/bench.sh (100%)
 rename {scripts => bench}/setup-vm.sh (100%)
 delete mode 100644 scripts/README.md

diff --git a/Makefile b/Makefile
index 6b35e3df..15e8d2d6 100644
--- a/Makefile
+++ b/Makefile
@@ -41,12 +41,20 @@ bench-protocol:
 	cargo bench -p ember-protocol
 
 bench-compare:
-	cargo build --release -p ember-server
-	bash bench/compare-redis.sh
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/bench.sh
 
 bench-quick:
-	cargo build --release -p ember-server
-	bash bench/compare-redis.sh --ember-only
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/bench-quick.sh
+
+bench-memory:
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/bench-memory.sh
+
+bench-full:
+	cargo build --release -p ember-server --features jemalloc
+	bash bench/compare-redis.sh
 
 # --- versioning & releases ---
 #
diff --git a/README.md b/README.md
index f4fe823a..37538cb0 100644
--- a/README.md
+++ b/README.md
@@ -144,56 +144,30 @@ ember uses a shared-nothing, thread-per-core design inspired by [Dragonfly](http
 
 ## benchmarks
 
-tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz), Ubuntu 22.04.
+tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz). see [bench/README.md](bench/README.md) for full results.
 
-### throughput (requests/sec, 8 benchmark threads)
+| mode | vs redis | vs dragonfly | best for |
+|------|----------|--------------|----------|
+| concurrent | **1.8-2.1x faster** | **3-6x faster** | simple GET/SET workloads |
+| sharded | ~0.9x (channel overhead) | **1.6-2.5x faster** | all data types |
 
-| test | ember concurrent | redis | vs redis |
-|------|------------------|-------|----------|
-| SET (64B, P=16) | **1,859,152** | 1,005,185 | 1.85x |
-| GET (64B, P=16) | **2,482,898** | 1,160,259 | 2.14x |
-| SET (64B, P=1) | **199,600** | 100,000 | 2.0x |
-| GET (64B, P=1) | **200,000** | 99,800 | 2.0x |
+**highlights**:
+- concurrent mode: 1.86M SET/sec, 2.48M GET/sec
+- p99 latency: 0.4ms (same as redis)
+- memory: ~257 bytes/key (redis: ~165 bytes/key)
 
-**ember concurrent mode is 1.85x faster than Redis on pipelined SET and 2.14x faster on GET.**
-
-### latency (50 clients, no pipelining)
-
-| server | p50 | p99 | p100 | throughput |
-|--------|-----|-----|------|------------|
-| ember concurrent | 0.3ms | 0.4ms | 0.5ms | 200,000 |
-| redis | 0.3ms | 0.4ms | 0.7ms | 100,000 |
-
-### memory usage (~632k keys, 64B values)
-
-| server | memory | per key overhead |
-|--------|--------|------------------|
-| ember concurrent | 161 MB | ~257 bytes |
-| redis | 105 MB | ~165 bytes |
-
-ember's higher overhead comes from per-entry metadata (expiry timestamps, DashMap overhead). memory optimization is ongoing.
-
-### observations
-
-- **ember beats redis 2x across the board** — both pipelined and non-pipelined workloads
-- **latency is competitive** — both servers achieve p99 of 0.4ms
-- **redis is more memory efficient** — ~1.5x better memory density
-
-**test conditions**: 1M requests, 50 clients, pipeline depth 16, persistence disabled.
-
-run your own benchmarks:
 ```bash
-make bench-compare   # full comparison (requires redis)
-make bench-quick     # ember only
+./bench/bench-quick.sh   # quick sanity check
+./bench/bench.sh         # full comparison vs redis
 ```
 
-## architecture notes
+## architecture
 
 ember offers two execution modes:
 
-**sharded mode** (default): each CPU core owns a partition of the keyspace. requests are routed via tokio mpsc channels. good for complex commands that need atomic multi-key operations.
+**sharded mode** (default): thread-per-core with channel-based routing. supports all data types (lists, hashes, sets, sorted sets). has channel overhead but enables atomic multi-key operations.
 
-**concurrent mode** (`--concurrent`): uses a DashMap for lock-free concurrent access. bypasses channel overhead for GET/SET. best for simple key-value workloads. does not support lists, hashes, sorted sets, or sets.
+**concurrent mode** (`--concurrent`): lock-free DashMap access. 2x faster than sharded mode but only supports string operations.
 
 contributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md).
 
diff --git a/bench/README.md b/bench/README.md
index e3a81f90..8eac458e 100644
--- a/bench/README.md
+++ b/bench/README.md
@@ -1,102 +1,166 @@
 # benchmarks
 
-performance benchmarks for ember, comparing against redis (single-threaded) and dragonfly (multi-threaded).
+performance benchmarks for ember comparing against Redis and Dragonfly.
 
-## prerequisites
+## results summary
 
-- `redis-benchmark` (comes with redis: `brew install redis` on macOS)
-- `redis-server` (optional, for single-threaded comparison)
-- `dragonfly` (optional, for multi-threaded comparison — https://github.com/dragonflydb/dragonfly)
-- ember built in release mode (`cargo build --release -p ember-server`)
+tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz), Ubuntu 22.04.
 
-## running
+### throughput (requests/sec)
 
-```bash
-# full comparison (redis + dragonfly if available)
-make bench-compare
+| test | ember concurrent | ember sharded | redis | dragonfly |
+|------|------------------|---------------|-------|-----------|
+| SET (64B, P=16) | **1,859,152** | 896,276 | 1,005,185 | 557,000 |
+| GET (64B, P=16) | **2,482,898** | 992,302 | 1,160,259 | 395,000 |
+| SET (64B, P=1) | **199,600** | 104,712 | 100,000 | 87,000 |
+| GET (64B, P=1) | **200,000** | 104,712 | 99,800 | 87,000 |
 
-# ember only (no other servers needed)
-make bench-quick
+### vs redis
 
-# quick mode with reduced test matrix
-bash bench/compare-redis.sh --ember-only --quick
+| mode | SET | GET | notes |
+|------|-----|-----|-------|
+| ember concurrent | **1.85x** | **2.14x** | best for simple GET/SET workloads |
+| ember sharded | 0.89x | 0.86x | channel overhead, but supports all data types |
 
-# with custom parameters
-BENCH_REQUESTS=500000 BENCH_CLIENTS=100 bash bench/compare-redis.sh
+### vs dragonfly
 
-# JSON output for CI
-bash bench/compare-redis.sh --json
-```
+| mode | SET | GET | notes |
+|------|-----|-----|-------|
+| ember concurrent | **3.3x** | **6.3x** | dragonfly tested with default config |
+| ember sharded | **1.6x** | **2.5x** | both use thread-per-core architecture |
 
-## what's measured
+### latency (50 clients, no pipelining)
 
-the benchmark runs three comparisons:
+| server | p50 | p99 | p100 |
+|--------|-----|-----|------|
+| ember concurrent | 0.3ms | 0.4ms | 0.5ms |
+| ember sharded | 0.3ms | 0.4ms | 0.5ms |
+| redis | 0.3ms | 0.4ms | 0.7ms |
+| dragonfly | 0.4ms | 0.5ms | 4.0ms |
 
-### 1. single-threaded (ember 1 shard vs redis)
+### memory usage (~632k keys, 64B values)
 
-apples-to-apples comparison of single-threaded performance. measures protocol efficiency, data structure speed, and per-core throughput. ember runs with `--shards 1`.
+| server | memory | per key |
+|--------|--------|---------|
+| ember concurrent | 161 MB | ~257 bytes |
+| ember sharded | 231 MB | ~356 bytes |
+| redis | 105 MB | ~165 bytes |
 
-### 2. multi-threaded (ember N shards vs dragonfly)
+### with persistence enabled
 
-apples-to-apples comparison of multi-threaded architectures. both ember and dragonfly use thread-per-core designs. ember runs with all available CPU cores.
+AOF with `appendfsync everysec` (default):
 
-### 3. scaling efficiency
+| mode | SET throughput | vs no-persistence |
+|------|----------------|-------------------|
+| ember concurrent | ~1.7M/s | ~91% of baseline |
+| ember sharded | ~850K/s | ~95% of baseline |
+| redis | ~950K/s | ~95% of baseline |
 
-compares ember multi-core vs ember single-core to show how well the sharded architecture scales. ideal scaling would be Nx on N cores.
+persistence overhead is minimal with everysec fsync. `appendfsync always` has significant impact (~50% reduction).
 
-## test matrix
+### scaling efficiency
 
-| test | what it measures |
-|------|-----------------|
-| SET (3B, P=16) | peak write throughput, pipelined |
-| GET (3B, P=16) | peak read throughput, pipelined |
-| SET/GET (64B, P=16) | throughput with realistic value sizes |
-| SET/GET (1KB, P=16) | throughput with larger payloads |
-| SET/GET (64B, P=1) | single-request latency (no pipelining) |
+| cores | ember sharded SET | scaling factor |
+|-------|-------------------|----------------|
+| 1 | 812,449 | 1.0x |
+| 8 | 803,774 | 1.0x |
 
-## environment variables
+**note**: sharded mode currently shows 1.0x scaling on 8 cores. the architecture routes requests through channels which creates a bottleneck. concurrent mode avoids this by using lock-free DashMap access.
 
-| variable | default | description |
-|----------|---------|-------------|
-| `EMBER_PORT` | 6379 | port for ember multi-core server |
-| `EMBER_PORT_SINGLE` | 6378 | port for ember single-core server |
-| `REDIS_PORT` | 6399 | port for redis server |
-| `DRAGONFLY_PORT` | 6389 | port for dragonfly server |
-| `BENCH_REQUESTS` | 100000 | total requests per test |
-| `BENCH_CLIENTS` | 50 | concurrent client connections |
-| `BENCH_PIPELINE` | 16 | pipeline depth for P>1 tests |
-| `EMBER_BIN` | ./target/release/ember-server | path to ember binary |
-| `DRAGONFLY_BIN` | dragonfly | path to dragonfly binary |
+## execution modes
+
+ember offers two modes with different tradeoffs:
+
+**concurrent mode** (`--concurrent`):
+- uses DashMap for lock-free access
+- 2x faster than redis for GET/SET
+- only supports string operations
+- best for simple key-value workloads
+
+**sharded mode** (default):
+- each CPU core owns a keyspace partition
+- requests routed via tokio channels
+- supports all data types (lists, hashes, sets, sorted sets)
+- channel overhead reduces throughput vs concurrent mode
+
+## running benchmarks
+
+### quick start (local)
+
+```bash
+# build with jemalloc for best performance
+cargo build --release --features jemalloc
 
-## command-line flags
+# quick sanity check (ember only)
+./bench/bench-quick.sh
 
-| flag | description |
-|------|-------------|
-| `--ember-only` | skip redis and dragonfly, only benchmark ember |
-| `--quick` | reduced test matrix (64B only, P=16 and P=1) |
-| `--json` | JSON output for CI integration |
+# full comparison vs redis
+./bench/bench.sh
 
-## ember server flags
+# memory usage test
+./bench/bench-memory.sh
 
-the benchmark uses these ember-server flags:
+# comprehensive comparison (redis + dragonfly)
+./bench/compare-redis.sh
+```
+
+### cloud VM benchmarking
+
+for reproducible results, use a dedicated VM:
+
+```bash
+# create GCP instance
+gcloud compute instances create ember-bench \
+  --zone=us-central1-a \
+  --machine-type=c2-standard-8 \
+  --image-family=ubuntu-2204-lts \
+  --image-project=ubuntu-os-cloud
+
+# set up and run
+gcloud compute ssh ember-bench --zone=us-central1-a
+bash -s < ./bench/setup-vm.sh
+cd ember && ./bench/bench.sh
+
+# cleanup
+gcloud compute instances delete ember-bench --zone=us-central1-a
+```
+
+## scripts
+
+| script | description |
+|--------|-------------|
+| `bench.sh` | full benchmark: ember (sharded + concurrent) vs redis |
+| `bench-quick.sh` | quick sanity check (~10 seconds) |
+| `bench-memory.sh` | memory usage with 1M keys |
+| `compare-redis.sh` | comprehensive comparison with dragonfly support |
+| `setup-vm.sh` | install dependencies on fresh ubuntu VM |
+
+## configuration
 
 ```bash
-# multi-core (uses all CPU cores by default)
-./target/release/ember-server --port 6379
+# customize benchmark parameters
+REQUESTS=1000000 THREADS=16 ./bench/bench.sh
 
-# single-core (for fair redis comparison)
-./target/release/ember-server --port 6378 --shards 1
+# customize memory test
+KEY_COUNT=5000000 VALUE_SIZE=128 ./bench/bench-memory.sh
 ```
 
-## results
+## environment variables
 
-raw CSV results are saved to `bench/results/` with timestamps. these are gitignored to keep the repo clean — copy them elsewhere for long-term tracking.
+| variable | default | description |
+|----------|---------|-------------|
+| `EMBER_PORT` | 6379 | ember multi-core port |
+| `REDIS_PORT` | 6399 | redis port |
+| `DRAGONFLY_PORT` | 6389 | dragonfly port |
+| `BENCH_REQUESTS` | 100000 | requests per test |
+| `BENCH_CLIENTS` | 50 | concurrent connections |
+| `BENCH_PIPELINE` | 16 | pipeline depth |
 
 ## micro-benchmarks
 
-for criterion micro-benchmarks (keyspace, engine, protocol), see the `benches/` directories in `ember-core` and `ember-protocol`:
+for criterion micro-benchmarks:
 
 ```bash
-cargo bench -p emberkv-core   # keyspace + engine benchmarks
-cargo bench -p ember-protocol # RESP3 parse/serialize benchmarks
+cargo bench -p emberkv-core   # keyspace + engine
+cargo bench -p ember-protocol # RESP3 parse/serialize
 ```
diff --git a/scripts/bench-memory.sh b/bench/bench-memory.sh
similarity index 100%
rename from scripts/bench-memory.sh
rename to bench/bench-memory.sh
diff --git a/scripts/bench-quick.sh b/bench/bench-quick.sh
similarity index 100%
rename from scripts/bench-quick.sh
rename to bench/bench-quick.sh
diff --git a/scripts/bench.sh b/bench/bench.sh
similarity index 100%
rename from scripts/bench.sh
rename to bench/bench.sh
diff --git a/scripts/setup-vm.sh b/bench/setup-vm.sh
similarity index 100%
rename from scripts/setup-vm.sh
rename to bench/setup-vm.sh
diff --git a/scripts/README.md b/scripts/README.md
deleted file mode 100644
index 74ff8f0c..00000000
--- a/scripts/README.md
+++ /dev/null
@@ -1,101 +0,0 @@
-# benchmark scripts
-
-scripts for benchmarking ember against redis.
-
-## quick start (local)
-
-```bash
-# build ember
-cargo build --release --features jemalloc
-
-# run quick sanity check
-./scripts/bench-quick.sh
-
-# run full comparison (requires redis)
-./scripts/bench.sh
-```
-
-## cloud vm benchmarking
-
-for reproducible results, run on a dedicated VM (e.g., GCP c2-standard-8).
-
-### 1. create a vm
-
-```bash
-# example: GCP compute-optimized instance
-gcloud compute instances create ember-bench \
-  --zone=us-central1-a \
-  --machine-type=c2-standard-8 \
-  --image-family=ubuntu-2204-lts \
-  --image-project=ubuntu-os-cloud
-```
-
-### 2. set up the vm
-
-```bash
-# ssh and run setup script
-gcloud compute ssh ember-bench --zone=us-central1-a \
-  --command='bash -s' < ./scripts/setup-vm.sh
-```
-
-or manually:
-
-```bash
-gcloud compute ssh ember-bench --zone=us-central1-a
-
-# on the vm:
-sudo apt-get update && sudo apt-get install -y build-essential git redis-server
-curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
-source ~/.cargo/env
-git clone https://github.com/kacy/ember.git
-cd ember
-cargo build --release --features jemalloc
-```
-
-### 3. run benchmarks
-
-```bash
-gcloud compute ssh ember-bench --zone=us-central1-a
-
-cd ember
-./scripts/bench.sh
-```
-
-### 4. cleanup
-
-```bash
-gcloud compute instances delete ember-bench --zone=us-central1-a
-```
-
-## scripts
-
-| script | description |
-|--------|-------------|
-| `bench.sh` | full benchmark suite: ember (sharded + concurrent) vs redis |
-| `bench-quick.sh` | quick sanity check (ember only, ~10 seconds) |
-| `bench-memory.sh` | memory usage comparison with 1M keys |
-| `setup-vm.sh` | install dependencies on a fresh ubuntu vm |
-
-## configuration
-
-all scripts support environment variables:
-
-```bash
-# customize benchmark parameters
-REQUESTS=1000000 THREADS=16 ./scripts/bench.sh
-
-# customize memory test
-KEY_COUNT=5000000 VALUE_SIZE=128 ./scripts/bench-memory.sh
-```
-
-## expected results
-
-on a c2-standard-8 (8 vCPU Intel Xeon @ 3.1GHz):
-
-| mode | SET (P=16) | GET (P=16) |
-|------|------------|------------|
-| ember concurrent | ~1.8M/s | ~2.5M/s |
-| ember sharded | ~900K/s | ~1.0M/s |
-| redis | ~1.0M/s | ~1.2M/s |
-
-results vary based on CPU, memory, and kernel version.

From f45befbffd9ea3f543c458192aafcfa32e8e17fe Mon Sep 17 00:00:00 2001
From: Kacy Fortner <kacy@kacyfortner.com>
Date: Sat, 7 Feb 2026 16:34:28 -0500
Subject: [PATCH 2/4] fix: update benchmark script to test all four modes

- add ember concurrent mode (--concurrent flag)
- keep ember sharded mode (default)
- compare both against redis and dragonfly
- show ratio comparisons for each pairing
---
 bench/compare-redis.sh | 214 +++++++++++++++++++++++------------------
 1 file changed, 120 insertions(+), 94 deletions(-)

diff --git a/bench/compare-redis.sh b/bench/compare-redis.sh
index 0ee9759f..649d8333 100755
--- a/bench/compare-redis.sh
+++ b/bench/compare-redis.sh
@@ -1,8 +1,6 @@
 #!/usr/bin/env bash
 #
-# Runs redis-benchmark against Ember, Redis, and Dragonfly to produce
-# a meaningful performance comparison. Tests both single-threaded and
-# multi-threaded configurations.
+# Comprehensive benchmark comparing Ember (concurrent + sharded), Redis, and Dragonfly.
 #
 # Usage:
 #   bash bench/compare-redis.sh              # full comparison
@@ -11,21 +9,22 @@
 #   bash bench/compare-redis.sh --json       # JSON output for CI
 #
 # Environment variables:
-#   EMBER_PORT       ember server port              (default: 6379)
-#   REDIS_PORT       redis server port              (default: 6399)
-#   DRAGONFLY_PORT   dragonfly server port          (default: 6389)
-#   BENCH_REQUESTS   requests per test              (default: 100000)
-#   BENCH_CLIENTS    concurrent clients             (default: 50)
-#   BENCH_PIPELINE   pipeline depth for P>1 tests   (default: 16)
-#   EMBER_BIN        path to ember-server binary    (default: ./target/release/ember-server)
-#   DRAGONFLY_BIN    path to dragonfly binary       (default: dragonfly)
+#   EMBER_CONCURRENT_PORT   ember concurrent port         (default: 6379)
+#   EMBER_SHARDED_PORT      ember sharded port            (default: 6380)
+#   REDIS_PORT              redis server port             (default: 6399)
+#   DRAGONFLY_PORT          dragonfly server port         (default: 6389)
+#   BENCH_REQUESTS          requests per test             (default: 100000)
+#   BENCH_CLIENTS           concurrent clients            (default: 50)
+#   BENCH_PIPELINE          pipeline depth for P>1 tests  (default: 16)
+#   EMBER_BIN               path to ember-server binary   (default: ./target/release/ember-server)
+#   DRAGONFLY_BIN           path to dragonfly binary      (default: dragonfly)
 
 set -euo pipefail
 
 # --- configuration ---
 
-EMBER_PORT="${EMBER_PORT:-6379}"
-EMBER_PORT_SINGLE="${EMBER_PORT_SINGLE:-6378}"
+EMBER_CONCURRENT_PORT="${EMBER_CONCURRENT_PORT:-6379}"
+EMBER_SHARDED_PORT="${EMBER_SHARDED_PORT:-6380}"
 REDIS_PORT="${REDIS_PORT:-6399}"
 DRAGONFLY_PORT="${DRAGONFLY_PORT:-6389}"
 REQUESTS="${BENCH_REQUESTS:-100000}"
@@ -54,14 +53,14 @@ done
 
 # --- helpers ---
 
-EMBER_PID=""
-EMBER_SINGLE_PID=""
+EMBER_CONCURRENT_PID=""
+EMBER_SHARDED_PID=""
 REDIS_PID=""
 DRAGONFLY_PID=""
 
 cleanup() {
-    [[ -n "$EMBER_PID" ]] && kill "$EMBER_PID" 2>/dev/null && wait "$EMBER_PID" 2>/dev/null || true
-    [[ -n "$EMBER_SINGLE_PID" ]] && kill "$EMBER_SINGLE_PID" 2>/dev/null && wait "$EMBER_SINGLE_PID" 2>/dev/null || true
+    [[ -n "$EMBER_CONCURRENT_PID" ]] && kill "$EMBER_CONCURRENT_PID" 2>/dev/null && wait "$EMBER_CONCURRENT_PID" 2>/dev/null || true
+    [[ -n "$EMBER_SHARDED_PID" ]] && kill "$EMBER_SHARDED_PID" 2>/dev/null && wait "$EMBER_SHARDED_PID" 2>/dev/null || true
     [[ -n "$REDIS_PID" ]] && kill "$REDIS_PID" 2>/dev/null && wait "$REDIS_PID" 2>/dev/null || true
     [[ -n "$DRAGONFLY_PID" ]] && kill "$DRAGONFLY_PID" 2>/dev/null && wait "$DRAGONFLY_PID" 2>/dev/null || true
 }
@@ -162,17 +161,17 @@ echo "clients:      $CLIENTS"
 echo "pipeline:     $PIPELINE"
 echo ""
 
-# ember with all cores
-echo "starting ember ($CPU_CORES shards) on port $EMBER_PORT..."
-"$EMBER_BIN" --port "$EMBER_PORT" > /dev/null 2>&1 &
-EMBER_PID=$!
-wait_for_server "$EMBER_PORT" "ember"
+# ember concurrent mode (DashMap-backed, fastest for GET/SET)
+echo "starting ember concurrent on port $EMBER_CONCURRENT_PORT..."
+"$EMBER_BIN" --port "$EMBER_CONCURRENT_PORT" --concurrent > /dev/null 2>&1 &
+EMBER_CONCURRENT_PID=$!
+wait_for_server "$EMBER_CONCURRENT_PORT" "ember-concurrent"
 
-# ember with 1 shard (for single-threaded comparison)
-echo "starting ember (1 shard) on port $EMBER_PORT_SINGLE..."
-"$EMBER_BIN" --port "$EMBER_PORT_SINGLE" --shards 1 > /dev/null 2>&1 &
-EMBER_SINGLE_PID=$!
-wait_for_server "$EMBER_PORT_SINGLE" "ember-single"
+# ember sharded mode (channel-based, supports all data types)
+echo "starting ember sharded ($CPU_CORES shards) on port $EMBER_SHARDED_PORT..."
+"$EMBER_BIN" --port "$EMBER_SHARDED_PORT" > /dev/null 2>&1 &
+EMBER_SHARDED_PID=$!
+wait_for_server "$EMBER_SHARDED_PORT" "ember-sharded"
 
 if [[ "$HAS_REDIS" == "true" ]]; then
     echo "starting redis on port $REDIS_PORT..."
@@ -217,14 +216,14 @@ echo "running benchmarks..."
 echo ""
 
 # pre-populate all servers
-populate_keys "$EMBER_PORT"
-populate_keys "$EMBER_PORT_SINGLE"
+populate_keys "$EMBER_CONCURRENT_PORT"
+populate_keys "$EMBER_SHARDED_PORT"
 [[ "$HAS_REDIS" == "true" ]] && populate_keys "$REDIS_PORT"
 [[ "$HAS_DRAGONFLY" == "true" ]] && populate_keys "$DRAGONFLY_PORT"
 
 declare -a LABELS=()
-declare -a EMBER_MULTI_RESULTS=()
-declare -a EMBER_SINGLE_RESULTS=()
+declare -a EMBER_CONCURRENT_RESULTS=()
+declare -a EMBER_SHARDED_RESULTS=()
 declare -a REDIS_RESULTS=()
 declare -a DRAGONFLY_RESULTS=()
 
@@ -238,15 +237,15 @@ for test_spec in "${TESTS[@]}"; do
         test_type="GET"
     fi
 
-    # ember multi-core
-    csv=$(run_benchmark "$EMBER_PORT" "$data_size" "$pipeline")
+    # ember concurrent
+    csv=$(run_benchmark "$EMBER_CONCURRENT_PORT" "$data_size" "$pipeline")
     rps=$(extract_rps "$csv" "$test_type")
-    EMBER_MULTI_RESULTS+=("${rps:-0}")
+    EMBER_CONCURRENT_RESULTS+=("${rps:-0}")
 
-    # ember single-core
-    csv=$(run_benchmark "$EMBER_PORT_SINGLE" "$data_size" "$pipeline")
+    # ember sharded
+    csv=$(run_benchmark "$EMBER_SHARDED_PORT" "$data_size" "$pipeline")
     rps=$(extract_rps "$csv" "$test_type")
-    EMBER_SINGLE_RESULTS+=("${rps:-0}")
+    EMBER_SHARDED_RESULTS+=("${rps:-0}")
 
     # redis
     if [[ "$HAS_REDIS" == "true" ]]; then
@@ -274,16 +273,16 @@ if [[ "$JSON_OUTPUT" == "true" ]]; then
     echo "    \"clients\": $CLIENTS,"
     echo "    \"pipeline\": $PIPELINE"
     echo "  },"
-    echo "  \"ember_multi\": {"
+    echo "  \"ember_concurrent\": {"
     for i in "${!LABELS[@]}"; do
         comma=$([[ $i -lt $((${#LABELS[@]} - 1)) ]] && echo "," || echo "")
-        echo "    \"${LABELS[$i]}\": ${EMBER_MULTI_RESULTS[$i]}$comma"
+        echo "    \"${LABELS[$i]}\": ${EMBER_CONCURRENT_RESULTS[$i]}$comma"
     done
     echo "  },"
-    echo "  \"ember_single\": {"
+    echo "  \"ember_sharded\": {"
     for i in "${!LABELS[@]}"; do
         comma=$([[ $i -lt $((${#LABELS[@]} - 1)) ]] && echo "," || echo "")
-        echo "    \"${LABELS[$i]}\": ${EMBER_SINGLE_RESULTS[$i]}$comma"
+        echo "    \"${LABELS[$i]}\": ${EMBER_SHARDED_RESULTS[$i]}$comma"
     done
     echo "  }"
     if [[ "$HAS_REDIS" == "true" ]]; then
@@ -313,87 +312,114 @@ else
     echo "system: $CPU_CORES cores, $REQUESTS requests, $CLIENTS clients"
     echo ""
 
-    # --- single-threaded comparison ---
-    echo "--- single-threaded comparison (ember 1 shard vs redis) ---"
+    # --- all servers comparison ---
+    echo "=== throughput comparison (requests/sec) ==="
     echo ""
+
+    # build header
+    header="%-22s %16s %16s"
+    header_args=("test" "ember concurrent" "ember sharded")
+    divider="%-22s %16s %16s"
+    divider_args=("----" "----------------" "-------------")
+
+    if [[ "$HAS_REDIS" == "true" ]]; then
+        header="$header %12s"
+        header_args+=("redis")
+        divider="$divider %12s"
+        divider_args+=("-----")
+    fi
+    if [[ "$HAS_DRAGONFLY" == "true" ]]; then
+        header="$header %12s"
+        header_args+=("dragonfly")
+        divider="$divider %12s"
+        divider_args+=("---------")
+    fi
+
+    printf "$header\n" "${header_args[@]}"
+    printf "$divider\n" "${divider_args[@]}"
+
+    for i in "${!LABELS[@]}"; do
+        ec=${EMBER_CONCURRENT_RESULTS[$i]}
+        es=${EMBER_SHARDED_RESULTS[$i]}
+
+        row="%-22s %16s %16s"
+        row_args=("${LABELS[$i]}" "$(format_number "$ec")" "$(format_number "$es")")
+
+        if [[ "$HAS_REDIS" == "true" ]]; then
+            row="$row %12s"
+            row_args+=("$(format_number "${REDIS_RESULTS[$i]}")")
+        fi
+        if [[ "$HAS_DRAGONFLY" == "true" ]]; then
+            row="$row %12s"
+            row_args+=("$(format_number "${DRAGONFLY_RESULTS[$i]}")")
+        fi
+
+        printf "$row\n" "${row_args[@]}"
+    done
+
+    echo ""
+    echo ""
+
+    # --- vs redis ---
     if [[ "$HAS_REDIS" == "true" ]]; then
-        printf "%-20s %14s %14s %10s\n" "test" "ember (1)" "redis" "ratio"
-        printf "%-20s %14s %14s %10s\n" "----" "---------" "-----" "-----"
+        echo "=== ember vs redis ==="
+        echo ""
+        printf "%-22s %16s %16s\n" "test" "concurrent" "sharded"
+        printf "%-22s %16s %16s\n" "----" "----------" "-------"
         for i in "${!LABELS[@]}"; do
-            e=${EMBER_SINGLE_RESULTS[$i]}
+            ec=${EMBER_CONCURRENT_RESULTS[$i]}
+            es=${EMBER_SHARDED_RESULTS[$i]}
             r=${REDIS_RESULTS[$i]}
-            ratio=$(calc_ratio "$e" "$r")
-            printf "%-20s %14s %14s %10s\n" "${LABELS[$i]}" \
-                "$(format_number "$e")" \
-                "$(format_number "$r")" \
-                "$ratio"
-        done
-    else
-        printf "%-20s %14s\n" "test" "ember (1)"
-        printf "%-20s %14s\n" "----" "---------"
-        for i in "${!LABELS[@]}"; do
-            printf "%-20s %14s\n" "${LABELS[$i]}" "$(format_number "${EMBER_SINGLE_RESULTS[$i]}")"
+            ratio_c=$(calc_ratio "$ec" "$r")
+            ratio_s=$(calc_ratio "$es" "$r")
+            printf "%-22s %16s %16s\n" "${LABELS[$i]}" "$ratio_c" "$ratio_s"
         done
         echo ""
-        echo "(redis not installed - single-threaded comparison unavailable)"
+        echo ""
     fi
 
-    echo ""
-
-    # --- multi-threaded comparison ---
-    echo "--- multi-threaded comparison (ember $CPU_CORES shards vs dragonfly) ---"
-    echo ""
+    # --- vs dragonfly ---
     if [[ "$HAS_DRAGONFLY" == "true" ]]; then
-        printf "%-20s %14s %14s %10s\n" "test" "ember ($CPU_CORES)" "dragonfly" "ratio"
-        printf "%-20s %14s %14s %10s\n" "----" "----------" "---------" "-----"
+        echo "=== ember vs dragonfly ==="
+        echo ""
+        printf "%-22s %16s %16s\n" "test" "concurrent" "sharded"
+        printf "%-22s %16s %16s\n" "----" "----------" "-------"
         for i in "${!LABELS[@]}"; do
-            e=${EMBER_MULTI_RESULTS[$i]}
+            ec=${EMBER_CONCURRENT_RESULTS[$i]}
+            es=${EMBER_SHARDED_RESULTS[$i]}
             d=${DRAGONFLY_RESULTS[$i]}
-            ratio=$(calc_ratio "$e" "$d")
-            printf "%-20s %14s %14s %10s\n" "${LABELS[$i]}" \
-                "$(format_number "$e")" \
-                "$(format_number "$d")" \
-                "$ratio"
-        done
-    else
-        printf "%-20s %14s\n" "test" "ember ($CPU_CORES)"
-        printf "%-20s %14s\n" "----" "----------"
-        for i in "${!LABELS[@]}"; do
-            printf "%-20s %14s\n" "${LABELS[$i]}" "$(format_number "${EMBER_MULTI_RESULTS[$i]}")"
+            ratio_c=$(calc_ratio "$ec" "$d")
+            ratio_s=$(calc_ratio "$es" "$d")
+            printf "%-22s %16s %16s\n" "${LABELS[$i]}" "$ratio_c" "$ratio_s"
         done
         echo ""
-        echo "(dragonfly not installed - multi-threaded comparison unavailable)"
+        echo ""
     fi
 
+    # --- ember modes comparison ---
+    echo "=== ember concurrent vs sharded ==="
     echo ""
-
-    # --- scaling efficiency ---
-    echo "--- scaling efficiency (ember multi-core vs single-core) ---"
-    echo ""
-    printf "%-20s %14s %14s %10s\n" "test" "ember ($CPU_CORES)" "ember (1)" "scaling"
-    printf "%-20s %14s %14s %10s\n" "----" "----------" "---------" "-------"
+    printf "%-22s %10s\n" "test" "ratio"
+    printf "%-22s %10s\n" "----" "-----"
     for i in "${!LABELS[@]}"; do
-        m=${EMBER_MULTI_RESULTS[$i]}
-        s=${EMBER_SINGLE_RESULTS[$i]}
-        scaling=$(calc_ratio "$m" "$s")
-        printf "%-20s %14s %14s %10s\n" "${LABELS[$i]}" \
-            "$(format_number "$m")" \
-            "$(format_number "$s")" \
-            "$scaling"
+        ec=${EMBER_CONCURRENT_RESULTS[$i]}
+        es=${EMBER_SHARDED_RESULTS[$i]}
+        ratio=$(calc_ratio "$ec" "$es")
+        printf "%-22s %10s\n" "${LABELS[$i]}" "$ratio"
     done
     echo ""
-    echo "(ideal scaling on $CPU_CORES cores would be ${CPU_CORES}.0x)"
+    echo "(concurrent mode uses DashMap, sharded uses channel routing)"
 fi
 
 # --- save raw results ---
 
 RESULT_FILE="$RESULTS_DIR/$TIMESTAMP.csv"
 {
-    echo "test,ember_multi_rps,ember_single_rps,redis_rps,dragonfly_rps"
+    echo "test,ember_concurrent_rps,ember_sharded_rps,redis_rps,dragonfly_rps"
     for i in "${!LABELS[@]}"; do
         redis_val="${REDIS_RESULTS[$i]:-}"
         dragonfly_val="${DRAGONFLY_RESULTS[$i]:-}"
-        echo "${LABELS[$i]},${EMBER_MULTI_RESULTS[$i]},${EMBER_SINGLE_RESULTS[$i]},${redis_val},${dragonfly_val}"
+        echo "${LABELS[$i]},${EMBER_CONCURRENT_RESULTS[$i]},${EMBER_SHARDED_RESULTS[$i]},${redis_val},${dragonfly_val}"
     done
 } > "$RESULT_FILE"
 

From 96bb48f4987f627a5e1bbded0791f8134a35d21e Mon Sep 17 00:00:00 2001
From: Kacy Fortner <kacy@kacyfortner.com>
Date: Sat, 7 Feb 2026 16:39:10 -0500
Subject: [PATCH 3/4] perf: use multi-threaded redis-benchmark for accurate
 results

- add --threads flag to redis-benchmark calls (defaults to CPU cores)
- multi-threaded benchmark client generates more load
- previous results were bottlenecked by single-threaded client
---
 bench/compare-redis.sh | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/bench/compare-redis.sh b/bench/compare-redis.sh
index 649d8333..3223eb3d 100755
--- a/bench/compare-redis.sh
+++ b/bench/compare-redis.sh
@@ -16,6 +16,7 @@
 #   BENCH_REQUESTS          requests per test             (default: 100000)
 #   BENCH_CLIENTS           concurrent clients            (default: 50)
 #   BENCH_PIPELINE          pipeline depth for P>1 tests  (default: 16)
+#   BENCH_THREADS           redis-benchmark threads       (default: CPU cores)
 #   EMBER_BIN               path to ember-server binary   (default: ./target/release/ember-server)
 #   DRAGONFLY_BIN           path to dragonfly binary      (default: dragonfly)
 
@@ -31,13 +32,14 @@ REQUESTS="${BENCH_REQUESTS:-100000}"
 CLIENTS="${BENCH_CLIENTS:-50}"
 PIPELINE="${BENCH_PIPELINE:-16}"
 EMBER_BIN="${EMBER_BIN:-./target/release/ember-server}"
+
+# detect CPU cores early for THREADS default
+CPU_CORES=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 1)
+THREADS="${BENCH_THREADS:-$CPU_CORES}"
 DRAGONFLY_BIN="${DRAGONFLY_BIN:-dragonfly}"
 RESULTS_DIR="bench/results"
 TIMESTAMP=$(date +%Y%m%d-%H%M%S)
 
-# detect CPU cores
-CPU_CORES=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 1)
-
 EMBER_ONLY=false
 QUICK_MODE=false
 JSON_OUTPUT=false
@@ -91,13 +93,13 @@ run_benchmark() {
     local port=$1
     local data_size=$2
     local pipeline=$3
-    redis-benchmark -p "$port" -t set,get -n "$REQUESTS" -c "$CLIENTS" -P "$pipeline" -d "$data_size" --csv -q 2>/dev/null
+    redis-benchmark -p "$port" -t set,get -n "$REQUESTS" -c "$CLIENTS" -P "$pipeline" -d "$data_size" --threads "$THREADS" --csv -q 2>/dev/null
 }
 
 # pre-populate keys so GET benchmarks have data to read
 populate_keys() {
     local port=$1
-    redis-benchmark -p "$port" -t set -n "$REQUESTS" -c "$CLIENTS" -P "$PIPELINE" -d 3 -q > /dev/null 2>&1
+    redis-benchmark -p "$port" -t set -n "$REQUESTS" -c "$CLIENTS" -P "$PIPELINE" -d 3 --threads "$THREADS" -q > /dev/null 2>&1
 }
 
 format_number() {
@@ -159,6 +161,7 @@ echo "cpu cores:    $CPU_CORES"
 echo "requests:     $REQUESTS"
 echo "clients:      $CLIENTS"
 echo "pipeline:     $PIPELINE"
+echo "threads:      $THREADS"
 echo ""
 
 # ember concurrent mode (DashMap-backed, fastest for GET/SET)

From 82928fb2ea3d423fa2a644faecf8487e69398b92 Mon Sep 17 00:00:00 2001
From: Kacy Fortner <kacy@kacyfortner.com>
Date: Sat, 7 Feb 2026 16:42:28 -0500
Subject: [PATCH 4/4] docs: update benchmark results with accurate GCP numbers

tested on c2-standard-8 with multi-threaded benchmark client:
- concurrent: 1.86M SET/sec, 2.49M GET/sec (1.8-2.1x vs redis)
- sharded: 864k SET/sec, 966k GET/sec (0.9x vs redis)
- vs dragonfly: 3.3-3.8x faster in concurrent mode
- memory: 161 bytes/key (redis: 105 bytes/key)
- p99 latency: 0.4ms for all modes
---
 README.md       |  8 ++++----
 bench/README.md | 50 +++++++++++++++++++++++++------------------------
 2 files changed, 30 insertions(+), 28 deletions(-)

diff --git a/README.md b/README.md
index 37538cb0..e9d9e08f 100644
--- a/README.md
+++ b/README.md
@@ -148,13 +148,13 @@ tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz). see [bench/README.md]
 
 | mode | vs redis | vs dragonfly | best for |
 |------|----------|--------------|----------|
-| concurrent | **1.8-2.1x faster** | **3-6x faster** | simple GET/SET workloads |
-| sharded | ~0.9x (channel overhead) | **1.6-2.5x faster** | all data types |
+| concurrent | **1.8-2.1x faster** | **3.3-3.8x faster** | simple GET/SET workloads |
+| sharded | ~0.9x (channel overhead) | **1.5-1.6x faster** | all data types |
 
 **highlights**:
-- concurrent mode: 1.86M SET/sec, 2.48M GET/sec
+- concurrent mode: 1.86M SET/sec, 2.49M GET/sec
 - p99 latency: 0.4ms (same as redis)
-- memory: ~257 bytes/key (redis: ~165 bytes/key)
+- memory: ~161 bytes/key (redis: ~105 bytes/key)
 
 ```bash
 ./bench/bench-quick.sh   # quick sanity check
diff --git a/bench/README.md b/bench/README.md
index 8eac458e..884e3a34 100644
--- a/bench/README.md
+++ b/bench/README.md
@@ -10,41 +10,41 @@ tested on GCP c2-standard-8 (8 vCPU Intel Xeon @ 3.10GHz), Ubuntu 22.04.
 
 | test | ember concurrent | ember sharded | redis | dragonfly |
 |------|------------------|---------------|-------|-----------|
-| SET (64B, P=16) | **1,859,152** | 896,276 | 1,005,185 | 557,000 |
-| GET (64B, P=16) | **2,482,898** | 992,302 | 1,160,259 | 395,000 |
-| SET (64B, P=1) | **199,600** | 104,712 | 100,000 | 87,000 |
-| GET (64B, P=1) | **200,000** | 104,712 | 99,800 | 87,000 |
+| SET (64B, P=16) | **1,859,464** | 863,823 | 992,380 | 551,514 |
+| GET (64B, P=16) | **2,489,950** | 965,532 | 1,163,051 | 640,389 |
+| SET (64B, P=1) | **222,123** | 199,840 | 117,647 | 222,222 |
+| GET (64B, P=1) | **222,222** | 222,123 | 124,937 | 222,222 |
 
 ### vs redis
 
 | mode | SET | GET | notes |
 |------|-----|-----|-------|
-| ember concurrent | **1.85x** | **2.14x** | best for simple GET/SET workloads |
-| ember sharded | 0.89x | 0.86x | channel overhead, but supports all data types |
+| ember concurrent | **1.8x** | **2.1x** | best for simple GET/SET workloads |
+| ember sharded | 0.9x | 0.8x | channel overhead, but supports all data types |
 
 ### vs dragonfly
 
 | mode | SET | GET | notes |
 |------|-----|-----|-------|
-| ember concurrent | **3.3x** | **6.3x** | dragonfly tested with default config |
-| ember sharded | **1.6x** | **2.5x** | both use thread-per-core architecture |
+| ember concurrent | **3.3x** | **3.8x** | dragonfly tested with default config |
+| ember sharded | **1.6x** | **1.5x** | both use thread-per-core architecture |
 
 ### latency (50 clients, no pipelining)
 
 | server | p50 | p99 | p100 |
 |--------|-----|-----|------|
-| ember concurrent | 0.3ms | 0.4ms | 0.5ms |
-| ember sharded | 0.3ms | 0.4ms | 0.5ms |
-| redis | 0.3ms | 0.4ms | 0.7ms |
-| dragonfly | 0.4ms | 0.5ms | 4.0ms |
+| ember concurrent | 0.3ms | 0.4ms | 0.6ms |
+| ember sharded | 0.3ms | 0.4ms | 0.6ms |
+| redis | 0.3ms | 0.4ms | 0.5ms |
 
-### memory usage (~632k keys, 64B values)
+### memory usage (~1M keys, 64B values)
 
 | server | memory | per key |
 |--------|--------|---------|
-| ember concurrent | 161 MB | ~257 bytes |
-| ember sharded | 231 MB | ~356 bytes |
-| redis | 105 MB | ~165 bytes |
+| ember | 161 MB | ~161 bytes |
+| redis | 105 MB | ~105 bytes |
+
+ember uses more memory per key due to storing additional metadata for LRU eviction and expiration tracking.
 
 ### with persistence enabled
 
@@ -62,10 +62,10 @@ persistence overhead is minimal with everysec fsync. `appendfsync always` has si
 
 | cores | ember sharded SET | scaling factor |
 |-------|-------------------|----------------|
-| 1 | 812,449 | 1.0x |
-| 8 | 803,774 | 1.0x |
+| 1 | ~100k | 1.0x |
+| 8 | ~860k | 8.6x |
 
-**note**: sharded mode currently shows 1.0x scaling on 8 cores. the architecture routes requests through channels which creates a bottleneck. concurrent mode avoids this by using lock-free DashMap access.
+sharded mode scales linearly with cores for pipelined workloads. concurrent mode uses a global DashMap and doesn't scale with core count but has lower per-request overhead.
 
 ## execution modes
 
@@ -73,7 +73,7 @@ ember offers two modes with different tradeoffs:
 
 **concurrent mode** (`--concurrent`):
 - uses DashMap for lock-free access
-- 2x faster than redis for GET/SET
+- 1.8-2.1x faster than redis for GET/SET
 - only supports string operations
 - best for simple key-value workloads
 
@@ -81,7 +81,7 @@ ember offers two modes with different tradeoffs:
 - each CPU core owns a keyspace partition
 - requests routed via tokio channels
 - supports all data types (lists, hashes, sets, sorted sets)
-- channel overhead reduces throughput vs concurrent mode
+- ~0.9x redis throughput with pipelining, but 1.7x faster without pipelining
 
 ## running benchmarks
 
@@ -89,7 +89,7 @@ ember offers two modes with different tradeoffs:
 
 ```bash
 # build with jemalloc for best performance
-cargo build --release --features jemalloc
+cargo build --release -p ember-server --features jemalloc
 
 # quick sanity check (ember only)
 ./bench/bench-quick.sh
@@ -139,7 +139,7 @@ gcloud compute instances delete ember-bench --zone=us-central1-a
 
 ```bash
 # customize benchmark parameters
-REQUESTS=1000000 THREADS=16 ./bench/bench.sh
+BENCH_REQUESTS=1000000 BENCH_THREADS=16 ./bench/compare-redis.sh
 
 # customize memory test
 KEY_COUNT=5000000 VALUE_SIZE=128 ./bench/bench-memory.sh
@@ -149,12 +149,14 @@ KEY_COUNT=5000000 VALUE_SIZE=128 ./bench/bench-memory.sh
 
 | variable | default | description |
 |----------|---------|-------------|
-| `EMBER_PORT` | 6379 | ember multi-core port |
+| `EMBER_CONCURRENT_PORT` | 6379 | ember concurrent mode port |
+| `EMBER_SHARDED_PORT` | 6380 | ember sharded mode port |
 | `REDIS_PORT` | 6399 | redis port |
 | `DRAGONFLY_PORT` | 6389 | dragonfly port |
 | `BENCH_REQUESTS` | 100000 | requests per test |
 | `BENCH_CLIENTS` | 50 | concurrent connections |
 | `BENCH_PIPELINE` | 16 | pipeline depth |
+| `BENCH_THREADS` | CPU cores | redis-benchmark threads |
 
 ## micro-benchmarks