Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ a low-latency, memory-efficient, distributed cache written in Rust. designed to
- **lru eviction** — approximate LRU via random sampling when memory pressure hits
- **persistence** — append-only file (AOF) and point-in-time snapshots
- **pipelining** — multiple commands per read for high throughput
- **graceful shutdown** — drains active connections on SIGINT/SIGTERM before exiting

## quickstart

Expand Down
4 changes: 2 additions & 2 deletions crates/ember-cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

interactive command-line client for [ember](https://github.com/kacy/ember).

> this crate is a stub — the CLI is planned for a future phase.
> this crate is a stub — the CLI is planned for phase 5.

## planned features

Expand All @@ -20,4 +20,4 @@ interactive command-line client for [ember](https://github.com/kacy/ember).
| [ember-protocol](../ember-protocol) | RESP3 parsing and command dispatch |
| [ember-persistence](../ember-persistence) | AOF, snapshots, and crash recovery |
| [ember-server](../ember-server) | TCP server and connection handling |
| [ember-cluster](../ember-cluster) | distributed coordination (WIP) |
| [ember-cluster](../ember-cluster) | distributed coordination |
2 changes: 1 addition & 1 deletion crates/ember-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,4 +74,4 @@ the following CLUSTER commands are supported at the protocol layer:
| [ember-protocol](../ember-protocol) | RESP3 parsing and command dispatch |
| [ember-persistence](../ember-persistence) | AOF, snapshots, and crash recovery |
| [ember-server](../ember-server) | TCP server and connection handling |
| [ember-cli](../ember-cli) | interactive command-line client |
| [ember-cli](../ember-cli) | interactive command-line client (planned) |
8 changes: 4 additions & 4 deletions crates/ember-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ this is the heart of ember — it implements the shared-nothing, shard-per-core

- **engine** — routes requests to shards by key hash, supports single-key, multi-key, and broadcast operations
- **shard** — the single-threaded event loop per partition: dispatch, AOF recording, expiration ticks, fsync ticks
- **keyspace** — the key-value store itself: strings, lists, sorted sets, TTL, LRU eviction
- **types** — `Value` enum with `String(Bytes)`, `List(VecDeque<Bytes>)`, `SortedSet` (BTreeMap + HashMap dual-index)
- **keyspace** — the key-value store itself: strings, lists, sorted sets, hashes, sets, TTL, LRU eviction
- **types** — `Value` enum with `String(Bytes)`, `List(VecDeque<Bytes>)`, `SortedSet` (BTreeMap + HashMap dual-index), `Hash(HashMap)`, `Set(HashSet)`
- **memory** — per-shard memory tracking and entry size estimation
- **expiry** — lazy (on access) and active (background sampling) TTL expiration

Expand All @@ -34,5 +34,5 @@ let response = engine.route("mykey", ShardRequest::Get {
| [ember-protocol](../ember-protocol) | RESP3 parsing and command dispatch |
| [ember-persistence](../ember-persistence) | AOF, snapshots, and crash recovery |
| [ember-server](../ember-server) | TCP server and connection handling |
| [ember-cluster](../ember-cluster) | distributed coordination (WIP) |
| [ember-cli](../ember-cli) | interactive command-line client (WIP) |
| [ember-cluster](../ember-cluster) | distributed coordination |
| [ember-cli](../ember-cli) | interactive command-line client (planned) |
8 changes: 4 additions & 4 deletions crates/ember-core/src/keyspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -558,7 +558,7 @@ impl Keyspace {
/// Scans keys starting from a cursor position.
///
/// Returns the next cursor (0 if scan complete) and a batch of keys.
/// The `pattern` argument supports glob-style matching (*, ?, [abc]).
/// The `pattern` argument supports glob-style matching (`*`, `?`, `[abc]`).
pub fn scan_keys(
&self,
cursor: u64,
Expand Down Expand Up @@ -1777,7 +1777,7 @@ mod tests {
Some(Duration::from_secs(100)),
);
match ks.ttl("key") {
TtlResult::Seconds(s) => assert!(s >= 98 && s <= 100),
TtlResult::Seconds(s) => assert!((98..=100).contains(&s)),
other => panic!("expected Seconds, got {other:?}"),
}
}
Expand All @@ -1800,7 +1800,7 @@ mod tests {
ks.set("key".into(), Bytes::from("val"), None);
assert!(ks.expire("key", 60));
match ks.ttl("key") {
TtlResult::Seconds(s) => assert!(s >= 58 && s <= 60),
TtlResult::Seconds(s) => assert!((58..=60).contains(&s)),
other => panic!("expected Seconds, got {other:?}"),
}
}
Expand Down Expand Up @@ -2584,7 +2584,7 @@ mod tests {
ks.set("n".into(), Bytes::from("5"), Some(Duration::from_secs(60)));
ks.incr("n").unwrap();
match ks.ttl("n") {
TtlResult::Seconds(s) => assert!(s >= 58 && s <= 60),
TtlResult::Seconds(s) => assert!((58..=60).contains(&s)),
other => panic!("expected TTL preserved, got {other:?}"),
}
}
Expand Down
2 changes: 1 addition & 1 deletion crates/ember-core/src/shard.rs
Original file line number Diff line number Diff line change
Expand Up @@ -967,7 +967,7 @@ mod tests {

let resp = dispatch(&mut ks, &ShardRequest::Ttl { key: "key".into() });
match resp {
ShardResponse::Ttl(TtlResult::Seconds(s)) => assert!(s >= 58 && s <= 60),
ShardResponse::Ttl(TtlResult::Seconds(s)) => assert!((58..=60).contains(&s)),
other => panic!("expected Ttl(Seconds), got {other:?}"),
}
}
Expand Down
8 changes: 5 additions & 3 deletions crates/ember-persistence/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@ each shard gets its own persistence files (`shard-{id}.aof` and `shard-{id}.snap

**AOF** — `[EAOF magic][version][record...]` where each record is `[tag][payload][crc32]`

**snapshot (v2)** — `[ESNP magic][version][shard_id][entry_count][entries...][footer_crc32]` where entries are type-tagged (string=0, list=1, sorted set=2). v1 snapshots (no type tags) are still readable.
supported record types: SET, DEL, EXPIRE, LPUSH, RPUSH, LPOP, RPOP, ZADD, ZREM, HSET, HDEL, HINCRBY, SADD, SREM

**snapshot (v2)** — `[ESNP magic][version][shard_id][entry_count][entries...][footer_crc32]` where entries are type-tagged (string=0, list=1, sorted set=2, hash=3, set=4). v1 snapshots (no type tags) are still readable.

## usage

Expand All @@ -38,5 +40,5 @@ for entry in result.entries {
| [emberkv-core](../ember-core) | storage engine, keyspace, sharding |
| [ember-protocol](../ember-protocol) | RESP3 parsing and command dispatch |
| [ember-server](../ember-server) | TCP server and connection handling |
| [ember-cluster](../ember-cluster) | distributed coordination (WIP) |
| [ember-cli](../ember-cli) | interactive command-line client (WIP) |
| [ember-cluster](../ember-cluster) | distributed coordination |
| [ember-cli](../ember-cli) | interactive command-line client (planned) |
4 changes: 3 additions & 1 deletion crates/ember-persistence/src/aof.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ const TAG_SREM: u8 = 18;
/// A single mutation record stored in the AOF.
#[derive(Debug, Clone, PartialEq)]
pub enum AofRecord {
/// SET key value [expire_ms]. expire_ms is -1 for no expiration.
/// SET key value \[expire_ms\]. expire_ms is -1 for no expiration.
Set {
key: String,
value: Bytes,
Expand Down Expand Up @@ -444,6 +444,8 @@ impl AofWriter {
let mut writer = BufWriter::new(file);
format::write_header(&mut writer, format::AOF_MAGIC)?;
writer.flush()?;
// ensure the fresh header is durable before we start appending
writer.get_ref().sync_all()?;
self.writer = writer;
Ok(())
}
Expand Down
2 changes: 1 addition & 1 deletion crates/ember-persistence/src/snapshot.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ pub struct SnapshotWriter {

impl SnapshotWriter {
/// Creates a new snapshot writer. The file won't appear at `path`
/// until [`finish`] is called successfully.
/// until [`Self::finish`] is called successfully.
pub fn create(path: impl Into<PathBuf>, shard_id: u16) -> Result<Self, FormatError> {
let final_path = path.into();
let tmp_path = final_path.with_extension("snap.tmp");
Expand Down
22 changes: 15 additions & 7 deletions crates/ember-protocol/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ RESP3 wire protocol implementation for [ember](https://github.com/kacy/ember). h
- **parse** — zero-copy RESP3 frame parser that works directly on byte slices, returning `(Frame, bytes_consumed)` for pipelining support
- **serialize** — writes frames directly into `BytesMut` with no intermediate allocations
- **command** — converts raw frames into typed `Command` enums with argument validation, arity checks, and flag parsing
- **types** — `Frame` enum: `Simple`, `Error`, `Integer`, `Bulk`, `Null`, `Array`
- **types** — `Frame` enum: `Simple`, `Error`, `Integer`, `Bulk`, `Null`, `Array`, `Map`

## quick start

Expand Down Expand Up @@ -36,13 +36,21 @@ let cmd = Command::from_frame(frame).unwrap();

## supported commands

strings: `GET`, `SET`, `DEL`, `EXISTS`, `EXPIRE`, `TTL`, `TYPE`
**strings**: `GET`, `SET` (with NX/XX/EX/PX), `INCR`, `DECR`, `MGET`, `MSET`

lists: `LPUSH`, `RPUSH`, `LPOP`, `RPOP`, `LRANGE`, `LLEN`
**lists**: `LPUSH`, `RPUSH`, `LPOP`, `RPOP`, `LRANGE`, `LLEN`

sorted sets: `ZADD` (with NX/XX/GT/LT/CH flags), `ZREM`, `ZSCORE`, `ZRANK`, `ZRANGE` (with WITHSCORES)
**sorted sets**: `ZADD` (with NX/XX/GT/LT/CH flags), `ZREM`, `ZSCORE`, `ZRANK`, `ZRANGE` (with WITHSCORES), `ZCARD`

server: `PING`, `ECHO`, `DBSIZE`, `INFO`, `BGSAVE`, `BGREWRITEAOF`
**hashes**: `HSET`, `HGET`, `HGETALL`, `HDEL`, `HEXISTS`, `HLEN`, `HINCRBY`, `HKEYS`, `HVALS`, `HMGET`

**sets**: `SADD`, `SREM`, `SMEMBERS`, `SISMEMBER`, `SCARD`

**keys**: `DEL`, `EXISTS`, `EXPIRE`, `PEXPIRE`, `TTL`, `PTTL`, `PERSIST`, `TYPE`, `SCAN`

**server**: `PING`, `ECHO`, `DBSIZE`, `INFO`, `BGSAVE`, `BGREWRITEAOF`, `FLUSHDB`

**cluster**: `CLUSTER INFO`, `NODES`, `SLOTS`, `KEYSLOT`, `MYID`, `MEET`, `ADDSLOTS`, `DELSLOTS`, `SETSLOT`, `FORGET`, `REPLICATE`, `FAILOVER`, `COUNTKEYSINSLOT`, `GETKEYSINSLOT`, `MIGRATE`, `ASKING`

## related crates

Expand All @@ -51,5 +59,5 @@ server: `PING`, `ECHO`, `DBSIZE`, `INFO`, `BGSAVE`, `BGREWRITEAOF`
| [emberkv-core](../ember-core) | storage engine, keyspace, sharding |
| [ember-persistence](../ember-persistence) | AOF, snapshots, and crash recovery |
| [ember-server](../ember-server) | TCP server and connection handling |
| [ember-cluster](../ember-cluster) | distributed coordination (WIP) |
| [ember-cli](../ember-cli) | interactive command-line client (WIP) |
| [ember-cluster](../ember-cluster) | distributed coordination |
| [ember-cli](../ember-cli) | interactive command-line client (planned) |
Loading