Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 72 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,62 +2,98 @@
<img src="ember-logo.png" alt="ember logo" width="200">
</p>

# ❤️‍🔥 ember
# ember

a low-latency, memory-efficient, distributed cache written in Rust. designed to outperform Redis on throughput, latency, and memory efficiency while keeping the codebase small and readable.

## features

- **resp3 protocol** — full compatibility with `redis-cli` and existing Redis clients
- **core commands** — GET, SET, DEL, EXISTS, EXPIRE, TTL, TYPE, DBSIZE, INFO with proper semantics
- **list operations** — LPUSH, RPUSH, LPOP, RPOP, LRANGE, LLEN with O(1) push/pop and auto-cleanup
- **sorted sets** — ZADD (with NX/XX/GT/LT/CH flags), ZREM, ZSCORE, ZRANK, ZRANGE with WITHSCORES
- **sharded engine** — shared-nothing, thread-per-core design with no cross-shard locking on the hot path
- **string commands** — GET, SET (with NX/XX/EX/PX), MGET, MSET, INCR, DECR
- **list operations** — LPUSH, RPUSH, LPOP, RPOP, LRANGE, LLEN
- **sorted sets** — ZADD (with NX/XX/GT/LT/CH), ZREM, ZSCORE, ZRANK, ZRANGE, ZCARD
- **key commands** — DEL, EXISTS, EXPIRE, TTL, PEXPIRE, PTTL, PERSIST, TYPE, SCAN
- **server commands** — PING, ECHO, INFO, DBSIZE, FLUSHDB, BGSAVE, BGREWRITEAOF
- **sharded engine** — shared-nothing, thread-per-core design with no cross-shard locking
- **active expiration** — background sampling cleans up expired keys without client access
- **memory tracking** — per-shard byte-level accounting with configurable memory limits
- **memory limits** — per-shard byte-level accounting with configurable limits
- **lru eviction** — approximate LRU via random sampling when memory pressure hits
- **pipelined connections** — multiple commands per read for high throughput
- **persistence** — append-only file (AOF) and point-in-time snapshots
- **pipelining** — multiple commands per read for high throughput

## quickstart

```bash
# build
cargo build --release

# run the server (defaults to 127.0.0.1:6379, no memory limit)
# run the server (defaults to 127.0.0.1:6379)
./target/release/ember-server

# or with a memory limit and eviction
# with memory limit and eviction
./target/release/ember-server --max-memory 256M --eviction-policy allkeys-lru

# with persistence
./target/release/ember-server --data-dir ./data --appendonly
```

```bash
# connect with redis-cli
redis-cli SET hello world # => OK
redis-cli GET hello # => "world"
redis-cli SET hello world # => OK
redis-cli GET hello # => "world"
redis-cli MSET a 1 b 2 c 3 # => OK
redis-cli MGET a b c # => 1) "1" 2) "2" 3) "3"

# expiration
redis-cli SET temp data EX 60
redis-cli TTL temp # => 59
redis-cli DBSIZE # => (integer) 2
redis-cli TTL temp # => 59
redis-cli PTTL temp # => 59000
redis-cli PERSIST temp # => (integer) 1

# counters
redis-cli SET counter 10
redis-cli INCR counter # => (integer) 11
redis-cli DECR counter # => (integer) 10

# lists
redis-cli LPUSH mylist a b c # => (integer) 3
redis-cli LRANGE mylist 0 -1 # => 1) "c" 2) "b" 3) "a"
redis-cli RPOP mylist # => "a"
redis-cli LPUSH mylist a b c # => (integer) 3
redis-cli LRANGE mylist 0 -1 # => 1) "c" 2) "b" 3) "a"

# sorted sets
redis-cli ZADD board 100 alice 200 bob 150 charlie
redis-cli ZADD board 100 alice 200 bob
redis-cli ZRANGE board 0 -1 WITHSCORES
redis-cli ZRANK board alice # => (integer) 0
redis-cli ZSCORE board bob # => "200"
redis-cli ZCARD board # => (integer) 2

# iteration
redis-cli SCAN 0 MATCH "user:*" COUNT 100
redis-cli DBSIZE # => (integer) 6
redis-cli FLUSHDB # => OK
```

## configuration

| flag | default | description |
|------|---------|-------------|
| `--host` | 127.0.0.1 | address to bind to |
| `--port` | 6379 | port to listen on |
| `--max-memory` | unlimited | memory limit (e.g., 256M, 1G) |
| `--eviction-policy` | noeviction | `noeviction` or `allkeys-lru` |
| `--data-dir` | — | directory for persistence files |
| `--appendonly` | false | enable append-only file logging |
| `--appendfsync` | everysec | fsync policy: `always`, `everysec`, `no` |

## build & development

```bash
make check # fmt, clippy, tests
make build # debug build
make release # release build
make test # run all tests
make check # fmt, clippy, tests
make build # debug build
make release # release build
make test # run all tests
make docker-build # build docker image
```

see [CONTRIBUTING.md](CONTRIBUTING.md) for development workflow and code standards.

## project structure

```
Expand All @@ -66,20 +102,29 @@ crates/
ember-core/ core engine (keyspace, types, sharding)
ember-protocol/ RESP3 wire protocol
ember-persistence/ AOF and snapshot durability
ember-cluster/ raft, gossip, slot management
ember-cluster/ raft, gossip, slot management (wip)
ember-cli/ interactive CLI tool
```

## architecture

ember uses a shared-nothing, thread-per-core design inspired by [Dragonfly](https://github.com/dragonflydb/dragonfly). each core owns a partition of the keyspace with no cross-thread synchronization on the hot path.
ember uses a shared-nothing, thread-per-core design inspired by [Dragonfly](https://github.com/dragonflydb/dragonfly). each cpu core owns a partition of the keyspace with no cross-thread synchronization on the hot path.

| target | redis baseline | ember goal |
|--------|---------------|------------|
| metric | redis baseline | ember target |
|--------|---------------|--------------|
| throughput | ~100k ops/sec/core | 500k+ ops/sec/core |
| p99 latency | ~1ms | <200µs |
| memory/key | ~90 bytes overhead | <40 bytes |

## security

see [SECURITY.md](SECURITY.md) for:
- reporting vulnerabilities
- security considerations for deployment
- recommended configuration

**note**: ember does not currently support authentication. always run behind a firewall or in a trusted network.

## license

MIT
57 changes: 57 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# security policy

## supported versions

| version | supported |
|---------|-----------|
| 0.1.x | yes |

## reporting a vulnerability

if you discover a security vulnerability in ember, please report it responsibly:

1. **do not** open a public issue
2. email the maintainers directly at kacy@kacyfortner.com (or open a private security advisory on github)
3. include:
- a description of the vulnerability
- steps to reproduce
- potential impact
- any suggested fixes (optional)

we aim to respond within 48 hours and will work with you to understand and address the issue.

## security considerations

### network exposure

ember binds to `127.0.0.1` by default. if you expose it to a network:

- use a firewall to restrict access to trusted clients
- consider running behind a reverse proxy with TLS termination
- ember does not currently support authentication (planned for future releases)

### memory limits

always configure `--max-memory` in production to prevent unbounded memory growth:

```bash
ember-server --max-memory 1G --eviction-policy allkeys-lru
```

### untrusted input

- command parsing is defensive and rejects malformed input
- buffer sizes are capped to prevent memory exhaustion attacks
- pattern matching (SCAN MATCH) uses an iterative algorithm to avoid regex DOS

### persistence

if using AOF or snapshots:

- ensure the data directory has appropriate permissions
- aof files contain all write commands in binary format
- snapshots contain the full keyspace state

## security updates

security fixes are released as patch versions. we recommend staying up to date with the latest release.