GORedis

A Redis-compatible, in-memory, multi-data-type database server with Pub/Sub, written from scratch in Go.

This project is a deep dive into building a high-performance, concurrent network server that speaks the Redis Serialization Protocol (RESP). It is a learning exercise focused on Go's core concepts, including TCP networking, concurrency with goroutines and channels, thread-safe data structures with mutexes, and system design for persistence.

Features

This server implements a significant subset of Redis's core functionality, providing a durable, multi-data-type storage engine and a real-time messaging system.

Core

• Concurrent TCP Server: Capable of handling multiple client connections simultaneously with a graceful shutdown mechanism.
• RESP Parser/Writer: Full implementation for parsing client commands and serializing server responses.
• Standalone Server & Client: The server and interactive client are separate applications, allowing multiple clients to connect from different terminals.

Data Types & Commands

• Strings: SET, GET, DEL, EXISTS, INCR, DECR
• Lists: LPUSH, RPUSH, LPOP, RPOP, LLEN
• Hashes: HSET, HGET, HGETALL
• Sets: SADD, SISMEMBER, SMEMBERS, SREM
• Transactions: MULTI, DISCARD, EXEC

Pub/Sub

• Real-time Messaging: A complete Publish/Subscribe system for broadcasting messages to clients.
• SUBSCRIBE channel [channel ...]
• PUBLISH channel message
• UNSUBSCRIBE [channel ...]

Key Management & Expiration

• EXPIRE key seconds: Set a time-to-live on a key.
• TTL key: Inspect the remaining time-to-live of a key.
• Lazy & Active Expiration: Expired keys are evicted both on access and by a background janitor process.

Persistence

• Asynchronous Append-Only File (AOF): All write commands are sent to a channel and persisted by a dedicated goroutine, preventing disk I/O from blocking client responses.
• AOF Loading: On startup, the server automatically checks for and reads the AOF file to restore its state.
• AOF Compaction: Supports the BGREWRITEAOF command to rewrite the AOF file into a minimal format in the background.

Getting Started

To get the server and client running locally, follow these simple steps.

Prerequisites

• Go (version 1.18 or newer)
• redis-tools (for benchmarking)

Installation & Usage

1. Clone the repository:

   git clone https://github.com/Souvik606/Go-redis.git
   cd Go-redis

2. Tidy dependencies:

   go mod tidy

3. Run the Server: Open a terminal and run the server application. For best performance, it's recommended to run this from within a Linux environment like WSL.

   go run ./cmd/goredis/server/main.go

   You will see a log message confirming the server has started on port 6379. Keep this terminal open.

4. Run the Interactive Client: Open a new, separate terminal and run the client application.

   go run ./cmd/goredis/client/main.go

   You will get an interactive prompt, ready to accept Redis commands.

Testing Pub/Sub

To see the Pub/Sub feature in action, you need at least two clients.

1. Subscriber Client (Terminal A): SUBSCRIBE news
2. Publisher Client (Terminal B): PUBLISH news "hello world"
3. Observe the message appear automatically in Terminal A.

Testing Transactions

You can test atomic transactions using the MULTI, EXEC, and DISCARD commands in any client terminal.

1. Start with MULTI.
2. Queue commands like INCR or SET. The server will reply QUEUED.
3. Run EXEC to perform all actions at once.

Benchmarking

The server's performance was benchmarked against the official Redis server (v7.0.15) on the same machine.

How to Run the Benchmark

1. Start the GORedis Server: For the best performance, run the server from within a native Linux environment (like WSL), ensuring the project files are also inside the Linux filesystem (e.g., in ~/go-redis).

   # In WSL Terminal 1
   go run ./cmd/goredis/server/main.go

2. Run redis-benchmark: Open a second WSL terminal and run the benchmark utility. The following command simulates 50 concurrent clients sending a total of 100,000 SET and GET requests.

   # In WSL Terminal 2
   redis-benchmark -p 6379 -t SET,GET -n 100000 -c 50

Performance Results

The following table shows the averaged results over 5 separate benchmark runs each containing 100000 set and get commands. The "Performance %" column shows GORedis's performance relative to the official Redis (where higher is better for all metrics).

Command	Server	Throughput (ops/sec)	Avg Latency (ms)	p99 Latency (ms)	Max Latency (ms)
SET	GORedis	37,116.71	0.834	2.201	5.087
	Official Redis	66,711.14	0.386	0.703	3.199
	Performance %	55.64%	46.28%	31.94%	62.89%
GET	GORedis	37,249.75	0.811	1.959	4.594
	Official Redis	66,313.00	0.391	0.719	9.007
	Performance %	56.17%	48.21%	36.70%	196.06%

Analysis

As the results show, GORedis achieves a very respectable ~55-60% of the throughput of the official Redis server for simple SET and GET operations.

The primary difference in performance is due to:

• Go vs. C: Redis is written in highly-optimized C with manual memory management.
• Runtime Overhead: Go's garbage collector and goroutine scheduler introduce small, non-deterministic pauses, which are most visible in the higher p99 latency figures.
• Concurrency Model: GORedis uses a mutex to protect shared data, while Redis's single-threaded command execution model avoids locking overhead entirely.