diff --git a/README.md b/README.md index be0452f..2c23364 100644 --- a/README.md +++ b/README.md @@ -15,37 +15,109 @@ A real-time multiplayer platformer game built with HTML5 Canvas, Socket.IO, and - **Spatial grid collision** โ€” O(1) platform lookup for server-side physics - **Particle system** โ€” pooled, zero-allocation particle effects on the client - **Configurable tick rate** โ€” server tick rate is tunable via environment variable +- **Horizontally scalable** โ€” game-server pods self-register with the server-manager; the gateway routes each room to the least-loaded server --- -## ๐Ÿ—‚๏ธ Architecture +## ๐Ÿ›๏ธ Architecture + +### Old Architecture + +All clients connected directly to a single monolithic game server. Simple, but impossible to scale. + +![Old Architecture](./archRef/ArchOld.png) + +### New Architecture + +A layered system with a **Gateway**, a **Server Manager (orchestrator)**, and a fleet of **game-server pods**. + +![New Architecture](./archRef/ArchNew.png) + +#### How a room is created + +1. **Client โ†’ Gateway** (`GET /createRoom`): The client asks the gateway to allocate a room. +2. **Gateway โ†’ Server Manager** (`GET /assign`): The gateway asks the orchestrator for the least-loaded game-server. The server-manager maintains a min-heap sorted by active connections and returns the best server's `hostIp` and internal port. +3. **Gateway โ†’ Client**: Returns `{ hostIp, port, roomID }`. The gateway stores the room-code โ†’ server mapping in memory (in-process map; Redis migration is a TODO). +4. **Client โ†’ Game Server** (WebSocket `wss://server.boxgame.shadyggs.xyz:`): The client opens a Socket.IO connection directly to the assigned game-server pod. + +> The client connects **directly** to the game-server pod over WebSocket โ€” it never proxies game traffic through the gateway. The gateway is only involved during matchmaking. + +#### TLS & port mapping (production) + +Game-server pods use `hostNetwork` and listen on **internal** ports **30000โ€“31000** on the VM. The browser requires `wss://` (mixed-content rules). A **host-level Nginx stream** block terminates TLS on **external** ports **20000โ€“21000** and forwards plaintext to the matching internal port (`external + 10 000 = internal`). + +``` +Browser โ†’ wss://server.boxgame.shadyggs.xyz:20426 (TLS) + โ†’ Nginx stream :20426 + โ†’ 127.0.0.1:30426 (game-server pod, plaintext) +``` + +Port blocks are generated by `scripts/gen-stream-ports.sh` and included by `nginx-gameserver-stream.conf`. + +--- + +## ๐Ÿ—‚๏ธ Repository Structure ``` BoxGame/ -โ”œโ”€โ”€ client/ # Static frontend (HTML + Canvas + Socket.IO client) -โ”‚ โ”œโ”€โ”€ index.html -โ”‚ โ”œโ”€โ”€ index.js # Lobby, socket connection, HUD logic -โ”‚ โ”œโ”€โ”€ game.js # Canvas rendering, particles, input handling -โ”‚ โ””โ”€โ”€ styles.css โ”‚ -โ”œโ”€โ”€ server/ # Game server (Node.js + Express + Socket.IO) -โ”‚ โ”œโ”€โ”€ server.js # Physics loop, room management, socket events +โ”œโ”€โ”€ client/ # Static frontend (HTML5 Canvas + Socket.IO) +โ”‚ โ”œโ”€โ”€ index.html # Entry point / lobby UI +โ”‚ โ”œโ”€โ”€ index.js # Lobby logic, room creation/joining, HUD +โ”‚ โ”œโ”€โ”€ game.js # Canvas renderer, particle system, input handling +โ”‚ โ”œโ”€โ”€ styles.css # Lobby & game styles +โ”‚ โ”œโ”€โ”€ icon.png # App icon / banner +โ”‚ โ””โ”€โ”€ heart.png # UI asset (lives / branding) +โ”‚ +โ”œโ”€โ”€ server/ # Game server (Node.js + Socket.IO) +โ”‚ โ”œโ”€โ”€ server.ts # Physics loop, room management, socket events, +โ”‚ โ”‚ # self-registers with server-manager on startup +โ”‚ โ”œโ”€โ”€ config.yaml # Example k8s Pod spec for a game-server pod +โ”‚ โ”œโ”€โ”€ package.json +โ”‚ โ””โ”€โ”€ Dockerfile +โ”‚ +โ”œโ”€โ”€ gateway/ # Matchmaking / room-allocation service (Bun + TypeScript) +โ”‚ โ”œโ”€โ”€ index.ts # Express app; exposes /createRoom and /add +โ”‚ โ”œโ”€โ”€ routes/ +โ”‚ โ”‚ โ”œโ”€โ”€ createRoom.ts # Allocates a room: queries server-manager, maps codeโ†’server +โ”‚ โ”‚ โ”œโ”€โ”€ addToRoom.ts # Adds a client to an existing room +โ”‚ โ”‚ โ””โ”€โ”€ removeFromRoom.ts # Removes a client from a room +โ”‚ โ”œโ”€โ”€ helper/ +โ”‚ โ”‚ โ””โ”€โ”€ generateRoom.ts # Generates random room codes +โ”‚ โ”œโ”€โ”€ types/ +โ”‚ โ”‚ โ””โ”€โ”€ server.ts # Shared Server type +โ”‚ โ”œโ”€โ”€ package.json +โ”‚ โ”œโ”€โ”€ tsconfig.json โ”‚ โ””โ”€โ”€ Dockerfile โ”‚ -โ””โ”€โ”€ gateway/ # Optional auth/routing gateway (Bun + TypeScript + Redis) - โ”œโ”€โ”€ index.ts - โ””โ”€โ”€ routes/ - โ”œโ”€โ”€ addToRoom.ts - โ””โ”€โ”€ removeFromRoom.ts +โ”œโ”€โ”€ server-manager/ # Orchestrator / load-balancer (Bun + TypeScript) +โ”‚ โ”œโ”€โ”€ index.ts # Express app; exposes /assign and /register +โ”‚ โ”œโ”€โ”€ routes/ +โ”‚ โ”‚ โ”œโ”€โ”€ assignServer.ts # Pops best server from heap, increments connections, returns it +โ”‚ โ”‚ โ””โ”€โ”€ handleRegister.ts # Game-server pods POST here on startup to join the pool +โ”‚ โ”œโ”€โ”€ helper/ +โ”‚ โ”‚ โ””โ”€โ”€ heap.ts # Min-heap sorted by active connections (O(log n) assign) +โ”‚ โ”œโ”€โ”€ types/ +โ”‚ โ”‚ โ””โ”€โ”€ server.ts # Shared Server type { hostIp, port, connections } +โ”‚ โ”œโ”€โ”€ package.json +โ”‚ โ”œโ”€โ”€ tsconfig.json +โ”‚ โ””โ”€โ”€ Dockerfile +โ”‚ +โ”œโ”€โ”€ scripts/ +โ”‚ โ””โ”€โ”€ gen-stream-ports.sh # Generates Nginx stream server blocks for ports 20000โ€“21000 +โ”‚ +โ”œโ”€โ”€ archRef/ +โ”‚ โ”œโ”€โ”€ ArchOld.png # Monolithic architecture diagram +โ”‚ โ””โ”€โ”€ ArchNew.png # Distributed architecture diagram +โ”‚ +โ”œโ”€โ”€ nginx-gameserver-stream.conf # Host-level Nginx stream config (TLS termination) +โ”œโ”€โ”€ k8s.yaml # Production Kubernetes manifests (all services) +โ”œโ”€โ”€ k8s.local.yaml # Local/dev Kubernetes manifests +โ”œโ”€โ”€ .env # Root-level env vars +โ”œโ”€โ”€ CONTRIBUTING.md +โ””โ”€โ”€ package-lock.json ``` -### How it works - -1. A player opens the client, enters a name, and **creates or joins** a room via the gateway/server. -2. The game server runs a fixed-interval **physics loop** at the configured tick rate (default 120 Hz). -3. Each tick the server broadcasts a **delta-state** diff to clients; a full sync is sent every ~2 seconds. -4. The client renders at `requestAnimationFrame` speed, interpolating the received state. - --- ## ๐Ÿš€ Getting Started @@ -56,9 +128,9 @@ BoxGame/ |------|---------| | Node.js | โ‰ฅ 18 | | npm | โ‰ฅ 9 | -| Bun *(gateway only)* | โ‰ฅ 1.1 | -| Redis *(gateway only)* | โ‰ฅ 7 | +| Bun *(gateway & server-manager)* | โ‰ฅ 1.1 | | Docker *(optional)* | โ‰ฅ 24 | +| kubectl + k8s cluster *(production)* | โ€” | ### 1. Clone the repository @@ -67,24 +139,62 @@ git clone https://github.com/your-username/BoxGame.git cd BoxGame ``` -### 2. Start the game server +### 2. Start the server-manager + +The server-manager must be running before game servers start, because each server registers itself on startup. + +```bash +cd server-manager +bun install +bun run index.ts +# Listens on http://localhost:4689 +``` + +**Environment variables (`server-manager/.env`):** + +| Variable | Default | Description | +|----------|---------|-------------| +| `PORT` | `4689` | HTTP port | + +### 3. Start one or more game servers + +Each server picks a random port, starts the Socket.IO game loop, and self-registers with the server-manager. ```bash cd server npm install -node server.js +# Point it at your server-manager +SERVER_MANAGER_URL=http://localhost:4689 npx ts-node server.ts ``` -The server listens on `http://localhost:3000` by default. - **Environment variables (`server/.env`):** | Variable | Default | Description | |----------|---------|-------------| -| `PORT` | `3000` | HTTP port | +| `SERVER_MANAGER_URL` | โ€” | URL of the server-manager | +| `HOST_IP` | โ€” | IP reported to server-manager (clients use this to connect) | +| `PORT_MIN` | `0` | Lower bound for random port selection | +| `PORT_MAX` | `0` | Upper bound (`0` = OS-assigned) | | `TICK` | `120` | Game tick rate (Hz) | -### 3. Serve the client +### 4. Start the gateway + +```bash +cd gateway +bun install +SERVER_MANAGER_URL=http://localhost:4689 bun run index.ts +# Listens on http://localhost:3000 +``` + +**Environment variables (`gateway/.env`):** + +| Variable | Default | Description | +|----------|---------|-------------| +| `PORT` | `3000` | HTTP port | +| `SERVER_MANAGER_URL` | โ€” | URL of the server-manager | +| `SERVER_HOST_DOMAIN` | `localhost` | Public hostname clients use to reach game servers (production: `server.boxgame.shadyggs.xyz`) | + +### 5. Serve the client The `client/` directory is plain static HTML โ€” serve it with any static file server: @@ -98,31 +208,48 @@ python3 -m http.server 8080 --directory client/ Then open `http://localhost:8080` in your browser. -> **Important:** Update the Socket.IO server URL in `client/index.js` to point to your running game server. +> **Important:** Update the gateway URL in `client/index.js` to point to your running gateway. -### 4. (Optional) Start the gateway - -The gateway provides a Redis-backed routing layer for assigning players to rooms. +### 6. Docker (individual services) ```bash -cd gateway -bun install +# Game server +cd server && docker build -t boxgame-server . && docker run -p 30000:30000 boxgame-server -# Set environment variables -export REDIS_URL=redis://localhost:6379 -export PORT=4000 +# Gateway +cd gateway && docker build -t boxgame-gateway . && docker run -p 3000:3000 boxgame-gateway -bun run index.ts +# Server-manager +cd server-manager && docker build -t boxgame-server-manager . && docker run -p 4689:4689 boxgame-server-manager ``` -### 5. Docker (server only) +--- + +## โ˜ธ๏ธ Kubernetes Deployment (Production) + +All manifests are in `k8s.yaml`. It provisions two namespaces: + +| Namespace | Services | +|-----------|----------| +| `box-game` | `gateway`, `server-manager` | +| `box-game-servers` | dynamically created `game-server` pods | ```bash -cd server -docker build -t boxgame-server . -docker run -p 6000:6000 boxgame-server +kubectl apply -f k8s.yaml ``` +Key design decisions: +- **Game-server pods use `hostNetwork: true`** so they bind directly on the VM's network interface and are reachable by the Nginx stream proxy on the host. +- **RBAC**: the `server-manager-sa` ServiceAccount is granted pod create/delete rights in `box-game-servers` so the server-manager can spawn new game-server pods dynamically. +- **Nginx stream** on the host VM (not inside k8s) terminates TLS for `server.boxgame.shadyggs.xyz` on external ports 20000โ€“21000 and forwards to internal pod ports 30000โ€“31000. Generate port blocks with: + + ```bash + sudo bash scripts/gen-stream-ports.sh + sudo nginx -t && sudo systemctl reload nginx + ``` + +- **Gateway Ingress** (`gateway.boxgame.shadyggs.xyz`) is handled by the nginx ingress controller + cert-manager (Let's Encrypt). + --- ## ๐ŸŽฎ Controls @@ -137,7 +264,7 @@ docker run -p 6000:6000 boxgame-server ## ๐Ÿ”ง Configuration -### Game constants (`server/server.js`) +### Game constants (`server/server.ts`) | Constant | Default | Description | |----------|---------|-------------| @@ -145,6 +272,11 @@ docker run -p 6000:6000 boxgame-server | `jumpPower` | `-15` | Vertical velocity applied on jump | | `PLAYER_SPEED` | `3` | Horizontal movement speed | | `MAX_PLAYERS_PER_ROOM` | `20` | Hard cap per room | +| `tick` | `120` | Physics + broadcast rate (Hz) | + +### Server-manager internals + +The server-manager uses an in-memory **min-heap** keyed on active connection count. Each `POST /register` from a game-server pod inserts into the heap in O(log n). Each `GET /assign` pops the minimum (least-loaded server), increments its connection count, re-inserts it, and returns the server's `hostIp` / `port` in response headers. --- diff --git a/archRef/ArchNew.png b/archRef/ArchNew.png new file mode 100644 index 0000000..239796e Binary files /dev/null and b/archRef/ArchNew.png differ diff --git a/archRef/ArchOld.png b/archRef/ArchOld.png new file mode 100644 index 0000000..673a094 Binary files /dev/null and b/archRef/ArchOld.png differ