API Load Balancer

A production-style monorepo showcasing a programmable API Load Balancer with weighted round-robin, least-connections, health checks, circuit breakers, retries, rate limiting, per-key quotas, canary/blue‑green controls, config hot‑reload, and a real dashboard.

Stack

• Edge: NGINX (reverse proxy) -> gateway/ (FastAPI-based LB & API gateway)
• Control plane: control/ (FastAPI admin API, config storage, WebSocket metrics)
• Dashboard: dashboard/ (React + Vite + Tailwind + shadcn/ui + Recharts)
• Sample backends: services/service_py/ (FastAPI), services/service_node/ (Express)
• Data: Redis (rate limiting & ephemeral state), SQLite (config) – Postgres ready via SQLAlchemy URL
• Observability: Prometheus metrics, structured logs, request tracing, basic Grafana dashboard JSON
• Orchestration: Docker Compose

Features

• Auth: JWT for admin endpoints; API keys for client traffic
• Algorithms: weighted round-robin, least-connections; runtime switch via admin API
• Health checks: active (HTTP pings) + passive (error rates), out-of-rotation, auto-rejoin
• Circuit breaker: half-open recovery, rolling window
• Rate limiting: token bucket per API key via Redis; burst + sustained limits
• Retries: exponential backoff + jitter, per-route policy
• Canary/Blue‑Green: weighted traffic splits per service version, runtime adjustable
• Config: hot-reload, persisted to SQLite by default
• Tracing: trace-id propagation (Zipkin-style headers), per-hop logs
• Metrics: Prometheus, WebSocket live feed to dashboard
• CI: GitHub Actions workflow (lint/test/build), pre-commit hooks
• Seeds: demo API keys, services, and rollout config

Quick start

Requirements: Docker + Docker Compose

cp .env.example .env
docker compose up --build

• Dashboard: http://localhost:5173
• Gateway (client traffic): http://localhost:8080
• Admin API: http://localhost:9000/docs
• Prometheus metrics (gateway): http://localhost:8080/metrics

Login (dashboard): use the seeded admin user from .env.example

Structure

api-load-balancer/
  control/             # Admin API & metrics hub
  dashboard/           # React dashboard
  gateway/             # FastAPI-based L7 gateway (balancer + auth + rate limit)
  services/
    service_py/        # FastAPI demo service
    service_node/      # Express demo service
  ops/
    nginx/             # NGINX reverse-proxy config
    grafana/           # Sample dashboard JSON
  .env.example
  docker-compose.yml
  Makefile

Notes

• This is intentionally verbose to mirror a realistic team repo. The defaults run entirely via docker compose.
• Swap SQLite with Postgres by setting DATABASE_URL in .env (control & gateway use SQLAlchemy URLs).
• The gateway proxies /api/* to registered services according to config and rollout weights.
• Use the Admin API to toggle algorithms, weights, and canary ratios at runtime.
• The Dashboard uses WebSocket to stream live backend health and traffic stats.
• See ./docs/ for API contracts and rollout playbooks.

License

MIT