Express + Prometheus + Grafana Monitoring System

A Node.js Express application with integrated monitoring using Prometheus for metrics collection and Grafana for visualization.

Architecture Overview

System Flow: Express → Prometheus → Grafana

┌─────────────┐      ┌──────────────┐      ┌─────────────┐
│   Express   │─────▶│  Prometheus  │─────▶│   Grafana   │
│   App       │      │  (Scraper)   │      │ (Dashboard) │
│  Port 3000  │      │  Port 9090   │      │  Port 3001  │
└─────────────┘      └──────────────┘      └─────────────┘
     │                       │                      │
     │                       │                      │
     └───────────────────────┴──────────────────────┘
                    Metrics Flow

Detailed Flow Description

1. Express Application (Metrics Collection)

• The Express app uses prom-client library to collect and expose metrics
• Metrics Collection:
  • HTTP request duration (latency) is tracked using a Histogram metric
  • Default Node.js metrics (CPU, memory, event loop, etc.) are automatically collected
  • Metrics are labeled by HTTP method, route, and status code
• Metrics Endpoint: The app exposes metrics at /metrics endpoint in Prometheus format
• Middleware: A custom middleware tracks every HTTP request and records its duration

2. Prometheus (Metrics Scraping & Storage)

• Prometheus acts as a time-series database and metrics collector
• Scraping: Prometheus periodically scrapes the /metrics endpoint from the Express app
• Configuration:
  • Scrape interval: Every 5 seconds (configured in prometheus.yml)
  • Target: app:3000/metrics
• Storage: Prometheus stores all collected metrics in its time-series database
• Query Interface: Prometheus provides a query language (PromQL) to query metrics

3. Grafana (Visualization & Dashboards)

• Grafana connects to Prometheus as a data source
• Data Source: Configured to pull metrics from Prometheus
• Dashboards: Creates visual dashboards with graphs, charts, and alerts
• Real-time Monitoring: Displays real-time metrics and historical trends
• Visualization: Converts raw metrics data into meaningful visual representations

Data Flow Sequence

1. User Request → Express app receives HTTP request
2. Middleware Tracking → Request duration timer starts
3. Response → Express sends response, timer stops and records metric
4. Metrics Exposure → Metrics available at http://app:3000/metrics
5. Prometheus Scrape → Prometheus fetches metrics every 5 seconds
6. Storage → Prometheus stores metrics in time-series database
7. Grafana Query → Grafana queries Prometheus for metrics data
8. Visualization → Grafana displays metrics in dashboards

Services

• Express App (app): Node.js application on port 3000
• Redis (redis): Redis server on port 6379
• Prometheus (prometheus): Metrics server on port 9090
• Grafana (grafana): Dashboard on port 3001

Getting Started

Prerequisites

• Docker and Docker Compose installed

Running the Application

1. Start all services:

   docker-compose up -d

2. Access the services:

   • Express App: http://localhost:3000
   • Prometheus: http://localhost:9090
   • Grafana: http://localhost:3001 (default login: admin/admin)
   • Redis: localhost:6379

3. View metrics:

   • Express metrics endpoint: http://localhost:3000/metrics
   • Prometheus query interface: http://localhost:9090

Setting up Grafana

1. Login to Grafana at http://localhost:3001
2. Add Prometheus as a data source:
   • URL: http://prometheus:9090
   • Access: Server (default)
3. Create dashboards to visualize your metrics

Metrics Available

• http_request_duration_seconds: HTTP request latency histogram
• Default Node.js metrics (CPU, memory, event loop, etc.)

Project Structure

.
├── app/
│   ├── db/
│   │   └── init.redis.js    # Redis client initialization
│   ├── metrics.js           # Prometheus metrics configuration
│   ├── server.js            # Express application
│   ├── Dockerfile           # App container definition
│   └── package.json         # Node.js dependencies
├── docker-compose.yml       # Service orchestration
├── prometheus.yml           # Prometheus configuration
└── README.md               # This file

Configuration

Prometheus Configuration (prometheus.yml)

• Scrapes Express app metrics every 5 seconds
• Target endpoint: app:3000/metrics

Environment Variables

• REDIS_URL: Redis connection URL (default: redis://redis:6379)

Stopping Services

docker-compose down

To remove volumes (including Redis data):

docker-compose down -v