A/B Testing Platform MVP

A lightweight, production-ready A/B testing platform with REST API.

Features

• Fast REST API built with FastAPI
• Statistical analysis (t-tests, z-tests, confidence intervals)
• PostgreSQL database
• API key authentication
• Docker & Docker Compose ready
• Experiment lifecycle management
• Traffic allocation & variant assignment

Quick Start

Using Docker (Recommended)

# Start all services
docker-compose up -d

# API available at http://localhost:8000
# Docs at http://localhost:8000/docs

Manual Setup

# Install dependencies
pip install -r requirements.txt

# Set up database
python scripts/init_db.py

# Run API server
uvicorn app.main:app --reload

API Usage

Create an Experiment

curl -X POST http://localhost:8000/api/v1/experiments \
 -H "X-API-Key: your-api-key" \
 -H "Content-Type: application/json" \
 -d '{
 "name": "Button Color Test",
 "description": "Testing red vs blue button",
 "variants": [
 {"name": "control", "traffic_percentage": 50},
 {"name": "variant_a", "traffic_percentage": 50}
 ],
 "metrics": [{
 "name": "conversion_rate",
 "type": "proportion"
 }]
 }'

Get Experiment Results

curl http://localhost:8000/api/v1/experiments/{id}/results \
 -H "X-API-Key: your-api-key"

Track Events

curl -X POST http://localhost:8000/api/v1/events \
 -H "X-API-Key: your-api-key" \
 -H "Content-Type: application/json" \
 -d '{
 "experiment_id": "exp_123",
 "user_id": "user_456",
 "variant": "variant_a",
 "metric": "conversion_rate",
 "value": 1
 }'

Project Structure

.
├── app/
│ ├── main.py # FastAPI application
│ ├── models.py # Database models
│ ├── schemas.py # Pydantic schemas
│ ├── api/
│ │ ├── experiments.py # Experiment endpoints
│ │ ├── events.py # Event tracking endpoints
│ │ └── auth.py # Authentication
│ ├── core/
│ │ ├── config.py # Configuration
│ │ ├── database.py # Database connection
│ │ └── security.py # Security utilities
│ └── services/
│ ├── experiment_service.py
│ ├── stats_service.py # Statistical analysis
│ └── assignment_service.py
├── tests/
├── scripts/
│ └── init_db.py
├── docker-compose.yml
├── Dockerfile
└── requirements.txt

Environment Variables

Create a .env file:

DATABASE_URL=postgresql://user:password@localhost:5432/abtest
SECRET_KEY=your-secret-key
API_KEYS=key1,key2,key3

Development

# Run tests
pytest

# Format code
black app/

# Lint
ruff check app/

Documentation

• API Docs: http://localhost:8000/docs
• Architecture: docs/ARCHITECTURE.md
• API Reference: docs/API_REFERENCE.md

License

MIT