Skip to content

README.md

Latest commit

 

History

History
463 lines (345 loc) · 9.89 KB

README.md

File metadata and controls

463 lines (345 loc) · 9.89 KB
Raw

UpCloud Infrastructure

Scalable cloud infrastructure with Pulumi (Python), supporting frontend and backend deployments on UpCloud.

⚠️ Note: Infrastructure code only - no actual deployment yet. GitHub Actions workflows are disabled by default.

🏗️ Architecture

Internet
   │
   ▼
[Load Balancer]
   │
   ├─────────┬─────────┐
   ▼         ▼         ▼
[API-1]  [API-2]  [API-N]  (Private Network)
   │         │         │
   └────┬────┴────┬────┘
        │         │
        ▼         ▼
[Managed PostgreSQL]  [Object Storage]

🎯 Components

  • Load Balancer: HTTP traffic distribution with health checks
  • Cloud Servers: 2+ API application servers (scalable)
  • Managed PostgreSQL: Fully managed database with automatic backups
  • Object Storage: S3-compatible storage for images/files
  • Private Network: Secure internal communication (10.0.0.0/24)
  • Firewall: Security rules for network isolation

📋 Prerequisites

1. UpCloud Account & API Credentials

  1. Sign up at upcloud.com
  2. Create API credentials:
    • Go to UpCloud Control Panel
    • Navigate to: Account → API Access
    • Click "Create API User" or use your account email
    • Generate an API password (not your account password!)
    • Save username and password securely

2. Install Pulumi

macOS:

brew install pulumi

Windows:

choco install pulumi

Linux:

curl -fsSL https://get.pulumi.com | sh

Verify installation:

pulumi version

3. Install Python

Python 3.8+ required:

python3 --version

🚀 Quick Start

1. Clone & Setup

cd upcloud
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

2. Login to Pulumi

Choose one option:

Option A: Pulumi Cloud (Free)

pulumi login

Option B: Local State

pulumi login --local

3. Create Stack

# Initialize stack (typically use 'dev' or 'prod')
pulumi stack init dev

4. Configure

Copy example configuration:

cp Pulumi.dev.yaml.example Pulumi.dev.yaml

Edit Pulumi.dev.yaml with your settings, or use CLI:

# Project settings
pulumi config set project_name myapp
pulumi config set environment production
pulumi config set zone de-fra1

# API servers
pulumi config set api_server_count 2
pulumi config set api_server_plan 2xCPU-4GB

# Database
pulumi config set database_plan 2x2xCPU-4GB
pulumi config set database_version 16
pulumi config set database_name appdb
pulumi config set database_username dbadmin

# UpCloud credentials (IMPORTANT: use --secret flag!)
pulumi config set upcloud:username your-upcloud-username
pulumi config set --secret upcloud:password your-upcloud-api-password

5. Preview & Deploy

Preview changes:

pulumi preview

Deploy infrastructure:

pulumi up

Type yes when prompted to confirm deployment.

6. Get Outputs

# View all outputs
pulumi stack output

# Get specific values
pulumi stack output load_balancer_endpoint
pulumi stack output database_host

# Get secrets (requires --show-secrets flag)
pulumi stack output database_password --show-secrets
pulumi stack output database_connection_string --show-secrets

⚙️ Configuration Options

Available Zones

  • de-fra1 - Frankfurt, Germany (default)
  • fi-hel1 - Helsinki, Finland
  • us-nyc1 - New York, USA
  • sg-sin1 - Singapore
  • nl-ams1 - Amsterdam, Netherlands

Server Plans (api_server_plan)

  • 1xCPU-2GB - Small (€15-20/month each)
  • 2xCPU-4GB - Medium (€30-35/month each) [Recommended]
  • 4xCPU-8GB - Large (€60-70/month each)
  • 8xCPU-16GB - X-Large (€120-140/month each)

Database Plans (database_plan)

  • 1x1xCPU-2GB - Small (€30-40/month)
  • 2x2xCPU-4GB - Medium (€50-70/month) [Recommended]
  • 4x4xCPU-8GB - Large (€100-140/month)

📊 Scaling

Scale API Servers

# Increase to 4 servers
pulumi config set api_server_count 4
pulumi up

Upgrade Database

# Upgrade to larger plan
pulumi config set database_plan 4x4xCPU-8GB
pulumi up

Upgrade Server Size

# Upgrade to more powerful servers
pulumi config set api_server_plan 4xCPU-8GB
pulumi up

🔐 Security

Credentials Management

Never commit secrets! Use Pulumi's secret management:

# Set secrets (encrypted in state)
pulumi config set --secret upcloud:password your-password
pulumi config set --secret api_key your-api-key

Firewall Rules

Built-in firewall rules:

  • ✅ Allow HTTP (8080) from load balancer
  • ✅ Allow SSH (22) for management
  • ✅ Allow outbound HTTP/HTTPS
  • ✅ Allow PostgreSQL (5432) to private network
  • ❌ Drop all other inbound traffic

Network Isolation

  • Database is on private network only (no public access)
  • API servers communicate internally via 10.0.0.0/24
  • Only load balancer exposed to internet

💾 Object Storage Setup

Object storage requires manual setup in UpCloud Control Panel:

# View instructions
pulumi stack output object_storage_instructions

Manual steps:

  1. Log into UpCloud Control Panel
  2. Go to Storage → Object Storage
  3. Create bucket (name from output: object_storage_bucket_name)
  4. Generate access keys
  5. Use S3-compatible client in your app

Example Python code:

import boto3

s3 = boto3.client('s3',
    endpoint_url='https://de-fra1.upcloudobjects.com',
    aws_access_key_id='YOUR_ACCESS_KEY',
    aws_secret_access_key='YOUR_SECRET_KEY'
)

# Upload file
s3.upload_file('local.jpg', 'myapp-production-storage', 'images/file.jpg')

# Public URL
url = 'https://de-fra1.upcloudobjects.com/myapp-production-storage/images/file.jpg'

🧪 Testing Infrastructure

# tests/test_infrastructure.py
import pulumi
import pytest

@pulumi.runtime.test
def test_api_server_count():
    # Add your infrastructure tests
    pass

Run tests:

pytest tests/

📈 Monitoring

View resource metrics in UpCloud Control Panel:

  • CPU usage
  • Memory usage
  • Network traffic
  • Database performance

💰 Cost Estimation

Starting setup (2 API servers):

  • 2x Cloud Servers (2xCPU-4GB): ~$60/month
  • Managed PostgreSQL (2x2xCPU-4GB): ~$60/month
  • Load Balancer: ~$15/month
  • Object Storage: Pay-as-you-go (~$0.01/GB)
  • Total: ~$135/month

Production setup (4 API servers):

  • 4x Cloud Servers (4xCPU-8GB): ~$280/month
  • Managed PostgreSQL (4x4xCPU-8GB): ~$140/month
  • Load Balancer: ~$15/month
  • Object Storage: ~$10-50/month
  • Total: ~$445-485/month

🔄 Update & Maintenance

Update infrastructure code

# Pull latest changes
git pull

# Update dependencies
pip install -r requirements.txt --upgrade

# Preview changes
pulumi preview

# Apply updates
pulumi up

View change history

pulumi stack history

Rollback

# View update history
pulumi stack history

# Rollback to specific version
pulumi stack select dev
pulumi refresh
# Manually revert code changes and run pulumi up

🗑️ Destroy Infrastructure

Warning: This will delete all resources including databases!

# Preview what will be destroyed
pulumi destroy --preview-only

# Destroy everything
pulumi destroy

🐛 Troubleshooting

API Authentication Failed

# Verify credentials
pulumi config get upcloud:username
pulumi config get upcloud:password --show-secrets

# Update if needed
pulumi config set upcloud:username your-username
pulumi config set --secret upcloud:password your-password

Database Creation Timeout

Database creation can take 15-30 minutes. If timeout occurs:

# Increase timeout or wait and refresh
pulumi refresh

State Conflicts

# Refresh state
pulumi refresh

# Cancel current update
pulumi cancel

View Detailed Logs

# Run with verbose logging
pulumi up --logtostderr -v=9

📚 Resources

🆘 Support

📝 Next Steps

  1. ✅ Deploy infrastructure: pulumi up
  2. ✅ Get database credentials: pulumi stack output database_connection_string --show-secrets
  3. ✅ Set up object storage (see instructions output)
  4. 🎨 Deploy your frontend: ./deploy-frontend.sh (see FRONTEND_DEPLOYMENT.md)
  5. 🔨 Deploy your API application to API servers
  6. 🌐 Configure DNS to point to load balancer and frontend
  7. 🔒 Set up SSL/TLS certificates
  8. 📊 Configure monitoring and alerts
  9. 🚀 Launch! � Frontend Deployment

This infrastructure supports hosting static frontends (SvelteKit, Next.js, Vite) via Object Storage:

Quick Deploy:

# Build your frontend
npm run build

# Deploy
./deploy-frontend.sh

Supported Frameworks:

  • SvelteKit (static adapter)
  • Next.js (static export)
  • Vite (React, Vue, Svelte)
  • Nuxt 3 (static generation)

See FRONTEND_DEPLOYMENT.md for complete guide including:

  • Framework-specific configuration
  • CI/CD setup (GitHub Actions)
  • Custom domain configuration
  • CDN integration
  • Performance optimization

�

🎉 Benefits of Pulumi vs Terraform

  • ✅ Real Python - Use Python features (loops, functions, classes)
  • ✅ Type Safety - IDE autocomplete and type checking
  • ✅ Testing - Standard pytest framework
  • ✅ Debugging - Use Python debugger
  • ✅ Packages - Import any Python package
  • ✅ Secrets - Built-in encrypted secret management
  • ✅ Better Outputs - Rich, formatted outputs
  • ✅ Component Resources - Create reusable components