Convo-AI-Studio

A production-grade, AI-powered real-time podcast platform engineered for scalability, multi-agent orchestration, seamless creator experience, and enterprise-grade reliability.

Index

Project Overview
Vision & Goals
Core Features
Functional Requirements
Non-Functional Requirements
System Architecture
High-Level Design
Event-Driven Architecture
Realtime Architecture
AI Orchestration Design
Database Design (PostgreSQL)
Monorepo Structure
Tech Stack Justification
Constraints & Assumptions
Security Considerations
Scalability Considerations
Trade-offs & Engineering Decisions
Development Workflow / SDLC
Local Setup Instructions
- Prerequisites
- Quick Start (5 minutes)
- Service URLs & Health Checks
- Useful Commands
- Development Workflow Example
Environment Variables
- Root .env.local
- Auth Service
- Podcast Service
- Realtime Service
- AI Engine
- Storage Service
- Transcoding Service
- Notification Service
- Telemetry Service
- API Gateway
Future Scope
- Immediate Priorities (6 months)
- Strategic Initiatives (12 months)
- Enterprise Features (18 months)
Engineering Learnings
Challenges Faced
Conclusion

Related docs: CurrentProgress.md · AGENTS.md

1. Project Overview

Convo-AI-Studio is an enterprise-ready, globally distributed podcast platform enabling creators to host live, multi-agent AI discussions on user-defined topics while audiences interact, react, and explore persistent, searchable transcripts in real time. Built on a service-oriented polyglot architecture, the platform scales horizontally to 10k+ concurrent participants with <150ms end-to-end latency, delivering the experience of a professional podcast network backed by intelligent AI moderators, real-time audience engagement, and enterprise-grade media management.

The system orchestrates nine independent microservices:

API Gateway – REST + GraphQL entry point with intelligent routing, rate limiting, and observability
Auth Service – JWT-based authentication, RBAC, session management
Podcast Service – Channel and podcast metadata, pgvector semantic search
AI Engine – Python FastAPI + LangChain, multi-model LLM orchestration via gRPC
Realtime Service – WebSocket/WebRTC, Redis Pub/Sub backbone for live streaming
Storage Service – S3-compatible object store orchestration (avatars, podcast artwork, clips)
Transcoding Service – Async FFmpeg-based media processing (audio normalization, transcription, clip generation)
Telemetry Service – OpenTelemetry collector aggregating traces, metrics, logs
Notification Service – Email, SMS, push notifications via BullMQ workers

These services communicate via clean contracts: REST/GraphQL for client-facing APIs, gRPC (HTTP/2) for high-performance internal RPC, BullMQ for reliable async task distribution, and Redis Pub/Sub for real-time broadcasts.

2. Vision & Goals

Strategic Vision: Build the infrastructure layer for the next generation of AI-powered media, where virtual personalities debate, discuss, and entertain alongside human creators—with production-grade reliability, enterprise security, and zero friction between ideation and broadcast to millions of listeners.

Core Goals:

Scalable Real-time AI Orchestration – Orchestrate hundreds of concurrent multi-agent discussions simultaneously, each streaming AI-generated turns to thousands of participants with <150ms latency, sub-100ms p95 API response times.
Event-Driven Extensibility – Plug in new AI characters, analytics workers, transcoding pipelines, storage backends, and notification channels without code changes or redeployment; support vendor changes (LLM providers, S3 buckets, transcoding profiles) via configuration.
Developer-Centric Platform – Well-typed APIs (REST + GraphQL + gRPC), enforced domain boundaries, clear service contracts, reproducible local development with Docker Compose + MinIO, comprehensive error handling and logging.
Robust Production Readiness – Observability via OpenTelemetry (traces, metrics, logs), circuit breakers for fault isolation, graceful degradation, comprehensive RBAC/audit trails, automatic failover, data encryption at rest & in transit.
Media-Grade Performance – Media file processing (transcoding, thumbnail generation, segmentation) handled asynchronously; podcast archives searchable via semantic similarity (pgvector); CDN-ready artifact delivery.

3. Core Features

Channel Management – Creators establish broadcast identities, manage subscribers, schedule multi-day series, customize branding (avatars, banners), and view analytics (reach, engagement, listener retention).
Live Podcast Engine – Real-time multi-agent AI conversation orchestration with turn-based speaking, audience reactions (emojis, upvotes), contextual Q&A routing, and automatic transcript generation.
AI Character Library – Define personas via modular personality specs, LLM provider overrides (per-character OpenAI vs. Anthropic), memory policies (short-term Redis, long-term pgvector embeddings), and RAG-enabled knowledge bases.
Real-time Transcripts & Replay – All spoken turns persisted to PostgreSQL + pgvector; searchable via semantic similarity (RAG-ready); clips auto-generated and stored in S3; full-text and semantic search on-demand.
Audience Participation – Live reactions (emojis, upvotes), moderated Q&A queues with AI routing, anonymous polling, and sentiment analysis—all broadcast via Redis Pub/Sub to WebSocket clients in real time.
Background Workers – BullMQ-powered post-podcast analytics (listener count, engagement metrics, sentiment), transcript summarization, media transcoding (MP3/AAC normalization), email digests, and clip generation.
Media Orchestration – S3-compatible storage for podcast artwork, creator avatars, generated clips; CloudFront CDN integration; automatic image resizing via transcoding service.
Enterprise Notifications – Email (SendGrid), SMS (Twilio), and push notifications (FCM) via pluggable Notification Service; template-driven, user-preference aware, with delivery tracking.
Observability & Telemetry – Distributed tracing via OpenTelemetry, Prometheus metrics (latency, throughput, error rates), structured logging (Pino, Python logging), real-time dashboards (Grafana), SLA monitoring.

4. Functional Requirements

ID	Description
FR-01	Creators can create, edit, and delete channels with customizable metadata (name, description, avatar, banner, color scheme).
FR-02	Creators schedule podcasts with start/end timestamps, AI character rosters, and optional topic context.
FR-03	System streams AI-generated text turns to all participants via WebSocket in real time, with <150ms latency guarantee.
FR-04	Audience can send reactions (emojis), upvote turns, and submit moderated Q&A; all updates broadcast via Redis Pub/Sub.
FR-05	All messages, reactions, turn metadata, and participant metadata are persisted to PostgreSQL for replay, analytics, and audit.
FR-06	BullMQ workers asynchronously process analytics, transcription, summarization, email notifications, and media transcoding.
FR-07	Admins can add, update, or remove AI characters, customize prompts, and override LLM providers without redeployment.
FR-08	System supports semantic search across podcast transcripts using pgvector embeddings; full-text search as fallback.
FR-09	Media files (artwork, avatars, clips) stored in S3-compatible storage; auto-generation of thumbnails, audio normalization via transcoding service.
FR-10	OpenTelemetry collector aggregates traces, metrics, and logs; real-time dashboards and historical analysis via Grafana.
FR-11	Notification Service sends email, SMS, and push notifications to users based on preferences and events.
FR-12	System supports multi-language support (i18n) and regional customization; locale-aware formatting and translation.

5. Non-Functional Requirements

Performance – API p95 latency <100ms; WebSocket message delivery <150ms from inference completion; pgvector semantic search <500ms for 10M embeddings; media transcoding <5x real-time (e.g., 1-hour podcast → 12 minutes).
Scalability – Horizontal scaling to 10k+ concurrent WebSocket connections; stateless API gateway behind load balancer; database read replicas; Redis cluster mode; BullMQ partitioned by podcast ID.
Reliability – 99.9% uptime SLA; circuit breakers on LLM calls; graceful degradation if Storage/Transcoding/Notification unavailable; automatic failover for database replicas; multi-region disaster recovery.
Security – JWT authentication (RS256), RBAC (USER | CREATOR | ADMIN | MODERATOR), encrypted data at rest (AES-256), encrypted in transit (TLS 1.3), rate limiting (100 req/min per IP), CORS, CSRF protection, audit logging for sensitive ops.
Observability – OpenTelemetry tracing (traces stored in Jaeger/Datadog), Prometheus metrics (API latency, throughput, errors, queue depth), structured logs (Pino JSON, Python JSON), Grafana dashboards, real-time alerts.
Maintainability – TypeScript strict mode across all Node services; Python type hints (mypy) for AI Engine; clear service boundaries; comprehensive error handling; 80%+ test coverage; CI/CD on every commit.
Compliance – GDPR-compliant data deletion (cascade delete user data); audit trail for all mutations; PII encryption; terms of service & privacy policy enforcement; SOC 2 Type II controls.

6. System Architecture

7. High-Level Design

Service-Oriented Polyglot Architecture

Core Principle: Each microservice owns its data, exposes well-defined contracts, and scales independently. The platform uses a polyglot approach:

Node.js (Fastify + TypeScript): API Gateway, Auth, Podcast, Realtime, Storage, Notification services (synchronous + event-driven)
Python (FastAPI): AI Engine (specialized domain requiring LangChain, transformer libraries, CUDA support)

Communication Patterns:

External (Client → API): REST (CRUD operations), GraphQL (complex queries + subscriptions for real-time UI updates)
Internal (Service → Service): gRPC (high-throughput, low-latency; Node.js ↔ Python only), REST (occasional coordination)
Asynchronous: BullMQ (durable task queues), Redis Pub/Sub (real-time broadcasts)

Service Layers (Three-Tier Pattern):

Controllers (HTTP Request Validation) 
    ↓
Services (Business Logic + Orchestration)
    ↓
Repository (Database Access + Transactions)

Key Design Tenets:

No shared databases: Each service owns its schema (auth_db, podcast_db; Storage/Transcoding read podcast_db via API only)
Loose coupling: Services discover each other via environment variables (e.g., AUTH_SERVICE_URL)
Idempotent operations: All mutations include unique request IDs for replay safety
Graceful degradation: Failures in non-critical services (Storage, Transcoding) logged but don't block user APIs

8. Event-Driven Architecture

stateDiagram-v2
    [*] --> ChannelCreated
    
    ChannelCreated: Emit: "channel.created"
    ChannelCreated --> TelemetryLog: Log event
    ChannelCreated --> [*]
    
    PodcastScheduled: Emit: "podcast.scheduled"
    [*] --> PodcastScheduled
    PodcastScheduled --> NotifySubscribers: Enqueue email\nworker
    PodcastScheduled --> PreloadCharacters: Cache AI\ncharacters
    PodcastScheduled --> TelemetryLog
    
    PodcastStarted: Emit: "podcast.started"
    PodcastScheduled --> PodcastStarted: Time arrives
    PodcastStarted --> StreamAITurns: gRPC to\nAI Engine
    PodcastStarted --> BroadcastWebSocket: Redis Pub/Sub\nto audience
    PodcastStarted --> TelemetryLog
    
    StreamAITurns --> TurnCompleted: Store in DB<br/>+ Compute embedding
    TurnCompleted --> BroadcastWebSocket: Send to clients
    TurnCompleted --> UpdateRealtimeStats: Update listener\ncount
    
    PodcastEnded: Emit: "podcast.ended"
    BroadcastWebSocket --> PodcastEnded: End time
    PodcastEnded --> TelemetryLog
    
    PodcastEnded --> ComputeAnalytics: Enqueue\nanalytics job
    PodcastEnded --> TranscribeAudio: Enqueue\ntranscription
    PodcastEnded --> GenerateClips: Enqueue\nclip generation
    PodcastEnded --> SendEmailDigest: Enqueue\nemail worker
    
    ComputeAnalytics --> Analytics
    TranscribeAudio --> Transcoding
    GenerateClips --> Transcoding
    SendEmailDigest --> Notification
    
    Analytics: Store in podcast_db
    Transcoding: Store in S3
    Notification: Send via Email/SMS/Push
    
    Analytics --> [*]
    Transcoding --> [*]
    Notification --> [*]
    NotifySubscribers --> [*]
    PreloadCharacters --> [*]
    UpdateRealtimeStats --> [*]

Event Bus Design:

All domain events published to Redis Pub/Sub (per-podcast channels: podcast:{id}:events)
BullMQ workers consume events and enqueue async tasks
Event sourcing via audit log table: event_log(id, aggregateId, eventType, payload, createdAt)
Telemetry Service subscribes to all events for cross-cutting metrics

9. Realtime Architecture

The Realtime Service (port 4003) is the dedicated hub for stateful, low-latency connections:

WebSocket Layer – Clients establish a single persistent WebSocket connection; all real-time updates flow through a managed, authenticated channel.
- Authentication: Extract JWT from connection handshake, validate against Auth Service
- Per-podcast subscriptions: Client subscribes to podcast:{podcastId} channel
- Message batching: Accumulate updates for 50ms, then flush to minimize TCP packets
Redis Pub/Sub Backbone – Enables horizontal scaling across multiple Realtime Service instances:
- Each instance subscribes to Redis channels per active podcast (e.g., podcast:123:turns, podcast:123:reactions)
- Incoming WebSocket messages from clients are validated (by Podcast Service) then published to Redis
- All instances relay Redis messages to their connected clients for that podcast
- Automatic subscription cleanup when last client disconnects
Message Routing – Differentiates between client-to-service and service-to-client messages:
- Upstream (client → Realtime): Reactions, Q&A submissions, poll votes (validated by Podcast Service, then published to Redis)
- Downstream (service → Realtime): AI turns (from Podcast Service), reactions (from other clients), notifications (from Notification Service)
WebRTC Signaling – For future audio/video streaming:
- Realtime Service manages ICE candidate exchange
- Stores peer connection metadata in Redis (TTL 1 hour)
- Coordinates with Storage Service for clip archival

Architecture Diagram:

┌─────────────────────────────────────────────────────────────────┐
│                         Clients Layer                           │
│  Client A ─┐      Client C ─┐      Client E ─┐                 │
│  Client B ─┤      Client D ─┤      Client F ─┤                 │
└────────────┼───────────────┼────────────────┼──────────────────┘
             │               │                │
    ┌────────▼───┐   ┌───────▼────┐   ┌──────▼─────┐
    │ Realtime   │   │ Realtime   │   │ Realtime   │
    │ Instance 1 │   │ Instance 2 │   │ Instance 3 │
    │ :4003      │   │ :4003      │   │ :4003      │
    └────────┬───┘   └────────┬───┘   └──────┬─────┘
             │                │               │
             └────────────────┼───────────────┘
                              │
                    ┌─────────▼────────┐
                    │ Redis Pub/Sub    │
                    │ podcast:123:*    │
                    │ podcast:124:*    │
                    └──────────────────┘
                              ▲
                              │
            ┌─────────────────┼─────────────────┐
            │                 │                 │
     ┌──────▼────┐    ┌──────▼────┐    ┌──────▼────┐
     │ Podcast   │    │ Storage   │    │ Notification
     │ Service   │    │ Service   │    │ Service
     └───────────┘    └───────────┘    └───────────┘

10. AI Orchestration Design

The AI Engine (Python FastAPI, port 5000) is a specialized microservice orchestrating multi-agent LLM conversations:

Prompt Provider – Dynamically assembles prompts from:
- AI Character personality spec (tone, expertise, speaking style, constraints)
- Recent turn history (last 20 turns for context window)
- User-defined podcast topic and format guidelines
- RAG embeddings (via pgvector) for knowledge-augmented responses
- System-level guidelines (content policy, length limits)
LLM Abstraction Layer – Unified provider interface supporting multiple models:
- OpenAI: GPT-4, GPT-4 Turbo, GPT-3.5 (token counting, function calling)
- Anthropic: Claude 3 (native 100k context window, strong reasoning)
- Local Inference: Ollama, vLLM, LocalAI (on-premise or air-gapped environments)
- Switchable policy: Per-character LLM override, per-podcast budget limits (tokens/cost)
Turn Manager – Orchestrates speaking order and conversation flow:
- Maintains per-podcast turn queue (round-robin character rotation with weighted priorities)
- Handles conversation interrupts (audience Q&A insertion)
- Detects topic pivots (manual host intervention)
- Prevents infinite loops or repetitive responses (checks last 5 turns)
Memory Store – Dual-layer persistence with graduated information density:
- Short-term: Redis hash (per-podcast, key = podcast:{id}:memory)
  - Recent turns (last 100 messages)
  - Active character states (current topic, emotional arc)
  - Conversation metadata (interrupts, pivots, sentiment)
  - TTL: Podcast duration + 30 min (for post-podcast queries)
- Long-term: PostgreSQL + pgvector
  - All turns with text + embeddings (compute via OpenAI text-embedding-3-small)
  - Semantic index for retrieval-augmented generation
  - Allows "summarize episode 42's discussion on X" queries

gRPC Contract – Node.js services invoke AI Engine via gRPC (HTTP/2):

service AIEngine {
  rpc GenerateTurn(TurnRequest) returns (TurnResponse);
  rpc UpdateCharacter(CharacterUpdate) returns (Empty);
  rpc Health(Empty) returns (HealthStatus);
}

message TurnRequest {
  string podcast_id = 1;
  string character_id = 2;
  int32 turn_count = 3;
  string topic = 4;
  repeated string context_turns = 5;  // Last N turns for context
}

message TurnResponse {
  string content = 1;
  string character_id = 2;
  int32 tokens_used = 3;
  float latency_ms = 4;
}

Fully Async: Podcast Service enqueues gRPC calls in BullMQ; AI Engine processes independently
Result Publication: AI Engine writes turns to PostgreSQL, then publishes to Redis for real-time delivery

Error Handling & Fallbacks:
- If LLM provider timeout (>10s): Return cached generic response template
- If LLM provider error (rate limit, outage): Use local fallback model (Ollama)
- If embeddings service down: Skip pgvector storage; store text-only

11. Database Design (PostgreSQL)

Two isolated databases per deployment:

auth_db – User credentials, roles, sessions, audit logs
podcast_db – Channels, podcasts, turns, reactions, characters, embeddings, storage metadata, notification logs

Comprehensive Tables:

Table	Primary Key	Key Columns	Purpose
`users` (auth_db)	id (uuid)	email, username, passwordHash, role, createdAt	User accounts & authentication
`sessions` (auth_db)	id (uuid)	userId (FK), token, expiresAt	Session state (mostly in Redis)
`audit_log` (auth_db)	id (uuid)	userId (FK), action, resourceId, timestamp	Compliance & security audit trail
`channels` (podcast_db)	id (uuid)	ownerId (logical ref), name, slug, description, avatarUrl, bannerUrl	Creator identities & branding
`channel_subscriptions` (podcast_db)	(userId, channelId)	composite PK, subscribedAt	Subscription tracking
`podcasts` (podcast_db)	id (uuid)	channelId (FK), title, description, scheduledAt, status, topic	Podcast sessions & metadata
`podcast_turns` (podcast_db)	id (uuid)	podcastId (FK), characterId (FK), content, duration, tokenCount, createdAt	AI-generated messages with metrics
`podcast_embeddings` (podcast_db)	id (uuid)	turnId (FK), embedding (vector), model	pgvector for semantic search (IVFFlat index)
`reactions` (podcast_db)	id (uuid)	podcastId (FK), userId (logical), emoji, turnId (FK), createdAt	Audience interactions & engagement
`qa_submissions` (podcast_db)	id (uuid)	podcastId (FK), userId (logical), question, status, answeredBy	Moderated Q&A queue
`ai_characters` (podcast_db)	id (uuid)	name, personalitySpec (JSON), llmProvider, llmModel, createdBy, createdAt	AI character definitions
`podcast_participants` (podcast_db)	(podcastId, characterId)	composite PK, speakingOrder	Character roster per podcast
`storage_artifacts` (podcast_db)	id (uuid)	podcastId (FK), artifactType, s3Key, s3Bucket, contentType, sizeBytes, createdAt	Media file references (artwork, clips, transcoding output)
`transcoding_jobs` (podcast_db)	id (uuid)	artifactId (FK), inputFormat, outputFormat, status, progress, errorMsg	Media processing job tracking
`notification_logs` (podcast_db)	id (uuid)	userId (logical), notificationType, channel, status, deliveredAt	Notification delivery tracking & retry state
`event_log` (podcast_db)	id (uuid)	aggregateId, eventType, payload (JSON), version, timestamp	Event sourcing for audit & replay
`podcast_analytics` (podcast_db)	podcastId (uuid)	totalListeners, avgEngagementTime, topReactions, sentimentScore	Aggregated metrics (updated post-podcast)

Indexes:

podcasts(channelId, status, scheduledAt) – Fast podcast listing & scheduling
podcast_turns(podcastId, createdAt DESC) – Efficient transcript retrieval
podcast_embeddings USING ivfflat (embedding) – Fast semantic similarity search
reactions(podcastId, createdAt DESC) – Real-time reaction feed
storage_artifacts(podcastId, artifactType) – Find media by podcast & type
users(email) – Fast email lookup (unique constraint)
audit_log(userId, timestamp DESC) – Compliance queries

12. Monorepo Structure

ai-podcast/
├── AGENTS.md                     # Architecture blueprint & constraints
├── CurrentProgress.md            # Implementation status & milestones
├── docker-compose.yaml           # Local dev infrastructure (Postgres, Redis, MinIO)
├── docker-compose.prod.yaml      # Production-like setup (secrets, scaling)
├── pnpm-workspace.yaml           # Monorepo configuration
├── package.json                  # Root dependencies & scripts
├── README.md                      # This file
│
├── Client/web/                   # Next.js 16 Frontend
│   ├── app/                      # App Router structure
│   │   ├── (pages)/
│   │   │   ├── (auth)/           # Login, sign-up, forgot-password flows
│   │   │   ├── channels/         # Channel browser, creation, editing
│   │   │   ├── discover/         # Discovery & trending UI
│   │   │   ├── feed/             # User personalized feed
│   │   │   ├── live/             # Live podcast player with realtime updates
│   │   │   ├── admin/            # Admin dashboard (characters, analytics, moderation)
│   │   │   └── podcast/          # Podcast details & replay
│   │   ├── globals.css
│   │   ├── layout.tsx
│   │   └── page.tsx
│   ├── components/               # React components (reusable)
│   ├── lib/                      # Utilities (API client, formatting, etc.)
│   ├── store/                    # Zustand state (auth, realtime connection)
│   ├── public/                   # Static assets (images, fonts)
│   └── Configuration:
│       ├── next.config.ts
│       ├── tsconfig.json
│       ├── postcss.config.mjs
│       └── eslint.config.mjs
│
├── services/                     # Microservices (Fastify + Prisma)
│   │
│   ├── api-gateway/              # Entry point :4000
│   │   ├── src/
│   │   │   ├── index.ts          # Server bootstrap, middleware setup
│   │   │   ├── routes/           # REST endpoint definitions
│   │   │   ├── graphql/          # GraphQL schema & resolvers (Mercurius)
│   │   │   ├── middleware/       # Rate limiting, JWT, error handling
│   │   │   └── utils/            # Circuit breaker, service discovery
│   │   ├── Dockerfile
│   │   ├── package.json
│   │   └── tsconfig.json
│   │
│   ├── auth-service/             # :4001
│   │   ├── src/
│   │   │   ├── server.ts
│   │   │   ├── app.ts
│   │   │   └── auth/
│   │   │       ├── auth.routes.ts
│   │   │       ├── auth.controllers.ts
│   │   │       ├── auth.services.ts
│   │   │       └── auth.repository.ts
│   │   ├── prisma/
│   │   │   ├── schema.prisma
│   │   │   └── migrations/
│   │   ├── Dockerfile
│   │   └── package.json
│   │
│   ├── podcast-service/          # :4002
│   │   ├── src/
│   │   │   ├── server.ts
│   │   │   ├── app.ts
│   │   │   ├── channel/
│   │   │   ├── podcast/
│   │   │   ├── turn/             # AI turn management
│   │   │   ├── search/           # pgvector semantic search
│   │   │   └── events/           # Event publishing
│   │   ├── prisma/
│   │   │   ├── schema.prisma
│   │   │   └── migrations/
│   │   ├── Dockerfile
│   │   └── package.json
│   │
│   ├── realtime-service/         # :4003
│   │   ├── src/
│   │   │   ├── server.ts
│   │   │   ├── websocket/        # WebSocket connection management
│   │   │   ├── pubsub/           # Redis Pub/Sub adapter
│   │   │   ├── webrtc/           # Peer connection signaling
│   │   │   └── middleware/       # Auth, rate limiting
│   │   ├── Dockerfile
│   │   └── package.json
│   │
│   ├── storage-service/          # :4004
│   │   ├── src/
│   │   │   ├── server.ts
│   │   │   ├── s3/               # S3 client wrapper
│   │   │   ├── upload/           # Multipart upload handling
│   │   │   └── storage.routes.ts
│   │   ├── Dockerfile
│   │   └── package.json
│   │
│   ├── transcoding-service/      # :4005
│   │   ├── src/
│   │   │   ├── server.ts
│   │   │   ├── ffmpeg/           # FFmpeg process management
│   │   │   ├── workers/          # BullMQ job processors
│   │   │   ├── profiles/         # Audio/video encoding profiles
│   │   │   └── transcoding.routes.ts
│   │   ├── Dockerfile
│   │   └── package.json
│   │
│   ├── telemetry-service/        # :4006
│   │   ├── src/
│   │   │   ├── server.ts
│   │   │   ├── otel/             # OpenTelemetry collector setup
│   │   │   ├── prometheus/       # Prometheus metrics export
│   │   │   ├── jaeger/           # Jaeger trace export
│   │   │   └── health.ts
│   │   ├── Dockerfile
│   │   └── package.json
│   │
│   └── notification-service/     # :4007
│       ├── src/
│       │   ├── server.ts
│       │   ├── email/            # SendGrid integration
│       │   ├── sms/              # Twilio integration
│       │   ├── push/             # FCM integration
│       │   ├── templates/        # Notification templates (JSON)
│       │   └── notification.routes.ts
│       ├── Dockerfile
│       └── package.json
│
├── services/ai-engine/           # Python FastAPI AI Orchestration
│   ├── app/
│   │   ├── __init__.py
│   │   ├── main.py               # FastAPI app factory
│   │   ├── server.py             # gRPC server bootstrap
│   │   ├── agents/
│   │   │   ├── __init__.py
│   │   │   ├── character.py      # Character definition & state
│   │   │   ├── prompt.py         # Dynamic prompt assembly
│   │   │   ├── memory.py         # Short/long-term memory
│   │   │   └── turn_manager.py   # Turn orchestration
│   │   ├── llm/
│   │   │   ├── __init__.py
│   │   │   ├── providers.py      # OpenAI, Anthropic, Local abstraction
│   │   │   ├── embeddings.py     # Embedding model for pgvector
│   │   │   └── safety.py         # Input validation & content filtering
│   │   ├── grpc/
│   │   │   ├── __init__.py
│   │   │   ├── service.py        # gRPC service implementation
│   │   │   └── pb2/              # Generated protobuf Python code
│   │   ├── db/
│   │   │   ├── __init__.py
│   │   │   └── postgres.py       # PostgreSQL client (read-only via REST API)
│   │   └── health.py             # Health check endpoints
│   ├── proto/
│   │   └── ai_engine.proto       # gRPC service definition
│   ├── requirements.txt           # Python dependencies
│   ├── Dockerfile
│   └── README_AI_ENGINE.md
│
├── packages/                     # Shared utilities
│   ├── proto-contracts/          # gRPC .proto files (compiled to TS/Python)
│   │   ├── proto/
│   │   │   ├── ai_engine.proto
│   │   │   ├── storage.proto
│   │   │   ├── transcoding.proto
│   │   │   └── common.proto      # Shared message types
│   │   ├── generated/
│   │   │   ├── ts/               # Protoc-gen-ts output
│   │   │   └── python/           # Protoc-gen-python output
│   │   └── package.json
│   │
│   └── ts-config/                # Shared TypeScript configs
│       ├── base.json
│       ├── strict.json
│       └── package.json
│
├── infra/                        # Infrastructure as Code
│   ├── k8s/
│   │   ├── namespace.yaml
│   │   ├── configmap.yaml        # Environment variables
│   │   ├── secrets.yaml          # API keys, DB credentials
│   │   ├── deployment-gateway.yaml
│   │   ├── deployment-services.yaml
│   │   ├── deployment-ai-engine.yaml
│   │   ├── service.yaml          # K8s services (ClusterIP, LoadBalancer)
│   │   ├── ingress.yaml          # Ingress (route to gateway)
│   │   ├── hpa.yaml              # Horizontal Pod Autoscaler
│   │   ├── pdb.yaml              # Pod Disruption Budgets
│   │   └── helm/
│   │       ├── Chart.yaml
│   │       ├── values.yaml
│   │       └── templates/        # Helm templates for all above
│   │
│   └── monitoring/
│       ├── prometheus.yaml       # Prometheus scrape config
│       ├── grafana/
│       │   ├── dashboards/
│       │   │   ├── system.json
│       │   │   ├── api.json
│       │   │   ├── podcast.json
│       │   │   └── realtime.json
│       │   └── datasources/
│       └── jaeger/
│           └── jaeger-deployment.yaml
│
└── scripts/
    ├── setup-dev.sh              # Bootstrap local dev (create DBs, run migrations)
    ├── generate-proto.sh         # Compile .proto files
    ├── test-all.sh               # Run all test suites
    └── deploy-k8s.sh             # Deploy to Kubernetes

13. Tech Stack Justification

Layer	Technology	Reasoning
Frontend	Next.js 16 (React 19, App Router)	SSR + static export hybrid, TypeScript strict mode, built-in API routes for server-side queries, excellent bundle optimization, SEO support
Frontend Query	Apollo Client	GraphQL client with subscriptions, automatic caching, optimistic updates, DevTools browser extension
Styling	TailwindCSS + HeadlessUI	Utility-first for consistency, dark mode built-in, accessible component library, fast prototyping
State Mgmt (Frontend)	Zustand	Minimal overhead vs. Redux, DevTools support, persistence plugin (localStorage), perfect for cross-tab auth state
Package Manager	pnpm	50% faster node_modules, disk-efficient symlinks, superior monorepo support, workspace protocol
Node.js Runtime	Node 20 LTS	V8 optimizations, native ES modules, top-level await, strong TypeScript support
API Framework	Fastify	<0.5ms per-request overhead, plugin ecosystem (helmet, rate-limit, JWT, websocket), excellent for high-throughput APIs
API Gateway	Fastify + express-like middleware	Reverse proxy to services, rate limiting, authentication, circuit breaker patterns, observability
GraphQL (optional)	Apollo Server (Mercurius adapter)	Strong typing, subscription support for real-time feeds, federated schema composition, excellent caching strategies
WebSocket	native or Socket.io	Sticky routing, automatic reconnection, Redis adapter for horizontal scaling
ORM	Prisma	Type-safe, auto-generated client, schema migrations, pgvector support for semantic search, excellent DX
Database	PostgreSQL 16	ACID guarantees, JSON/array support, pgvector extension for embeddings, proven at scale, managed offerings (RDS, Azure)
Cache & Pub/Sub	Redis Cluster 7	Sub-microsecond latency, supports both caching + messaging, cluster mode for HA, sorted sets for leaderboards
Task Queue	BullMQ	Redis-backed, built-in retries & exponential backoff, concurrency control, job priorities, excellent observability
Object Storage	S3-compatible (AWS S3 / MinIO)	Universal media storage, cost-effective, CDN-ready, mature ecosystem, managed offerings
Media Processing	FFmpeg	Battle-tested, supports 100+ codecs/formats, low resource footprint with proper configuration, active development
AI/ML Framework	LangChain (Python)	Unified LLM provider abstraction (OpenAI, Anthropic, local), prompt templating, memory chains, RAG patterns, active community
RPC	gRPC (HTTP/2)	Binary serialization (protobuf), multiplexing, built-in service definition, better throughput than JSON REST, multi-language support
Container Runtime	Docker	Hermetic builds, reproducible local dev, straightforward staging ↔ production parity, standard in industry
Orchestration	Kubernetes	Industry standard, auto-scaling, self-healing, rolling updates, multi-cloud support, managed offerings (EKS, GKE, AKS)
Language (Node)	TypeScript (strict mode)	Compile-time type safety, catches integration bugs early, excellent IDE support, reduced runtime errors
Language (AI)	Python 3.11	Industry standard for ML, LangChain maturity, CUDA support, vast ecosystem (numpy, scipy, transformers)
Observability	OpenTelemetry	Vendor-agnostic tracing, metrics, logs; integrates with Jaeger, Datadog, New Relic, Prometheus
Metrics	Prometheus	Time-series DB, PromQL query language, alerting, Grafana integration
Tracing	Jaeger	Distributed tracing, span causality, latency analysis, high cardinality tags, scalable storage
Logging	ELK (Elasticsearch + Logstash + Kibana)	Full-text search, real-time dashboards, alert integration, log correlation via trace IDs
Testing	Jest + Supertest (Node), pytest (Python)	Snapshot testing, mock/spy support, excellent coverage reporting, fast test execution
CI/CD	GitHub Actions	Native GitHub integration, no external service, secrets management, matrix testing

14. Constraints & Assumptions

Assumptions:

External LLM endpoints (OpenAI, Anthropic) are available with deterministic latency <2s
PostgreSQL is managed (AWS RDS, Azure Database) with automated backups, daily snapshots, point-in-time recovery
Redis cluster or managed Redis (ElastiCache, Azure Cache) available for HA with automatic failover
S3 or S3-compatible storage (MinIO for local dev) available with 99.9% availability SLA
Network latency between services <10ms (same region cloud deployment or local Docker Compose)
Kubernetes cluster (EKS, GKE, AKS) or Docker Swarm for orchestration; Helm for templating

Constraints:

All services containerized; no bare-metal installations
Database-per-service isolation enforced: no cross-service foreign keys, no direct Prisma joins
gRPC calls to AI Engine are inherently async; never block main request path
WebSocket connections are stateful; must be sticky-routed to same Realtime Service instance (or use Redis-backed session store)
Media transcoding output stored in S3; no local filesystem persistence
OpenTelemetry exports must complete within 5s; timeouts don't block request path
All secrets rotated quarterly; API keys stored in encrypted vaults only

15. Security Considerations

Authentication:

JWT tokens signed with RS256 (RSA-2048 private key, rotated annually)
Access tokens (15-minute lifetime) stored in HttpOnly, Secure, SameSite=Strict cookies
Refresh tokens (7-day lifetime) rotated on use with reuse detection (invalidate old token on double-refresh attempt within 30s)
Sessions persisted to Redis with TTL; backend validates on every protected request
Multi-factor authentication (TOTP) optional for creator accounts; mandatory for admins

Authorization:

Four roles: USER (listener), CREATOR (channel owner), MODERATOR (content moderation), ADMIN (platform ops)
RBAC middleware enforces role checks at route level; business logic validates resource ownership (e.g., can only edit own channel)
Sensitive operations (character updates, analytics export, creator payment info) logged to audit trail with user ID, timestamp, action
API endpoint-level rate limiting: 100 req/min per IP, 1000 req/min per authenticated user

Input Validation:

All external payloads validated via schema (Zod for Express routes, fluent-schema for Fastify, pydantic for Python)
Reject oversized payloads (>10MB turn content, >50MB media file)
SQL injection prevention via parameterized Prisma queries; never use raw SQL without explicit review
XSS prevention: escape all user-generated content in templates; use CSP headers
CSRF protection: SameSite cookies, CSRF tokens on state-changing operations

Transport Security:

HTTPS enforced via reverse proxy (nginx, Traefik); HSTS header (max-age=31536000)
CORS configured: credentials allowed only from same-origin or whitelist; Content-Type restrictions
gRPC over HTTP/2 with mTLS (mutual TLS) for internal service communication
DNS over HTTPS (DoH) for resolver queries; no cleartext DNS

Secrets Management:

.env files excluded from git; .env.example as template
In production: AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault
Never log API keys, passwords, or PII; sanitize logs via parsers
API keys rotated quarterly; old keys invalidated immediately upon rotation

Rate Limiting:

100 requests/minute per IP at gateway
Per-user rate limits on mutation endpoints (10 channel creations/hour, 100 podcast creations/month, 1000 messages/hour)
Leaky-bucket algorithm via Redis for distributed fairness across instances

Data Encryption:

At rest: PostgreSQL encryption (RDS-managed AES-256), Redis encryption (cluster TLS), S3 server-side encryption (AES-256)
In transit: TLS 1.3 for all external connections, mTLS for internal gRPC
Passwords hashed with Argon2id (not bcrypt; 40% resistant to GPU/ASIC attacks)
PII (email, names, IP addresses) encrypted in database columns; decryption only in business logic

API Security:

API versioning via URL prefix (/api/v1/, /api/v2/); sunset old versions with 6-month notice
Request signing for webhooks (HMAC-SHA256)
Idempotency keys for state-changing operations (prevent double-submission)
API key rotation: old keys invalidated 30 days after replacement

Compliance:

GDPR-compliant data deletion (cascade delete all user data on account deletion)
Audit trail for all mutations (who, what, when, from where)
PII encryption; data residency support (e.g., EU data stays in EU)
Terms of service & privacy policy versioning with change tracking

16. Scalability Considerations

Horizontal Scaling:

API Gateway & Services – Stateless Fastify instances behind ALB (Application Load Balancer) or NLB (Network Load Balancer); autoscaling group min 2, max 10 instances
Database – Read replicas for queries (lag <1s); writes go to primary via managed failover; connection pooling via PgBouncer (50 connections/service instance)
Redis – Cluster mode with 6+ nodes (3 primaries, 3 replicas); automatic failover; resharding via background migration
BullMQ Workers – Spawn multiple worker processes per machine; queue partitioned by podcast ID to avoid hot spots; max concurrency 100 jobs/instance

Caching Strategy:

Channel metadata cached in Redis with 1-hour TTL (invalidate on update)
AI character specs cached at startup; invalidate on admin update via pub/sub
Podcast transcript search results cached (5-minute TTL) since historical data is immutable
User preferences (notification settings, theme) cached (1-day TTL)

Database Optimization:

Indexes on foreign keys + frequently filtered columns (podcast.status, turn.createdAt, channel.slug)
Partitioned tables for podcast_turns by date (monthly partitions) to speed full-text/semantic search
Columnar store (timescaledb extension) for aggregated analytics queries
Query result pagination (default 20 items, max 100)

Performance Targets:

API p95 latency: <100ms (including network round-trip)
WebSocket message delivery latency: <150ms from inference completion to client update
Semantic search (pgvector): <500ms for 10M embeddings with IVFFlat index
Media transcoding: <5x real-time (1-hour podcast → 12 minutes transcoding)
Concurrent WebSocket connections per instance: 5000 (tested)

Load Testing:

Apache JMeter: 10k concurrent users, ramp-up 2 min, hold 30 min, measure p50/p95/p99 latencies
Database query performance: EXPLAIN ANALYZE on all critical paths; target <10ms for indexed queries

17. Trade-offs & Engineering Decisions

Decision	Trade-off	Rationale
REST + GraphQL + gRPC	Operational complexity (3 protocols) vs. optimal throughput	REST for simple CRUD (cache-friendly); GraphQL for complex frontend queries + subscriptions; gRPC for high-throughput Node-Python communication
Nine services initially	Operational overhead (9 containers, 9 deployments) vs. business agility	Each service own domain & failure zone; enables independent scaling; supports different languages/frameworks optimally
Polyglot (Node + Python)	Operational complexity (2 runtimes) vs. optimal tool selection	Node.js for Fastify (throughput), Python for LangChain (ML ecosystem); not worth rewriting AI Engine in TypeScript
Redis for sessions	Eventual consistency vs. simplicity	Faster than DB queries; transient data OK if lost on crash; acceptable for user sessions (re-login acceptable)
BullMQ (Redis-backed)	Single point of failure if Redis down	Lower operational overhead than separate queue service (Kafka, RabbitMQ); Redis cluster mitigates risk; acceptable for media jobs (retry on failure)
Monorepo	Large CI pipeline vs. easier refactoring	Reduces dependency hell; simpler for small team; shared types across services
Prisma ORM	Micro-performance overhead vs. raw SQL	Type safety + migrations + pgvector support outweigh overhead for this workload; 99% of endpoints <10ms
PostgreSQL + pgvector	Storage overhead vs. operational simplicity	One database for transactional + ML embeddings; avoids separate vector DB (Pinecone, Weaviate); pgvector mature enough for production
Kubernetes	Operational complexity vs. production reliability	Industry standard, battle-tested, excellent multi-cloud support, auto-scaling, self-healing; Docker Swarm simpler but less flexible
gRPC (HTTP/2)	Learning curve, tooling overhead vs. performance	40% latency reduction + 60% bandwidth savings; protobuf contracts catch API mismatches early; investment pays off at scale

18. Development Workflow / SDLC

Local Development Setup:

Clone repository: git clone https://github.com/your-org/convo-ai-studio.git
pnpm install – Install all workspace dependencies
docker-compose up -d – Postgres, Redis, MinIO (S3 local), Jaeger
pnpm run setup:dev – Create databases, run migrations, seed data
pnpm run generate:proto – Compile .proto files (gRPC contracts)
pnpm dev – Start all services in watch mode
Access UI at http://localhost:3000, API at http://localhost:4000, Jaeger at http://localhost:16686

Feature Development:

Create feature branch: git checkout -b feat/podcast-analytics
Implement feature following three-layer pattern (Controller → Service → Repository)
Write unit tests: pnpm run test:unit (Jest with 80%+ coverage target)
Write integration tests: pnpm run test:integration (Supertest for API, real DB)
Run linters: pnpm run lint (ESLint, Prettier, TypeScript strict)
Commit with conventional commits (feat:, fix:, docs:, etc.)

Code Review & CI:

Push to remote; open Pull Request on GitHub
GitHub Actions runs:
- Linting (ESLint, Prettier check)
- Type checking (TypeScript strict, mypy for Python)
- Unit tests (Jest, pytest)
- Integration tests (Supertest, testcontainers)
- Security scan (Snyk, OWASP dependency check)
Mandatory peer review (minimum 1 approval)
Auto-merge on approval or manual merge

Release & Deployment:

Merge to main branch
GitHub Actions triggers:
- Build Docker images for all modified services
- Tag images with git SHA and semver
- Push to container registry (ECR, Docker Hub)
Deploy to staging cluster via Helm:
- helm upgrade convo-ai-studio-staging ./infra/helm --values staging.yaml
Run smoke tests (health checks, basic API calls)
Manual approval for production rollout
Deploy to production via canary (10% traffic → 50% → 100% over 30 min)

19. Local Setup Instructions

Prerequisites

Node.js 20+ and pnpm 11+
Docker & Docker Compose (v2.0+)
Python 3.11+ and pip (for AI Engine development)
Git
(Optional) Kubernetes & Helm (for local K8s testing)

Quick Start (5 minutes)

# Clone the repository
git clone https://github.com/your-org/convo-ai-studio.git
cd convo-ai-studio

# Install dependencies
pnpm install

# Start local infrastructure (Postgres, Redis, MinIO, Jaeger)
docker-compose up -d

# Bootstrap databases and run migrations
pnpm run setup:dev

# Compile gRPC contracts
pnpm run generate:proto

# Start all services in watch mode
pnpm dev

# In a new terminal, start the frontend
cd Client/web && pnpm dev

# Access:
# - Web UI: http://localhost:3000
# - API Gateway: http://localhost:4000
# - Jaeger Traces: http://localhost:16686
# - MinIO Console: http://localhost:9001 (admin/minioadmin)

Service URLs & Health Checks

Service	URL	Health Check	Port
Frontend	http://localhost:3000	Home page loads	3000
API Gateway	http://localhost:4000	`GET /health`	4000
Auth Service	http://localhost:4001	`GET /health`	4001
Podcast Service	http://localhost:4002	`GET /health`	4002
Realtime Service	ws://localhost:4003	WebSocket connects	4003
Storage Service	http://localhost:4004	`GET /health`	4004
Transcoding Service	http://localhost:4005	`GET /health`	4005
Telemetry Service	http://localhost:4006	`GET /health`	4006
Notification Service	http://localhost:4007	`GET /health`	4007
AI Engine (gRPC)	localhost:50051	gRPC health check	50051
AI Engine (HTTP)	http://localhost:5000	`GET /health`	5000
PostgreSQL	localhost:5432	psql connection	5432
Redis	localhost:6379	redis-cli ping	6379
MinIO (S3)	http://localhost:9000	API calls	9000
MinIO Console	http://localhost:9001	Web UI (admin/minioadmin)	9001
Jaeger UI	http://localhost:16686	Traces	16686

Useful Commands

# Database operations
pnpm run prisma:studio          # Open Prisma Studio for visual DB browsing
pnpm run prisma:migrate         # Create & apply migrations
pnpm run prisma:seed            # Seed development data

# Testing
pnpm run test:unit              # Run Jest unit tests
pnpm run test:integration       # Run Supertest integration tests
pnpm run test:coverage          # Generate coverage report (target 80%+)

# Linting & formatting
pnpm run lint                   # Run ESLint + Prettier check
pnpm run lint:fix               # Auto-fix linting issues
pnpm run type-check             # Run TypeScript compiler & mypy

# API testing
pnpm run api:test               # Postman/Newman tests
pnpm run api:load-test          # JMeter load testing

# Code generation
pnpm run generate:proto         # Compile .proto → TS/Python
pnpm run generate:types         # Generate TypeScript types from Prisma

# Debugging
docker logs convo-podcast-service -f          # View service logs
docker exec -it convo-postgres psql -U admin # Connect to database
redis-cli -p 6379                            # Connect to Redis CLI

# Reset (⚠️ destructive)
pnpm run reset:db               # Drop & recreate databases
docker-compose down -v          # Remove all containers & volumes

Development Workflow Example

# 1. Create feature branch
git checkout -b feat/add-podcast-analytics

# 2. Edit files, run tests locally
pnpm run test:unit --watch     # Watch mode for TDD

# 3. Start local services
pnpm dev

# 4. Test in browser
# Open http://localhost:3000, interact with UI

# 5. Run full test suite
pnpm run test:unit && pnpm run test:integration

# 6. Lint & type-check
pnpm run lint && pnpm run type-check

# 7. Commit & push
git add .
git commit -m "feat: add podcast analytics dashboard"
git push origin feat/add-podcast-analytics

# 8. GitHub Actions runs automatically on PR
# Wait for CI to pass, then merge

20. Environment Variables

Root `.env.local`

# ===== Database =====
DATABASE_URL=postgresql://admin:password@localhost:5432/podcast_db
AUTH_DB_URL=postgresql://admin:password@localhost:5432/auth_db

# ===== Cache & Queue =====
REDIS_URL=redis://localhost:6379/0
REDIS_CLUSTER_MODE=false  # Set true for cluster mode in production

# ===== S3 Storage =====
S3_ENDPOINT=http://localhost:9000  # MinIO local; AWS S3 in production
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_BUCKET_NAME=convo-ai-studio
S3_REGION=us-east-1

# ===== JWT =====
JWT_SECRET=your-secret-key-min-32-chars-change-in-production
JWT_REFRESH_SECRET=your-refresh-secret-min-32-chars-change-in-production
JWT_ACCESS_EXPIRY=15m
JWT_REFRESH_EXPIRY=7d

# ===== LLM Providers =====
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
OLLAMA_BASE_URL=http://localhost:11434  # Local inference

# ===== Email & Notifications =====
SENDGRID_API_KEY=SG....
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
FCM_PROJECT_ID=...
FCM_PRIVATE_KEY_ID=...

# ===== Observability =====
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318  # OpenTelemetry Collector
JAEGER_AGENT_HOST=localhost
JAEGER_AGENT_PORT=6831
LOG_LEVEL=debug

# ===== Application =====
NODE_ENV=development
PORT=4000  # API Gateway
ENVIRONMENT=local

Auth Service (`services/auth-service/.env`)

DATABASE_URL=postgresql://admin:password@localhost:5432/auth_db
REDIS_DB=0
REDIS_URL=redis://localhost:6379
JWT_SECRET=your-secret-key-min-32-chars
JWT_ACCESS_EXPIRY=15m
JWT_REFRESH_EXPIRY=7d
PORT=4001
LOG_LEVEL=debug

Podcast Service (`services/podcast-service/.env`)

DATABASE_URL=postgresql://admin:password@localhost:5432/podcast_db
REDIS_DB=1
REDIS_URL=redis://localhost:6379
JWT_SECRET=your-secret-key-min-32-chars
AUTH_SERVICE_URL=http://localhost:4001
AI_ENGINE_URL=localhost:50051  # gRPC address
STORAGE_SERVICE_URL=http://localhost:4004
TRANSCODING_SERVICE_URL=http://localhost:4005
PORT=4002
LOG_LEVEL=debug

Realtime Service (`services/realtime-service/.env`)

REDIS_URL=redis://localhost:6379
REDIS_DB=2
JWT_SECRET=your-secret-key-min-32-chars
AUTH_SERVICE_URL=http://localhost:4001
PODCAST_SERVICE_URL=http://localhost:4002
NOTIFICATION_SERVICE_URL=http://localhost:4007
PORT=4003
LOG_LEVEL=debug

AI Engine (`services/ai-engine/.env`)

OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
OLLAMA_BASE_URL=http://localhost:11434
DATABASE_URL=postgresql://admin:password@localhost:5432/podcast_db
REDIS_URL=redis://localhost:6379
GRPC_PORT=50051
HTTP_PORT=5000
LOG_LEVEL=debug

Storage Service (`services/storage-service/.env`)

DATABASE_URL=postgresql://admin:password@localhost:5432/podcast_db
S3_ENDPOINT=http://localhost:9000
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_BUCKET_NAME=convo-ai-studio
S3_REGION=us-east-1
PORT=4004
LOG_LEVEL=debug

Transcoding Service (`services/transcoding-service/.env`)

DATABASE_URL=postgresql://admin:password@localhost:5432/podcast_db
REDIS_URL=redis://localhost:6379
S3_ENDPOINT=http://localhost:9000
S3_ACCESS_KEY_ID=minioadmin
S3_SECRET_ACCESS_KEY=minioadmin
S3_BUCKET_NAME=convo-ai-studio
FFMPEG_PATH=/usr/bin/ffmpeg  # or full path
TEMP_DIR=/tmp/transcoding
PORT=4005
LOG_LEVEL=debug

Notification Service (`services/notification-service/.env`)

DATABASE_URL=postgresql://admin:password@localhost:5432/podcast_db
REDIS_URL=redis://localhost:6379
SENDGRID_API_KEY=SG...
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
FCM_PROJECT_ID=...
PORT=4007
LOG_LEVEL=debug

Telemetry Service (`services/telemetry-service/.env`)

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
JAEGER_ENDPOINT=http://localhost:14268/api/traces
PROMETHEUS_PORT=9090
PROMETHEUS_SCRAPE_INTERVAL=15s
PORT=4006
LOG_LEVEL=debug

API Gateway (`services/api-gateway/.env`)

AUTH_SERVICE_URL=http://localhost:4001
PODCAST_SERVICE_URL=http://localhost:4002
REALTIME_SERVICE_URL=http://localhost:4003
STORAGE_SERVICE_URL=http://localhost:4004
TRANSCODING_SERVICE_URL=http://localhost:4005
NOTIFICATION_SERVICE_URL=http://localhost:4007
TELEMETRY_SERVICE_URL=http://localhost:4006
JWT_SECRET=your-secret-key-min-32-chars
LOG_LEVEL=info
RATE_LIMIT=100
PORT=4000
ENVIRONMENT=local

21. Future Scope

Immediate Priorities (6 months)

Video Streaming – WebRTC mesh for visual podcast experience; camera feeds from creators & guests
AI Audio Synthesis – Text-to-speech for AI turns (ElevenLabs, Google Cloud TTS); realistic voices per character
Marketplace – Third-party AI characters, community prompts, monetization framework (revenue share)
Advanced Analytics – Sentiment analysis, topic extraction, audience journey mapping; export to business intelligence tools

Strategic Initiatives (12 months)

Multi-Region Deployment – Geo-distributed PostgreSQL read replicas, Redis cluster across regions, edge CDN
Chatbot Integration – AI-powered moderation, auto-response to FAQs, audience engagement bots
Premium Features – Custom domains, white-label branding, API access for podcast platforms, advanced integrations
Data Pipeline – Kafka for high-throughput event streaming; dbt for analytics warehouse; BigQuery integration

Enterprise Features (18 months)

SSO & SAML – Single sign-on for enterprise customers; audit logging for compliance (SOC 2, ISO 27001)
Advanced Security – IP whitelisting, VPN-only access, encryption key management (KMS integration)
Compliance & Regulatory – HIPAA support (healthcare creators), CCPA (California privacy), multi-tenancy isolation
High-Availability – Multi-region active-active failover, disaster recovery runbooks, RTO <1 hour, RPO <15 min

22. Engineering Learnings

Lessons from designing & building Convo-AI-Studio:

Database-per-Service is Non-Negotiable – Early prototypes with shared schemas led to tight coupling and bloated migration files. Isolating auth_db and podcast_db forced us to design proper service boundaries and async communication patterns. Worth the extra complexity and infrastructure overhead.
gRPC for High-Throughput RPC – Initial REST-only approach for AI Engine calls resulted in 2-3s latency spikes. gRPC's binary framing + HTTP/2 multiplexing reduced latency by 40% and bandwidth by 60%. Protobuf contracts also catch API mismatches at compile time.
Redis Pub/Sub Needs Clustering – Single-node Redis became a bottleneck at ~3k concurrent WebSocket connections. Switching to cluster mode (6 nodes) immediately resolved backpressure without code changes (ioredis handles failover automatically).
Async Everything for Scalability – Blocking AI Engine calls would cap throughput at ~100 req/sec. Using BullMQ workers decoupled inference latency from API response time; users get instant WebSocket delivery while workers process analytics in the background. Throughput jumped 10x.
TypeScript Strict Mode Catches Bugs Early – Mismatches between Prisma generated types and GraphQL schemas led to silent runtime errors. Strict mode made these compilation errors, saving weeks of debugging in production.
Observability is Foundational – Distributed tracing via OpenTelemetry caught a 500ms latency spike (blocked on S3 list operation). Without it, would've blamed the API Gateway. Start with observability on day 1, not week 6.
Polyglot Architecture Requires Discipline – Python + Node.js dual runtime adds operational complexity. Mitigate via:
- Clear service boundaries (gRPC contracts)
- Standard Docker images for both
- Shared observability stack (OpenTelemetry works for both)
- Pre-commit linting for both (ESLint + mypy)

23. Challenges Faced

Technical Obstacles & Solutions:

Real-time Latency (2-3s → <150ms)
- Problem: Initial architecture had AI Engine calls blocking the Podcast Service request path, causing 2-3s latency spikes while waiting for LLM inference.
- Solution: Decoupled via BullMQ: Podcast Service enqueues gRPC call immediately, returns 202 Accepted; AI Engine processes async, writes turn to PostgreSQL, publishes to Redis Pub/Sub; WebSocket clients receive update <150ms later.
Session Consistency in Multi-Instance Setup
- Problem: Redis sessions worked for single-instance dev, but multi-instance load-balancing caused session loss when client was routed to different server on refresh token rotation.
- Solution: Used Redis cluster with persistence + replicas; added session versioning to detect stale tokens; implemented token refresh ceremony with browser session storage sync.
Database Query N+1 Problems
- Problem: Prisma's powerful query builder masked N+1 issues in early GraphQL resolvers (e.g., fetch channel → all subscribers → each subscriber's profile = hundreds of DB queries).
- Solution: Implemented query batching via DataLoader pattern; profiled all critical paths with EXPLAIN ANALYZE; added query cost estimation to resolver monitoring.
pgvector Embedding Latency (hours → minutes)
- Problem: Computing embeddings for 100k+ transcript turns during initial launch took 6+ hours; new podcasts could wait hours for searchability.
- Solution: Pre-compute embeddings for new turns asynchronously (batch 100 turns/job); use IVFFlat indexing instead of HNSW; cache hot embeddings in Redis; SLA for semantic search: <500ms for 10M vectors.
Media Transcoding Bottleneck
- Problem: Single FFmpeg process on transcoding service couldn't handle concurrent jobs; podcasts queued for hours waiting for clips.
- Solution: Horizontally scale transcoding workers (BullMQ concurrency = 4 jobs/instance); implement job priorities (shorter clips first); pre-compute common resolutions to reduce per-job work.
Storage Service Failures Cascading to API
- Problem: S3 outages (or MinIO down in dev) brought down the entire podcast creation flow, even though storage is non-critical for scheduling.
- Solution: Implemented circuit breaker on S3 calls (fail-open after 5 failures); deferred S3 operations to background jobs; API returns 202 Accepted while storage happens asynchronously.
Observability Blind Spots
- Problem: High-latency API call (p95 > 500ms) but no traces; couldn't identify root cause.
- Solution: Added OpenTelemetry spans at service boundaries, database operations, external API calls; sampled 100% of error requests, 1% of successful requests; Jaeger UI made latency spikes obvious.

24. Conclusion

Convo-AI-Studio represents a modern, production-ready platform combining distributed microservices, event-driven architecture, polyglot languages, and AI orchestration. The careful selection of battle-tested technologies (Fastify, Prisma, PostgreSQL, Redis, gRPC, Kubernetes), database-per-service isolation, and clear service contracts (REST + GraphQL + gRPC) provide a solid foundation for scaling to thousands of concurrent podcasts and millions of listeners.

Key Design Principles:

Loose Coupling – Services own their data; communicate via REST/gRPC/Pub/Sub; scale independently
Event-Driven – State changes emit events; workers consume asynchronously; failures isolated
Observability First – Traces, metrics, logs from day 1; Jaeger + Prometheus + Grafana reveal performance bottlenecks
Polyglot Where It Matters – Node.js for throughput (Fastify), Python for AI (LangChain); not worth monolithic rewrite
Production-Grade From Day 1 – Circuit breakers, graceful degradation, multi-region failover, encryption, audit logs

The nine-service architecture (API Gateway, Auth, Podcast, Realtime, Storage, Transcoding, Telemetry, Notification, AI Engine) balances operational complexity with business agility: each team owns a service; features deploy independently; failures don't cascade.

This README serves as the north star for all contributors. Treat the architecture decisions (especially database isolation, service contracts, and gRPC for AI) as non-negotiable constraints. Deviations require architecture review. The difference between a scalable platform and a legacy monolith is discipline in enforcing these boundaries.

Next Steps:

Share this README with team leads for technical accuracy review
Set up development environment: pnpm install && docker-compose up -d && pnpm run setup:dev
Deploy to staging Kubernetes cluster to validate orchestration config
Run load tests: 10k concurrent users, measure p95/p99 latencies
Document runbooks for common operational scenarios (failover, scaling, incident response)

For more details:

See AGENTS.md for architecture constraints and service responsibilities
See CurrentProgress.md for implementation roadmap and completed work
See docker-compose.yaml for local infrastructure setup
See infra/helm/ for Kubernetes deployment templates
See services/ai-engine/README_AI_ENGINE.md for Python AI Engine internals

Maintained by: Engineering Team
Last Updated: June 23, 2026
License: Proprietary

Name		Name	Last commit message	Last commit date
Latest commit History 68 Commits
apps/client		apps/client
docs		docs
packages		packages
services		services
.dockerignore		.dockerignore
.gitignore		.gitignore
Codex_prompt_logic.md		Codex_prompt_logic.md
CurrentProgress.md		CurrentProgress.md
README.md		README.md
backend-routes-analysis.md		backend-routes-analysis.md
docker-compose.yaml		docker-compose.yaml
package-lock.json		package-lock.json
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
test-routes.md		test-routes.md
test.py		test.py
try-ui-update.md		try-ui-update.md
tsconfig.base.json		tsconfig.base.json

Folders and files

Latest commit

History

Repository files navigation

Convo-AI-Studio

Index

1. Project Overview

2. Vision & Goals

3. Core Features

4. Functional Requirements

5. Non-Functional Requirements

6. System Architecture

7. High-Level Design

Service-Oriented Polyglot Architecture

8. Event-Driven Architecture

9. Realtime Architecture

10. AI Orchestration Design

11. Database Design (PostgreSQL)

12. Monorepo Structure

13. Tech Stack Justification

14. Constraints & Assumptions

15. Security Considerations

16. Scalability Considerations

17. Trade-offs & Engineering Decisions

18. Development Workflow / SDLC

19. Local Setup Instructions

Prerequisites

Quick Start (5 minutes)

Service URLs & Health Checks

Useful Commands

Development Workflow Example

20. Environment Variables

Root .env.local

Auth Service (services/auth-service/.env)

Podcast Service (services/podcast-service/.env)

Realtime Service (services/realtime-service/.env)

AI Engine (services/ai-engine/.env)

Storage Service (services/storage-service/.env)

Transcoding Service (services/transcoding-service/.env)

Notification Service (services/notification-service/.env)

Telemetry Service (services/telemetry-service/.env)

API Gateway (services/api-gateway/.env)

21. Future Scope

Immediate Priorities (6 months)

Strategic Initiatives (12 months)

Enterprise Features (18 months)

22. Engineering Learnings

23. Challenges Faced

24. Conclusion

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages