Orqestra Assets

Monorepo for the Orqestra Assets subsystem, a DAM (Digital Asset Management) focused on creative pieces.

Overview

Service	Port	Description
orqestra-assets	8020	DAM API (FastAPI): upload, hybrid search, metadata
indexing-consumer	—	SQS worker: generates LLM descriptions + embeddings → OpenSearch
frontend	5173	Vite/React interface
auth-service	8002	JWT authentication
api-gateway	8000	HTTP gateway with rate limiting and JWT validation
opensearch	9200	k-NN vector index + BM25 hybrid search
opensearch-dashboards	5601	OpenSearch UI
localstack	4567	Local emulation of S3, SQS, Lambda and CloudWatch
db	5433	PostgreSQL — orqestra_assets database
orqestra-db	5434	PostgreSQL — auth + campaigns database

Prerequisites

• Docker Desktop 4.x+
• OpenAI API key (embeddings and LLM descriptions)
• (Optional) LocalStack Pro token for Lambda support

Quick start

# 1. Clone the repository
git clone <repo-url>
cd creative-dam

# 2. Set up environment variables
cp env.example .env
# Edit .env and fill in at least OPENAI_API_KEY

# 3. Start all services
docker compose up --build

Wait for all services to become healthy (may take 2–3 minutes on first run).

Open:

• Frontend: http://localhost:5173
• DAM API docs: http://localhost:8020/docs
• OpenSearch Dashboards: http://localhost:5601

Configuration

All required and optional variables are documented in env.example.

Required variables

Variable	Description
OPENAI_API_KEY	OpenAI key for embeddings and LLM descriptions
SECRET_KEY	JWT secret shared between auth-service and api-gateway

Optional variables

Variable	Default	Description
LOCALSTACK_AUTH_TOKEN	—	LocalStack Pro token (leave empty for Community edition)
INDEXING_MAX_WORKERS	5	Indexing worker parallelism
SEMANTIC_SEARCH_MIN_SCORE	0.8	Minimum score for pure k-NN search
HYBRID_SEARCH_MIN_SCORE	0.8	Minimum score for hybrid search

Architecture

Browser → api-gateway:8000 → auth-service:8002
                           → orqestra-assets:8020 → PostgreSQL
                                                   → S3 (LocalStack)
                                                   → SQS (LocalStack)
                                                         ↓
                                              indexing-consumer
                                                         ↓
                                                    OpenSearch

Indexing flow

1. Upload via POST /assets/upload-app, /assets/upload-email or /assets/text
2. File saved to S3 + row inserted in the assets table (status pending)
3. Message published to the assets-indexing SQS queue
4. Worker consumes the queue: generates description (LLM for PNG/HTML), embedding (OpenAI text-embedding-3-small), and upserts into OpenSearch
5. Status updated to indexed in PostgreSQL

Search

• SQL (no query text): paginated listing with filters by channel, campaign, status
• Hybrid (with query text): BM25 + k-NN via assets-hybrid-pipeline in OpenSearch 3.5

Evaluation

The official evaluation benchmark is NLP-only:

• Query specs in evals/assets/query_specs.json must contain only query, expected_ids, and type
• Deterministic filters (channel, format, tags, locale, etc.) are not allowed in the benchmark
• run_eval.py sends only query and limit to POST /assets/search

Run evaluation

docker compose --profile eval run --rm eval-run

Primary KPIs:

• MRR
• Success@1
• Success@3
• Recall@3

Local development (outside Docker)

# Install Python dependencies
pip install -r requirements.txt

# Export variables
export PYTHONPATH=orqestra-assets
export $(cat .env | xargs)

# Run the API
uvicorn main:app --app-dir orqestra-assets --reload --port 8020

# Run Alembic migrations
alembic upgrade head

Repository structure

.
├── orqestra-assets/        # DAM API + indexing worker
│   ├── app/                # FastAPI: routes, schemas, services, OpenSearch client
│   ├── indexing_worker/    # SQS consumer: LLM description, embedding, upsert
│   ├── alembic/            # PostgreSQL migrations
│   └── opensearch/         # OpenSearch configuration
├── auth-service/           # JWT authentication (FastAPI)
├── api-gateway/            # HTTP gateway (FastAPI + slowapi)
├── frontend/               # React/Vite interface
├── grafana/                # Provisioned dashboards and datasources
├── scripts/                # init-localstack.sh, build-lambda-zip.sh
├── docker-compose.yml
├── env.example             # Environment variables template
└── alembic.ini

Postman collection

Import orqestra-assets.postman_collection.json to test all DAM endpoints directly.

Security notes

• Never commit the .env file — it is listed in .gitignore
• Credentials in docker-compose.yml (postgres / orqestra_password) are for local development only
• AWS keys (test / test) are LocalStack placeholders with no real access
• In production, replace SECRET_KEY, database passwords, and use real IAM roles for AWS