CloudSpend Guardian

CloudSpend Guardian is a full-stack FinOps application that ingests cloud billing files, detects spending anomalies, explains root causes with LLM-generated insights, and delivers actionable dashboards and reports.

Project Overview

This project helps teams monitor cloud spend proactively by combining:

• Data ingestion and normalization for billing files
• Hybrid storage for analytics and context-heavy records
• Rule-based anomaly detection
• AI-assisted anomaly explanations
• Streamlit dashboards and reporting tools for fast decision-making

Business Problem

Cloud costs are often reviewed too late, after overspending has already happened. Teams typically face:

• Delayed visibility into daily/service-level spend spikes
• Limited context when anomalies are detected
• Fragmented reporting across tools and formats
• Difficulty translating technical anomalies into business actions

CloudSpend Guardian addresses this by centralizing ingestion, analytics, anomaly detection, and AI-generated recommendations in one lightweight workflow.

Features

• Upload billing files via API and Streamlit (CSV and JSON)
• Normalize and persist billing records
• Aggregate daily and service-level cost summaries
• Manage budgets by cloud account
• Detect anomalies using rolling-average and threshold logic
• Generate AI explanations for anomalies (OpenAI or Gemini)
• Convert USD spend to local currency (e.g., MYR) with FX API
• View dashboards, anomalies, AI insights, and monthly reports
• Export report summaries as CSV

Architecture

Backend

• Framework: FastAPI
• Layers:
  • api/ route handlers
  • services/ business logic
  • repositories/ database access
  • db/ connection + SQLAlchemy models
  • schemas/ Pydantic request/response contracts

Frontend

• Framework: Streamlit
• Multi-page UI:
  • Upload Data
  • Dashboard
  • Anomalies
  • AI Insights
  • Budgets
  • Reports

Hybrid Database Design

CloudSpend Guardian uses a hybrid persistence strategy:

• SQLite (SQLAlchemy) for structured analytics and operational entities:

  • budgets
  • anomalies
  • daily_cost_summaries
  • service_cost_summaries
  • reports (snapshot-ready model)

• MongoDB Atlas for flexible/raw/contextual data:

  • raw_billing_records (raw + normalized ingestion rows)
  • ingestion_logs (upload status and metadata)
  • anomaly_diagnostics (detection context)
  • ai_insights (full AI responses + prompt context)

This design keeps analytics fast and predictable while preserving rich diagnostic/AI context.

APIs Used

• OpenAI API (optional): anomaly explanation generation
• Google Gemini API (optional): alternative LLM provider
• Exchange Rate API (open.er-api.com): USD to local currency conversion

Local Setup

1. Create and activate a Python virtual environment
2. Install dependencies:

   pip install -r requirements.txt

3. Configure environment:

   cp .env.example .env

4. Update .env values (MongoDB URI, AI keys, FX settings) as needed

Run Instructions

Backend

cd backend
uvicorn app.main:app --reload

Frontend

cd frontend
streamlit run streamlit_app/Home.py

Demo Flow

1. Open the Streamlit app home page
2. Go to Upload Data and upload a sample billing file
3. Open Dashboard to view total spend, trend, and service breakdown
4. Open Budgets and create a budget for global or a cloud account
5. Open Anomalies and run detection
6. Open AI Insights and generate explanation for an anomaly
7. Open Reports to review monthly summary and export CSV

Sample Billing Files (Suggested Names)

• aws_quickstart_usage.csv
• gcp_quickstart_usage.json
• baseline_usage.csv
• service_spike_usage.csv
• budget_breach_usage.csv
• multi_anomaly_usage.csv

Demo Script (Short)

Use this script during presentation:

1. “I’ll upload a monthly billing file and ingest it.”
2. “Now we can see daily and service-level spend in the dashboard.”
3. “I’ll set a budget threshold and run anomaly detection.”
4. “Here’s a detected spike; now I’ll generate an AI explanation.”
5. “Finally, I’ll open Reports and export a CSV summary for stakeholders.”

Resume-Ready Project Bullets

• Built a full-stack FinOps platform using FastAPI, Streamlit, SQLAlchemy/SQLite, and MongoDB Atlas for cloud cost ingestion, monitoring, and reporting.
• Designed a hybrid database architecture separating analytical summaries (SQL) from raw/diagnostic/AI context (NoSQL) to improve maintainability and query performance.
• Implemented rule-based anomaly detection (rolling averages + budget thresholds) and integrated LLM-powered incident explanations using OpenAI/Gemini APIs.
• Added currency conversion support with external FX API integration and delivered exportable monthly reporting for stakeholder-friendly cost visibility.

Future Improvements

• Scheduled ingestion and detection jobs
• User authentication and role-based access
• Cloud-provider-native connectors (AWS CUR, GCP Billing Export, Azure Cost Management)
• Alerting channels (email/Slack/Teams)
• Historical model-based anomaly scoring (beyond rule-based detection)
• CI pipeline with automated test execution and coverage reporting