🤖 AI Support Copilot

A RAG-powered customer support API that answers from a knowledge base with cited sources, confidence scoring, and automatic escalation.

---

🔗 Live Demo

▶ Live Chat (Vercel) · 🤗 Backend API (HF Spaces) · 📖 API Docs · GitHub

⚠️ Backend on free tier — may take ~40s to wake up on first load.

The demo is themed around TechMart, a fictional electronics retailer used as the knowledge-base domain (the chat UI brands itself "AI Support Copilot — TechMart's AI assistant").

---

Problem Statement

Support teams spend most of their time answering the same repetitive, tier-1 questions, while naive chatbots hallucinate plausible-but-wrong answers and erode customer trust. What's needed is a grounded, source-cited system that knows when it doesn't know — and escalates to a human instead of guessing.

---

Business Impact

Auto-resolves common support queries in real time with source-cited answers, and safely escalates low-confidence cases to humans instead of guessing — cutting tier-1 support load while protecting customer trust.

---

Engineering Highlights

• RAG pipeline architecture: document chunking → local embedding → semantic retrieval → grounded LLM generation, organized into clean modules (config.py, rag/pipeline.py, routes/) rather than one monolithic script. The FAISS index is built once at startup and reused across requests.
• REST API design: FastAPI POST /chat endpoint with strict Pydantic request/response models, returning answer + sources + confidence + escalated flag + latency_ms on every call, plus proper status codes (503 when the pipeline is unavailable, 422 on invalid input).
• Confidence-based escalation logic: when the top retrieval similarity falls below a tunable threshold (0.32), the API returns a human-escalation response instead of hallucinating — a deliberate reliability decision, not an afterthought.
• Local embeddings, no paid embedding API: sentence-transformers (all-MiniLM-L6-v2, 384-dim) runs locally and indexes into FAISS; Groq (Llama 3.3 70B) handles only the final generation, keeping latency low and cost off the embedding path.
• Containerized deployment: Dockerized FastAPI on Hugging Face Spaces (port 7860, CPU-only PyTorch, embedding model pre-cached at build time); the React frontend ships on Vercel's edge — a deliberate split because the RAG/ML dependency footprint (torch, faiss-cpu, sentence-transformers) is too heavy for serverless cold-starts.

---

Tech Stack

Layer	Technology
API	FastAPI · Uvicorn · Pydantic v2 (Python 3.11)
RAG	sentence-transformers (all-MiniLM-L6-v2) · FAISS (IndexFlatIP, cosine)
LLM	Groq — Llama 3.3 70B Versatile (OpenAI-compatible API)
Frontend	React 18 · Vite · Tailwind CSS
Container	Docker (python:3.10-slim)
Deployment	Hugging Face Spaces (backend) · Vercel (frontend)

---

Architecture

┌──────────────────────────────┐   REST / JSON   ┌────────────────────────────────────────┐
│  React + Vite Chat UI         │ ──────────────► │  FastAPI RAG Backend (Docker)            │
│  Vercel edge network          │ ◄────────────── │  Hugging Face Spaces · port 7860         │
│  chat • sources • confidence  │   cited answer  │  sentence-transformers + FAISS + Groq    │
└──────────────────────────────┘                 └────────────────────────────────────────┘

POST /chat  ▼
  1. Embed query        → sentence-transformers (all-MiniLM-L6-v2), runs locally
  2. Retrieve top-k=4   → FAISS cosine similarity over the chunked knowledge base
  3. Confidence gate    → top score < 0.32  →  escalate to a human (no generation)
  4. Grounded prompt    → inject retrieved chunks as cited context for the LLM
  5. Generate           → Groq · Llama 3.3 70B → grounded, source-cited answer
       ▼
{ answer, sources[], confidence, escalated, latency_ms }

• Presentation layer — React/Vite chat UI on Vercel; renders answers, source chips, confidence, and live stats.
• API layer — FastAPI with Pydantic validation, CORS, and per-request latency tracking; the boundary between client and pipeline.
• Retrieval layer — query embedded locally and matched against the FAISS index (top-k cosine) built from chunked .txt knowledge-base docs.
• Reliability gate — confidence (top similarity) is checked before generation; low scores short-circuit to a human-escalation response.
• Generation layer — retrieved chunks are injected into a grounded prompt sent to Groq (Llama 3.3 70B), which answers using only the provided context.

---

API Reference

Method	Endpoint	Description
POST	/chat	Answer a user message → answer, sources, confidence, escalated, latency_ms
GET	/stats	Live in-memory metrics — total queries, avg latency, escalation rate
GET	/health	Liveness check + loaded embedding model / LLM info
GET	/docs	Auto-generated interactive Swagger UI

POST /chat request body

{ "message": "How do I track my order?" }

Response

{
  "answer": "You can track your order from your account under 'Orders' …",
  "sources": ["Order Tracking", "Shipping Info"],
  "confidence": 0.71,
  "escalated": false,
  "latency_ms": 412.3
}

---

How It Works

1. Chunk + embed — knowledge-base .txt docs are split into overlapping chunks (400 chars / 80 overlap) and embedded locally with all-MiniLM-L6-v2 into a FAISS index.
2. Retrieve — the user query is embedded and the top-k=4 most similar chunks are pulled via cosine similarity.
3. Grounded generation — retrieved chunks are injected into a constrained prompt; Groq (Llama 3.3 70B) answers using only that context and returns the source documents.
4. Confidence-based escalation — if the top similarity is below the 0.32 threshold, the request is escalated to a human instead of generating a low-quality answer.

---

⚙️ Local Setup

Prerequisites: Python 3.11+, Node.js 18+, a free Groq API key.

Backend

cd backend
pip install -r requirements.txt
cp .env.example .env          # then add your GROQ_API_KEY
uvicorn main:app --reload --port 7860

API → http://localhost:7860 · Swagger → http://localhost:7860/docs

Frontend

cd frontend
npm install
cp .env.example .env.local    # set VITE_API_URL=http://localhost:7860
npm run dev

Variable	File	Description
GROQ_API_KEY	backend/.env	Groq API key for LLM generation
VITE_API_URL	frontend/.env.local	Backend base URL

---

🚀 Deployment

• Backend → Hugging Face Spaces (Docker). backend/Dockerfile builds from python:3.10-slim, installs CPU-only PyTorch, and pre-caches the embedding model at build time; it runs uvicorn main:app --host 0.0.0.0 --port 7860. GROQ_API_KEY is set in HF Spaces → Settings → Repository Secrets.
• Frontend → Vercel. Built with npm run build; set VITE_API_URL to the live Hugging Face Space URL via Vercel environment variables.
• Secrets (.env files, API keys) are git-ignored and never committed.

---

License

MIT — see LICENSE