🚀 Agentic Production RAG API

This is a high-performance, asynchronous, and agentic Retrieval-Augmented Generation (RAG) system built with FastAPI, Gradio, and Ollama.

Unlike standard RAG prototypes, this system is engineered for Production Accuracy using a multi-stage retrieval pipeline and Agentic Self-Correction.

---

🏗️ Core Architecture & Features

1. 🧠 Advanced Hybrid Retrieval (Ensemble)

We don't just rely on vectors. Our system uses a Hybrid Search approach:

• Semantic Search (Dense): Captures conceptual meaning using ChromaDB and all-MiniLM-L6-v2.
• Keyword Search (Sparse - BM25): Ensures precise matching for specific names, dates, and rare terminology.
• Cross-Encoder Re-ranking: All candidates are re-scored by a secondary model (bge-reranker-base) to filter out noise and ensure only the Top 3 highest-relevance chunks reach the AI.

2. 🛡️ Agentic Self-Correction (Self-RAG)

The system evaluates its own search results. If the re-ranker score falls below a set Relevance Threshold, the AI triggers a "Safe Fallback" mode, explaining to the user exactly why it hasn't answered, rather than fabricating an answer (hallucination).

3. 🔥 Real-time Performance

• Global Model Caching: Heavy models (embeddings, reranker) are loaded at startup into RAM, reducing per-request latency from 8s to sub-1s.
• Streaming Tokens: Responses are streamed token-by-token directly to the UI using a Python generator.
• PyMuPDF Ingestion: Industrial-grade PDF parsing (10-20x faster than standard PyPDF).

4. 🗄️ Production Persistence

• SQL Session Memory: Conversational history is stored in SQLite (via SQLAlchemy) to provide persistent context across sessions.
• Alembic Migrations: Full database version control to manage schema evolution.
• Celery + Redis: Asynchronous background processing ensures the UI stays responsive while large documents are being vectorized.

---

🛠️ Technical Stack

• Backend: FastAPI
• Intelligence: Ollama (Llama 3.2), LangChain, Sentence-Transformers
• Vector DB: ChromaDB
• Relational DB: SQLite + SQLAlchemy + Alembic
• Task Queue: Celery + Redis
• Frontend: Gradio (Mounted to FastAPI)

---

🚀 Installation & Setup

1. Prerequisites

• Ollama (Run ollama serve and ollama pull llama3.2)
• Redis (Run via docker-compose up -d)

2. Install Dependencies

python -m venv .venv
source .venv/bin/activate
uv pip install -r requirements.txt

3. Run Database Migrations

alembic upgrade head

4. Running the System

You need three terminal windows open:

Window 1: Start Redis (Backend Queue)

docker-compose up -d

Window 2: Start the Celery Worker (The "Laborer")

source .venv/bin/activate
celery -A worker.celery_app worker --loglevel=info --pool=solo

Window 3: Start the FastAPI App (The "Brain")

source .venv/bin/activate
uvicorn main:app --reload

---

🌐 How to Use

1. Navigate to http://localhost:8000/chat-ui.
2. Upload your PDF (Wait for the worker to finish).
3. Select your document in the dropdown.
4. Chat with your book! (Notice the real-time streaming and citations).

---

🧪 Testing & Evaluation

This project uses a two-tier testing strategy to ensure both functional reliability and AI accuracy.

1. Functional Tests (Pytest)

Verifies that the API endpoints, Gradio UI, and database connections are working.

• Run command: pytest tests/test_api.py -v

2. RAG Evaluation (DeepEval)

Uses DeepEval with a local Llama 3.2 (Ollama) as a judge to scientifically score the RAG pipeline. We test two stages independently:

A. Retriever Evaluation (tests/test_retriever.py)

Tests if the search engine finds the right context without using the LLM.

• Metrics: Contextual Precision, Contextual Recall, Contextual Relevancy.
• Run command: deepeval test run tests/test_retriever.py

B. Generator Evaluation (tests/test_generator.py)

Tests if the LLM provides truthful answers given perfect context (isolates the "Brain").

• Metrics: Faithfulness (No hallucination), Answer Relevancy.
• Run command: deepeval test run tests/test_generator.py

---

🧪 Future Roadmap

• Implement DeepEval for automated accuracy scoring (Retriever & Generator).
• Add JWT-based Authentication for private multi-user access.
• Dockerize the entire deployment for cloud production.

---

Created by Rahul Thakur 🚀