Enterprise Agentic RAG

A production-style Retrieval-Augmented Generation system built on an agentic LangGraph workflow with safety guardrails, semantic reranking, and page-accurate citations. It runs fully locally and for free — Groq for inference, sentence-transformers for embeddings, and an embedded Qdrant vector store — yet keeps the seams (gateway, persistent memory, semantic cache) needed to scale out.

Upload any document — PDFs, Word, PowerPoint, HTML, or plain text — and ask questions about it. The agent grounds every answer in the retrieved context and cites the pages it used.

---

Table of contents

• Highlights
• Architecture
• Quickstart
• Ingesting documents
• API
• Configuration
• Project structure
• Tech stack
• Contributing
• License

---

Highlights

• Agentic pipeline (LangGraph) — a Planner -> Retriever -> Responder graph decides whether a question needs document retrieval or can be answered conversationally, with memory across turns.
• Document-agnostic ingestion — PDF, DOCX, PPTX, HTML, and TXT. PDFs are chunked page-by-page so answers can cite exact page numbers.
• Two-stage retrieval — fast vector search over Qdrant, then a local FlashRank cross-encoder reranks for precision.
• Grounded, faithful answers — the responder is constrained to the retrieved context and instructed to say when something isn't in the documents, instead of hallucinating.
• Safety guardrails (NeMo Guardrails) — blocks off-topic prompts and jailbreak attempts before they reach the pipeline.
• Runs free & offline-friendly — no paid embedding API, no managed database required to start.
• Built to scale — optional Postgres-backed persistent memory and a Redis semantic cache are wired in behind feature flags.

---

Architecture

graph TD
    User((User)) --> UI[Streamlit UI]
    UI -->|POST /query| API[FastAPI]
    API --> Guard{NeMo Guardrails}
    Guard -->|blocked| UI
    Guard -->|pass| Planner{Planner}
    Planner -->|conversational| Responder[Responder]
    Planner -->|needs documents| Retriever[Retriever]
    Retriever --> Qdrant[(Qdrant Vector Store)]
    Retriever --> Reranker[FlashRank Reranker]
    Reranker --> Responder
    Responder -->|grounded answer + pages| UI

    UI -->|POST /ingest| Ingest[Ingestion]
    Ingest --> Parse[Parse + page-aware chunk] --> Embed[sentence-transformers] --> Qdrant

Loading

Request flow: a query passes the guardrails gate, the planner classifies intent, the retriever pulls and reranks the most relevant chunks, and the responder synthesizes an answer strictly from that context — citing pages where available.

---

Quickstart

1. Prerequisites

• Python 3.10+
• A free Groq API key

2. Install

git clone https://github.com/Abhishek500/EnterpriseRag.git
cd EnterpriseRag
python -m venv .venv
# Windows:  .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt

3. Configure

cp .env.example .env      # Windows: copy .env.example .env

Edit .env and set GROQ_API_KEY. Everything else has a working default.

4. Run the backend

uvicorn app.main:app --host 0.0.0.0 --port 8080

5. Run the UI (in a second terminal)

streamlit run ui/app.py

Open http://localhost:8501, upload a document from the sidebar, and start asking questions.

The embedded Qdrant store is held by a single process. If you use ingest_local.py for bulk loading, stop the backend first, then restart it.

---

Ingesting documents

Via the UI: drag files into the sidebar uploader and click Ingest.

Via the API:

curl -F "files=@/path/to/document.pdf" http://localhost:8080/ingest

In bulk from a folder:

python ingest_local.py ./documents

---

API

Method	Endpoint	Description
POST	/query	Ask a question. Body: { "q": "...", "thread_id": "..." }
POST	/ingest	Upload one or more files (multipart) to index
POST	/delete_source	Remove all chunks for a filename. Body: { "source": "..." }
GET	/graph	PNG of the LangGraph agent topology
GET	/health	Health check

Example:

curl -X POST http://localhost:8080/query \
  -H "Content-Type: application/json" \
  -d '{"q": "Summarize the key points of the document", "thread_id": "demo"}'

---

Configuration

All settings are read from environment variables (see .env.example). The most useful:

Variable	Default	Purpose
GROQ_API_KEY	(required)	Groq inference key
GROQ_MODEL	llama-3.3-70b-versatile	Main answer-generation model
EMBEDDING_MODEL	all-MiniLM-L6-v2	sentence-transformers embedding model
QDRANT_URL	(empty -> embedded)	Set to use a Qdrant server / Qdrant Cloud
LOCAL_MODE	true	false enables Postgres-backed persistent memory
USE_SEMANTIC_CACHE	false	true enables the Redis semantic cache
LOGFIRE_TOKEN	(empty)	Set to stream traces to Logfire

---

Project structure

app/
  main.py                  # FastAPI app: /query, /ingest, /delete_source
  config.py                # Environment-driven settings
  agents/
    graph.py               # LangGraph definition + checkpointer
    state.py               # Agent state schema
    nodes/                 # planner, retriever, responder
  guardrails/              # NeMo Guardrails config + gate
  gateway/                 # Provider-agnostic LLM client
  ingestion/chunking/      # Page-aware + plain-text chunkers
  services/
    retrieval/             # Embeddings, Qdrant client, FlashRank reranker
    infra/                 # Optional Postgres memory + Redis cache
ui/app.py                  # Streamlit chat interface
ingest_local.py            # Bulk folder ingestion CLI

---

Tech stack

LangGraph · LangChain · Groq (Llama 3.x) · NeMo Guardrails · Qdrant · sentence-transformers · FlashRank · FastAPI · Streamlit · Logfire

---

Contributing

Contributions are welcome! Please read the contributing guidelines and our Code of Conduct before opening an issue or pull request.

Quick start for contributors:

pip install -r requirements-dev.txt
ruff check .          # lint
pytest -q             # tests

CI runs both on every push and pull request, so please make sure they pass locally before submitting.

Security

Found a vulnerability? Please see SECURITY.md for how to report it responsibly.

License

MIT