Agentic RAG Knowledge Assistant

A course-material Q&A assistant that decides for itself whether a question needs document search before answering — instead of running every question through the same retrieve-then-answer pipeline like a typical RAG chatbot.

The problem

Course syllabi are long, dense PDFs. Students end up scanning through pages to find a single answer — "how much is the final worth?", "what textbook do I need?" — that's buried in a wall of text. A simple keyword search (Ctrl+F) fails when the student's wording doesn't match the document's wording.

Target user: a university student who wants direct answers to course logistics questions, with a pointer to exactly where in the syllabus that answer came from — so they can verify it themselves.

What makes this "agentic" rather than a standard RAG chatbot

A standard RAG pipeline sends every question through retrieval, no matter what. This assistant instead gives Claude a search tool and lets it decide, per question, whether searching is warranted. For example:

• "What should a syllabus include?" → Claude calls the retrieval tool, pulls matching passages, and answers with (Source: file, page N) citations.
• "What is the capital of France?" → Claude answers directly and notes that the answer isn't from the course materials — no unnecessary retrieval call, no irrelevant context wasted.

This decision isn't hardcoded with if/else logic — it's driven entirely by the system prompt instructing the model on when to use the tool, which the model then follows autonomously.

How it works

flowchart TD
    subgraph Ingest["Offline: ingest.py"]
        PDF[PDF in data/] --> Chunk[Page-based chunking + overlap]
        Chunk --> Store[(ChromaDB<br/>local vector store)]
    end

    subgraph Runtime["Runtime"]
        User[User: CLI or Web UI] --> Agent[Claude Agent SDK<br/>system prompt decides]
        Agent -- "needs course info?" --> Tool[retrieve_documents tool<br/>via in-process MCP server]
        Tool --> Store
        Store -- "chunks + distance score" --> Tool
        Tool --> Agent
        Agent -- "weak match?" --> Agent
        Agent -- "answer + Source, page N" --> User
    end

Loading

Tech stack

Component	Choice	Why
Agent runtime	Claude Agent SDK (Python)	Built-in agentic loop and tool-calling — no need to hand-roll a decision loop.
Vector store	ChromaDB	Runs locally with zero setup, includes a built-in local embedding model, so the MVP needs no separate embedding API key or cost.
PDF parsing	pypdf	Simple, dependency-light text extraction with per-page access.
Chunking	Page-based, with character overlap	Keeps page-number metadata for free (needed for citations) while still handling long pages via overlap-based sub-splitting.

Project structure

.
├── data/                # source PDFs (not committed except a sample)
├── chroma_db/            # persisted vector store (git-ignored, generated by ingest.py)
├── ingest.py             # load PDFs -> chunk -> embed -> store in ChromaDB
├── tools.py              # retrieve_documents tool, exposed via an in-process MCP server
├── main.py                # CLI chat loop using the Claude Agent SDK
├── app.py                 # Gradio web chat interface
└── requirements.txt

Setup

git clone https://github.com/burakcoleman/agentic-rag-knowledge-assistant.git
cd agentic-rag-knowledge-assistant

python3 -m venv venv
source venv/bin/activate      # Windows: venv\Scripts\activate

pip install -r requirements.txt

Create a .env file in the project root with your Anthropic API key:

ANTHROPIC_API_KEY=sk-ant-...

Usage

1. Drop one or more PDFs into data/ (a sample syllabus is included).
2. Build the vector store:

   python3 ingest.py

3. Start the assistant:

   python3 main.py

4. Ask questions. Type exit to quit.

Example

You: What should a syllabus include?
Assistant: Based on the course materials, a syllabus should include the
following required elements (per the syllabus checklist): Course Title,
Course Number, Course Credits, Course Description, Course Goals, Student
Learning Outcomes, Textbook and Supplies, Course Requirements, Grading...
(Source: sample-syllabus.pdf, page 7; Source: sample-syllabus.pdf, page 1)

You: What is the capital of France?
Assistant: The capital of France is Paris.
Note: this is general knowledge and isn't drawn from the ingested course
materials, which cover syllabus/course documentation.