A production-quality, fully open-source Retrieval-Augmented Generation (RAG) chatbot.
- 100% Open Source: No proprietary dependencies or SaaS tools
- Model Agnostic: Works with Claude, Gemini, GPT, or local Ollama models
- Grounded Answers: Strict prompts prevent hallucination
- Citation Support: Every answer includes source references
- Local Embeddings: BAAI/bge-base-en-v1.5 runs on your machine
- Multiple Formats: Ingest PDF, Markdown, and plain text
- Production Ready: FastAPI, type hints, logging, and clean architecture
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Document │────▶│ Embedding │────▶│ ChromaDB │
│ Ingestion │ │ Service │ │ Vector Store │
└─────────────────┘ └──────────────────┘ └────────┬────────┘
│
┌─────────────────┐ ┌──────────────────┐ ┌────────▼────────┐
│ FastAPI │────▶│ RAG Pipeline │────▶│ Retriever │
│ /chat │ │ │ │ │
└─────────────────┘ └────────┬─────────┘ └─────────────────┘
│
┌────────▼─────────┐
│ LLM Provider │
│ (Pluggable) │
└──────────────────┘
# Clone the repository
cd C:\Users\dhev0\.gemini\antigravity\scratch\rag-chatbot
# Create virtual environment
python -m venv venv
venv\Scripts\activate # Windows
# source venv/bin/activate # Linux/Mac
# Install dependencies
pip install -e .# Copy example config
copy .env.example .env
# Edit .env with your settings
# For local operation (recommended), use Ollama:
# LLM_PROVIDER=ollama
# OLLAMA_MODEL=llama3.2# Install Ollama from https://ollama.ai/download
# Then pull a model:
ollama pull llama3.2
# Ollama runs automatically after installation# Run demo ingestion
python examples/ingest_documents.py --demo
# Or ingest your own files
python examples/ingest_documents.py ./docs/manual.pdf ./docs/guide.mdpython -m uvicorn src.rag.main:app --reload --port 8000# Via curl
curl -X POST http://localhost:8000/api/v1/chat \
-H "Content-Type: application/json" \
-d '{"query": "What is RAG?"}'
# Via example script
python examples/chat_example.py "What is RAG?"
# Interactive mode
python examples/chat_example.py| Endpoint | Method | Description |
|---|---|---|
/api/v1/health |
GET | Health check |
/api/v1/stats |
GET | Vector store statistics |
/api/v1/ingest |
POST | Ingest documents |
/api/v1/chat |
POST | RAG query |
/api/v1/clear |
DELETE | Clear vector store |
/docs |
GET | Interactive API docs |
{
"file_path": "./documents/manual.pdf"
}
// OR
{
"content": "Your document text here...",
"source": "document_name"
}{
"query": "What is RAG?",
"top_k": 5,
"include_context": true
}{
"answer": "RAG (Retrieval-Augmented Generation) is a technique that... [Source: DOC-1]",
"citations": [
{"source_id": "DOC-1", "source_name": "rag_guide.md", "page": null}
],
"context_chunks": [
{
"chunk_id": "rag_guide_abc123",
"source": "rag_guide.md",
"score": 0.89,
"content_preview": "RAG combines retrieval with generation..."
}
],
"query": "What is RAG?",
"llm_provider": "Ollama"
}All settings via environment variables (.env file):
| Variable | Default | Description |
|---|---|---|
LLM_PROVIDER |
ollama |
ollama, claude, gemini, openai |
OLLAMA_MODEL |
llama3.2 |
Model for Ollama |
EMBEDDING_MODEL |
BAAI/bge-base-en-v1.5 |
HuggingFace embedding model |
CHROMA_PERSIST_PATH |
./data/chromadb |
Vector store location |
CHUNK_SIZE |
1500 |
Chunk size in characters |
CHUNK_OVERLAP |
200 |
Overlap between chunks |
TOP_K_RESULTS |
5 |
Number of chunks to retrieve |
# Install from https://ollama.ai/download
ollama pull llama3.2
# That's it! Server runs automatically# Set in .env
LLM_PROVIDER=claude
ANTHROPIC_API_KEY=your_key_here
CLAUDE_MODEL=claude-3-haiku-20240307# Set in .env
LLM_PROVIDER=gemini
GOOGLE_API_KEY=your_key_here
GEMINI_MODEL=gemini-1.5-flash# Set in .env
LLM_PROVIDER=openai
OPENAI_API_KEY=your_key_here
OPENAI_MODEL=gpt-4o-miniThis system uses strict grounding to prevent hallucinations:
- System Prompt: Explicitly forbids using external knowledge
- Citation Requirement: Every claim must reference a source
- "I Don't Know": Model instructed to admit when context is insufficient
- Metadata Tracking: Full source traceability for verification
The grounding prompt is defined in src/rag/rag/prompts.py and can be customized.
# src/rag/ingestion/loaders.py
class DocxLoader(BaseLoader):
def load(self, source: str | Path) -> list[Document]:
# Your implementation
pass# src/rag/llm/my_provider.py
class MyLLM(BaseLLM):
@property
def provider_name(self) -> str:
return "MyProvider"
async def generate(self, system_prompt: str, user_message: str) -> str:
# Your implementation
passThe system uses ChromaDB, but you can implement a new vector store:
# src/rag/vectorstore/qdrant.py
class QdrantVectorStore:
def add_chunks(self, chunks: list[Chunk]) -> int: ...
def search(self, query: str, top_k: int) -> list[SearchResult]: ...rag-chatbot/
├── src/rag/
│ ├── api/ # FastAPI routes and schemas
│ ├── embeddings/ # Local embedding service
│ ├── ingestion/ # Document loaders, cleaners, chunkers
│ ├── llm/ # LLM provider abstraction
│ ├── rag/ # Core RAG pipeline
│ ├── vectorstore/ # ChromaDB integration
│ ├── config.py # Configuration management
│ └── main.py # FastAPI application
├── examples/ # Usage examples
├── tests/ # Test suite
├── pyproject.toml # Python dependencies
└── .env.example # Configuration template
# Run tests
pip install -e ".[dev]"
python -m pytest tests/ -v
# Type checking
python -m mypy src/MIT License - see LICENSE for details.
- ChromaDB - Open-source vector database
- sentence-transformers - Local embeddings
- Ollama - Local LLM inference
- FastAPI - Modern API framework