Chat PDF: LLM RAG Backend

A FastAPI backend that ingests text/PDFs into Weaviate, uses OpenAI embeddings for retrieval, and answers questions via a chat model using retrieved context.

Features

• Ingest raw text or upload PDFs and store chunked vectors in Weaviate
• Semantic retrieval with OpenAI embeddings
• RAG endpoint to answer questions grounded by retrieved context
• OpenAPI docs available at /docs and /redoc

Quickstart (Docker Compose)

1. Create a .env file in the repo root (copy from example and edit):

cp env.example .env
# then edit .env to set OPENAI_API_KEY and any overrides

Optional overrides (defaults shown):

OPENAI_BASE_URL=https://api.openai.com/v1
WEAVIATE_HOST=weaviate
WEAVIATE_PORT=8080
# Host port mapping for Weaviate (change if 8080 is in use on host)
WEAVIATE_HOST_PORT=8080
API_HOST=0.0.0.0
API_PORT=8000
WORKERS=2
LOG_LEVEL=info
EMBEDDING_MODEL=text-embedding-3-large
CHAT_MODEL=gpt-5-nano
EMBEDDINGS_PROVIDER=openai
FASTEMBED_MODEL=BAAI/bge-small-en-v1.5
TEMPERATURE=0.4

2. Start services:

docker compose up --build

• API: http://localhost:8000
• Docs: http://localhost:8000/docs
• UI: http://localhost:5173
• Weaviate: http://localhost:<WEAVIATE_HOST_PORT> (default 8080)

Data persists in the Docker named volume weaviate_data. To reset Weaviate data (helpful if the container is unhealthy due to stale data):

docker compose down -v
docker volume rm chat-pdf_weaviate_data || true
docker compose up -d --build weaviate

If your UI needs to point to a different API URL, set VITE_API_BASE_URL in .env and rebuild the UI service:

echo 'VITE_API_BASE_URL=http://localhost:8000' >> .env
docker compose up -d --build ui

Web UI (React + Vite + Tailwind)

A lightweight frontend lives in ui/:

Dev setup:

cd ui
npm install
npm run dev

• UI: http://localhost:5173
• API: http://localhost:8000

If your API is not at the default, set:

echo 'VITE_API_BASE_URL=http://localhost:8000' > .env

For Docker builds, set VITE_API_BASE_URL in the repo root .env (not the ui/ folder) and rebuild just the UI service:

docker compose up -d --build ui

Features:

• Upload PDF with drag & drop or file picker (shows progress and success)
• See a local list of uploaded files (client-side)
• Ask Questions in a chat-style UI (answers show sources)
• Chat history is stored in localStorage

Routes:

• / Upload PDF
• /chat Ask Questions

Local Development (without Docker for API)

Run Weaviate via Docker, then run the API locally.

1. Start Weaviate:

docker compose up -d weaviate

2. Create virtualenv & install deps:

cd api
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

3. Set environment variables (at minimum OPENAI_API_KEY):

export OPENAI_API_KEY=your_api_key_here
export OPENAI_BASE_URL=https://api.openai.com/v1
export WEAVIATE_HOST=localhost
export WEAVIATE_PORT=8080
export API_HOST=0.0.0.0
export API_PORT=8000

4. Run the API (hot-reload):

uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

API Overview

1) Ingest raw text

POST /api/ingest

Body:

{
  "id": "optional-id-string",
  "text": "Your document content here...",
  "source": "my-source-label.txt"
}

Example:

curl -X POST http://localhost:8000/api/ingest \
  -H "Content-Type: application/json" \
  -d '{"text":"Hello world text to index","source":"example"}'

2) Upload PDF

POST /api/upload-pdf (multipart form)

Example:

curl -X POST http://localhost:8000/api/upload-pdf \
  -F "file=@/path/to/file.pdf"

The server extracts text, chunks it, embeds, and stores vectors in Weaviate.

3) RAG question answering

POST /api/rag

Body:

{
  "question": "What is discussed in the document?",
  "top_k": 4
}

Example:

curl -X POST http://localhost:8000/api/rag \
  -H "Content-Type: application/json" \
  -d '{"question":"Summarize the key points","top_k":4}'

Configuration

Environment variables (read via Pydantic settings):

• OPENAI_API_KEY (required)
• OPENAI_BASE_URL (default https://api.openai.com/v1)
• WEAVIATE_HOST (default weaviate in Docker; use localhost for local dev)
• WEAVIATE_PORT (default 8080)
• API_HOST (default 0.0.0.0)
• API_PORT (default 8000)
• WORKERS (default 2)
• LOG_LEVEL (default info)
• EMBEDDING_MODEL (default text-embedding-3-large)
• CHAT_MODEL (default gpt-5-nano)
• EMBEDDINGS_PROVIDER (default openai, options: openai or fastembed)
• FASTEMBED_MODEL (default BAAI/bge-small-en-v1.5 when using fastembed)
• TEMPERATURE (default 0.4)
• VITE_API_BASE_URL (UI build-time; default http://localhost:8000 if unset)

Tip: You can also create a .env file (root) to store these values.

Notes

• Schema is ensured at API startup (Document class in Weaviate).
• If you see 502 errors from embedding/chat, verify OPENAI_API_KEY and network egress.
• OpenAPI docs are available at /docs and /redoc once the API is running.

Project Structure

api/
  app/
    main.py           # FastAPI app & endpoints
    settings.py       # Pydantic settings
    rag.py            # OpenAI calls (embeddings/chat)
    weaviate_client.py# Weaviate client & schema/batch ops
    ingest.py         # Text ingestion helper
    pdf_utils.py      # PDF extraction & chunking
  Dockerfile
  requirements.txt
docker-compose.yml