Not theory. Not tutorials. Engineering.
Curated guides · Production patterns · Real code · Battle-tested architecture
🚀 Start Here · 📖 Browse Topics · 🤝 Contribute · ⭐ Star this repo
Every AI tutorial teaches you to build a chatbot in 10 minutes.
Nobody teaches you what happens at minute 11 — when it hallucinates, costs $4,000/month, fails silently, and your boss is asking why.
This handbook covers the gap between "it works on my laptop" and "it works at 3am under load."
| I want to... | Go here |
|---|---|
| Build my first production RAG system | 📂 RAG Engineering |
| Design a reliable AI agent | 📂 Agent Architecture |
| Stop my LLM from costing a fortune | 📂 Cost & Performance |
| Evaluate my model properly | 📂 Evals & Observability |
| Secure my AI application | 📂 AI Security |
| Clone a working production app | 📂 Starter Kits |
- 🧠 Core LLM Engineering
- 📂 RAG & Retrieval Systems
- 🤖 Agent Engineering
- 📊 Evals & Observability
- 🔒 AI Security
- 💰 Cost & Performance
- 🏗️ Infrastructure & Deployment
- 🎯 Fine-Tuning & Alignment
- 🚀 Starter Kits & Templates
- 📚 Curated Resources
- 🤝 Contributing
- 🏆 Contributors
TL;DR: Master the primitives before you build the system.
| Guide | Description | Difficulty |
|---|---|---|
| Model Comparison Matrix 2026 | GPT-4o vs Claude vs Gemini vs OSS — real benchmarks | 🟢 Beginner |
| Intelligent Model Routing | Route by task type to cut costs 60%+ | 🟡 Intermediate · 🚧 Coming Soon |
| Prompt Architecture Patterns | System prompts, few-shot design, chain-of-thought | 🟡 Intermediate · 🚧 Coming Soon |
| Structured Output Reliability | JSON mode, tool use, schema enforcement | 🟡 Intermediate · 🚧 Coming Soon |
| Context Window Management | Chunking, summarization, long-context strategies | 🔴 Advanced · 🚧 Coming Soon |
| LLM API Error Handling | Rate limits, timeouts, fallbacks, retries | 🟢 Beginner · 🚧 Coming Soon |
📬 Want to write one of these? Claim a topic
TL;DR: RAG is 80% retrieval engineering. Get the pipeline right — the LLM is the easy part.
├── Data changes frequently? → YES: RAG is right
├── Knowledge base > 128k tokens? → YES: RAG is right
├── Need source citations? → YES: RAG is right
├── Static, small, well-defined data? → MAYBE: Consider fine-tuning
└── Real-time data required? → YES: RAG + streaming ingestion
| Guide | Description | Difficulty |
|---|---|---|
| RAG System Design | End-to-end architecture for production RAG | 🟡 Intermediate |
| Chunking Strategies | Fixed, semantic, hierarchical — tradeoffs | 🟡 Intermediate · 🚧 Coming Soon |
| Embedding Model Selection | OpenAI vs Cohere vs BGE vs local models | 🟡 Intermediate · 🚧 Coming Soon |
| Vector Database Comparison | Pinecone vs Weaviate vs Qdrant vs pgvector | 🟡 Intermediate · 🚧 Coming Soon |
| Hybrid Search | BM25 + dense vectors, re-ranking, fusion | 🔴 Advanced · 🚧 Coming Soon |
| Advanced RAG Patterns | HyDE, FLARE, Self-RAG, Corrective RAG | 🔴 Advanced · 🚧 Coming Soon |
Full working kit in starter-kits/rag-production/ — FastAPI + Qdrant + BGE + Claude.
git clone https://github.com/pranavjangam57/ai-engineering-handbook.git
cd ai-engineering-handbook/starter-kits/rag-production
cp .env.example .env # paste your ANTHROPIC_API_KEY
docker-compose up --build # starts the API on :8000 + Qdrant on :6333# Ingest a document
curl -X POST http://localhost:8000/ingest \
-H "Content-Type: application/json" \
-d '{"text": "Your document text here", "source": "my-doc.txt"}'
# Ask a question — get a cited answer
curl -X POST http://localhost:8000/query \
-H "Content-Type: application/json" \
-d '{"question": "What does the document say about X?"}'TL;DR: An agent is an LLM in a loop with tools. The hard part is making the loop reliable.
┌──────────────────────────────────────┐
│ PLANNING LAYER │ ← ReAct / CoT / ToT
└──────────────┬───────────────────────┘
▼
┌──────────────────────────────────────┐
│ TOOL LAYER │ ← APIs, Code exec, Search
└──────────────┬───────────────────────┘
▼
┌──────────────────────────────────────┐
│ MEMORY LAYER │ ← Short + Long term
└──────────────┬───────────────────────┘
▼
┌──────────────────────────────────────┐
│ SAFETY LAYER │ ← Budget caps + human-in-loop
└──────────────────────────────────────┘
| Guide | Description | Difficulty |
|---|---|---|
| Agent Architecture Patterns | ReAct, Plan-and-Execute, Multi-agent, Reflexion | 🟡 Intermediate |
| Tool Design for Agents | Schemas, error handling, safe execution | 🟡 Intermediate · 🚧 Coming Soon |
| Agent Memory Systems | In-context vs vector vs episodic memory | 🔴 Advanced · 🚧 Coming Soon |
| Multi-Agent Orchestration | Supervisor patterns, agent communication | 🔴 Advanced · 🚧 Coming Soon |
| Human-in-the-Loop Design | When to pause, how to escalate | 🟡 Intermediate · 🚧 Coming Soon |
TL;DR: If you can't measure it, you can't ship it. Evals are the unit tests of AI engineering.
┌──────────┐
│ A/B in │ ← Production traffic
│ prod │
/└──────────┘\
/ ┌──────────┐ \
/ │ LLM-as- │ \ ← Automated evals
/ │ judge │ \
/ └──────────┘ \
/ ┌──────────────────┐ \
/ │ Unit evals on │ \ ← Deterministic checks
/ │ golden dataset │ \
/ └──────────────────┘ \
└────────────────────────────────┘
| Guide | Description | Difficulty |
|---|---|---|
| Eval Framework Design | Golden datasets, metrics, LLM-as-judge pipeline | 🟡 Intermediate |
| RAG-Specific Evals | Faithfulness, relevance, context recall | 🔴 Advanced · 🚧 Coming Soon |
| Observability Stack | Langfuse, Helicone, Arize, custom logging | 🟡 Intermediate · 🚧 Coming Soon |
| Regression Testing for LLMs | Catching prompt regressions on deploy | 🔴 Advanced · 🚧 Coming Soon |
TL;DR: AI systems have an entirely new attack surface. Most teams don't know it exists.
| # | Vulnerability | Severity | Guide |
|---|---|---|---|
| 1 | Prompt Injection | 🔴 Critical | Guide → |
| 2 | Insecure Output Handling | 🔴 Critical | 🚧 Coming Soon |
| 3 | Training Data Poisoning | 🟠 High | 🚧 Coming Soon |
| 4 | Model Denial of Service | 🟠 High | 🚧 Coming Soon |
| 5 | Supply Chain Vulnerabilities | 🟠 High | 🚧 Coming Soon |
| 6 | Sensitive Information Disclosure | 🟠 High | 🚧 Coming Soon |
| 7 | Insecure Plugin Design | 🟡 Medium | 🚧 Coming Soon |
| 8 | Excessive Agency | 🟡 Medium | 🚧 Coming Soon |
| 9 | Overreliance | 🟡 Medium | 🚧 Coming Soon |
| 10 | Model Theft | 🟡 Medium | 🚧 Coming Soon |
TL;DR: The best AI system is the one that doesn't bankrupt you.
| Guide | Description | Potential Savings |
|---|---|---|
| Model Routing by Task | Use small models for simple tasks | 40–70% · 🚧 Coming Soon |
| Prompt Caching | Cache system prompts, reduce input tokens | 20–50% · 🚧 Coming Soon |
| Semantic Caching | Cache similar queries with vector lookup | 30–60% · 🚧 Coming Soon |
| Batching Strategies | Async batch APIs for non-real-time workloads | 50% flat · 🚧 Coming Soon |
| Guide | Description | Difficulty |
|---|---|---|
| LLM Gateway Architecture | Rate limiting, auth, routing, logging | 🔴 Advanced · 🚧 Coming Soon |
| Self-Hosting OSS Models | vLLM, Ollama, TGI on GPU infra | 🔴 Advanced · 🚧 Coming Soon |
| CI/CD for AI Systems | Eval gates, prompt versioning, rollbacks | 🟡 Intermediate · 🚧 Coming Soon |
| Guide | Description | Difficulty |
|---|---|---|
| When NOT to Fine-Tune | The decision tree before you spend $10k | 🟢 Beginner · 🚧 Coming Soon |
| LoRA & QLoRA Guide | Parameter-efficient fine-tuning in practice | 🔴 Advanced · 🚧 Coming Soon |
| Dataset Curation | Building high-quality training sets | 🟡 Intermediate · 🚧 Coming Soon |
Clone → Configure → Deploy. No boilerplate. No setup hell.
| Kit | Stack | Status | Deploy |
|---|---|---|---|
| Production RAG App | FastAPI + Qdrant + BGE + Claude | ✅ Ready |  |
| AI Chat with Memory | Next.js + Supabase + Claude | 🚧 Coming Soon | — |
| Agent Backend API | FastAPI + Celery + Redis | 🚧 Coming Soon | — |
| LLM Eval Pipeline | Python + Langfuse + RAGAS | 🚧 Coming Soon | — |
A fully working REST API that you can point at your own documents:
starter-kits/rag-production/
├── main.py ← FastAPI: /ingest, /ingest/batch, /query, /health
├── src/rag.py ← Pipeline: embed → retrieve → rerank → generate
├── src/chunker.py ← Token-based + markdown-aware chunking
├── src/models.py ← Typed Pydantic request/response schemas
├── docker-compose.yml ← One command: API + Qdrant vector DB
├── Dockerfile
├── requirements.txt
└── .env.example ← Copy to .env, add your API key, done
Real use cases you can build on top of this today:
- Internal knowledge base Q&A
- Document search with citations
- Customer support bot grounded in your docs
- Research assistant over a PDF library
📰 Must-Read Papers
| Paper | Year | Why It Matters |
|---|---|---|
| Attention Is All You Need | 2017 | The transformer architecture that started everything |
| RAG: Retrieval-Augmented Generation | 2020 | Foundational RAG paper |
| ReAct: Reasoning and Acting | 2022 | The basis of most agent frameworks today |
| Constitutional AI | 2022 | How to align models at scale |
| LLM-as-a-Judge | 2023 | Using LLMs to evaluate LLMs |
| Self-RAG | 2023 | Adaptive retrieval augmentation |
🎥 Best Video Courses
| Course | Provider | Level | Cost |
|---|---|---|---|
| Deep Learning Specialization | DeepLearning.AI | Beginner | Paid |
| LLMOps | DeepLearning.AI | Intermediate | Free |
| Building with Anthropic | Anthropic | Intermediate | Free |
| Full Stack LLM Bootcamp | FSDL | Advanced | Free |
🛠️ Essential Tools & Frameworks
| Tool | Category | Use Case |
|---|---|---|
| LangChain | Orchestration | Chains, agents, RAG pipelines |
| LlamaIndex | RAG | Data indexing and retrieval |
| Langfuse | Observability | Tracing, evals, prompt management |
| RAGAS | Evaluation | RAG-specific evaluation metrics |
| Qdrant | Vector DB | High-performance vector search |
| vLLM | Inference | High-throughput LLM serving |
| Instructor | Structured Output | Type-safe LLM outputs |
| DSPy | Optimization | Programmatic prompt optimization |
This handbook grows through community contributions. Every guide written, every bug fixed, every resource added helps thousands of engineers.
Read CONTRIBUTING.md before opening a PR.
| How to contribute | What it takes |
|---|---|
| ✍️ Write a 🚧 Coming Soon guide | 2–4 hours |
| 🐛 Fix incorrect content | 15 minutes |
| 🔗 Add a curated resource | 5 minutes |
| 💬 Start a discussion | 2 minutes |
Every contributor is listed here automatically.
MIT — see LICENSE. Share freely. Credit kindly. Build boldly.
If this saved you time, saved you money, or saved you from a 3am incident — ⭐ star this repo.
It takes 2 seconds and helps thousands of engineers find it.