Evals Observability

Minimal OpenAI API examples using uv.

Setup

uv sync
cp .env.example .env

Edit .env and set OPENAI_API_KEY.

Run

uv run src/01_basics/01_basic_completion.py

Basic Langfuse Eval Cases

uv run src/01_basics/01_basic_completion.py
uv run src/01_basics/02_boolean_score.py
uv run src/01_basics/03_categorical_score.py
uv run src/01_basics/04_multi_case_eval.py

Structured Output Eval Cases

uv run src/02_structured_output/01_basic_structured_completion.py
uv run src/02_structured_output/02_boolean_structured_score.py
uv run src/02_structured_output/03_categorical_structured_score.py
uv run src/02_structured_output/04_multi_case_structured_eval.py

Chunking Eval Cases

uv run src/03_chunking/01_compare_chunkers.py
uv run src/03_chunking/02_eval_chunkers.py

OpenAI Agents Eval Cases

uv run src/04_openai_agents/01_basic_agent.py
uv run src/04_openai_agents/02_agent_with_tool.py
uv run src/04_openai_agents/03_eval_agent_answer.py
uv run src/04_openai_agents/04_eval_agent_tool_use.py

Set these values in .env before running:

OPENAI_API_KEY=...
LANGFUSE_PUBLIC_KEY=...
LANGFUSE_SECRET_KEY=...
LANGFUSE_HOST=https://cloud.langfuse.com