Pinned Loading

1. trailhead-travel-agent-eval trailhead-travel-agent-eval Public

   LLM-agent quality suite for a fictional travel-support bot (single-turn Q&A, multi-turn chatbot, RAG) using DeepEval — correctness, safety, and hallucination metrics scored by a local judge model.

   Python

2. trailhead-travel-rag-eval trailhead-travel-rag-eval Public

   RAG-pipeline evaluation for a fictional travel-support agent using Ragas — faithfulness, context precision/recall, and answer relevancy, with noise-aware CI thresholds.

   Python

3. trailhead-travel-observability trailhead-travel-observability Public

   Observability and CI regression gating for an LLM agent using LangSmith — tracing, versioned eval datasets, a GitHub Actions prompt-regression gate, and production drift monitoring.

   Python

4. trailhead-travel-red-team trailhead-travel-red-team Public

   Automated red-teaming and safety layering for a RAG agent — Promptfoo attack generation, Guardrails AI validation, and Llama Guard classification, with before/after break-rate comparison.

   Python