Core engine behind Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations

An end-to-end AI agent project that transcribes audio files, embeds user queries, and searches in Qdrant and web browser via the Brave API. A Streamlit interface powered by OpenAI GPT models delivers actionable health insights from both the archive and the latest research.

Evaluation of Multi-Agent Systems on Cloud Run with the GenAI Client in Vertex AI SDK

Frontend for Calibrate, a framework for evaluating AI agents: speech-to-text, text-to-speech, LLM evaluation, end-to-end simulations

Test, record, and replay AI agents locally and in CI with a private pytest-style framework

Specification and documentation for the Evaluation Context Protocol

Compare OpenClaw setups against the same scenario suite. Run prompts across multiple configurations, capture answers, latency, token usage, tool calls, and file reads, then generate a single comparison report.

Agentic Workflow Evaluation: Text Summarization Agent. This project includes an AI agent evaluation workflow using a text summarization model with OpenAI API and Transformers library. It follows an iterative approach: generate summaries, analyze metrics, adjust parameters, and retest to refine AI agents for accuracy, readability, and performance.

Open-source framework for evaluating AI agent performance: task completion rate, accuracy, latency, cost efficiency, and reliability metrics for enterprise workflows.

Run AI agent evaluations in your CI/CD pipeline using Calibrate and catch regressions

Reproducible benchmark framework for testing hypotheses about AI coding agents