Skip to content

Repository files navigation

PGAI Patient Voice Bot

Automated voice bot that calls Pretty Good AI's test line and simulates realistic patient conversations to evaluate agent quality and find bugs.

Architecture

See ARCHITECTURE.md for full system design and decision rationale.

Quick Start

python make_calls.py

Setup

1. Accounts needed

  • Twilio: twilio.com/try-twilio
  • Deepgram: console.deepgram.com
  • Groq: console.groq.com
  • ngrok: ngrok.com/download

2. Install

pip install -r requirements.txt
cp .env.example .env
# Fill in your keys

3. Start ngrok

ngrok http 8000
# Copy the https URL into PUBLIC_URL in .env

4. Start server

uvicorn main:app --host 0.0.0.0 --port 8000

5. Run all 10 calls

python make_calls.py

6. Download recordings

python download_recordings.py

Scenarios

# Name Tests
1 new_appointment Standard scheduling
2 reschedule Change existing appointment
3 medication_refill Refill workflow
4 cancel_appointment Cancellation handling
5 practice_info Hours and insurance questions
6 weekend_request Edge: closed on weekends
7 elderly_patient Edge: slow confused speaker
8 barge_in_impatient Edge: interruptions
9 reluctant_caller Edge: private health concern
10 controlled_substance Edge: lost prescription

Cost

Under $1 total for all 10 calls.

About

An LLM-driven testing harness that places real phone calls through Twilio, scripts 10 adversarial caller personas via Groq/Llama, and evaluates how a live voice agent actually responds. Full suite runs for under $1.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages