QualiaQC

Audio Intelligence QA Platform for Support Calls

Features • Screenshots • Installation • Usage • Architecture • Rubric • LLM Integration • Contributing

---

🎯 Overview

QualiaQC is an open-source, audio-first AI QA reviewer for customer support calls. It ingests call recordings, performs local transcription and sentiment analysis, and uses a local LLM to generate structured QA scores and coaching feedback.

Key Highlights

• 🔒 100% Local – No cloud APIs, your data never leaves your machine
• 🎯 Evidence-Linked Findings – Every score is backed by specific transcript moments
• 📊 Sentiment Timeline – Track emotional trajectory for both agent and customer
• 🎓 Actionable Coaching – Not just scores, but specific improvement suggestions
• 🔌 Modular Rubrics – Customize evaluation criteria with versioned YAML configs

---

📸 Screenshots

Dashboard

The main dashboard showing call quality metrics, score distribution, and recent calls overview.

Calls List

Browse and search all processed calls with status, duration, and score information.

Upload Modal

Drag & drop audio file upload with supported format validation and progress tracking.

Note: Additional screenshots showing call analysis, sentiment timeline, and coaching insights are available after processing a call.

---

✨ Features

Audio Ingestion

• Drag & Drop Upload – Modern UI with progress indicators
• Local API – REST endpoints for programmatic ingestion
• Watch Folder – Automatic processing of new audio files
• Format Support – WAV, MP3, M4A

Transcription

• Local Whisper – Using faster-whisper for optimized performance
• Speaker Diarization – Separate agent vs customer speech
• Timestamped Segments – Click-to-jump navigation

Sentiment Analysis

• Dual-Track Analysis – Agent and customer sentiment separately
• Timeline Visualization – See emotional trajectory over time
• Trend Detection – Improving, stable, or declining sentiment
• Confidence Scores – Know how reliable each assessment is

QA Evaluation

• Local LLM – Powered by Ollama (Llama, Mistral, etc.)
• Strict JSON Output – Structured, parseable results
• Category Scoring – 7 categories with weighted scores
• Evidence-Linked Findings – Every finding cites specific quotes
• Coaching Insights – Specific suggestions with improved phrasing

UI/UX

• Modern Dark Theme – Premium SaaS-quality design
• Timeline Transcript – Color-coded by speaker with sentiment
• Score Badges – Visual indicators for performance levels
• Expandable Findings – Drill into evidence and coaching tips
• Keyboard Accessible – Full keyboard navigation support

---

🚀 Installation

Prerequisites

• Python 3.10+ (required for backend)
• Node.js 18+ (required for frontend)
• Ollama (required for LLM evaluation) – Install Ollama
• FFmpeg (required for audio processing) – Install FFmpeg

Step 1: Clone the Repository

git clone https://github.com/ryanshatz/QualiaQC.git
cd QualiaQC

Step 2: Install Ollama Model

# Pull a capable model (Llama 3 recommended)
ollama pull llama3:8b

# Or for better quality (requires more RAM)
ollama pull llama3:70b

# Ensure Ollama is running
ollama serve

Step 3: Start the Backend

cd backend

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Copy environment config
cp .env.example .env

# Initialize database
python -c "from app.database import init_db; init_db()"

# Start server
uvicorn app.main:app --reload --port 8000

Step 4: Start the Frontend

cd frontend

# Install dependencies
npm install

# Start development server
npm run dev

Step 5: Access the Application

Open http://localhost:3000 in your browser.

---

💻 Usage

Processing a Call

1. Click "Upload Call" in the header
2. Drag & drop an audio file or click to browse
3. Select the rubric to use (default provided)
4. Click "Upload & Analyze"
5. Wait for processing to complete (status updates shown)

Viewing Results

Once processing completes:

1. Transcript Tab – View conversation with speaker labels and sentiment
2. Analysis Tab – See category scores with expandable findings
3. Coaching Tab – Get actionable improvement suggestions

Demo Mode

Process sample audio to test the system:

cd backend
python scripts/process_demo.py

---

🏗️ Architecture

QualiaQC/
├── backend/                    # Python FastAPI backend
│   ├── app/
│   │   ├── api/               # REST API endpoints
│   │   ├── core/              # Configuration, rubrics
│   │   ├── models/            # SQLAlchemy models
│   │   ├── processors/        # Audio, transcription, sentiment
│   │   ├── schemas/           # Pydantic schemas
│   │   └── services/          # Business logic
│   ├── rubrics/               # YAML rubric definitions
│   ├── tests/                 # Backend tests
│   └── scripts/               # Utility scripts
│
├── frontend/                   # Next.js frontend
│   ├── src/
│   │   ├── app/               # Next.js app router
│   │   ├── components/        # React components
│   │   └── lib/               # API client, types, utils
│   └── public/                # Static assets
│
├── samples/                    # Sample audio files
└── screenshots/                # Application screenshots

Technology Stack

Layer	Technology
Frontend	Next.js 14, React 18, TypeScript
Styling	CSS Modules, Custom Properties
Icons	Lucide React
Backend	FastAPI, SQLAlchemy, Pydantic
Database	SQLite
Transcription	faster-whisper
Sentiment	HuggingFace Transformers
LLM	Ollama (local)
Audio	pydub, librosa

Data Flow

┌──────────┐     ┌──────────────┐     ┌───────────────┐     ┌──────────────┐
│  Audio   │────►│ Transcription│────►│   Sentiment   │────►│    LLM QA    │
│  Input   │     │   (Whisper)  │     │   Analysis    │     │  Evaluation  │
└──────────┘     └──────────────┘     └───────────────┘     └──────────────┘
                        │                     │                     │
                        ▼                     ▼                     ▼
                 ┌──────────────────────────────────────────────────────┐
                 │                     SQLite DB                        │
                 │  • calls • transcripts • sentiments • evaluations    │
                 └──────────────────────────────────────────────────────┘

---

📜 Rubric

QualiaQC includes a comprehensive default rubric with 7 evaluation categories:

Category	Weight	Description
Greeting & Professionalism	10%	Proper greeting and professional tone
Issue Understanding	15%	Thoroughly understands customer issue
Accuracy & Clarity	20%	Provides accurate, clear information
Empathy & Tone	20%	Demonstrates understanding of emotions
Call Structure & Control	10%	Maintains appropriate conversation flow
Resolution Effectiveness	15%	Achieves appropriate resolution
Closing & Next Steps	10%	Proper closing with follow-up

Customizing Rubrics

Edit backend/rubrics/default.yaml:

version: "1.0"
name: "Custom Rubric"
categories:
  - id: EMPATHY
    name: "Empathy & Tone"
    weight: 20
    description: "Agent demonstrates understanding and care"
    criteria:
      - "Acknowledges customer emotions"
      - "Uses empathetic language"
    examples:
      strong:
        - "I understand how frustrating that must be..."
      weak:
        - "That's just our policy."
    coaching_guidance: |
      Acknowledge emotions before solving problems.

---

🤖 LLM Integration

QualiaQC uses Ollama for local LLM-powered evaluation:

How It Works

1. Prompt Construction – Transcript + sentiment + rubric combined
2. JSON Mode – LLM outputs strictly structured JSON
3. Schema Validation – Output validated with Pydantic
4. Evidence Linking – Findings reference specific transcript moments

Required Output Schema

{
  "overall_score": 78,
  "category_scores": [
    {
      "category_id": "EMPATHY",
      "score": 65,
      "confidence": 88,
      "findings": [
        {
          "summary": "Agent did not acknowledge customer frustration",
          "evidence": {
            "quote": "That's just our policy.",
            "start_time": "00:04:12",
            "end_time": "00:04:15"
          },
          "impact": "Customer felt dismissed",
          "coaching_tip": "Acknowledge emotion before explaining policy"
        }
      ]
    }
  ],
  "strengths": ["Clear explanation of next steps"],
  "improvements": ["Show more empathy"],
  "suggested_phrases": [
    {
      "text": "I understand why that's frustrating...",
      "confidence": 91
    }
  ]
}

Supported Models

• llama3:8b (recommended)
• llama3:70b (better quality, requires more RAM)
• mistral
• mixtral

---

🧪 Testing

Backend Tests

cd backend

# Run all tests
python -m pytest

# Run with coverage
python -m pytest --cov=app --cov-report=html

# Run specific tests
python -m pytest tests/test_rubric_parsing.py -v

Test Coverage

• Rubric Parsing – YAML loading, validation, weight calculation
• Sentiment Output – Score ranges, trend detection, aggregation
• LLM Schema – Response validation, malformed handling

---

🔌 API Reference

Method	Endpoint	Description
POST	/api/calls/upload	Upload audio file
POST	/api/calls/submit-path	Submit local file path
GET	/api/calls	List all calls
GET	/api/calls/{id}	Get call details
GET	/api/calls/{id}/transcript	Get transcript
GET	/api/calls/{id}/sentiment	Get sentiment data
GET	/api/calls/{id}/evaluation	Get QA evaluation
GET	/api/jobs/{id}/status	Check processing status
DELETE	/api/calls/{id}	Delete a call
GET	/api/rubrics	List available rubrics
GET	/api/rubrics/{name}	Get rubric details
GET	/api/health	Health check

---

📄 License

MIT License – See LICENSE for details.

---

🙏 Acknowledgments

• faster-whisper – Optimized Whisper implementation
• Ollama – Local LLM runtime
• HuggingFace Transformers – NLP models
• Lucide – Beautiful icons
• Next.js – React framework

---

Built with ❤️ for better customer service

© 2026 QualiaQC • v0.1.0