Local Interview Helper with LLM

A local, real-time interview copilot. Listens to your meeting audio, transcribes via Whisper, and a local LLM ("Sofia") suggests answers in a floating always-on-top window — personalized to your CV.

No meeting bot. No cloud. No external API keys. Everything runs on your hardware.

What it does

• Captures the system audio (whatever Zoom/Meet/Teams is playing) and optionally your microphone, in parallel — using PulseAudio/PipeWire monitor sources.
• Streams chunks through a VAD (voice activity detector) to a self-hosted faster-whisper-server running on your GPU.
• Sends the rolling transcript to a self-hosted Ollama model (the "Sofia" persona) with your resume injected into the system prompt.
• Renders Sofia's streamed answer in a frameless always-on-top PyQt6 window.
• Quick-action buttons to refine the last answer (Aprofundar / Mais curto / STAR / Reformular).
• Saves every session to sessions/YYYY-MM-DD-HHMM.md for later review.

Architecture

   ┌────────────────────────┐
   │  Floating UI (PyQt6)   │  ← you interact here
   │  drag, always-on-top   │
   └────────┬───────────────┘
            │ chunks (16kHz mono int16)
   ┌────────▼───────────────┐
   │  parec (PulseAudio)    │  ← captures monitor + mic in parallel
   │  + WebRTC VAD          │
   └────────┬───────────────┘
            │ POST /v1/audio/transcriptions
   ┌────────▼───────────────┐
   │ faster-whisper-server  │  ← Docker, CUDA, OpenAI-compatible API
   │ (your GPU host)        │
   └────────┬───────────────┘
            │ transcribed text → appended to transcript buffer
   ┌────────▼───────────────┐
   │  You click ✨ Responder │
   └────────┬───────────────┘
            │ POST /api/chat (streaming)
   ┌────────▼───────────────┐
   │  Ollama  (your GPU)    │  ← Sofia = persona on a chat model
   └────────────────────────┘

Single-machine setup is fine if you have a GPU. The author runs Whisper + Ollama on one GPU host and the UI on the same machine — but the UI talks to both via HTTP, so you can split them across machines on your LAN.

Prerequisites

Hardware on the GPU host:

• NVIDIA GPU with at least 10 GB VRAM (Whisper turbo ≈ 3 GB + Qwen 9B Q4 ≈ 6 GB).
• 16 GB system RAM, ~10 GB free disk.

Software on the GPU host:

• Linux with NVIDIA drivers + CUDA runtime.
• Docker + Docker Compose.
• nvidia-container-toolkit configured as a docker runtime.
• Ollama installed and listening on 0.0.0.0:11434.

Software on the UI host (same or different machine):

• Linux with PulseAudio or PipeWire (PipeWire with pipewire-pulse works too).
• pulseaudio-utils (provides pactl and parec).
• X11 session (the floating-window hint is X11-flavored — Wayland support is compositor-dependent).
• Python 3.11+ and uv.
• Qt6 system libs (libqt6gui6, libxkbcommon-x11-0, libxcb-cursor0).
• portaudio (libportaudio2).

Setup

1. Whisper (on the GPU host)

mkdir -p ~/whisper-server && cd ~/whisper-server
cat > docker-compose.yml <<'EOF'
services:
  whisper:
    image: fedirz/faster-whisper-server:latest-cuda
    container_name: whisper-server
    restart: unless-stopped
    ports:
      - "0.0.0.0:8000:8000"
    volumes:
      - whisper-cache:/root/.cache/huggingface
    environment:
      - WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
      - WHISPER__COMPUTE_TYPE=float16
      - WHISPER__DEVICE=cuda
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
volumes:
  whisper-cache:
EOF
docker compose up -d

The first transcription request triggers the model download (~1.5 GB) and load (~15 s cold start). Subsequent requests run in 200–500 ms.

Verify:

curl http://localhost:8000/health      # 200 OK
curl http://localhost:8000/v1/models   # JSON list

2. Ollama + Qwen (on the GPU host)

ollama pull qwen3.5:9b
# Make sure ollama listens on all interfaces if the UI is on another machine:
#   sudo systemctl edit ollama
#   [Service]
#   Environment="OLLAMA_HOST=0.0.0.0"

Other Ollama-compatible chat models work too — just change SOFIA_MODEL in .env. Models with a think: true default (Qwen3-thinking, etc.) work because llm.py already sends think: false to keep latency interview-grade.

3. The app (on the UI host)

git clone https://github.com/<you>/interview_helper.git
cd interview_helper

./scripts/setup.sh        # installs apt deps + uv + Python deps
cp .env.example .env      # then edit endpoints if Whisper/Ollama are on another machine
cp profile.example.md profile.md
# Edit profile.md with your actual CV — Sofia uses this verbatim.

./scripts/run.sh

Usage

The window is frameless and stays on top. Drag it from anywhere on its surface.

Control	What it does
Idioma	Output language: en-US (default), pt-BR, pt-PT, auto.
Fonte	PulseAudio source for the interviewer's voice — pick the 🔊 …monitor of whichever sink your meeting audio is playing out on.
Eu também	Also capture your microphone as a second stream. Lines are tagged [Eu] vs [Outro] so Sofia knows who said what.
● Gravar	Start/stop capture. Transcript appends in real time.
✨ Responder	Send the current transcript to Sofia. Streamed answer fills the lower panel.
🗑	Clear transcript + answer.
Aprofundar / Mais curto / STAR / Reformular	One-click refinements of the previous answer.

Picking the right audio source

The monitor source captures whatever audio is playing out of a given sink. To get the interviewer's voice:

pactl info | grep "Default Sink"
# pick the corresponding "<that-sink>.monitor" in the Fonte dropdown

If your headphones/speakers are on a different sink than the one Zoom/Meet is playing to, you'll capture nothing. Either change your meeting output, or pick the monitor of the sink the meeting actually uses.

Tuning

All knobs live in .env. The defaults are reasonable; common tweaks:

• SILENCE_TIMEOUT — lower = snappier, higher = waits for full sentences (default 0.6).
• MAX_SPEECH — force-flush after this much continuous speech, otherwise the buffer never closes on non-stop talkers (default 8.0).
• MIN_RMS — chunks quieter than this RMS are dropped before they reach Whisper. Raises if you see Whisper hallucinating "Thank you." / "Subscribe." on quiet audio.
• WHISPER_MODEL — try Systran/faster-whisper-medium.en for snappier English-only transcription, Systran/faster-whisper-large-v3 for max quality.

Troubleshooting

Window appears but does nothing when I click Record. Capture is running but no chunks reach Whisper. Likely causes: wrong Fonte source (try a 🔊 ...monitor not a 🎤 ... mic), or MIN_RMS too high.

Whisper returns 500 with Requested float16 compute type, but the target device does not support efficient float16 computation. The container lost GPU access — common after systemctl daemon-reload or installing packages on the host. Fix:

cd ~/whisper-server && docker compose restart

Transcription is full of Thank you. / Subscribe. lines. Whisper hallucinations on near-silence. Raise MIN_RMS in .env (try 120 or 150). The pipeline also drops a known list of these strings.

Sofia's responses are slow / take >5 s for the first token. Cold start of the chat model. Pre-warm it once at the start:

curl http://<ollama-host>:11434/api/chat \
  -d '{"model":"qwen3.5:9b","messages":[{"role":"user","content":"hi"}],"think":false,"stream":false}'

Wayland session — window doesn't stay on top. The WindowStaysOnTopHint flag is a hint that some Wayland compositors honor and some don't. X11 always works.

Layout

interview_helper/
├── README.md
├── pyproject.toml
├── .env.example
├── profile.example.md      ← copy to profile.md and edit
├── src/interview_helper/
│   ├── config.py           pydantic-settings, reads .env
│   ├── persona.py          Sofia's system prompt (loads profile.md)
│   ├── audio.py            parec + WebRTC VAD, multi-source with labels
│   ├── stt.py              faster-whisper-server client (OpenAI-compat)
│   ├── llm.py              Ollama streaming client (think: false)
│   ├── pipeline.py         async orchestration + hallucination filter
│   ├── session.py          append-only Markdown session log
│   ├── ui.py               PyQt6 floating overlay
│   └── __main__.py
├── scripts/
│   ├── setup.sh            one-time install
│   └── run.sh              launch the UI
└── sessions/               (gitignored) per-run transcripts + Q&A

License

MIT. See LICENSE.

Acknowledgements

• faster-whisper-server — the OpenAI-compatible Whisper container.
• faster-whisper / CTranslate2 under it.
• Ollama for local LLM serving.
• WebRTC VAD.