diarizer

A small, self-contained post-call diarization microservice: give it an audio file, get back a speaker-annotated transcript (who said what, with timestamps). Batch, on-prem, GPU.

audio ──▶ ffmpeg (16 kHz mono) ──▶ German-CT2 faster-whisper (STT)
                                       │ segments
                                       ▼
                              pyannote community-1 (exclusive diarization)
                                       │ speaker turns
                                       ▼
                     segment-level max-overlap merge ──▶ [{start, end, text, speaker}]

Built for speaker-attributed meeting protocols (accuracy over latency). Speaker diarization is inherently global/offline — it needs the whole recording to cluster a speaker consistently — so this is a batch service, deliberately decoupled from any real-time transcription path.

API

• GET /health — readiness + config (cuda, models, token present).
• POST /diarize — JSON {audio_url, meeting_id?, segments?, num_speakers?, min_speakers?, max_speakers?, language?}. Fetches audio_url, transcribes + diarizes, returns the annotated transcript. Pass segments (your own [{start,end,text}]) to skip STT and only diarize onto them.
• POST /diarize_upload — the same, but as a multipart upload (file=<audio>, plus optional form fields). Handy when the caller already holds the bytes and would rather not expose a URL.

Response:

{ "meeting_id": "...", "language": "de", "model": "...", "diarizer": "...",
  "num_speakers": 4, "speakers": ["SPEAKER_00", ...],
  "segments": [{ "start": 0.7, "end": 34.5, "text": "...", "speaker": "SPEAKER_00" }],
  "elapsed_s": 138.1 }

See sample-result.example.json for the shape. Real transcripts are gitignored — never commit them.

Models

Stage	Default	Why
STT	Reality-Interface/whisper-large-v3-german-faster-whisper	German-tuned CT2 finetune (drop-in for faster-whisper); far better on German + domain terms than plain large-v3
Diarization	pyannote/speaker-diarization-community-1	exclusive_speaker_diarization → non-overlapping regions, trivial to merge at segment level

Both are gated on Hugging Face — accept their licenses, then supply an HF_TOKEN (see below). Override models via DIA_WHISPER_MODEL / DIA_PYANNOTE_MODEL.

Run

Needs an NVIDIA GPU + nvidia-container-toolkit. Peak VRAM ~8 GB (models load sequentially — Whisper → release → pyannote → release — behind a single-flight lock, so it fits a small slice).

cp .env.example .env         # put your HF_TOKEN in .env (never commit it)
docker compose up -d
curl -s localhost:8008/health | jq

Config (env)

Var	Default	Notes
HF_TOKEN	—	required; access to the two gated models
DIA_WHISPER_MODEL	German CT2 large-v3	any faster-whisper / CT2 model id
DIA_WHISPER_COMPUTE	int8_float16	CTranslate2 compute type
DIA_PYANNOTE_MODEL	community-1	pyannote pipeline id
DIA_PYANNOTE_DEVICE	cuda	cpu to run diarization off-GPU
DIA_ALLOWED_LANGUAGES	de,en	log a warning if the detected language is outside this set
HF_HUB_DISABLE_XET	1 (compose)	gated weights are Xet-backed; the Xet path 403s → use classic HTTPS

Build gotchas (hard-won)

• CUDA / torch: pin torch+torchaudio+torchvision to a cu-build your driver supports in one resolve. PyPI's default torch tracks the newest CUDA; on an older driver you get "driver too old". This repo pins cu128 (CUDA 12.8). torchvision must match the torch minor or lightning trips "torchvision has no attribute extension".
• torchcodec may fail to load its native libs; pyannote then can't decode a file path. The service feeds pyannote an in-memory waveform (stdlib wave → tensor) to sidestep it.
• faster-whisper condition_on_previous_text=False is mandatory — the default collapses long audio to a single segment on some finetunes.

License

TBD — set before any external distribution.