Skip to content

feat: add local realtime voice agent stack - #144

Draft
cvsz wants to merge 21 commits into
mainfrom
feat/local-realtime-voice-agent
Draft

feat: add local realtime voice agent stack#144
cvsz wants to merge 21 commits into
mainfrom
feat/local-realtime-voice-agent

Conversation

@cvsz

@cvsz cvsz commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a production-oriented local realtime voice vertical slice to Z Platform:

  • apps/zvoice: browser voice UI with AudioWorklet microphone capture, 16 kHz PCM16 streaming, live transcripts, audio playback, cancellation, and barge-in.
  • services/voice-gateway: service-authenticated ticket issuance, short-lived HMAC one-time WebSocket tickets, replay rejection, concurrency admission, raw WebSocket tunneling, health, metrics, and redacted structured logs.
  • services/voice-agent: pinned Hugging Face speech-to-speech runtime using Faster Whisper STT, the OpenAI-compatible chat-completions backend, and Qwen3-TTS.
  • compose.voice.yml: optional Ollama, llama.cpp, and vLLM profiles while keeping services/ai-gateway as the sole LLM policy boundary.
  • Architecture, operations runbook, environment template, and smoke checks.

Architecture

Browser -> apps/zvoice -> services/voice-gateway -> services/voice-agent
                                                  -> services/ai-gateway
                                                     -> Ollama / llama.cpp / vLLM

Security properties

  • Browser never receives Z_PLATFORM_SERVICE_TOKEN or provider keys.
  • WebSocket access uses a 10–300 second signed ticket transported through Sec-WebSocket-Protocol.
  • Tickets are single-use and reject tampering, expiry, and replay.
  • Speech runtime has no published host port.
  • Published application/gateway/runtime-provider ports bind to loopback by default.
  • External production exposure remains gated on reviewed identity, TLS, Redis-backed replay state, privacy/retention controls, load tests, and human release approval.

Validation completed

  • node --check for gateway, ZVoice server, browser client, and AudioWorklet.
  • Node tests: 6 passed, 0 failed.
    • ticket signing, expiry, single-use, tamper rejection, endpoint authentication, browser-safe response contract;
    • ZVoice health secret non-disclosure and identity/service-token proxy boundary.
  • Shell syntax checks for runtime entrypoint and smoke script.
  • Python bytecode compilation for the health check.
  • JSON and YAML parsing for root package configuration and Compose overlay.
  • Git comparison: branch is 21 commits ahead and 0 behind main.

Local run — Ollama

cp configs/voice-agent.env.example .env.voice
# Set Z_PLATFORM_SERVICE_TOKEN and VOICE_TICKET_SECRET with independent random values.

docker compose --env-file .env.voice \
  -f compose.yml -f compose.voice.yml \
  --profile voice-ollama up -d --build

docker compose --env-file .env.voice \
  -f compose.yml -f compose.voice.yml \
  exec ollama ollama pull qwen3:8b

Open http://127.0.0.1:3022.

Remaining release checks

This PR is intentionally draft because the current execution environment could not perform Docker image builds, model downloads, GPU compatibility validation, or an end-to-end browser microphone/speaker test. Those checks are documented in docs/operations/voice-agent.md and must pass on the target host before merge or external exposure.

The first implementation stores consumed ticket nonces in memory and is therefore single-replica. Move replay/admission state to Redis before horizontal scaling.

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@ecc-tools

ecc-tools Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR.


export function parseBearer(header) {
if (typeof header !== "string") return null;
const match = /^Bearer\s+(.+)$/i.exec(header.trim());
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants