feat: add local realtime voice agent stack - #144
Draft
cvsz wants to merge 21 commits into
Draft
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Contributor
|
ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR. |
|
|
||
| export function parseBearer(header) { | ||
| if (typeof header !== "string") return null; | ||
| const match = /^Bearer\s+(.+)$/i.exec(header.trim()); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a production-oriented local realtime voice vertical slice to Z Platform:
apps/zvoice: browser voice UI with AudioWorklet microphone capture, 16 kHz PCM16 streaming, live transcripts, audio playback, cancellation, and barge-in.services/voice-gateway: service-authenticated ticket issuance, short-lived HMAC one-time WebSocket tickets, replay rejection, concurrency admission, raw WebSocket tunneling, health, metrics, and redacted structured logs.services/voice-agent: pinned Hugging Facespeech-to-speechruntime using Faster Whisper STT, the OpenAI-compatible chat-completions backend, and Qwen3-TTS.compose.voice.yml: optional Ollama, llama.cpp, and vLLM profiles while keepingservices/ai-gatewayas the sole LLM policy boundary.Architecture
Security properties
Z_PLATFORM_SERVICE_TOKENor provider keys.Sec-WebSocket-Protocol.Validation completed
node --checkfor gateway, ZVoice server, browser client, and AudioWorklet.main.Local run — Ollama
Open
http://127.0.0.1:3022.Remaining release checks
This PR is intentionally draft because the current execution environment could not perform Docker image builds, model downloads, GPU compatibility validation, or an end-to-end browser microphone/speaker test. Those checks are documented in
docs/operations/voice-agent.mdand must pass on the target host before merge or external exposure.The first implementation stores consumed ticket nonces in memory and is therefore single-replica. Move replay/admission state to Redis before horizontal scaling.