Skip to content

feat: Kith voice sidecar + backend TTS wiring for Kaori#1

Merged
wbaxterh merged 6 commits into
masterfrom
feature/kaori-kith-integration
Apr 28, 2026
Merged

feat: Kith voice sidecar + backend TTS wiring for Kaori#1
wbaxterh merged 6 commits into
masterfrom
feature/kaori-kith-integration

Conversation

@wbaxterh

Copy link
Copy Markdown
Owner

Summary

  • Adds kith-voice/ — a Bun sidecar service that bridges Kith's PipecatRuntime (ElevenLabs TTS) to browser WebSocket clients for streaming audio
  • Wires routes/dm.js to fire-and-forget POST bot responses to the Kith voice service when the client sends an x-kith-session header
  • Adds KITH_VOICE_URL to .env.example

Architecture

Browser ←— WS ——→ kith-voice (Bun :3040) ←— WS ——→ Pipecat Python sidecar
Backend ——→ POST /speak/:sessionId ——→ kith-voice

Test plan

  • Start kith-voice sidecar (cd kith-voice && bun dev) with valid ELEVENLABS_API_KEY in .env
  • Set KITH_VOICE_URL=http://localhost:3040 in Backend .env
  • Send a DM to Kaori from the web client with x-kith-session header — verify Backend logs [Bot] Kith voice request and kith-voice logs [kith] speak
  • Verify WebSocket delivers tts_audio_chunk events with base64 audio to the connected browser
  • Verify barge-in message from browser triggers runtime.bargeIn() and stops TTS
  • Verify /health endpoint returns session count
  • Verify session teardown on WebSocket close

🤖 Generated with Claude Code

wbaxterh and others added 4 commits April 23, 2026 15:07
Adds kith-voice/ Bun service that bridges Kith's PipecatRuntime to
browser WebSocket clients for streaming ElevenLabs TTS. The backend
dm.js route now fires a POST to the Kith voice service after emitting
a bot message, keyed by the x-kith-session header from the web client.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lockfile was missing transitive deps (https-proxy-agent, agent-base,
debug, ms), causing `npm ci` to fail in CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mechanical auto-fixes from `biome check --write --unsafe`:
- Prefix unused params/vars with underscore
- Convert string concatenation to template literals
- Apply formatting (trailing commas, quotes, semicolons)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Convert assignment expressions in arrow callbacks to block bodies
and re-format to satisfy biome check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wbaxterh
wbaxterh marked this pull request as ready for review April 23, 2026 23:26
wbaxterh and others added 2 commits April 26, 2026 22:08
…orms

- Voice tuning: stability 0.55, style 0.45, speed 1.05 for smoother output
- Expanded slang dict with gen-z terms, laugh tags, extended vowel cleanup
- Added cleanForTTS text transform (strips markdown, collapses punctuation)
- Added OpenRouter/kaori-ai-response.js fallback when kaori-bot on :3001 unavailable
- Added Kith session debug logging

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove Kith debug console.log from dm.js
- Switch kith-voice deps from absolute file: paths to @kithjs/* npm packages
- Rename imports from @kith/ to @kithjs/
- Make Python sidecar paths configurable via env vars for production
- Update .env.example with new voice ID and optional path overrides

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@wbaxterh
wbaxterh merged commit f2a7ace into master Apr 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant