diff --git a/api-reference/realtime.mdx b/api-reference/realtime.mdx
index be02ec0..537b314 100644
--- a/api-reference/realtime.mdx
+++ b/api-reference/realtime.mdx
@@ -617,7 +617,7 @@ Optional text-to-speech configuration. If omitted, raw text fragments are sent.
```
- Use 20 to 25 seconds of clean speech for `voice_clone_v1.audio_data`. In JSON, protobuf `bytes` fields are base64-encoded; binary protobuf clients send the raw audio bytes.
+ Use 10 to 15 seconds of clean speech for `voice_clone_v1.audio_data`. In JSON, protobuf `bytes` fields are base64-encoded; binary protobuf clients send the raw audio bytes.
@@ -658,7 +658,7 @@ Provide an inline custom voice clone for hosted TTS using reference audio and it
| Field | Type | Description |
|-------|------|-------------|
-| `audio_data` | bytes | Raw reference audio bytes. Use 20 to 25 seconds of clean speech. |
+| `audio_data` | bytes | Raw reference audio bytes. Use 10 to 15 seconds of clean speech. |
| `audio_format` | AudioLineConfiguration | Format of `audio_data`, including sample rate, channel count, and sample format |
| `ref_text` | string | Exact transcript of the reference audio, including disfluencies and false starts |
diff --git a/livekit.mdx b/livekit.mdx
index e2cf359..966874e 100644
--- a/livekit.mdx
+++ b/livekit.mdx
@@ -91,7 +91,7 @@ if __name__ == "__main__":
| `temperature` | `float` | `1.0` | Sampling temperature (0.0–2.0) |
| `generate_reply_timeout` | `float` | `30.0` | Timeout in seconds for `generate_reply` (0 = no timeout) |
| `ws_url` | `str \| None` | `None` | Direct WebSocket URL override — useful for local development |
- | `tts_config` | `ElevenLabsTtsConfig \| HostedTtsConfig \| None` | `None` | TTS configuration (enables audio output). Use `ElevenLabsTtsConfig` for ElevenLabs synthesis or `HostedTtsConfig` for Deepslate-hosted cloned voices. |
+ | `tts_config` | `ElevenLabsTtsConfig \| HostedTtsConfig \| HostedVoiceCloneConfig \| None` | `None` | TTS configuration (enables audio output). Use `ElevenLabsTtsConfig` for ElevenLabs synthesis, `HostedTtsConfig` for a pre-cloned Deepslate-hosted voice, or `HostedVoiceCloneConfig` to clone a voice on the fly from an audio sample. |
| `http_session` | `aiohttp.ClientSession \| None` | `None` | Shared aiohttp session |
| `vad_confidence_threshold` | `float` | `0.5` | Minimum confidence to consider audio as speech (0.0–1.0) |
| `vad_min_volume` | `float` | `0.01` | Minimum volume threshold (0.0–1.0) |
@@ -145,6 +145,38 @@ if __name__ == "__main__":
```
+
+ Clone a voice on the fly from a raw audio sample, with no pre-uploaded voice ID required. The sample is sent as part of the session initialization request. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output.
+
+ | Parameter | Type | Default | Description |
+ |-----------|------|---------|-------------|
+ | `audio_data` | `bytes` | required | Raw PCM audio data of the voice sample (10 to 15 seconds of speech) |
+ | `audio_sample_rate` | `int` | required | Sample rate of the audio sample in Hz |
+ | `audio_channels` | `int` | required | Number of audio channels in the sample |
+ | `ref_text` | `str` | required | Exact transcript of the voice sample, including disfluencies and false starts |
+ | `mode` | `HostedTtsMode` | `HostedTtsMode.HIGH_QUALITY` | Quality/latency tradeoff for synthesis |
+
+ The `mode` parameter accepts the same `HostedTtsMode` values as `HostedTtsConfig` above.
+
+ ```python
+ from pathlib import Path
+ from deepslate.livekit import HostedVoiceCloneConfig
+
+ llm = deepslate.livekit.RealtimeModel(
+ tts_config=HostedVoiceCloneConfig(
+ audio_data=Path("voice_sample.pcm").read_bytes(),
+ audio_sample_rate=24000,
+ audio_channels=1,
+ ref_text="Hi there, um, thanks for calling. How can I help you today?",
+ )
+ )
+ ```
+
+
+ For best results, provide a clean 10 to 15 second sample and a `ref_text` that transcribes the audio exactly, including filler words and false starts. An accurate transcript improves the quality of the cloned voice.
+
+
+
Configure server-side text-to-speech with ElevenLabs. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output and automatic interruption handling.
@@ -209,6 +241,9 @@ if __name__ == "__main__":
Hosted voice TTS supports a low latency mode for fastest possible response at the cost of some output quality
+
+ Use a pre-cloned hosted voice or clone one on the fly from a raw audio sample, with no external TTS credentials
+
Speak text directly via TTS without routing through the LLM
diff --git a/pipecat.mdx b/pipecat.mdx
index 1c4bb5d..c2ce5a9 100644
--- a/pipecat.mdx
+++ b/pipecat.mdx
@@ -177,6 +177,39 @@ if __name__ == "__main__":
```
+
+ Clone a voice on the fly from a raw audio sample, with no pre-uploaded voice ID required. The sample is sent as part of the session initialization request. Pass an instance to `DeepslateRealtimeLLMService(tts_config=...)` to enable PCM audio output.
+
+ | Parameter | Type | Default | Description |
+ |-----------|------|---------|-------------|
+ | `audio_data` | `bytes` | required | Raw PCM audio data of the voice sample (10 to 15 seconds of speech) |
+ | `audio_sample_rate` | `int` | required | Sample rate of the audio sample in Hz |
+ | `audio_channels` | `int` | required | Number of audio channels in the sample |
+ | `ref_text` | `str` | required | Exact transcript of the voice sample, including disfluencies and false starts |
+ | `mode` | `HostedTtsMode` | `HostedTtsMode.HIGH_QUALITY` | Quality/latency tradeoff for synthesis |
+
+ The `mode` parameter accepts the same `HostedTtsMode` values as `HostedTtsConfig` above.
+
+ ```python
+ from pathlib import Path
+ from deepslate.pipecat import HostedVoiceCloneConfig, DeepslateRealtimeLLMService
+
+ llm = DeepslateRealtimeLLMService(
+ options=opts,
+ tts_config=HostedVoiceCloneConfig(
+ audio_data=Path("voice_sample.pcm").read_bytes(),
+ audio_sample_rate=24000,
+ audio_channels=1,
+ ref_text="Hi there, um, thanks for calling. How can I help you today?",
+ ),
+ )
+ ```
+
+
+ For best results, provide a clean 10 to 15 second sample and a `ref_text` that transcribes the audio exactly, including filler words and false starts. An accurate transcript improves the quality of the cloned voice.
+
+
+
Configure server-side text-to-speech with ElevenLabs via Deepslate. Pass an instance to `DeepslateRealtimeLLMService(tts_config=...)` to enable PCM audio output.
@@ -241,6 +274,9 @@ if __name__ == "__main__":
Hosted voice TTS supports a low latency mode for fastest possible response at the cost of some output quality
+
+ Use a pre-cloned hosted voice or clone one on the fly from a raw audio sample, with no external TTS credentials
+
Speak text directly via TTS without routing through the LLM