Skip to content

Document HostedVoiceCloneConfig for pipecat and livekit plugins #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
serhatzx wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Document HostedVoiceCloneConfig for pipecat and livekit plugins #26

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fa8fe74
Document HostedVoiceCloneConfig for pipecat and livekit plugins
serhatzx Jun 4, 2026
321317b
Update recommended audio duration for `voice_clone_v1.audio_data` to …
serhatzx Jun 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions api-reference/realtime.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
The Opal WebSocket Protocol uses Protocol Buffers for message encoding. All messages are wrapped in either `ServiceBoundMessage` (client to server) or `ClientBoundMessage` (server to client).

<Note>
Messages are binary-encoded protobuf. JSON examples below are shown for readability. [Download the proto file](https://raw.githubusercontent.com/deepslate-labs/deepslate-docs/refs/heads/main/api-reference/realtime.proto).

Check warning on line 10 in api-reference/realtime.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

api-reference/realtime.mdx#L10 

Did you really mean 'protobuf'?

Check warning on line 10 in api-reference/realtime.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

api-reference/realtime.mdx#L10 

Did you really mean 'proto'?
</Note>

<Info>
Expand Down Expand Up @@ -572,7 +572,7 @@
```
</Tab>
<Tab title="Hosted">
Uses Deepslate-hosted TTS. You can reference a hosted voice by ID, or provide an inline voice clone from reference audio and exact reference text for this realtime session.

Check warning on line 575 in api-reference/realtime.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

api-reference/realtime.mdx#L575 

Did you really mean 'realtime'?

| Field | Type | Description |
|-------|------|-------------|
Expand Down Expand Up @@ -617,7 +617,7 @@
```

<Note>
Use 20 to 25 seconds of clean speech for `voice_clone_v1.audio_data`. In JSON, protobuf `bytes` fields are base64-encoded; binary protobuf clients send the raw audio bytes.
Use 10 to 15 seconds of clean speech for `voice_clone_v1.audio_data`. In JSON, protobuf `bytes` fields are base64-encoded; binary protobuf clients send the raw audio bytes.

@coderabbitai coderabbitai Bot Jun 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Align HostedVoiceCloneV1.audio_data duration with the protobuf contract.

Line 620 and Line 661 now document 10–15 seconds, but api-reference/realtime.proto for HostedVoiceCloneV1.audio_data still specifies 20–25 seconds. This creates a contract mismatch between API reference and schema for the same field.

Please make one source of truth and update the other side in the same change (either docs back to 20–25 or proto+related docs to 10–15 consistently).

Also applies to: 661-661

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@api-reference/realtime.mdx` at line 620, The documentation changes at lines
620 and 661 in api-reference/realtime.mdx now specify "10 to 15 seconds" for
HostedVoiceCloneV1.audio_data, but the HostedVoiceCloneV1.audio_data field
definition in api-reference/realtime.proto still documents "20 to 25 seconds".
Resolve this contract mismatch by choosing one as the source of truth: either
update the protobuf schema specification for HostedVoiceCloneV1.audio_data in
api-reference/realtime.proto to "10 to 15 seconds" to match the documentation,
or revert the documentation changes at lines 620 and 661 back to "20 to 25
seconds" to match the proto specification. Ensure all references across both
files use the same duration range consistently.

</Note>
</Tab>
</Tabs>
Expand Down Expand Up @@ -658,9 +658,9 @@

| Field | Type | Description |
|-------|------|-------------|
| `audio_data` | bytes | Raw reference audio bytes. Use 20 to 25 seconds of clean speech. |
| `audio_data` | bytes | Raw reference audio bytes. Use 10 to 15 seconds of clean speech. |
| `audio_format` | AudioLineConfiguration | Format of `audio_data`, including sample rate, channel count, and sample format |
| `ref_text` | string | Exact transcript of the reference audio, including disfluencies and false starts |

Check warning on line 663 in api-reference/realtime.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

api-reference/realtime.mdx#L663 

Did you really mean 'disfluencies'?

### HostedTtsMode

Expand Down Expand Up @@ -713,7 +713,7 @@
| `role` | ChatMessageRole | Role of the entity this message is attributed to |
| `content` | ChatMessageContent[] | Ordered content blocks of this message |
| `delivery_status` | ChatDeliveryStatus | Delivery status of this message |
| `ephemeral` | bool | `true` when the message was spoken via `DirectSpeech` with `include_in_history: false` — audible to the user but not in the LLM's context |

Check warning on line 716 in api-reference/realtime.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

api-reference/realtime.mdx#L716 

Did you really mean 'LLM's'?

### ChatMessageRole

Expand Down Expand Up @@ -778,6 +778,6 @@
| `ERROR_CONFIGURATION` | Configuration errors (invalid audio format, missing required fields) |
| `ERROR_PROTOCOL` | Protocol errors (malformed packets, unexpected message types) |
| `ERROR_INFERENCE` | Inference/AI processing errors (model unavailable, processing failed, timeout) |
| `ERROR_AUDIO` | Audio pipeline errors (codec failure, VAD errors) |

Check warning on line 781 in api-reference/realtime.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

api-reference/realtime.mdx#L781 

Did you really mean 'codec'?
| `ERROR_TTS` | TTS synthesis errors |
| `ERROR_INTERNAL` | Internal service errors (catch-all for server-side issues) |
37 changes: 36 additions & 1 deletion livekit.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: 'LiveKit Plugin'
description: 'Integrate Deepslate voice AI with LiveKit Agents for real-time voice applications'

Check warning on line 3 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L3 

Did you really mean 'Deepslate'?
icon: 'microphone-lines'
---

The `deepslate-livekit` package provides a `RealtimeModel` implementation for the [LiveKit Agents](https://github.com/livekit/agents) framework, enabling seamless integration with Deepslate's unified voice AI infrastructure.

Check warning on line 7 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L7 

Did you really mean 'Deepslate's'?

<Note>
This plugin lives in the [deepslate-sdks monorepo](https://github.com/deepslate-labs/deepslate-sdks). We welcome contributions — feel free to open issues or pull requests there.
Expand All @@ -12,7 +12,7 @@

## Prerequisites

- A Deepslate account with API credentials

Check warning on line 15 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L15 

Did you really mean 'Deepslate'?
- Python 3.11+
- LiveKit server and API credentials
- (Optional) ElevenLabs API key for server-side TTS
Expand All @@ -29,15 +29,15 @@

| Variable | Required | Description |
|----------|----------|-------------|
| `DEEPSLATE_VENDOR_ID` | Yes | Your Deepslate vendor ID |

Check warning on line 32 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L32 

Did you really mean 'Deepslate'?
| `DEEPSLATE_ORGANIZATION_ID` | Yes | Your Deepslate organization ID |

Check warning on line 33 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L33 

Did you really mean 'Deepslate'?
| `DEEPSLATE_API_KEY` | Yes | Your Deepslate API key |

Check warning on line 34 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L34 

Did you really mean 'Deepslate'?
| `ELEVENLABS_API_KEY` | No | ElevenLabs API key for server-side TTS |
| `ELEVENLABS_VOICE_ID` | No | ElevenLabs voice ID |
| `ELEVENLABS_MODEL_ID` | No | ElevenLabs model (e.g., `eleven_turbo_v2`) |

<Warning>
Never expose your Deepslate or ElevenLabs API keys to clients. This plugin is for **server-side use** with LiveKit Agents.

Check warning on line 40 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L40 

Did you really mean 'Deepslate'?
</Warning>

## Quick Start
Expand Down Expand Up @@ -83,16 +83,16 @@
<Accordion title="RealtimeModel Parameters">
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `vendor_id` | `str \| None` | env: `DEEPSLATE_VENDOR_ID` | Deepslate vendor ID |

Check warning on line 86 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L86 

Did you really mean 'Deepslate'?
| `organization_id` | `str \| None` | env: `DEEPSLATE_ORGANIZATION_ID` | Deepslate organization ID |

Check warning on line 87 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L87 

Did you really mean 'Deepslate'?
| `api_key` | `str \| None` | env: `DEEPSLATE_API_KEY` | Deepslate API key |

Check warning on line 88 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L88 

Did you really mean 'Deepslate'?
| `base_url` | `str` | `https://app.deepslate.eu` | Base URL for the Deepslate API |

Check warning on line 89 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L89 

Did you really mean 'Deepslate'?
| `system_prompt` | `str` | `"You are a helpful assistant."` | Default system prompt |
| `temperature` | `float` | `1.0` | Sampling temperature (0.0–2.0) |
| `generate_reply_timeout` | `float` | `30.0` | Timeout in seconds for `generate_reply` (0 = no timeout) |
| `ws_url` | `str \| None` | `None` | Direct WebSocket URL override — useful for local development |
| `tts_config` | `ElevenLabsTtsConfig \| HostedTtsConfig \| None` | `None` | TTS configuration (enables audio output). Use `ElevenLabsTtsConfig` for ElevenLabs synthesis or `HostedTtsConfig` for Deepslate-hosted cloned voices. |
| `tts_config` | `ElevenLabsTtsConfig \| HostedTtsConfig \| HostedVoiceCloneConfig \| None` | `None` | TTS configuration (enables audio output). Use `ElevenLabsTtsConfig` for ElevenLabs synthesis, `HostedTtsConfig` for a pre-cloned Deepslate-hosted voice, or `HostedVoiceCloneConfig` to clone a voice on the fly from an audio sample. |
| `http_session` | `aiohttp.ClientSession \| None` | `None` | Shared aiohttp session |

Check warning on line 95 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L95 

Did you really mean 'aiohttp'?
| `vad_confidence_threshold` | `float` | `0.5` | Minimum confidence to consider audio as speech (0.0–1.0) |
| `vad_min_volume` | `float` | `0.01` | Minimum volume threshold (0.0–1.0) |
| `vad_start_duration_ms` | `int` | `200` | Duration of speech to detect start (ms) |
Expand All @@ -101,7 +101,7 @@
</Accordion>

<Accordion title="VAD Configuration">
Voice Activity Detection is handled **server-side** by Deepslate. You tune it via the `vad_*` parameters on `RealtimeModel` — no client-side VAD pipeline is needed.

Check warning on line 104 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L104 

Did you really mean 'Deepslate'?

| Parameter | Default | Description |
|-----------|---------|-------------|
Expand All @@ -113,7 +113,7 @@
</Accordion>

<Accordion title="HostedTtsConfig">
Use a voice cloned and hosted within Deepslate — no external TTS provider credentials required. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output.

Check warning on line 116 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L116 

Did you really mean 'Deepslate'?

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
Expand Down Expand Up @@ -145,6 +145,38 @@
```
</Accordion>

<Accordion title="HostedVoiceCloneConfig">
Clone a voice on the fly from a raw audio sample, with no pre-uploaded voice ID required. The sample is sent as part of the session initialization request. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `audio_data` | `bytes` | required | Raw PCM audio data of the voice sample (10 to 15 seconds of speech) |
Comment thread
serhatzx marked this conversation as resolved.
| `audio_sample_rate` | `int` | required | Sample rate of the audio sample in Hz |
| `audio_channels` | `int` | required | Number of audio channels in the sample |
| `ref_text` | `str` | required | Exact transcript of the voice sample, including disfluencies and false starts |
| `mode` | `HostedTtsMode` | `HostedTtsMode.HIGH_QUALITY` | Quality/latency tradeoff for synthesis |

The `mode` parameter accepts the same `HostedTtsMode` values as `HostedTtsConfig` above.

```python
from pathlib import Path
from deepslate.livekit import HostedVoiceCloneConfig

llm = deepslate.livekit.RealtimeModel(
tts_config=HostedVoiceCloneConfig(
audio_data=Path("voice_sample.pcm").read_bytes(),
audio_sample_rate=24000,
audio_channels=1,
ref_text="Hi there, um, thanks for calling. How can I help you today?",
)
)
```

<Tip>
For best results, provide a clean 10 to 15 second sample and a `ref_text` that transcribes the audio exactly, including filler words and false starts. An accurate transcript improves the quality of the cloned voice.
</Tip>
</Accordion>

<Accordion title="ElevenLabsTtsConfig">
Configure server-side text-to-speech with ElevenLabs. Pass an instance to `RealtimeModel(tts_config=...)` to enable audio output and automatic interruption handling.

Expand Down Expand Up @@ -209,6 +241,9 @@
<Card title="Low Latency Mode" icon="bolt">
Hosted voice TTS supports a low latency mode for fastest possible response at the cost of some output quality
</Card>
<Card title="Voice Cloning" icon="clone">
Use a pre-cloned hosted voice or clone one on the fly from a raw audio sample, with no external TTS credentials
</Card>
<Card title="Direct Speech" icon="comment-dots">
Speak text directly via TTS without routing through the LLM
</Card>
Expand Down Expand Up @@ -311,7 +346,7 @@

## Chat History Export

Export the full conversation history as a list of structured message dicts at any point during a session:

Check warning on line 349 in livekit.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

livekit.mdx#L349 

Did you really mean 'dicts'?

```python
from livekit.agents import EventTypes
Expand Down
36 changes: 36 additions & 0 deletions pipecat.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: 'Pipecat Plugin'

Check warning on line 2 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L2 

Did you really mean 'Pipecat'?
description: 'Integrate Deepslate voice AI with Pipecat for real-time voice applications'

Check warning on line 3 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L3 

Did you really mean 'Deepslate'?

Check warning on line 3 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L3 

Did you really mean 'Pipecat'?
icon: 'microphone-lines'
---

The `deepslate-pipecat` package provides a `DeepslateRealtimeLLMService` implementation for the [Pipecat](https://github.com/pipecat-ai/pipecat) framework, enabling seamless integration with Deepslate's unified voice AI infrastructure.

Check warning on line 7 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L7 

Did you really mean 'Deepslate's'?

<Note>
This plugin lives in the [deepslate-sdks monorepo](https://github.com/deepslate-labs/deepslate-sdks). We welcome contributions — feel free to open issues or pull requests there.
Expand All @@ -12,9 +12,9 @@

## Prerequisites

- A Deepslate account with API credentials

Check warning on line 15 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L15 

Did you really mean 'Deepslate'?
- Python 3.11+
- A Pipecat-compatible transport (e.g. Daily.co, Twilio, generic WebSocket)

Check warning on line 17 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L17 

Did you really mean 'Twilio'?
- (Optional) ElevenLabs API key for server-side TTS

## Installation
Expand All @@ -29,15 +29,15 @@

| Variable | Required | Description |
|----------|----------|-------------|
| `DEEPSLATE_VENDOR_ID` | Yes | Your Deepslate vendor ID |

Check warning on line 32 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L32 

Did you really mean 'Deepslate'?
| `DEEPSLATE_ORGANIZATION_ID` | Yes | Your Deepslate organization ID |

Check warning on line 33 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L33 

Did you really mean 'Deepslate'?
| `DEEPSLATE_API_KEY` | Yes | Your Deepslate API key |

Check warning on line 34 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L34 

Did you really mean 'Deepslate'?
| `ELEVENLABS_API_KEY` | No | ElevenLabs API key for server-side TTS |
| `ELEVENLABS_VOICE_ID` | No | ElevenLabs voice ID |
| `ELEVENLABS_MODEL_ID` | No | ElevenLabs model (e.g., `eleven_turbo_v2`) |

<Warning>
Never expose your Deepslate or ElevenLabs API keys to clients. This plugin is for **server-side use** only.

Check warning on line 40 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L40 

Did you really mean 'Deepslate'?
</Warning>

## Quick Start
Expand Down Expand Up @@ -103,7 +103,7 @@

<AccordionGroup>
<Accordion title="DeepslateOptions">
The main configuration class for connecting to the Deepslate API. Use `DeepslateOptions.from_env()` to load credentials from environment variables, with optional keyword overrides.

Check warning on line 106 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L106 

Did you really mean 'Deepslate'?

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
Expand All @@ -119,7 +119,7 @@
</Accordion>

<Accordion title="VAD Configuration">
Pass a `VadConfig` to `DeepslateRealtimeLLMService` to tune server-side Voice Activity Detection. Disable client-side VAD on your transport since Deepslate handles it.

Check warning on line 122 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L122 

Did you really mean 'Deepslate'?

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
Expand All @@ -143,7 +143,7 @@
</Accordion>

<Accordion title="HostedTtsConfig">
Use a voice cloned and hosted within Deepslate — no external TTS provider credentials required. Pass an instance to `DeepslateRealtimeLLMService(tts_config=...)` to enable PCM audio output.

Check warning on line 146 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L146 

Did you really mean 'Deepslate'?

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
Expand Down Expand Up @@ -177,8 +177,41 @@
```
</Accordion>

<Accordion title="HostedVoiceCloneConfig">
Clone a voice on the fly from a raw audio sample, with no pre-uploaded voice ID required. The sample is sent as part of the session initialization request. Pass an instance to `DeepslateRealtimeLLMService(tts_config=...)` to enable PCM audio output.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `audio_data` | `bytes` | required | Raw PCM audio data of the voice sample (10 to 15 seconds of speech) |
Comment thread
serhatzx marked this conversation as resolved.
| `audio_sample_rate` | `int` | required | Sample rate of the audio sample in Hz |
| `audio_channels` | `int` | required | Number of audio channels in the sample |
| `ref_text` | `str` | required | Exact transcript of the voice sample, including disfluencies and false starts |
| `mode` | `HostedTtsMode` | `HostedTtsMode.HIGH_QUALITY` | Quality/latency tradeoff for synthesis |

The `mode` parameter accepts the same `HostedTtsMode` values as `HostedTtsConfig` above.

```python
from pathlib import Path
from deepslate.pipecat import HostedVoiceCloneConfig, DeepslateRealtimeLLMService

llm = DeepslateRealtimeLLMService(
options=opts,
tts_config=HostedVoiceCloneConfig(
audio_data=Path("voice_sample.pcm").read_bytes(),
audio_sample_rate=24000,
audio_channels=1,
ref_text="Hi there, um, thanks for calling. How can I help you today?",
),
)
```

<Tip>
For best results, provide a clean 10 to 15 second sample and a `ref_text` that transcribes the audio exactly, including filler words and false starts. An accurate transcript improves the quality of the cloned voice.
</Tip>
</Accordion>

<Accordion title="ElevenLabsTtsConfig">
Configure server-side text-to-speech with ElevenLabs via Deepslate. Pass an instance to `DeepslateRealtimeLLMService(tts_config=...)` to enable PCM audio output.

Check warning on line 214 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L214 

Did you really mean 'Deepslate'?

| Parameter | Type | Description |
|-----------|------|-------------|
Expand Down Expand Up @@ -241,6 +274,9 @@
<Card title="Low Latency Mode" icon="bolt">
Hosted voice TTS supports a low latency mode for fastest possible response at the cost of some output quality
</Card>
<Card title="Voice Cloning" icon="clone">
Use a pre-cloned hosted voice or clone one on the fly from a raw audio sample, with no external TTS credentials
</Card>
<Card title="Direct Speech" icon="comment-dots">
Speak text directly via TTS without routing through the LLM
</Card>
Expand All @@ -257,7 +293,7 @@
Exponential-backoff reconnection with a configurable retry limit
</Card>
<Card title="Transport Agnostic" icon="plug">
Works with any Pipecat transport: Daily.co, Twilio, generic WebSocket, and more

Check warning on line 296 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L296 

Did you really mean 'Pipecat'?

Check warning on line 296 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L296 

Did you really mean 'Twilio'?
</Card>
</CardGroup>

Expand Down Expand Up @@ -337,7 +373,7 @@
)
```

Use `LLMMessagesUpdateFrame` to resync the full context and optionally trigger an immediate model reply.

Check warning on line 376 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L376 

Did you really mean 'resync'?

## Direct Speech

Expand Down Expand Up @@ -406,7 +442,7 @@

## Custom Frames Reference

In addition to standard Pipecat frames, `deepslate-pipecat` exposes the following frames for controlling and observing Deepslate-specific behaviour.

Check warning on line 445 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L445 

Did you really mean 'Pipecat'?

### Input Frames (push into the pipeline)

Expand All @@ -423,12 +459,12 @@
| `DeepslateSessionInitializedFrame` | Emitted once when the session is fully initialized and ready to accept messages. |
| `DeepslateChatHistoryFrame` | Chat history export result. `messages: list[ChatMessageDict]`. |
| `DeepslateConversationQueryResultFrame` | Side-channel query result. `text: str`. |
| `DeepslateUserTranscriptionFrame` | User speech-to-text transcription from Deepslate. |

Check warning on line 462 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L462 

Did you really mean 'Deepslate'?
| `DeepslateModelTranscriptionFrame` | Word-aligned transcription for the model's TTS audio. `text: str`. |

## Transport Examples

The Deepslate service is transport-agnostic. Swap the transport to suit your deployment.

Check warning on line 467 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L467 

Did you really mean 'Deepslate'?

<AccordionGroup>
<Accordion title="Daily.co (WebRTC)">
Expand Down Expand Up @@ -497,7 +533,7 @@
Full message schemas and configuration options
</Card>
<Card title="Pipecat Docs" icon="book" href="https://docs.pipecat.ai/">
Pipecat framework documentation

Check warning on line 536 in pipecat.mdx

View check run for this annotation

Mintlify / Mintlify Validation (roomstechnologygmbh) - vale-spellcheck

pipecat.mdx#L536 

Did you really mean 'Pipecat'?
</Card>
<Card title="GitHub Repository" icon="github" href="https://github.com/deepslate-labs/deepslate-sdks">
Source code, issues, and contributions
Expand Down