> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tryhamsa.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Text to speech

> Speak text over the realtime WebSocket: complete or streamed text, request order, interruptions, audio formats, pronunciation fixes and timing.

Send text and get speech back on the same session, while speech to text keeps listening. The voice comes from the handshake's `tts` settings, and each request can use another one.

## Speak a complete text

```json theme={null}
{ "type": "tts", "id": "greeting", "text": "أهلا وسهلا، كيف يمكنني مساعدتك؟" }
```

| Field | Required | Meaning |
| - | - | - |
| `text` | Yes | 1 to 5,000 characters |
| `id` | No | Your ID for the request, up to 64 characters, echoed in its replies. Default: `tts_1`, `tts_2`, ... It can't be the ID of a request that is still waiting or speaking |
| `voice`, `dialect`, `language`, `model` | No | This request only: replaces the session's [`tts` settings](https://docs.tryhamsa.com/websocket/realtime/configuration#tts) |

## Speak text as it arrives

When an LLM writes the reply, send each piece with `tts.append`, and `tts.end` after the last one:

```json theme={null}
{ "type": "tts.append", "id": "reply-1", "text": "أهلا وسهلا، " }
{ "type": "tts.append", "id": "reply-1", "text": "كيف يمكنني " }
{ "type": "tts.append", "id": "reply-1", "text": "مساعدتك اليوم؟" }
{ "type": "tts.end", "id": "reply-1" }
```

* The first `tts.append` with an `id` starts the request. Voice settings are read from that first message.
* A word is used once a space follows it, or at `tts.end`.
* For quick first audio, the first two pieces are spoken as soon as each has 5 words or more and ends with `.` `!` `?` `؟` `،` or `,`. Everything after them is spoken as one piece at `tts.end`, so the speech flows naturally.
* Always send `tts.end`. Until it arrives the request isn't finished, and later requests wait behind it.
* `tts.started` comes once the request is first in line, often before its first piece of text is ready.
* A `tts.append` to a request that is still waiting or speaking but already got `tts.end`, or to a `tts` request, gets an `invalid_message` error.
* Once a request has finished or been stopped, a `tts.append` with the same `id` starts a **new** request. Use a new `id` for each reply, and after `tts.cancel`, stop sending the rest of that reply.

## Request order

Requests on a session run one at a time, in the order you send them. Every audio frame belongs to the request announced by the last `tts.started`, so you never need to sort audio by request. `queue_ms` in `timing` shows how long a request waited.

## Audio

Each request starts with `tts.started`, which gives the audio format:

```json theme={null}
{ "type": "tts.started", "id": "greeting", "encoding": "pcm16", "sample_rate": 16000, "channels": 1, "output": "binary" }
```

| Settings | Audio |
| - | - |
| Arabic voices (default) | PCM16 little-endian, mono, 16 kHz |
| `language: "en"` | PCM16 little-endian, mono, 16 or 24 kHz (as `tts.started` says) |
| `mulaw: true` | 8-bit µ-law, mono, 8 kHz (phone audio) |

Always play the audio at the `sample_rate` from `tts.started`. Frames hold whole samples, and the audio has no WAV header.

> [accordion] Play the audio in a browser
>
> ```javascript theme={null}
> const player = new AudioContext();
> let format = { encoding: 'pcm16', sample_rate: 16000 };
> let nextTime = 0;
> let sources = [];
>
> // Call on tts.started.
> function setFormat(msg) { format = { encoding: msg.encoding, sample_rate: msg.sample_rate }; }
>
> // Call with each binary frame (pcm16; for mulaw, decode the bytes first).
> function playPcm16(arrayBuffer) {
>   const pcm = new Int16Array(arrayBuffer);
>   const buffer = player.createBuffer(1, pcm.length, format.sample_rate);
>   const channel = buffer.getChannelData(0);
>   for (let i = 0; i < pcm.length; i++) channel[i] = pcm[i] / 32768;
>   const source = player.createBufferSource();
>   source.buffer = buffer;
>   source.connect(player.destination);
>   nextTime = Math.max(nextTime, player.currentTime + 0.04); // queue frames back to back
>   source.start(nextTime);
>   nextTime += buffer.duration;
>   sources.push(source);
> }
>
> // Call when the caller interrupts.
> function stopPlayback() {
>   sources.forEach((s) => s.stop());
>   sources = [];
>   nextTime = 0;
> }
> ```

With `tts.output: "base64"`, audio comes in JSON messages instead of binary frames, for clients that can't read binary frames:

```json theme={null}
{ "type": "tts.audio", "id": "greeting", "seq": 0, "data": "AAABAAIAAwA=" }
```

`seq` counts from 0 within the request.

If your app reads slowly, Hamsa waits instead of sending audio faster than you can take it.

## How a request ends

| Message | When |
| - | - |
| `tts.done` | All the audio was sent. `audio_ms` is its length and `characters` the length of your text |
| `tts.cancelled` | You stopped it. `audio_ms` is the audio sent before it stopped. A request stopped before it started has only `id` |
| `tts.error` | Speech couldn't be generated. `code` is `upstream_error` (for example an unknown voice name or dialect; it can come after `tts.started` and some audio), `voice_not_available` (the voice ID isn't one of your voices or a public voice; nothing was generated), or `internal_error` (the voice couldn't be checked: try again). Later requests still run |

```json theme={null}
{
  "type": "tts.done",
  "id": "greeting",
  "audio_ms": 2840,
  "characters": 31,
  "timing": { "queue_ms": 0, "text_wait_ms": 0, "upstream_ttfb_ms": 171, "first_audio_ms": 172, "total_ms": 1050 }
}
```

Every request ends with exactly one of these three. `tts.end` or `tts.cancel` for an `id` that is unknown or already finished does nothing and gets no reply.

`timing` is measured on Hamsa's side, in milliseconds:

| Field | Meaning |
| - | - |
| `queue_ms` | From the request arriving to it starting: waiting behind earlier requests of the session |
| `text_wait_ms` | Streamed text: from the request starting to its first piece being ready (5 words ending with punctuation, or `tts.end`) |
| `upstream_ttfb_ms` | From the first piece being ready to its first audio: the voice model's time |
| `first_audio_ms` | Request received to first audio sent. Your own time to first audio minus this is network time |
| `total_ms` | Request received to last audio sent |

A field is `null` when it doesn't apply, for example `first_audio_ms` for a request stopped before any audio.

## Stop speaking (interruptions)

```json theme={null}
{ "type": "tts.cancel", "id": "reply-1" }
{ "type": "tts.cancel" }
```

With `id`, it stops that request. Without it, it stops the current request and every waiting one. Speech generation stops at once. Requests that hadn't started get `tts.cancelled` right away; the current one gets it once it has stopped.

To let the caller interrupt your agent:

1. On `vad.speech_started`, stop playing audio in your app, send `tts.cancel`, and send `playback: idle`.
2. Ignore audio frames until the next `tts.started`: frames already on their way may still arrive.
3. Stop sending `tts.append` for the reply you stopped.
4. Wait for the caller's `final` transcript, then answer with a new `id`.

So that short replies like "mhm" don't interrupt, lower the volume on `vad.speech_started` instead, keep sending `playback: playing`, and decide on the transcript:

* `backchannel: true`: restore the volume and keep speaking.
* Any other transcript: stop and send `tts.cancel`.
* No transcript within about 2 seconds of `vad.speech_stopped` (it was noise): restore the volume.

See [Backchannels](https://docs.tryhamsa.com/websocket/realtime/speech-to-text.md#backchannels). The [Playground](https://docs.tryhamsa.com/websocket/realtime/playground.md) does this with **Interrupt when I talk: Smart**.

## Voices

Set the session's voice in the handshake, and change it with `config.update` or for one request:

```json theme={null}
{ "type": "tts", "text": "Hello! How can I help you today?", "voice": "Emma", "dialect": "en", "language": "en" }
```

* `voice` is a voice name of the dialect (any letter case). Names: [Realtime voice names](https://docs.tryhamsa.com/text-to-speech/voices.md#realtime-voice-names). The [Playground](https://docs.tryhamsa.com/websocket/realtime/playground.md) lists them by dialect.
* `dialect`: `pls`, `egy`, `ksa`, `uae`, `jor`, `leb`, `syr`, `irq`, `kuw`, `qat`, `bah`, `oma`, `msa`, `ar-sa`, or `en` for English voices (with `language: "en"`).
* Some names exist in more than one dialect, so always send `voice` and `dialect` together.
* `expressiveness`: **deprecated**. Still accepted, from `0` (flat, steady) to `2` (very expressive), default `1`; don't build on it.
* A **voice by ID**: pass the ID of one of your cloned voices, or of a public voice, as `voice`. It speaks in its own language and dialect unless the request sets `language` or `dialect`. Another account's voice ID gets `tts.error` with `voice_not_available`. [Preload a cloned voice](https://docs.tryhamsa.com/api-reference/endpoint/preload-cloned-tts-voice.md) before you use it.
* `model`: a custom voice model made for your account, if you have one.

## Pronunciation fixes

`tts.dictionary` replaces words or phrases before they are spoken:

```json theme={null}
"tts": { "dictionary": { "همسة": "هَمْسَة", "API": "إيه بي آي", "Hamsa AI": "همسة إيه آي" } }
```

* Only whole words and phrases match; a word inside a longer word doesn't.
* Keys are words separated by spaces. A key with punctuation in it, such as "Dr." or "e-mail", never matches, and a phrase doesn't match across punctuation in your text.
* Arabic diacritics are ignored when matching. Letter case isn't.
* When phrases overlap, the longest one wins.
* Keys up to 200 characters, replacements up to 500.
* Set it in the handshake or with `config.update` (the new dictionary replaces the old one, and applies to the rest of a streamed request that is speaking). `handshake_ack` shows how many entries it has, as `dictionary_entries`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.