> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tryhamsa.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime WebSocket

> One WebSocket session for streaming speech to text and text to speech: continuous audio in, live transcripts with end of speech, turns, gender and speakers, and streamed speech out.

One WebSocket session does streaming speech to text (STT) and streaming text to speech (TTS) at the same time. Send the caller's audio as it is captured and get transcripts as they speak; send text, even word by word from an LLM, and get speech back right away. It is built for voice agents: it tells you when the caller starts and stops talking, when their turn is over, and lets you cut your agent off when they interrupt.

```text theme={null}
wss://realtime.tryhamsa.com/v1
```

Connect to exactly this address (a trailing `/` is fine); any other path is answered with HTTP `404`.

> [card] [Playground](https://docs.tryhamsa.com/websocket/realtime/playground.md)
>
> Talk to it from your browser: microphone, transcripts, voices and live timings.

> [card] [Speech to text](https://docs.tryhamsa.com/websocket/realtime/speech-to-text.md)
>
> Audio formats, speech detection, turns, transcripts.

> [card] [Text to speech](https://docs.tryhamsa.com/websocket/realtime/text-to-speech.md)
>
> Whole and streamed text, interruptions, audio formats.

> [card] [Settings](https://docs.tryhamsa.com/websocket/realtime/configuration.md)
>
> Every setting, its range and default.

## How a session works

```mermaid theme={null}
sequenceDiagram
    participant App as Your app
    participant H as Hamsa
    App->>H: Connect to wss://realtime.tryhamsa.com/v1
    App->>H: handshake (key + settings)
    H-->>App: handshake_ack (session_id + settings)
    loop While the call lasts
        App->>H: Audio (binary frames)
        H-->>App: vad.speech_started / vad.speech_stopped
        H-->>App: transcript (interim, preflight, final)
        App->>H: tts or tts.append ... tts.end
        H-->>App: tts.started, audio (binary frames), tts.done
    end
    App->>H: close
```

* **Text frames** are JSON messages with a `type`.
* **Binary frames** are audio: from you, the caller's audio; from Hamsa, speech.
* Speech to text and text to speech run at the same time on the session, so you hear the caller while your agent speaks.

## Quick start

```javascript Browser theme={null}
// startMicrophone: see "Capture the microphone" in Speech to text.
// playPcm16, setFormat and stopPlayback: see "Play the audio in a browser" in Text to speech.
const ws = new WebSocket('wss://realtime.tryhamsa.com/v1');
ws.binaryType = 'arraybuffer';

ws.onopen = () => {
  ws.send(JSON.stringify({
    type: 'handshake',
    api_key: 'YOUR_API_KEY',
    config: {
      audio: { encoding: 'pcm16', sample_rate: 16000 },
      stt: { language: 'ar', gender_detection: true },
      tts: { voice: 'Amjad', dialect: 'pls' },
    },
  }));
};

ws.onmessage = (event) => {
  if (event.data instanceof ArrayBuffer) {
    playPcm16(event.data); // speech: 16-bit PCM at the sample_rate from tts.started
    return;
  }
  const msg = JSON.parse(event.data);
  if (msg.type === 'handshake_ack') {
    startMicrophone((pcm16Frame) => ws.send(pcm16Frame)); // 20 to 100 ms per frame
    ws.send(JSON.stringify({ type: 'tts', id: 'greeting', text: 'أهلا وسهلا، كيف يمكنني مساعدتك؟' }));
  }
  if (msg.type === 'tts.started') setFormat(msg); // the audio format of the speech that follows
  if (msg.type === 'vad.speech_started') { // the caller interrupted
    stopPlayback();
    ws.send(JSON.stringify({ type: 'tts.cancel' }));
  }
  if (msg.type === 'transcript' && msg.is_final) console.log('Caller:', msg.text);
  if (msg.type === 'error') console.error(msg.code, msg.message);
};
```

```python Python theme={null}
# pip install websockets
import asyncio, json, wave
import websockets

API_KEY = "YOUR_API_KEY"

async def main():
    async with websockets.connect("wss://realtime.tryhamsa.com/v1") as ws:
        await ws.send(json.dumps({
            "type": "handshake",
            "api_key": API_KEY,
            "config": {
                "audio": {"encoding": "pcm16", "sample_rate": 16000},
                "stt": {"language": "ar"},
                "tts": {"voice": "Amjad", "dialect": "pls"},
            },
        }))
        ack = json.loads(await ws.recv())
        if ack["type"] != "handshake_ack":
            raise RuntimeError(ack)  # an error message: code, message, reason

        async def send_audio():
            # A 16 kHz, 16-bit mono WAV, sent at real-time pace in 40 ms frames.
            with wave.open("caller.wav", "rb") as wav:
                while frame := wav.readframes(640):
                    await ws.send(frame)
                    await asyncio.sleep(0.04)
            await ws.send(json.dumps({"type": "stt.flush"}))  # end the last turn now

        sender = asyncio.create_task(send_audio())
        speech = bytearray()
        await ws.send(json.dumps({"type": "tts", "id": "greeting", "text": "أهلا وسهلا، كيف يمكنني مساعدتك؟"}))

        async for message in ws:
            if isinstance(message, bytes):
                speech += message  # 16 kHz PCM16 for Arabic voices
                continue
            msg = json.loads(message)
            if msg["type"] == "transcript":
                print(msg["kind"], msg["text"])
                if msg["is_final"] and sender.done():
                    break
            elif msg["type"] == "tts.done":
                print("speech ready:", msg["audio_ms"], "ms")
            elif msg["type"] == "error":
                print("error:", msg["code"], msg["message"])

        await ws.send(json.dumps({"type": "close"}))

asyncio.run(main())
```

## Authentication

Send your key as `api_key` in the `handshake` message. If your client can't, send it in the `x-api-key` header of the connection request, or in the URL as `?api_key=` (URLs can end up in logs, so avoid it).

* **Secret keys** (`sk_`) work. During a [rotation](https://docs.tryhamsa.com/overview/create-api-keys.md), the old value keeps working until its overlap ends.
* **Older (legacy) keys** work.
* **Public keys** (`pk_`) are refused: they only create call tokens for agents.
* A limited key needs the **Create** scope on **Realtime** (`realtime:create`).
* A key with allowed origins only works from a browser page on one of those websites. A server sends no origin, so it is refused.
* The key's owner must still be a member of the project, and your account balance must be positive when the session starts.

Keep secret keys on your server. For a browser or mobile app, open the session from your server, or use a key limited to **Realtime** with allowed origins. See [API keys](https://docs.tryhamsa.com/overview/create-api-keys.md).

## Handshake

The first message must be a `handshake`, within 5 seconds of connecting. Every setting is optional; the defaults suit a phone or web voice agent.

```json theme={null}
{
  "type": "handshake",
  "api_key": "YOUR_API_KEY",
  "config": {
    "audio": { "encoding": "pcm16", "sample_rate": 16000, "channels": 1 },
    "vad": { "min_silence_ms": 550 },
    "stt": { "model": "Hamsa-STT-S4", "language": "ar", "eos_enabled": true },
    "tts": { "voice": "Amjad", "dialect": "pls" }
  }
}
```

The reply is `handshake_ack`, with the session's ID and every setting after defaults:

```json theme={null}
{
  "type": "handshake_ack",
  "session_id": "6f1c2b8e-3a4d-4f0e-9b7a-1c2d3e4f5a6b",
  "config": { "audio": { ... }, "vad": { ... }, "stt": { ... }, "tts": { ... } }
}
```

Send audio and requests only after it: anything sent before `handshake_ack` is ignored. Some settings can change during the session with `config.update`. See [Settings](https://docs.tryhamsa.com/websocket/realtime/configuration.md).

## Messages

| You send | What it does |
| - | - |
| Binary frames | The caller's audio. See [Speech to text](https://docs.tryhamsa.com/websocket/realtime/speech-to-text.md) |
| `handshake` | Authenticates and configures the session. First message only |
| `config.update` | Changes settings during the session |
| `tts` | Speaks a complete text |
| `tts.append`, `tts.end` | Speaks text that arrives in pieces |
| `tts.cancel` | Stops speaking |
| `stt.flush` | Ends the caller's turn now |
| `playback` | Says whether your app is still playing speech |
| `ping` | Checks the connection |
| `close` | Ends the session |

| Hamsa sends | What it means |
| - | - |
| `handshake_ack` | The session is open |
| `config.updated` | Settings after a `config.update` |
| `vad.speech_started`, `vad.speech_stopped` | The caller started or stopped talking |
| `transcript` | What the caller said: `interim`, `preflight` or `final` |
| `tts.started` | Speech is starting, in this audio format |
| Binary frames, or `tts.audio` | Speech audio |
| `tts.done`, `tts.cancelled`, `tts.error` | How a speech request ended |
| `pong` | Reply to `ping` |
| `error` | Something went wrong |
| `session.draining` | The server is restarting: open a new session |

Every message and field: [Message reference](https://docs.tryhamsa.com/websocket/realtime/messages.md).

## Errors

An `error` message has a `code` and a `message`:

```json theme={null}
{ "type": "error", "code": "unauthorized", "message": "API key was revoked", "reason": "ApiKeyRevoked" }
```

**Before the session opens**, the socket closes right after the error:

| `code` | Close code | Why |
| - | - | - |
| `handshake_required` | `4002` | The first message wasn't a valid `handshake` (bad JSON, a binary frame, or a field out of range: `message` names it) |
| `handshake_timeout` | `4002` | No `handshake` within 5 seconds |
| `unauthorized` | `4001` | No key, an unknown, revoked, disabled or expired key, a rotated key's old value after its overlap, or the key's owner is inactive |
| `forbidden` | `4004` | A public key, a key without the `realtime:create` scope, an origin the key doesn't allow, a deactivated project, or the key's owner is no longer a project member |
| `insufficient_balance` | `4003` | Your balance is zero or less |
| `rate_limited` | `4029` | Over 120 handshakes with this key in the current minute, refused ones included |
| `internal_error` | `1011` | The session couldn't start. Try again |

Some key refusals also have a `reason`: `PublicKeyMintOnly`, `KeyRotated`, `ApiKeyRevoked`, `ApiKeyDisabled` or `ApiKeyExpired`.

**During the session**, an error doesn't close it:

| `code` | Why |
| - | - |
| `invalid_message` | Bad JSON, an unknown `type`, a field out of range (`message` names it), a `tts` ID still in use, or a `tts.append` to a request that already got `tts.end`. An ID error also has `id`. Unknown field names aren't errors: they are ignored |
| `stt_disabled` | Audio sent to a session with `stt.enabled: false` |
| `upstream_error` | A transcription failed. Keep sending audio: the next speech is transcribed as usual |

A failed speech request gets a `tts.error` instead. See [Text to speech](https://docs.tryhamsa.com/websocket/realtime/text-to-speech.md#how-a-request-ends).

## Close codes

| Code | Meaning | What to do |
| - | - | - |
| `1000` | You sent `close`, or the session ended normally | |
| `1006` | The connection dropped without a close message, for example after 20 to 40 seconds without answering pings | Reconnect |
| `1009` | A frame was over 1 MB | Send smaller frames |
| `1011` | The session couldn't start | Reconnect |
| `1012` | The server restarted | Reconnect now |
| `4001` | Unauthorized | Check the key |
| `4002` | Handshake missing, invalid or late | Send a valid `handshake` first, within 5 seconds |
| `4003` | Balance is zero or less | Top up your balance |
| `4004` | Forbidden | Check the key's scope, allowed origins and project |
| `4008` | Idle: no audio and no messages for 5 minutes | Reconnect when you need it |
| `4029` | Too many new sessions for this key | Wait a minute |

## Limits

| | Limit |
| - | - |
| Handshake | Within 5 seconds of connecting |
| Handshakes | 120 per key per calendar minute, refused ones included |
| Frame size | 1 MB |
| Text in a `tts` or `tts.append` | 5,000 characters |
| Request `id` | 64 characters |
| A caller's turn | 30 seconds of audio. Longer turns are split |
| One speech segment | `vad.max_buffered_speech_s`, up to 30 seconds |
| Idle | 5 minutes with no audio and no valid messages from you. Speech you receive doesn't count: send `ping` to keep a quiet session open |

## Staying connected

* **Pings.** Hamsa pings the socket every 20 seconds. Browsers and WebSocket libraries answer on their own; a socket that doesn't answer is closed. To check the connection yourself, send `ping` and wait for `pong`: it echoes your `ts`, or has the server time (Unix milliseconds) if you sent none.
* **Restarts.** When a server restarts, you get `session.draining` with `reconnect_within_ms`. Finish the current turn or speech, then open a new session. At the end of that time the socket closes with `1012`.
* **Busy.** If the server is full, the connection request is answered with HTTP `503` and `Retry-After: 1`. Try again after a second.

## Latency tips

* Send `pcm16` at 16 kHz in 20 to 40 ms frames, as the audio is captured.
* `vad.min_silence_ms` is the floor for transcript latency: 300 to 400 ms suits quick agents, 550 ms or more suits callers who pause to think.
* Start your LLM on `preflight`. Drop its answer only if a newer `interim` or `preflight` arrives for the same `turn_id`; the `final` that follows has the same text.
* Stream your LLM's reply with `tts.append` instead of waiting for all of it.
* Measured on one production session with the defaults: about 700 ms from the caller stopping to the first transcript (550 ms of it is `min_silence_ms`), and about 190 ms to the first speech audio, whatever the text length.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.