> For the full page index, fetch https://echonova.ai/llms.txt # Stream speech as it is generated Keep a WebSocket open, send text, and play PCM audio while it is still being generated, with sentence-by-sentence replies, barge-in with cancel, billing, limits and a complete Python client. The Text to Speech stream sends audio while it is still being generated, so a listener hears the first words long before the whole reply is ready. One socket stays open for a whole conversation and takes one text after another. This page shows how to connect, play the audio as it arrives, feed a reply sentence by sentence, and stop speaking when the listener interrupts. Every message and field is on [Stream text to speech](/docs/reference/tts-stream). ## When to stream | | `POST /v1/text-to-speech/sync` | Stream | | ----------------- | ------------------------------ | --------------------------------------------------- | | What you get | A link to a finished WAV file | Raw 16-bit mono PCM frames while they are generated | | First audio | When the whole text is spoken | After the first few seconds of speech are generated | | Texts per request | One | Any number, one at a time, on one socket | | Stopping midway | Not possible | `cancel`, at the next chunk boundary | | Price | 1 credit per character | 1 credit per character | Stream when someone is waiting to hear the reply: a voice agent, a phone call, a live assistant. For audio you store or serve later, the synchronous endpoint gives you a file and a job. ## How a session works ```text you server |-- upgrade: Authorization --------------->| |-- {"type":"synthesize", ...} ----------->| |<-------------------------------- start | |<---------------- binary PCM frames ... | |<-------------------------------- done | |-- {"type":"synthesize", ...} ----------->| the next text, same socket |<-------------------------------- start | |<---------------- binary PCM frames ... | |-- {"type":"cancel"} -------------------->| the listener interrupts |<---------------------------- cancelled | ``` ## What you need - An API key whose services include Text to Speech, or that has no service restriction. See [Get an API key](/docs/developers/get-an-api-key). - A `voiceId` from [List available voices](/docs/reference/tts-list-voices). - A server. Browsers cannot set the `Authorization` header on a WebSocket, so a web page cannot connect. Stream on your server and pass the audio on to your player; never put the key in a page. - Credits on the workspace. See [Credits](/docs/developers/credits). - For the Python client below: Python 3.9 or later and `pip install "websockets>=14"`. ## Step 1: Connect Open a WebSocket to `wss://api.echonova.ai/v1/text-to-speech/stream` with `Authorization: Bearer YOUR_API_KEY` on the upgrade request. The stream takes no query parameters. When something is wrong before the upgrade, the server answers with plain HTTP and the usual error envelope, and nothing is charged; see [Errors](/docs/developers/errors#streaming). Open the socket before you need it, for example when a call starts, and keep it for the whole conversation. Reusing it saves the connection time on every reply and counts as one of your concurrent streams. ## Step 2: Send a synthesize message ```json { "type": "synthesize", "requestId": "reply-1", "text": "أهلاً بك، كيف يمكنني مساعدتك اليوم؟", "voiceId": "66e5ef3a9c1d2b4f8a7e6c01" } ``` | Field | Required | Value | | ----------- | -------- | ----------------------------------------------------------------------------------------------- | | `type` | Yes | `synthesize` | | `text` | Yes | 1 to 500 characters. | | `voiceId` | Yes | A voice from the voice list. The voice fixes the dialect. | | `requestId` | No | Your id for this text, up to 64 characters, echoed on `start`, `done`, `cancelled` and `error`. | | `speed` | No | 0.5 to 2, default 1. | | `cfg` | No | 1 to 5, default 2. | | `numSteps` | No | 16, 32, 48, 64 or 128, default 48. | | `tashkeel` | No | Adds the vowel marks to Arabic text before it is read. Fusha voices only. Default `false`. | | `normalize` | No | Reads numbers, dates, money and the like as a speaker of the dialect says them. Default `true`. | The fields mean what they mean on [Generate speech synchronously](/docs/reference/tts-generate-sync); see [Text to Speech voices and options](/docs/developers/tts-voices-and-options). Only one text is spoken at a time on a socket. A `synthesize` sent while another is still running cancels the running one first, exactly like `cancel` followed by `synthesize`. ## Step 3: Play the audio as it arrives The server answers with `start`, then binary frames, then `done`: ```json { "type": "start", "requestId": "reply-1", "sampleRate": 24000, "encoding": "pcm_s16le", "channels": 1 } ``` Each binary frame is 16-bit signed little-endian mono PCM at the `sampleRate` from `start`. That is 24 kHz today, and `synthesize` takes no rate of its own, but read it from `start` rather than hard-coding it. Play the frames in the order they arrive, or append them to a file. Audio is generated a few seconds of speech at a time, so frames come in bursts: give your player a small buffer, a few hundred milliseconds, so it does not run dry between bursts. ```json { "type": "done", "requestId": "reply-1", "durationSeconds": 2.9, "characters": 36, "creditsConsumed": 36 } ``` `done` means every frame of that text has been sent. Binary frames carry no `requestId`; the frames between a `start` and its `done` or `cancelled` belong to that request. With the `sounddevice` package, for example, playback is a stream you write the frames to: ```python import sounddevice # pip install sounddevice player = sounddevice.RawOutputStream(samplerate=24000, channels=1, dtype="int16") player.start() # for every binary frame received: player.write(frame) ``` ## Step 4: Speak a reply sentence by sentence When the text comes from a language model, do not wait for the whole reply. Cut it into sentences as it is generated and send each sentence as its own `synthesize`, so the listener hears the first sentence while the model is still writing the rest. Send the next sentence only after the `done` of the one before. A `synthesize` sent earlier cancels the sentence still being spoken. Keep a queue on your side: 1. When a sentence is complete (it ends with `.`, `?`, `!`, `؟` or a line break), add it to the queue. 2. When the socket is idle, send the first sentence in the queue. 3. On `done`, send the next one. **Recommended** - One `synthesize` per sentence or short paragraph, each under 500 characters, sent one after the other on one socket. **Less effective** - One `synthesize` for every few words. Each piece is spoken on its own, so the voice loses the rhythm of the sentence. - A new socket for every sentence. Each one adds a connection and counts against the limit of concurrent streams. ## Step 5: Stop speaking when the listener interrupts When the listener starts talking over the reply, send: ```json { "type": "cancel" } ``` The server stops the synthesis and answers with `{"type":"cancelled","requestId":"reply-1"}`. No frame for that request follows `cancelled`. Cancelling takes effect at the next chunk boundary: a few frames already generated can still arrive between your `cancel` and `cancelled`. Stop your player and clear its buffer at once, on your side, and drop every frame until `cancelled` arrives. Then send the next `synthesize`, or send it right away: a new `synthesize` cancels the running one too. > [!NOTE] > A cancelled synthesis is charged in full. The charge is taken when the > synthesis starts, for every character of its `text`, whatever part of it > was spoken. Sentence-sized messages keep what an interruption wastes small. ## Complete Python client This client sends one text, collects the PCM frames and writes them to `speech.wav`. The same client in JavaScript, TypeScript, Go and C# is on [Stream text to speech](/docs/reference/tts-stream). ```python import asyncio import json import wave import websockets # pip install "websockets>=14" URL = "wss://api.echonova.ai/v1/text-to-speech/stream" API_KEY = "YOUR_API_KEY" async def main(): pcm = bytearray() rate = 24000 async with websockets.connect( URL, additional_headers={"Authorization": f"Bearer {API_KEY}"} ) as ws: await ws.send(json.dumps({ "type": "synthesize", "requestId": "reply-1", "text": "أهلاً بك، كيف يمكنني مساعدتك اليوم؟", "voiceId": "66e5ef3a9c1d2b4f8a7e6c01", })) while True: message = await ws.recv() if isinstance(message, bytes): # a binary frame of PCM16 pcm.extend(message) continue event = json.loads(message) if event["type"] == "start": rate = event["sampleRate"] elif event["type"] == "done": print("seconds:", event["durationSeconds"], "credits:", event["creditsConsumed"]) break elif event["type"] in ("error", "cancelled"): raise SystemExit(f"{event['type']}: {event.get('message', '')}") with wave.open("speech.wav", "wb") as wav: wav.setnchannels(1) wav.setsampwidth(2) wav.setframerate(rate) wav.writeframes(bytes(pcm)) asyncio.run(main()) ``` ## Errors on an open socket An `error` message carries `code`, `message` and, when it concerns one text, its `requestId`: | `code` | What happened | What to do | | ---------------------- | --------------------------------------------------------- | --------------------------------------------------------------------------------------- | | `validation_failed` | A field of `synthesize` is missing or out of range. | Fix the message and send it again. The socket stays open. | | `voice_not_found` | No voice has this `voiceId`. | Take an id from the voice list. | | `invalid_message` | A text frame was not valid JSON or had an unknown `type`. | The socket closes with `1008`. Fix the message and reconnect. | | `frame_too_large` | A frame was larger than 64 KiB. | The socket closes with `1008`, or `1009` with no message over 1 MiB. Send shorter text. | | `insufficient_balance` | The workspace ran out of credits. | The socket closes with `4002`. Add credits. | | `spending_limit` | The key reached its **Credit Limit**. | The socket closes with `4029`. Raise the limit. | | `upstream_error` | Speech generation failed. | Send the text again. The socket stays open. | Every close code is listed on [Errors](/docs/developers/errors#streaming). ## Billing Each `synthesize` costs 1 credit per character of `text`, spaces included, the same rate as the other Text to Speech endpoints. The credits are charged when the synthesis starts, before the first frame, and `done` repeats the amount in `creditsConsumed`. A cancelled synthesis, by `cancel` or by a newer `synthesize`, is still charged. If the synthesis fails before any audio is sent, the charge is refunded. A key's **Credit Limit** applies to every `synthesize`. See [Credits](/docs/developers/credits). ## Limits | Limit | Value | | ------------------- | ------------------------------------------------------------------------- | | Text length | 500 characters per `synthesize` | | Syntheses at a time | 1 per socket; a new `synthesize` cancels the current one | | Session length | 30 minutes | | Idle timeout | 120 seconds without any frame; send a `ping` to keep a quiet socket open | | Concurrent streams | 10 open streams per workspace, Speech to Text and Text to Speech together | | Connection rate | 100 connection attempts per minute per key | See [Rate limits](/docs/developers/rate-limits#streams) for how the stream limits are counted. ## Troubleshooting ### My second sentence cut off the first A `synthesize` sent before the previous one's `done` cancels it. Queue the sentences and send each one on the previous `done`, as in Step 4. ### Frames keep arriving after I sent cancel The synthesis stops at the next chunk boundary, so a few frames can follow your `cancel`. Drop every frame until `cancelled` arrives. ### The audio plays too fast, too slow or as noise Play it at the `sampleRate` from `start`, as 16-bit signed little-endian mono samples. A player set to another rate, to stereo or to 8-bit misreads the frames. ### The socket closes after two minutes of silence The idle timeout is 120 seconds without any frame. Send `{"type":"ping"}` every minute or so while the conversation is quiet; the server answers with `pong`. ## FAQ ### Can I get MP3 or Opus instead of PCM? No. The stream sends raw 16-bit mono PCM. Encode it yourself if you need another format, or use [Generate speech synchronously](/docs/reference/tts-generate-sync) for a WAV file. ### Can one socket use several voices? Yes. `voiceId` belongs to each `synthesize`, so every text can use another voice, one text at a time. ### Can I connect from a browser? No. A browser cannot set the `Authorization` header on a WebSocket, and the key must never reach a web page. Stream on your server and pass the audio on. ## Related - [Stream text to speech](/docs/reference/tts-stream) - [Text to Speech quickstart](/docs/developers/tts-quickstart) - [Text to Speech voices and options](/docs/developers/tts-voices-and-options) - [List available voices](/docs/reference/tts-list-voices) - [Synchronous and asynchronous calls](/docs/developers/sync-vs-async) - [Rate limits](/docs/developers/rate-limits) - [Credits](/docs/developers/credits)