> For the full page index, fetch https://echonova.ai/llms.txt # Transcribe live audio Stream live audio over a WebSocket and get a final transcript for each segment, with the audio format, frame size, endpointing, billing, limits and a complete Python client. The Speech to Text stream takes audio while it is being spoken, from a microphone, a phone call or a live broadcast, and sends back the transcript of each piece of speech as soon as that piece ends. This page shows how to prepare the audio, connect, send it, decide where each piece ends, and read the results. Every message and field is on [Stream speech to text](/docs/reference/stt-stream). > [!WARNING] > Each transcript is final. The stream sends no interim or partial results > while someone is still speaking: the text of a segment arrives once, after > the segment ends. Plan your interface around whole utterances, not > word-by-word captions. ## When to stream | | `POST /v1/speech-to-text/sync` | `POST /v1/speech-to-text` | Stream | | ------------ | --------------------------------- | --------------------------------- | -------------------------------------------------- | | Input | A finished file, up to 10 minutes | A finished file, up to 10 minutes | Live audio, up to 25 minutes per session | | Result | One transcript for the file | One transcript for the file | One transcript per segment, while audio flows | | Audio format | Any supported file type | Any supported file type | Raw 16-bit mono PCM only | | Price | 100 credits a minute | 100 credits a minute | 200 credits a minute, the Live Speech to Text rate | Stream when the audio does not exist yet as a file and you need the words while the conversation is still going: a voice agent that must answer a caller, or captions for a live event. For a recording that is already on disk, upload it: it costs half as much. ## How a session works ```text you server |-- upgrade: auth, ?sample_rate, ?language >| |<------------------------- session.started | |-- binary PCM frames (100 ms each) ------->| |-- {"type":"end_segment"} ---------------->| |<------------------------------ transcript | |-- more frames, more end_segment ... ----->| |-- {"type":"stop"} ----------------------->| |<----------------- last transcripts, then | |<--------------------------- session.ended | |<------------------------------ close 1000 | ``` ## What you need - An API key whose services include Speech to Text, or that has no service restriction. See [Get an API key](/docs/developers/get-an-api-key). - A server. Browsers cannot set the `Authorization` header on a WebSocket, so a web page cannot connect. If the audio comes from a browser, send it to your own server and open the stream from there; never put the key in the page. - Credits on the workspace. See [Credits](/docs/developers/credits). - For the Python client below: Python 3.9 or later and `pip install "websockets>=14"`. ## Step 1: Prepare the audio The stream accepts one format: 16-bit signed little-endian PCM, one channel (`pcm_s16le`), with no file header. Choose a sample rate from 8,000 to 48,000 Hz and tell the server with `sample_rate` when you connect. 16,000 Hz is enough for speech and keeps frames small; 8,000 Hz suits telephone audio that was never wider. Convert stereo to mono and compressed audio to PCM before you send it. To turn a file into a WAV file the samples can read: ```bash ffmpeg -i input.mp3 -ac 1 -ar 16000 -c:a pcm_s16le audio.wav ``` A WAV file of this kind is a short header followed by exactly these samples, which is why the samples read the rate from the header and send the rest. A live source such as a microphone gives you the samples directly: capture at the rate you connect with, as 16-bit integers, one channel. ## Step 2: Connect Open a WebSocket to `wss://api.echonova.ai/v1/speech-to-text/stream` with your key on the upgrade request: | Part | Value | | --------------- | ------------------------------------------------------------------------- | | `Authorization` | Header. `Bearer YOUR_API_KEY`. | | `sample_rate` | Query parameter, required. The rate of the audio you send, 8000 to 48000. | | `language` | Query parameter, required. `ar` or `en`. | | `encoding` | Query parameter. `pcm_s16le`, the default and the only value. | When the key, the parameters and the balance are in order, the server accepts the upgrade and sends `session.started`. Wait for it before you send audio: ```json { "type": "session.started", "sessionId": "c8a1f2e4-5b6d-4e7f-9a0b-1c2d3e4f5a6b", "language": "ar", "sampleRate": 16000, "maxSessionSeconds": 1500 } ``` Keep `sessionId` in your logs: it identifies the session if you contact support. When something is wrong before the upgrade, the server answers with plain HTTP and the usual error envelope instead, and nothing is charged. The statuses are under [Errors](/docs/developers/errors#streaming). ## Step 3: Send audio frames Send the PCM as binary WebSocket frames, in order. About 100 ms of audio per frame works well: | Sample rate | Samples in 100 ms | Bytes per 100 ms frame | | ----------- | ----------------- | ---------------------- | | 8,000 Hz | 800 | 1,600 | | 16,000 Hz | 1,600 | 3,200 | | 24,000 Hz | 2,400 | 4,800 | | 48,000 Hz | 4,800 | 9,600 | A frame may not be larger than 64 KiB. A larger one gets a `frame_too_large` error and closes the session with `1008`; one over 1 MiB closes it with `1009` and no error message. Keep every frame an even number of bytes, so no sample is split across two frames. Send live audio as it is captured. When you stream a file, as the samples do, pause about 100 ms between frames so the server sees it the way it would see a live source. ## Step 4: End each segment The server transcribes audio in segments. A segment ends when you send `{"type":"end_segment"}`, or after 15 seconds of audio if you send nothing, and its transcript follows shortly after. Because there are no partial results, where the segments end decides how soon you see text and how well it reads. **Recommended** - Detect pauses on your side, with a voice activity detector or a simple energy threshold, and send `end_segment` when the speaker has been silent for 300 to 700 ms. Each transcript is then one whole sentence or turn. - In a voice agent, send `end_segment` as soon as your turn detection decides the caller has finished, and answer from that transcript. **Less effective** - Never sending `end_segment`. Speech is then cut every 15 seconds, often in the middle of a word, and the first words wait up to 15 seconds. - Sending `end_segment` on a fixed timer, such as every second. Short segments lose the context that makes a transcript accurate, and each segment is a separate charge rounded to a whole credit. ## Step 5: Read the transcripts Each segment comes back as one `transcript` message: ```json { "type": "transcript", "segmentId": "6720f3a1c4b5d6e7f8a9b0c1", "text": "مرحبا، كيف يمكنني مساعدتك اليوم؟", "language": "ar", "durationSeconds": 2.4, "creditsConsumed": 8, "isFinal": true } ``` | Field | What it holds | | ----------------- | ------------------------------------------------------------------------ | | `segmentId` | Identifies the segment. Use it to match transcripts to your own records. | | `text` | The transcript. Empty when the segment held no speech. | | `language` | The language detected, `ar` or `en`. | | `durationSeconds` | The length of the segment's audio, in seconds. | | `creditsConsumed` | The credits charged for this segment. | | `isFinal` | Always `true`. No later message changes this text. | A transcript can arrive a moment after you sent `end_segment` and while you keep sending audio for the next segment. Keep reading and sending at the same time, as the samples do with one task for each. ## Step 6: Stop Send `{"type":"stop"}` when the audio ends. The server transcribes what is left, sends the last `transcript` messages, then `session.ended`, then closes the socket with code `1000`: ```json { "type": "session.ended", "reason": "stopped", "totalSeconds": 42.6, "creditsConsumed": 142 } ``` `totalSeconds` and `creditsConsumed` are the totals of the whole session. Closing the socket yourself also ends the session, but audio that was not yet transcribed is then dropped. ## Complete Python client This client streams `audio.wav` in 100 ms frames, ends the segment, prints each transcript and stops. The same client in JavaScript, TypeScript, Go and C# is on [Stream speech to text](/docs/reference/stt-stream). ```python import asyncio import json import wave import websockets # pip install "websockets>=14" URL = "wss://api.echonova.ai/v1/speech-to-text/stream" API_KEY = "YOUR_API_KEY" async def main(): # A 16-bit mono PCM WAV file; its header gives the sample rate. with wave.open("audio.wav", "rb") as wav: if wav.getsampwidth() != 2 or wav.getnchannels() != 1: raise SystemExit("audio.wav must be 16-bit mono PCM") rate = wav.getframerate() pcm = wav.readframes(wav.getnframes()) async with websockets.connect( f"{URL}?sample_rate={rate}&language=ar", additional_headers={"Authorization": f"Bearer {API_KEY}"}, ) as ws: print(json.loads(await ws.recv())) # session.started async def send_audio(): step = rate // 10 * 2 # 100 ms of 16-bit samples, in bytes for i in range(0, len(pcm), step): await ws.send(pcm[i : i + step]) # bytes: a binary frame await asyncio.sleep(0.1) # real time, like a live source await ws.send(json.dumps({"type": "end_segment"})) await ws.send(json.dumps({"type": "stop"})) sender = asyncio.create_task(send_audio()) async for raw in ws: # ends when the server closes after session.ended message = json.loads(raw) if message["type"] == "transcript": print(message["text"]) elif message["type"] == "error": print("error:", message["code"], message["message"]) elif message["type"] == "session.ended": print("ended:", message["reason"], "credits:", message["creditsConsumed"]) await sender asyncio.run(main()) ``` ## Reconnect after a failure A session can end before you send `stop`. `session.ended` gives the reason and the close code says what to do: | Close code | Reason | What to do | | ---------- | -------------------------------- | -------------------------------------------------------------------------------------------------------- | | `1000` | `stopped` | Nothing: you sent `stop` or closed the socket. | | `1001` | none | The server is restarting. Reconnect and carry on. | | `1008` | none | Fix the frame: invalid JSON, an unknown `type`, or more than 64 KiB. | | `1009` | none | Send frames of 64 KiB or less; this one was over 1 MiB. | | `1011` | `upstream_error` | Reconnect after a short wait and carry on. | | `4002` | `insufficient_balance` | Add credits before you reconnect. | | `4008` | `idle_timeout` or `max_duration` | Reconnect. Send audio or a `ping` at least every 120 seconds, and start a new session before 25 minutes. | | `4029` | `spending_limit` | Raise the key's **Credit Limit**, or wait for the next billing cycle. | Audio sent after the last transcript you received is not transcribed when a session ends with an error, and it is not charged. If those words matter, keep the last few seconds of audio on your side and send them again on the new session. ## Billing The stream is charged at the Live Speech to Text rate, 200 credits a minute of audio, per second. Each segment is charged when its transcript is sent and rounded to a whole credit; the 2.4-second segment above costs 8 credits. A segment with no speech in it comes back with empty `text` and is still charged for its length, so do not stream long silences you do not need transcribed. A key's **Credit Limit** applies to every segment. See [Credits](/docs/developers/credits). ## Limits | Limit | Value | | ------------------ | ------------------------------------------------------------------------- | | Frame size | 64 KiB per binary frame; about 100 ms of audio is recommended | | Segment length | 15 seconds; longer speech is cut and transcribed in parts | | Session length | 25 minutes; the session then ends with reason `max_duration` | | Idle timeout | 120 seconds without any frame | | Concurrent streams | 10 open streams per workspace, Speech to Text and Text to Speech together | | Connection rate | 100 connection attempts per minute per key | See [Rate limits](/docs/developers/rate-limits#streams) for how the stream limits are counted. ## Troubleshooting ### The upgrade fails with 400 sample_rate must be an integer from 8000 to 48000 `sample_rate` is missing or out of range. Connect with `?sample_rate=` set to the rate of your audio. ### The upgrade fails with 400 language must be one of ar, en `language` is missing from the query, or holds something other than `ar` or `en`. Connect with `?language=ar` for Arabic and mixed speech, or `?language=en` for speech that is all English. ### The upgrade fails with 429 Too many concurrent streams The workspace already has 10 streams open, counting Text to Speech streams. Close one, or check that your code closes a socket when a call or session ends. ### No transcript arrives Nothing is transcribed until a segment ends. Send `end_segment` when the speaker pauses, or `stop` at the end. ### Every transcript is empty or garbled The audio does not match what you declared. Check that it is 16-bit, little-endian and mono, that it has no file header, and that `sample_rate` is the rate it was recorded at. A wrong rate makes speech sound too fast or too slow to the model. ### The socket closes with 1008 A frame was larger than 64 KiB, a text frame was not valid JSON, or its `type` is not one the stream knows. Read the `error` message sent just before the close for the details. ## FAQ ### Can I get words while someone is still speaking? No. Each transcript covers a whole segment and is sent once, when the segment ends. Send `end_segment` at every pause to get text sooner. ### Can I connect from a browser? No. A browser cannot set the `Authorization` header on a WebSocket, and the key must never reach a web page. Capture audio in the browser, send it to your server, and open the stream from there. ### Can I send MP3, Opus or stereo audio? No. Decode and convert it to 16-bit mono PCM first, as in Step 1. ### Is a session also a job? No. A stream creates no job, so it is not listed by `GET /v1/speech-to-text/jobs`, and it sends no webhook. The transcripts are only in the messages you receive, so store them as they arrive. ## Related - [Stream speech to text](/docs/reference/stt-stream) - [Speech to Text quickstart](/docs/developers/stt-quickstart) - [Synchronous and asynchronous calls](/docs/developers/sync-vs-async) - [Rate limits](/docs/developers/rate-limits) - [Errors](/docs/developers/errors) - [Credits](/docs/developers/credits)