Developers / Speech to Text

Stream speech to text

WS wss://api.echonova.ai/v1/speech-to-text/stream

Open one socket per live audio source. Send mono PCM16 little-endian audio as binary frames. Send end_segment when the speaker pauses to get that segment's transcript right away; without it the server cuts a segment every 15 seconds. Each transcript is final: there are no interim or partial results. Each segment is charged at the Live Speech to Text rate when its transcript is sent.

Try it is not available here: a browser cannot send an API key on a WebSocket, so connect from your server. Read the streaming guide · Try live transcription in the portal

Connect

GET Upgrade: websocket

Authorization string Required

Bearer, a space, then your API key, which starts with ek_. Create keys in the portal under Developers, API Keys.

Query parameters

sample_rate integer Required

Sample rate of the audio you send, in Hz.

8000 to 48000
language string Required

Spoken language. Required. ar for Arabic and mixed Arabic/English; en only for wholly English speech.

ar or en
encoding string Optional

Audio encoding. Only 16-bit signed little-endian mono PCM is accepted.

pcm_s16ledefault pcm_s16le

Messages you send

audio Binary frame

Binary frame of mono PCM16 little-endian audio at the sample_rate you connected with. At most 64 KiB.

end_segment

Transcribe the audio sent since the last segment now. Send it when the speaker pauses.

type string Required
end_segment
{
  "type": "end_segment"
}

stop

Transcribe any remaining audio, send the last transcripts and session.ended, then close.

type string Required
stop
{
  "type": "stop"
}

ping

Optional keep-alive. The server answers with pong.

type string Required
ping
{
  "type": "ping"
}

Messages you receive

session.started

Sent once after the socket opens. Start sending audio after it arrives.

type string Required
session.started
sessionId string Required

Identifies this session in support requests.

language string Required

The language you connected with.

ar or en
sampleRate integer Required

The sample rate the server expects, in Hz.

maxSessionSeconds integer Required

Seconds until the server ends the session.

{
  "type": "session.started",
  "sessionId": "c8a1f2e4-5b6d-4e7f-9a0b-1c2d3e4f5a6b",
  "language": "ar",
  "sampleRate": 16000,
  "maxSessionSeconds": 1500
}

transcript

The final transcript of one segment. Segments can arrive slightly after you send end_segment; match them by segmentId.

type string Required
transcript
segmentId string Required

Identifies the segment.

text string Required

The transcript. Empty when the segment held no speech.

language string Optional

The detected language, ar or en.

durationSeconds number Required

Length of the segment's audio, in seconds.

creditsConsumed number Required

Credits charged for this segment.

isFinal boolean Required

Always true: there are no interim results.

true
{
  "type": "transcript",
  "segmentId": "6720f3a1c4b5d6e7f8a9b0c1",
  "text": "مرحبا، كيف يمكنني مساعدتك اليوم؟",
  "language": "ar",
  "durationSeconds": 2.4,
  "creditsConsumed": 8,
  "isFinal": true
}

session.ended

The last message. The server closes the socket right after it.

type string Required
session.ended
reason string Required

Why the session ended.

stopped, max_duration, idle_timeout, insufficient_balance, spending_limit or upstream_error
totalSeconds number Required

Seconds of audio transcribed in this session.

creditsConsumed number Required

Credits charged in this session.

{
  "type": "session.ended",
  "reason": "stopped",
  "totalSeconds": 42.6,
  "creditsConsumed": 142
}

error

Something went wrong. Fatal errors are followed by a close frame; others leave the socket open.

type string Required
error
requestId string Optional

Text to Speech only: the synthesis that failed.

code string Required

Machine-readable reason.

invalid_message, frame_too_large, validation_failed, voice_not_found, insufficient_balance, spending_limit or upstream_error
message string Required

Human-readable explanation.

{
  "type": "error",
  "code": "validation_failed",
  "message": "text must be shorter than or equal to 500 characters"
}

pong

The answer to ping.

type string Required
pong
{
  "type": "pong"
}

Close codes

1000 Normal
You sent stop, or you closed the socket.
1001 Going away
The server is restarting; reconnect.
1008 Policy violation
A frame was not valid JSON, was an unknown type, or was larger than 64 KiB.
1009 Message too big
A frame was larger than 1 MiB; the socket closes without an error message.
1011 Server error
Speech to Text: the speech service failed; reconnect and retry. On Text to Speech a failed synthesis sends error and the socket stays open.
4002 Insufficient balance
The workspace ran out of credits mid-session.
4008 Timeout
No frame for 120 seconds, or the session reached its maximum length.
4029 Spending limit
The key reached its Credit Limit mid-session.

Limits

Frame size
64 KiB per binary frame; about 100 ms of audio per frame is recommended.
Segment length
15 seconds; longer speech is cut and transcribed in parts.
Session length
25 minutes; the server then ends the session with reason max_duration.
Idle timeout
120 seconds without any frame.
Concurrent streams
10 open streams per workspace, STT and TTS together.
Connection rate
100 connection attempts per minute per key.

Handshake errors

Plain HTTP answers to the upgrade request, in the usual envelope. Nothing is charged. All errors

WS /v1/speech-to-text/stream

import asyncio
import json
import wave

import websockets  # pip install "websockets>=14"

URL = "wss://api.echonova.ai/v1/speech-to-text/stream"
API_KEY = "YOUR_API_KEY"


async def main():
    # A 16-bit mono PCM WAV file; its header gives the sample rate.
    with wave.open("audio.wav", "rb") as wav:
        if wav.getsampwidth() != 2 or wav.getnchannels() != 1:
            raise SystemExit("audio.wav must be 16-bit mono PCM")
        rate = wav.getframerate()
        pcm = wav.readframes(wav.getnframes())

    async with websockets.connect(
        f"{URL}?sample_rate={rate}&language=ar",
        additional_headers={"Authorization": f"Bearer {API_KEY}"},
    ) as ws:
        print(json.loads(await ws.recv()))  # session.started

        async def send_audio():
            step = rate // 10 * 2  # 100 ms of 16-bit samples, in bytes
            for i in range(0, len(pcm), step):
                await ws.send(pcm[i : i + step])  # bytes: a binary frame
                await asyncio.sleep(0.1)  # real time, like a live source
            await ws.send(json.dumps({"type": "end_segment"}))
            await ws.send(json.dumps({"type": "stop"}))

        sender = asyncio.create_task(send_audio())
        async for raw in ws:  # ends when the server closes after session.ended
            message = json.loads(raw)
            if message["type"] == "transcript":
                print(message["text"])
            elif message["type"] == "error":
                print("error:", message["code"], message["message"])
            elif message["type"] == "session.ended":
                print("ended:", message["reason"], "credits:", message["creditsConsumed"])
        await sender


asyncio.run(main())
Example session

  1. Server{"type":"session.started","sessionId":"c8a1f2e4-5b6d-4e7f-9a0b-1c2d3e4f5a6b","language":"ar","sampleRate":16000,"maxSessionSeconds":1500}
  2. Youbinary: PCM16 frame
  3. Youbinary: PCM16 frame
  4. You{"type":"end_segment"}
  5. Server{"type":"transcript","segmentId":"6720f3a1c4b5d6e7f8a9b0c1","text":"مرحبا، كيف يمكنني مساعدتك اليوم؟","language":"ar","durationSeconds":2.4,"creditsConsumed":8,"isFinal":true}
  6. You{"type":"stop"}
  7. Server{"type":"session.ended","reason":"stopped","totalSeconds":42.6,"creditsConsumed":142}
  8. Serverclose 1000