Developers / Text to Speech

Stream text to speech

WS wss://api.echonova.ai/v1/text-to-speech/stream

Keep one socket open and send a synthesize message for each sentence or reply. The server answers with start, then binary PCM16 frames, then done. Audio is generated a few seconds of speech at a time, so the first audio arrives well before the whole text is spoken. Only one synthesis runs at a time: send cancel, or a new synthesize, to stop the current one (barge-in). Each synthesize is charged per character when it starts; a cancelled synthesis is still charged.

Try it is not available here: a browser cannot send an API key on a WebSocket, so connect from your server. Read the streaming guide · Try voices in the portal

Connect

GET Upgrade: websocket

Authorization string Required

Bearer, a space, then your API key, which starts with ek_. Create keys in the portal under Developers, API Keys.

Messages you send

synthesize

Speak this text. Cancels any synthesis still running on this socket.

type string Required
synthesize
requestId string Optional

Your id for this synthesis, echoed in start, done, cancelled and error. Generated when you leave it out.

Up to 64 characters
text string Required

The text to speak, 1 to 500 characters.

1 to 500 characters
voiceId string Required

The id of a voice from List available voices. The voice fixes the dialect.

speed number Optional

How fast the voice speaks.

0.5 to 2default 1
cfg number Optional

How closely the voice follows the text. Leave it at the default unless you are tuning.

1 to 5default 2
numSteps integer Optional

Diffusion steps. More steps are slower and cleaner.

16, 32, 48, 64 or 128default 48
tashkeel boolean Optional

Adds the vowel marks to Arabic text before it is read. Applies to Fusha voices only.

default false
normalize boolean Optional

Reads numbers, dates, times, money, units, URLs and e-mails the way a speaker of the voice's dialect says them.

default true
{
  "type": "synthesize",
  "requestId": "reply-1",
  "text": "أهلاً بك، كيف يمكنني مساعدتك اليوم؟",
  "voiceId": "66e5ef3a9c1d2b4f8a7e6c01"
}

cancel

Stop the running synthesis. The server answers with cancelled and sends no more audio for it.

type string Required
cancel
{
  "type": "cancel"
}

ping

Optional keep-alive. The server answers with pong.

type string Required
ping
{
  "type": "ping"
}

Messages you receive

start

Sent before the first audio frame of a synthesis. Use its sampleRate to play the audio.

type string Required
start
requestId string Required
sampleRate integer Required

Sample rate of the audio frames, in Hz.

encoding string Required
pcm_s16le
channels integer Required
1
{
  "type": "start",
  "requestId": "reply-1",
  "sampleRate": 24000,
  "encoding": "pcm_s16le",
  "channels": 1
}

audio Binary frame

Binary frame of mono PCM16 little-endian audio at the sampleRate from start. Play frames in the order they arrive.

done

All audio for the synthesis has been sent.

type string Required
done
requestId string Required
durationSeconds number Required

Length of the audio sent, in seconds.

characters integer Required

Characters charged.

creditsConsumed number Required

Credits charged for this synthesis.

{
  "type": "done",
  "requestId": "reply-1",
  "durationSeconds": 2.9,
  "characters": 36,
  "creditsConsumed": 36
}

cancelled

The synthesis stopped. No more audio follows for this requestId.

type string Required
cancelled
requestId string Required
{
  "type": "cancelled",
  "requestId": "reply-1"
}

error

Something went wrong. Fatal errors are followed by a close frame; others leave the socket open.

type string Required
error
requestId string Optional

Text to Speech only: the synthesis that failed.

code string Required

Machine-readable reason.

invalid_message, frame_too_large, validation_failed, voice_not_found, insufficient_balance, spending_limit or upstream_error
message string Required

Human-readable explanation.

{
  "type": "error",
  "code": "validation_failed",
  "message": "text must be shorter than or equal to 500 characters"
}

pong

The answer to ping.

type string Required
pong
{
  "type": "pong"
}

Close codes

1000 Normal
You sent stop, or you closed the socket.
1001 Going away
The server is restarting; reconnect.
1008 Policy violation
A frame was not valid JSON, was an unknown type, or was larger than 64 KiB.
1009 Message too big
A frame was larger than 1 MiB; the socket closes without an error message.
1011 Server error
Speech to Text: the speech service failed; reconnect and retry. On Text to Speech a failed synthesis sends error and the socket stays open.
4002 Insufficient balance
The workspace ran out of credits mid-session.
4008 Timeout
No frame for 120 seconds, or the session reached its maximum length.
4029 Spending limit
The key reached its Credit Limit mid-session.

Limits

Text length
500 characters per synthesize message.
Syntheses at a time
1 per socket; a new synthesize cancels the current one.
Session length
30 minutes.
Idle timeout
120 seconds without any frame.
Concurrent streams
10 open streams per workspace, STT and TTS together.
Connection rate
100 connection attempts per minute per key.

Handshake errors

Plain HTTP answers to the upgrade request, in the usual envelope. Nothing is charged. All errors

WS /v1/text-to-speech/stream

import asyncio
import json
import wave

import websockets  # pip install "websockets>=14"

URL = "wss://api.echonova.ai/v1/text-to-speech/stream"
API_KEY = "YOUR_API_KEY"


async def main():
    pcm = bytearray()
    rate = 24000
    async with websockets.connect(
        URL, additional_headers={"Authorization": f"Bearer {API_KEY}"}
    ) as ws:
        await ws.send(json.dumps({
            "type": "synthesize",
            "requestId": "reply-1",
            "text": "أهلاً بك، كيف يمكنني مساعدتك اليوم؟",
            "voiceId": "66e5ef3a9c1d2b4f8a7e6c01",
        }))
        while True:
            message = await ws.recv()
            if isinstance(message, bytes):  # a binary frame of PCM16
                pcm.extend(message)
                continue
            event = json.loads(message)
            if event["type"] == "start":
                rate = event["sampleRate"]
            elif event["type"] == "done":
                print("seconds:", event["durationSeconds"], "credits:", event["creditsConsumed"])
                break
            elif event["type"] in ("error", "cancelled"):
                raise SystemExit(f"{event['type']}: {event.get('message', '')}")

    with wave.open("speech.wav", "wb") as wav:
        wav.setnchannels(1)
        wav.setsampwidth(2)
        wav.setframerate(rate)
        wav.writeframes(bytes(pcm))


asyncio.run(main())
Example session

  1. You{"type":"synthesize","requestId":"reply-1","text":"أهلاً بك، كيف يمكنني مساعدتك اليوم؟","voiceId":"66e5ef3a9c1d2b4f8a7e6c01"}
  2. Server{"type":"start","requestId":"reply-1","sampleRate":24000,"encoding":"pcm_s16le","channels":1}
  3. Serverbinary: PCM16 frame
  4. Serverbinary: PCM16 frame
  5. Server{"type":"done","requestId":"reply-1","durationSeconds":2.9,"characters":36,"creditsConsumed":36}