add-read-aloud

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Add Read Aloud

添加朗读功能

Add Grok Text to Speech to an existing app: a speaker button on assistant replies, auto-speak, or narration of any text. Run on
/add-read-aloud
, typed Read aloud, or clear “speak this” / “TTS” intent. Cursor has no speaker; wire the app, not the IDE.
为现有应用添加 Grok 文本转语音功能:助手回复上的扬声器按钮、自动朗读,或任意文本的旁白。在触发
/add-read-aloud
、输入朗读,或明确的“朗读这段”/“TTS”意图时执行。Cursor 没有扬声器功能;请对接应用本身,而非 IDE。

Docs

文档

Pick the path

选择实现路径

NeedPath
Tap speaker, hear the finished reply. Narrate a page. Generate a file.Batch
POST https://api.x.ai/v1/tts
(default)
Audio starts while the LLM is still streaming; barge-in; texts over 15,000 charsStreaming
wss://api.x.ai/v1/tts
through a backend relay
Batch is the default for a read-aloud button: one request, one MP3, cacheable, the key never leaves the server. Go streaming only when the UX needs audio before the text is complete.
POST /v1/tts
has no documented streaming flag; do not invent one.
需求路径
点击扬声器,收听完整回复。朗读页面内容。生成音频文件。批量模式
POST https://api.x.ai/v1/tts
(默认)
LLM 还在流式生成文本时就开始播放音频;支持打断;文本超过 15,000 字符流式模式 通过后端中继连接
wss://api.x.ai/v1/tts
批量模式是朗读按钮的默认实现:一次请求返回一个 MP3 文件,可缓存,密钥不会暴露到服务端之外。仅当 UX 需要在文本生成完成前就开始播放音频时,才使用流式模式。
POST /v1/tts
没有官方文档记载的流式标识,请勿自行虚构。

Auth

鉴权

  • Bearer
    XAI_API_KEY
    , server side only. The TTS docs document no ephemeral-token flow, and browsers cannot set WebSocket headers, so browser streaming goes through your backend relay.
  • Never put the key in a client bundle. Do not paste keys in chat.
  • 采用 Bearer
    XAI_API_KEY
    鉴权,仅在服务端使用。TTS 文档未记载临时令牌流程,且浏览器无法设置 WebSocket 请求头,因此浏览器端的流式请求需通过你的后端中继转发。
  • 切勿将密钥放入客户端构建包中。请勿在聊天中粘贴密钥。

Steps

实现步骤

  1. Map the app
    • Where assistant messages render, where per-message actions live (copy, regenerate), how the reply stream ends, server framework, package manager.
    • The speaker icon belongs to read aloud. Waveform is voice mode (
      /add-voice
      ), microphone is dictation (
      /add-dictation
      ). Put a ghost speaker button in the message action row; loading shows a spinner, playing shows a stop square. One utterance at a time: starting a new one stops the current one.
    • Align the action row to the reply’s text edge, not the button’s box: an icon button centres its glyph, so if the assistant bubble has no padding pull the row left by that inset (e.g.
      -ml-[5px]
      for a 12–14 px icon in a 24 px button). Measure in the browser;
      getBoundingClientRect
      on the
      <p>
      and the
      <svg>
      should share a left edge.
    • Render the button only once the reply has finished streaming; on a live message it would speak a partial reply.
    • With many messages on screen, keep player state (active message id,
      loading | playing
      , last error) in one shared store (
      useSyncExternalStore
      , a signal, whatever the app uses) so every button reflects it and errors can surface in the app's existing status area. A per-button
      let current
      is not enough.
    • Auto-speak: opt-in toggle, off by default, and only after a user gesture on the page (autoplay policy). Never auto-speak on load.
    • If
      /add-voice
      is installed, its
      AudioContext
      and PCM player can play streaming TTS; do not add a second audio graph.
  2. Prepare the text
    • Speak prose, not markup. Strip markdown: headings → text,
      **bold**
      → text, links → link text, inline code → the code, fenced blocks →
      [pause] Code block omitted.
      , tables → one sentence per row or omit. Keep punctuation; it drives pacing.
    • Neutralise speech tags that arrive inside the reply (
      [laugh]
      ,
      <whisper>
      …) so the model’s text cannot steer delivery. Strip only the documented tag names (list in step 5), not every bracket:
      [1]
      citations and
      [note]
      must survive.
    • Batch limit is 15,000 characters per request. Split longer text on paragraph, then sentence, then word boundaries and play the parts in order; fetch part N+1 while N plays or there is a silent gap at every boundary. Or use streaming.
    • Cache by
      hash(text + voice_id + language + speed)
      ; the same reply is often replayed.
  3. Batch path (default)
    • Server: your route takes
      { text, voice_id?, language? }
      , validates the shape of each (
      voice_id
      ^[a-z0-9-]{1,64}$
      ,
      language
      BCP-47 or
      auto
      ), forwards JSON, streams the body back with the upstream
      Content-Type
      and
      Cache-Control: no-store
      . Map upstream 404 to “unknown voice” so the client gets a readable error. Default output is MP3 at 24 kHz / 128 kbps, playable everywhere in the browser.
    • language
      : default to
      "auto"
      for a chat app, where replies follow the user’s language; pin
      "en"
      etc. only for fixed-language products.
ts
// server (any runtime with fetch)
export async function speak(text: string, voice_id = "eve", language = "auto") {
  if (!text.trim() || text.length > 15_000) throw new Error("TTS text must be 1–15,000 chars");
  const res = await fetch("https://api.x.ai/v1/tts", {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({
      text,
      voice_id,
      language,                       // required: `auto` or BCP-47 (`en`, `pt-BR`); omitting it → 422
      // output_format: { codec: "mp3", sample_rate: 24000, bit_rate: 128000 }, // default
      // speed: 1.0,                  // 0.7–1.5
      // text_normalization: true,    // "$5" → "five dollars"
      // replace: { nginx: "/ˈɛndʒɪn ˈɛks/" },
    }),
  });
  if (!res.ok) throw new Error(`TTS ${res.status}`); // 400 bad text/format, 401 key, 404 unknown voice_id, 422 missing required field (e.g. language), 429/500/503 back off and retry
  return new Response(res.body, { headers: { "Content-Type": res.headers.get("content-type") ?? "audio/mpeg" } });
}
ts
// client
let current: HTMLAudioElement | null = null;
async function readAloud(text: string, voiceId = "eve") {
  current?.pause(); current = null;                       // one utterance at a time
  const res = await fetch("/api/tts", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ text, voice_id: voiceId }) });
  if (!res.ok) throw new Error("TTS request failed");
  const url = URL.createObjectURL(await res.blob());
  const audio = new Audio(url);
  audio.addEventListener("ended", () => URL.revokeObjectURL(url)); // avoid blob leaks
  current = audio;
  await audio.play();                                      // call from the click handler’s promise chain
}
function stop() { current?.pause(); current = null; }
  • Safari:
    audio.duration
    is
    Infinity
    on blob URLs. If you need a progress bar, decode with
    AudioContext.decodeAudioData(buf.slice(0))
    or request
    with_timestamps: true
    and read
    duration
    from the JSON envelope (audio is then base64 in
    audio
    ).
  • Safari suspends an
    AudioContext
    created outside a gesture for good. Create it synchronously in the click handler, before any
    await
    .
  1. Streaming path
    • Relay: server holds the key, upgrades the browser socket, builds the query string, forwards client JSON up and server JSON down. Request
      codec=pcm
      for the browser: raw PCM16 chunks can be scheduled as they arrive, while MP3 chunks cannot be decoded piecemeal without
      MediaSource
      .
ts
import { WebSocketServer, WebSocket } from "ws";

new WebSocketServer({ port: 8789 }).on("connection", (client) => {
  const q = new URLSearchParams({ language: "en", voice: "eve", codec: "pcm", sample_rate: "24000" /* optimize_streaming_latency: "1" */ });
  const up = new WebSocket(`wss://api.x.ai/v1/tts?${q}`, { headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` } });
  up.on("message", (d) => client.send(d.toString()));                                   // audio.delta, audio.done, audio.clear, session.updated, error
  client.on("message", (d) => up.readyState === WebSocket.OPEN && up.send(d.toString())); // text.delta, text.done, text.clear, session.update
  const end = () => { client.close(); up.close(); };
  up.on("close", end); up.on("error", end); client.on("close", end);
});
  • Client: one socket per chat session (it stays open across utterances; 50 concurrent sessions per team, permit TTL 600 s, so reconnect on close). Forward LLM tokens as
    text.delta
    (each ≤ 15,000 chars), send
    text.done
    when the reply finishes,
    text.clear
    on stop or when a new reply starts; drop queued audio on
    audio.clear
    .
ts
const ws = new WebSocket(relayUrl);
const ctx = new AudioContext({ sampleRate: 24000 }); // create in the click handler; resume if suspended
let playhead = 0, sources: AudioBufferSourceNode[] = [];

ws.addEventListener("message", (e) => {
  const ev = JSON.parse(e.data);
  if (ev.type === "audio.delta") {
    const bytes = Uint8Array.from(atob(ev.delta), (c) => c.charCodeAt(0));
    const pcm = new Int16Array(bytes.buffer, 0, bytes.byteLength >> 1);
    const buf = ctx.createBuffer(1, pcm.length, 24000);
    const ch = buf.getChannelData(0);
    for (let i = 0; i < pcm.length; i++) ch[i] = pcm[i] / 32768;
    const src = ctx.createBufferSource(); src.buffer = buf; src.connect(ctx.destination);
    playhead = Math.max(playhead, ctx.currentTime + 0.15);   // ~150 ms lead so chunks butt together
    src.start(playhead); playhead += buf.duration; sources.push(src);
  } else if (ev.type === "audio.done") { /* utterance finished; socket stays open */ }
  else if (ev.type === "audio.clear") { sources.forEach((s) => s.stop()); sources = []; playhead = 0; }
  else if (ev.type === "error") showError(ev.message);
});
// on each LLM token:  ws.send(JSON.stringify({ type: "text.delta", delta: token }))
// on reply finished:  ws.send(JSON.stringify({ type: "text.done" }))
// on stop / barge-in: ws.send(JSON.stringify({ type: "text.clear" }))  → wait for audio.clear before the next text.delta
  • Words split across
    text.delta
    boundaries are fine; matching and synthesis run across deltas.
  1. Options (JSON fields for batch, query params for streaming)
WantSet
Different voice
voice_id
(batch) /
voice
(streaming). Built-ins from
GET /v1/tts/voices
:
eve
(default),
ara
,
rex
,
leo
,
luna
,
atlas
,
aurora
,
orion
, … 28 total, all multilingual, case-insensitive. Custom voice: 8-char id from the console or
GET /v1/custom-voices
Non-English or mixed
language
:
en
,
ar-EG
,
ar-SA
,
ar-AE
,
bn
,
zh
,
fr
,
de
,
hi
,
id
,
it
,
ja
,
ko
,
pt-BR
,
pt-PT
,
ru
,
es-MX
,
es-ES
,
tr
,
vi
, or
auto
Faster or slower
speed
0.7–1.5
“$5”, “Dr.”, “3/4” spoken as words
text_normalization: true
Brand names, acronyms, jargon
replace: { "Acme Mobile": "Acme Mobull", "nginx": "/ˈɛndʒɪn ˈɛks/" }
; ≤200 entries, keys ≤100 chars (letters, digits, apostrophes, spaces), values ≤128; whole-word, case-insensitive, longest match wins. Streaming:
session.update { replace }
before the first
text.delta
Expressive deliveryInline
[pause]
,
[long-pause]
,
[laugh]
,
[chuckle]
,
[giggle]
,
[cry]
,
[sigh]
,
[breath]
,
[inhale]
,
[exhale]
,
[tsk]
,
[tongue-click]
,
[lip-smack]
,
[hum-tune]
. Wrapping
<whisper>
,
<soft>
,
<loud>
,
<emphasis>
,
<build-intensity>
,
<decrease-intensity>
,
<slow>
,
<fast>
,
<higher-pitch>
,
<lower-pitch>
,
<singing>
,
<sing-song>
around whole phrases
Captions, karaoke, lip-sync
with_timestamps: true
→ JSON
{ audio (base64), content_type, duration, audio_timestamps: { graph_chars[], graph_times[][start,end] } }
; step through
graph_chars
in order, never slice input by index
First audio sooner (streaming)
optimize_streaming_latency=1
(docs also list
2
; API reference lists
0
/
1
)
Telephony / IVR
output_format: { codec: "mulaw" | "alaw", sample_rate: 8000 }
; not playable in browsers
Editing, post-production
codec: "wav"
,
sample_rate: 44100
or
48000
Smaller files
codec: "mp3"
,
bit_rate: 64000
  1. Python twin (only if the server is Python)
python
import os, requests
r = requests.post(
    "https://api.x.ai/v1/tts",
    headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
    json={"text": text, "voice_id": "eve", "language": "en"},
)
r.raise_for_status(); audio_bytes = r.content  # audio/mpeg
  1. 梳理应用结构
    • 明确助手消息的渲染位置、每条消息的操作按钮位置(复制、重新生成)、回复流的结束方式、服务端框架、包管理器。
    • 扬声器图标属于朗读功能。波形图标对应语音模式(
      /add-voice
      ),麦克风图标对应听写功能(
      /add-dictation
      )。在消息操作栏中放置一个幽灵样式的扬声器按钮;加载时显示旋转加载器,播放时显示停止方块。同一时间仅播放一条语音:启动新的朗读会停止当前正在播放的内容。
    • 将操作栏与回复的文本边缘对齐,而非按钮的盒子边缘:图标按钮的图标是居中的,因此如果助手气泡没有内边距,需要将操作栏向左偏移对应内边距的距离(例如,对于 24px 按钮内的 12–14px 图标,使用
      -ml-[5px]
      )。在浏览器中实际测量:
      <p>
      标签和
      <svg>
      标签的
      getBoundingClientRect
      左边缘应该对齐。
    • 仅在回复流式传输完成后才渲染按钮;如果在消息还在生成时点击,会朗读不完整的回复。
    • 当屏幕上有大量消息时,将播放器状态(当前激活的消息 ID、
      loading | playing
      、最近一次错误)保存在统一的共享存储中(
      useSyncExternalStore
      、signal,或应用使用的任意状态管理方案),这样所有按钮都能同步状态,错误也能展示在应用现有的状态区域中。每个按钮单独用
      let current
      保存状态是不够的。
    • 自动朗读:采用用户主动开启的开关,默认关闭,且仅在页面上有用户手势触发后才生效(受自动播放策略限制)。切勿在页面加载时自动朗读。
    • 如果已安装
      /add-voice
      ,其自带的
      AudioContext
      和 PCM 播放器可以播放流式 TTS;请勿重复添加音频图。
  2. 文本预处理
    • 朗读的是纯文本内容,而非标记语言。去除 Markdown 格式:标题→纯文本、
      **粗体**
      →纯文本、链接→链接文本、行内代码→代码内容、代码块→
      [pause] 代码块已省略。
      、表格→每行一句话或直接省略。保留标点符号;标点会影响朗读的节奏。
    • 处理掉回复中自带的语音标签(
      [laugh]
      <whisper>
      等),避免模型生成的文本控制朗读效果。仅移除官方文档记载的标签名(列表见第5步),不要移除所有括号内容:
      [1]
      这类引用和
      [note]
      这类注释必须保留。
    • 批量模式的限制是每次请求最多 15,000 个字符。更长的文本需要先按段落拆分,再按句子、最后按单词边界拆分,然后按顺序播放;可以在播放第 N 段时就请求第 N+1 段,避免每段之间出现静音间隙。也可以使用流式模式。
    • hash(text + voice_id + language + speed)
      为键做缓存;同一条回复经常会被重复播放。
  3. 批量模式(默认)
    • 服务端:你的接口接收
      { text, voice_id?, language? }
      参数,校验每个参数的格式(
      voice_id
      符合
      ^[a-z0-9-]{1,64}$
      language
      为 BCP-47 格式或
      auto
      ),转发 JSON 请求,将上游返回的响应体流式返回,同时带上游的
      Content-Type
      Cache-Control: no-store
      响应头。将上游的 404 错误映射为“未知语音”,让客户端得到可读的错误信息。默认输出为 24 kHz / 128 kbps 的 MP3 格式,可在所有浏览器中播放。
ts
// server (any runtime with fetch)
export async function speak(text: string, voice_id = "eve", language = "auto") {
  if (!text.trim() || text.length > 15_000) throw new Error("TTS text must be 1–15,000 chars");
  const res = await fetch("https://api.x.ai/v1/tts", {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({
      text,
      voice_id,
      language,                       // required: `auto` or BCP-47 (`en`, `pt-BR`); omitting it → 422
      // output_format: { codec: "mp3", sample_rate: 24000, bit_rate: 128000 }, // default
      // speed: 1.0,                  // 0.7–1.5
      // text_normalization: true,    // "$5" → "five dollars"
      // replace: { nginx: "/ˈɛndʒɪn ˈɛks/" },
    }),
  });
  if (!res.ok) throw new Error(`TTS ${res.status}`); // 400 bad text/format, 401 key, 404 unknown voice_id, 422 missing required field (e.g. language), 429/500/503 back off and retry
  return new Response(res.body, { headers: { "Content-Type": res.headers.get("content-type") ?? "audio/mpeg" } });
}
ts
// client
let current: HTMLAudioElement | null = null;
async function readAloud(text: string, voiceId = "eve") {
  current?.pause(); current = null;                       // one utterance at a time
  const res = await fetch("/api/tts", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ text, voice_id: voiceId }) });
  if (!res.ok) throw new Error("TTS request failed");
  const url = URL.createObjectURL(await res.blob());
  const audio = new Audio(url);
  audio.addEventListener("ended", () => URL.revokeObjectURL(url)); // avoid blob leaks
  current = audio;
  await audio.play();                                      // call from the click handler’s promise chain
}
function stop() { current?.pause(); current = null; }
  • Safari 浏览器:blob URL 的
    audio.duration
    会返回
    Infinity
    。如果需要进度条,可以用
    AudioContext.decodeAudioData(buf.slice(0))
    解码,或者请求时带上
    with_timestamps: true
    ,从 JSON 响应包中读取
    duration
    (此时音频会以 base64 格式存在
    audio
    字段中)。
  • Safari 会永久挂起非用户手势触发创建的
    AudioContext
    。请在点击处理函数中、任何
    await
    之前同步创建它。
  1. 流式模式
    • 中继:服务端保管密钥,升级浏览器的 WebSocket 连接,构建查询字符串,将客户端的 JSON 向上游转发,将服务端的 JSON 向下转发给客户端。浏览器端请求时使用
      codec=pcm
      :原始 PCM16 数据块可以在到达时直接调度播放,而 MP3 数据块在没有
      MediaSource
      的情况下无法逐块解码。
ts
import { WebSocketServer, WebSocket } from "ws";

new WebSocketServer({ port: 8789 }).on("connection", (client) => {
  const q = new URLSearchParams({ language: "en", voice: "eve", codec: "pcm", sample_rate: "24000" /* optimize_streaming_latency: "1" */ });
  const up = new WebSocket(`wss://api.x.ai/v1/tts?${q}`, { headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` } });
  up.on("message", (d) => client.send(d.toString()));                                   // audio.delta, audio.done, audio.clear, session.updated, error
  client.on("message", (d) => up.readyState === WebSocket.OPEN && up.send(d.toString())); // text.delta, text.done, text.clear, session.update
  const end = () => { client.close(); up.close(); };
  up.on("close", end); up.on("error", end); client.on("close", end);
});
  • 客户端:每个聊天会话使用一个 WebSocket 连接(连接在多次朗读之间保持开启;每个团队最多 50 个并发会话,许可 TTL 为 600 秒,因此连接关闭时需要重连)。将 LLM 的 token 以
    text.delta
    事件转发(每个 ≤ 15,000 字符),回复生成完成时发送
    text.done
    ,停止朗读或新回复开始时发送
    text.clear
    ;收到
    audio.clear
    时丢弃已排队的音频。
ts
const ws = new WebSocket(relayUrl);
const ctx = new AudioContext({ sampleRate: 24000 }); // create in the click handler; resume if suspended
let playhead = 0, sources: AudioBufferSourceNode[] = [];

ws.addEventListener("message", (e) => {
  const ev = JSON.parse(e.data);
  if (ev.type === "audio.delta") {
    const bytes = Uint8Array.from(atob(ev.delta), (c) => c.charCodeAt(0));
    const pcm = new Int16Array(bytes.buffer, 0, bytes.byteLength >> 1);
    const buf = ctx.createBuffer(1, pcm.length, 24000);
    const ch = buf.getChannelData(0);
    for (let i = 0; i < pcm.length; i++) ch[i] = pcm[i] / 32768;
    const src = ctx.createBufferSource(); src.buffer = buf; src.connect(ctx.destination);
    playhead = Math.max(playhead, ctx.currentTime + 0.15);   // ~150 ms lead so chunks butt together
    src.start(playhead); playhead += buf.duration; sources.push(src);
  } else if (ev.type === "audio.done") { /* utterance finished; socket stays open */ }
  else if (ev.type === "audio.clear") { sources.forEach((s) => s.stop()); sources = []; playhead = 0; }
  else if (ev.type === "error") showError(ev.message);
});
// on each LLM token:  ws.send(JSON.stringify({ type: "text.delta", delta: token }))
// on reply finished:  ws.send(JSON.stringify({ type: "text.done" }))
// on stop / barge-in: ws.send(JSON.stringify({ type: "text.clear" }))  → wait for audio.clear before the next text.delta
  • 单词被拆分到不同
    text.delta
    边界中是没问题的;匹配和合成会跨多个 delta 进行。
  1. 配置选项(批量模式用 JSON 字段,流式模式用查询参数)
需求配置项
更换语音
voice_id
(批量模式)/
voice
(流式模式)。内置语音可通过
GET /v1/tts/voices
获取:
eve
(默认)、
ara
rex
leo
luna
atlas
aurora
orion
等,共 28 种,均支持多语言,不区分大小写。自定义语音:从控制台或
GET /v1/custom-voices
获取的 8 字符 ID。
非英语或混合语言
language
en
ar-EG
ar-SA
ar-AE
bn
zh
fr
de
hi
id
it
ja
ko
pt-BR
pt-PT
ru
es-MX
es-ES
tr
vi
auto
调整语速
speed
0.7–1.5
将“$5”、“Dr.”、“3/4”等朗读为完整单词
text_normalization: true
品牌名、缩写词、专业术语
replace: { "Acme Mobile": "Acme Mobull", "nginx": "/ˈɛndʒɪn ˈɛks/" }
;最多 200 条条目,键最长 100 字符(支持字母、数字、撇号、空格),值最长 128 字符;全词匹配,不区分大小写,最长匹配优先。流式模式:在第一个
text.delta
之前发送
session.update { replace }
富有表现力的朗读效果行内标签:
[pause]
[long-pause]
[laugh]
[chuckle]
[giggle]
[cry]
[sigh]
[breath]
[inhale]
[exhale]
[tsk]
[tongue-click]
[lip-smack]
[hum-tune]
。包裹整段短语的标签:
<whisper>
<soft>
<loud>
<emphasis>
<build-intensity>
<decrease-intensity>
<slow>
<fast>
<higher-pitch>
<lower-pitch>
<singing>
<sing-song>
字幕、卡拉OK、口型同步
with_timestamps: true
→ 返回 JSON
{ audio (base64), content_type, duration, audio_timestamps: { graph_chars[], graph_times[][start,end] } }
;请按顺序遍历
graph_chars
,切勿按索引切片输入内容
更快输出首段音频(流式模式)
optimize_streaming_latency=1
(文档还列出了
2
;API 参考中列出的是
0
/
1
电话 / IVR`output_format: { codec: "mulaw"
编辑、后期制作
codec: "wav"
sample_rate: 44100
48000
更小的文件体积
codec: "mp3"
bit_rate: 64000
  1. Python 版本实现(仅当服务端使用 Python 时)
python
import os, requests
r = requests.post(
    "https://api.x.ai/v1/tts",
    headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
    json={"text": text, "voice_id": "eve", "language": "en"},
)
r.raise_for_status(); audio_bytes = r.content  # audio/mpeg

streaming: websockets.connect(url, additional_headers={"Authorization": f"Bearer {key}"}); send {"type":"text.delta",...}, {"type":"text.done"}; read audio.delta / audio.done

streaming: websockets.connect(url, additional_headers={"Authorization": f"Bearer {key}"}); send {"type":"text.delta",...}, {"type":"text.done"}; read audio.delta / audio.done


7. **Smoke**
   - Batch: `curl -X POST https://api.x.ai/v1/tts -H "Authorization: Bearer $XAI_API_KEY" -H "Content-Type: application/json" -d '{"text":"Hello from read aloud.","voice_id":"eve","language":"en"}' --output /tmp/hello.mp3` → 200 `audio/mpeg` (MP3, 24 kHz, 128 kbps, mono), plays. Omit `language` → 422 (observed; the docs’ table only lists 400). `voice_id: "nope"` → 404.
   - In the app: no speaker while a reply streams; it appears when the reply ends. Click → spinner → stop square → back to speaker when audio ends; click again mid-play → stops at once; click a second reply while the first plays → first stops, second plays. Safari: first play works from the click, `URL.revokeObjectURL` fires on `ended`.
   - Text prep: a reply with a fenced block, a table, a `[1]` citation and a stray `[laugh]` → spoken as “Code block omitted”, one sentence per row, the citation intact, the tag gone. Unit-test this; it is pure.
   - Streaming: send a two-sentence reply token by token; audio should start before `text.done`. Send `text.clear` mid-utterance → `audio.clear`, playback stops with nothing stale. Second utterance on the same socket → fresh `audio.delta`s, no bleed from the first.
   - Search the client bundle for `XAI_API_KEY`; it must not be there. Against a running dev server, fetch `/`, collect the `/_next/static/chunks/*.js` (or equivalent) URLs it references, and grep each; do not rely on a production build you have not made.
   - Debug from logs with `/debug-voice`; swap its hook points to `audio.delta` (byte counts), `audio.done`, `audio.clear`, `error`.

7. **冒烟测试**
   - 批量模式:执行 `curl -X POST https://api.x.ai/v1/tts -H "Authorization: Bearer $XAI_API_KEY" -H "Content-Type: application/json" -d '{"text":"Hello from read aloud.","voice_id":"eve","language":"en"}' --output /tmp/hello.mp3` → 返回 200 `audio/mpeg`(MP3 格式,24 kHz,128 kbps,单声道),可正常播放。省略 `language` 参数 → 返回 422(实际观测结果;文档表格仅列出了 400)。`voice_id: "nope"` → 返回 404。
   - 应用内验证:回复流式生成时不显示扬声器按钮;回复生成完成后才显示。点击 → 显示加载旋转器 → 显示停止方块 → 音频播放结束后恢复为扬声器图标;播放中途再次点击 → 立即停止;第一个回复播放中点击第二个回复 → 第一个停止,第二个开始播放。Safari 浏览器:首次点击可正常播放,`ended` 事件触发时 `URL.revokeObjectURL` 正常执行。
   - 文本预处理验证:包含代码块、表格、`[1]` 引用和多余的 `[laugh]` 标签的回复 → 朗读时为“代码块已省略”、表格每行一句话、引用保留、标签被移除。为此编写单元测试;这部分逻辑是纯函数。
   - 流式模式验证:逐 token 发送一个包含两句话的回复;音频应该在 `text.done` 之前就开始播放。朗读中途发送 `text.clear` → 收到 `audio.clear`,播放停止,没有残留的旧音频。同一个连接上的第二次朗读 → 返回全新的 `audio.delta` 数据,不会混入第一次的内容。
   - 在客户端构建包中搜索 `XAI_API_KEY`;绝对不能出现。针对运行中的开发服务器,请求 `/` 页面,收集其引用的 `/_next/static/chunks/*.js`(或等效路径)URL,逐个 grep 检查;不要依赖你还没实际构建的生产版本。
   - 使用 `/debug-voice` 通过日志调试;将其钩子点替换为 `audio.delta`(字节数)、`audio.done`、`audio.clear`、`error`。

Out of scope

不在范围内

  • A voice agent that listens and answers (
    /add-voice
    ), speech to text (
    /add-dictation
    )
  • Voice cloning beyond passing an existing custom
    voice_id
  • Inventing a TTS token flow, a
    stream
    flag on
    POST /v1/tts
    , or event names not in the docs
  • 可收听并回答的语音 Agent(
    /add-voice
    )、语音转文字(
    /add-dictation
  • 除了传递现有自定义
    voice_id
    之外的语音克隆功能
  • 虚构 TTS token 流程、
    POST /v1/tts
    stream
    标识,或文档中未记载的事件名