elevenlabs-agents
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseElevenLabs Agents Platform
ElevenLabs Agents平台
Overview
概述
ElevenLabs Agents Platform is a comprehensive solution for building production-ready conversational AI voice agents. The platform coordinates four core components:
- ASR (Automatic Speech Recognition) - Converts speech to text (32+ languages, sub-second latency)
- LLM (Large Language Model) - Reasoning and response generation (GPT, Claude, Gemini, custom models)
- TTS (Text-to-Speech) - Converts text to speech (5000+ voices, 31 languages, low latency)
- Turn-Taking Model - Proprietary model that handles conversation timing and interruptions
ElevenLabs Agents平台是构建生产级对话式AI语音代理的综合性解决方案。该平台协调四大核心组件:
- ASR(自动语音识别) - 将语音转换为文本(支持32+种语言,亚秒级延迟)
- LLM(大语言模型) - 推理与响应生成(支持GPT、Claude、Gemini及自定义模型)
- TTS(文本转语音) - 将文本转换为语音(5000+种音色,支持31种语言,低延迟)
- Turn-Taking Model(话轮转换模型) - 自研模型,处理对话时序与打断逻辑
🚨 Package Updates (January 2026)
🚨 包更新(2026年1月)
ElevenLabs migrated to new scoped packages in August 2025. Current packages:
bash
npm install @elevenlabs/react@0.12.3 # React SDK (Dec 2025: localization, Scribe fixes)
npm install @elevenlabs/client@0.12.2 # JavaScript SDK (Dec 2025: localization)
npm install @elevenlabs/react-native@0.5.7 # React Native SDK (Dec 2025: mic fixes, speed param)
npm install @elevenlabs/elevenlabs-js@2.30.0 # Base SDK (Jan 2026: latest)
npm install -g @elevenlabs/agents-cli@0.6.1 # CLIDEPRECATED: , (uninstall if present)
@11labs/react@11labs/client⚠️ CRITICAL: v1 TTS models were removed on 2025-12-15. Use Turbo v2/v2.5 only.
ElevenLabs于2025年8月迁移至新的作用域包。当前可用包:
bash
npm install @elevenlabs/react@0.12.3 # React SDK(2025年12月:本地化、Scribe修复)
npm install @elevenlabs/client@0.12.2 # JavaScript SDK(2025年12月:本地化)
npm install @elevenlabs/react-native@0.5.7 # React Native SDK(2025年12月:麦克风修复、语速参数支持)
npm install @elevenlabs/elevenlabs-js@2.30.0 # 基础SDK(2026年1月:最新版本)
npm install -g @elevenlabs/agents-cli@0.6.1 # CLI工具已废弃包: 、(若已安装请卸载)
@11labs/react@11labs/client⚠️ 重要提示: v1 TTS模型已于2025年12月15日移除,仅可使用Turbo v2/v2.5版本。
December 2025 Updates
2025年12月更新
Widget Improvements (v0.5.5):
- Microphone permission handling improvements (better UX for permission requests)
- Text-only mode () no longer requires microphone access
chat_mode: true - system tool fix (no longer omits last message)
end_call
SDK Fixes:
- Scribe audio format parameter now correctly transmitted (v2.32.0, Jan 2026)
- React Native infinite loop fix in useEffect dependencies (v0.5.6)
- Speed parameter support in TTS overrides (v0.5.7)
- Localization support for chat UI terms (v0.12.3)
组件改进(v0.5.5):
- 麦克风权限处理优化(权限请求的用户体验提升)
- 纯文本模式()不再要求麦克风权限
chat_mode: true - 系统工具修复(不再遗漏最后一条消息)
end_call
SDK修复:
- Scribe音频格式参数现在可正确传输(v2.32.0,2026年1月)
- React Native中useEffect依赖项的无限循环问题修复(v0.5.6)
- TTS覆盖配置中支持语速参数(v0.5.7)
- 聊天UI术语的本地化支持(v0.12.3)
Package Selection Guide
包选择指南
Which ElevenLabs package should I use?
| Package | Environment | Use Case |
|---|---|---|
| Server only (Node.js) | Full API access, TTS, voices, models |
| Browser + Server | Agents SDK, WebSocket, lightweight |
| React apps | Conversational AI hooks |
| Mobile | iOS/Android agents |
⚠️ Why elevenlabs-js doesn't work in browser:
- Depends on Node.js module (by design)
child_process - Error:
Module not found: Can't resolve 'child_process' - Workaround for browser API access: Create proxy server endpoint using , call proxy from browser
elevenlabs-js
Affected Frameworks:
- Next.js client components
- Vite browser builds
- Electron renderer process
- Tauri webview
Source: GitHub Issue #293
应选择哪个ElevenLabs包?
| 包名称 | 运行环境 | 适用场景 |
|---|---|---|
| 仅服务器端(Node.js) | 完整API访问、TTS、音色、模型管理 |
| 浏览器 + 服务器端 | Agents SDK、WebSocket、轻量级集成 |
| React应用 | 对话式AI钩子函数 |
| 移动端 | iOS/Android代理开发 |
⚠️ 为什么elevenlabs-js无法在浏览器中运行?
- 依赖Node.js的模块(设计如此)
child_process - 错误提示:
Module not found: Can't resolve 'child_process' - 浏览器API访问的替代方案: 使用`elevenlabs-js创建代理服务器端点,从浏览器调用该代理
受影响的框架:
- Next.js客户端组件
- Vite浏览器构建
- Electron渲染进程
- Tauri网页视图
1. Quick Start
1. 快速开始
React SDK
React SDK
bash
npm install @elevenlabs/react zodtypescript
import { useConversation } from '@elevenlabs/react';
const { startConversation, stopConversation, status } = useConversation({
agentId: 'your-agent-id',
signedUrl: '/api/elevenlabs/auth', // Recommended (secure)
// OR apiKey: process.env.NEXT_PUBLIC_ELEVENLABS_API_KEY,
clientTools: { /* browser-side tools */ },
onEvent: (event) => { /* transcript, agent_response, tool_call */ },
serverLocation: 'us' // 'eu-residency' | 'in-residency' | 'global'
});bash
npm install @elevenlabs/react zodtypescript
import { useConversation } from '@elevenlabs/react';
const { startConversation, stopConversation, status } = useConversation({
agentId: 'your-agent-id',
signedUrl: '/api/elevenlabs/auth', // 推荐方式(更安全)
// 或 apiKey: process.env.NEXT_PUBLIC_ELEVENLABS_API_KEY,
clientTools: { /* 浏览器端工具 */ },
onEvent: (event) => { /* 处理转录文本、agent响应、工具调用等事件 */ },
serverLocation: 'us' // 可选值:'eu-residency' | 'in-residency' | 'global'
});CLI ("Agents as Code")
CLI("代理即代码")
bash
npm install -g @elevenlabs/agents-cli
elevenlabs auth login
elevenlabs agents init # Creates agents.json, tools.json, tests.json
elevenlabs agents add "Bot" --template customer-service
elevenlabs agents push --env dev # Deploy
elevenlabs agents test "Bot" # Testbash
npm install -g @elevenlabs/agents-cli
elevenlabs auth login
elevenlabs agents init // 创建agents.json、tools.json、tests.json配置文件
elevenlabs agents add "Bot" --template customer-service
elevenlabs agents push --env dev // 部署到开发环境
elevenlabs agents test "Bot" // 测试代理API (Programmatic)
API(编程式调用)
typescript
import { ElevenLabsClient } from 'elevenlabs';
const client = new ElevenLabsClient({ apiKey: process.env.ELEVENLABS_API_KEY });
const agent = await client.agents.create({
name: 'Support Bot',
conversation_config: {
agent: { prompt: { prompt: "...", llm: "gpt-4o" }, language: "en" },
tts: { model_id: "eleven_turbo_v2_5", voice_id: "your-voice-id" }
}
});typescript
import { ElevenLabsClient } from 'elevenlabs';
const client = new ElevenLabsClient({ apiKey: process.env.ELEVENLABS_API_KEY });
const agent = await client.agents.create({
name: 'Support Bot',
conversation_config: {
agent: { prompt: { prompt: "...", llm: "gpt-4o" }, language: "en" },
tts: { model_id: "eleven_turbo_v2_5", voice_id: "your-voice-id" }
}
});2. SDK Parameter Naming (camelCase vs snake_case)
2. SDK参数命名(驼峰式 vs 蛇形命名)
CRITICAL: The JS SDK uses camelCase for parameters while the Python SDK and API use snake_case. Using snake_case in JS causes silent failures where parameters are ignored.
Common Parameters:
| API/Python (snake_case) | JS SDK (camelCase) |
|---|---|
| |
| |
| |
| |
Example:
typescript
// ❌ WRONG - parameter ignored (snake_case):
const stream = await elevenlabs.textToSpeech.convert(voiceId, {
model_id: "eleven_v3", // Silently ignored!
text: "Hello"
});
// ✅ CORRECT - use camelCase:
const stream = await elevenlabs.textToSpeech.convert(voiceId, {
modelId: "eleven_v3", // Works!
text: "Hello"
});Tip: Always check TypeScript types for correct parameter names. This is the most common error when migrating from Python SDK.
Source: GitHub Issue #300
重要提示: JS SDK使用驼峰式(camelCase)命名参数,而Python SDK和API使用蛇形命名(snake_case)。在JS中使用蛇形命名会导致参数被静默忽略,无错误提示。
常见参数对比:
| API/Python(蛇形命名) | JS SDK(驼峰式) |
|---|---|
| |
| |
| |
| |
示例:
typescript
// ❌ 错误 - 参数被忽略(蛇形命名):
const stream = await elevenlabs.textToSpeech.convert(voiceId, {
model_id: "eleven_v3", // 被静默忽略!
text: "Hello"
});
// ✅ 正确 - 使用驼峰式:
const stream = await elevenlabs.textToSpeech.convert(voiceId, {
modelId: "eleven_v3", // 正常工作!
text: "Hello"
});提示: 始终参考TypeScript类型定义确认正确的参数名称。这是从Python SDK迁移时最常见的错误。
3. Agent Configuration
3. Agent配置
System Prompt Architecture (6 Components)
系统提示词架构(6个组件)
1. Personality - Identity, role, character traits
2. Environment - Communication context (phone, web, video)
3. Tone - Formality, speech patterns, verbosity
4. Goal - Objectives and success criteria
5. Guardrails - Boundaries, prohibited topics, ethical constraints
6. Tools - Available capabilities and when to use them
Template:
json
{
"agent": {
"prompt": {
"prompt": "Personality:\n[Agent identity and role]\n\nEnvironment:\n[Communication context]\n\nTone:\n[Speech style]\n\nGoal:\n[Primary objectives]\n\nGuardrails:\n[Boundaries and constraints]\n\nTools:\n[Available tools and usage]",
"llm": "gpt-4o", // gpt-5.1, claude-sonnet-4-5, gemini-3-pro-preview
"temperature": 0.7
}
}
}2025 LLM Models:
- ,
gpt-5.1(Oct 2025)gpt-5.1-2025-11-13 - ,
claude-sonnet-4-5(Oct 2025)claude-sonnet-4-5@20250929 - (2025)
gemini-3-pro-preview - (Oct 2025)
gemini-2.5-flash-preview-09-2025
1. 人格设定 - 身份、角色、性格特征
2. 环境上下文 - 沟通场景(电话、网页、视频)
3. 语气风格 - 正式程度、说话方式、 verbose程度
4. 目标设定 - 任务目标与成功标准
5. 防护规则 - 边界限制、禁止话题、伦理约束
6. 工具配置 - 可用能力及使用场景
模板:
json
{
"agent": {
"prompt": {
"prompt": "人格设定:\n[Agent身份与角色]\n\n环境上下文:\n[沟通场景]\n\n语气风格:\n[说话方式]\n\n目标设定:\n[核心任务]\n\n防护规则:\n[边界与约束]\n\n工具配置:\n[可用工具及用法]",
"llm": "gpt-4o", // 可选值:gpt-5.1、claude-sonnet-4-5、gemini-3-pro-preview
"temperature": 0.7
}
}
}2025年可用LLM模型:
- 、
gpt-5.1(2025年10月)gpt-5.1-2025-11-13 - 、
claude-sonnet-4-5(2025年10月)claude-sonnet-4-5@20250929 - (2025年)
gemini-3-pro-preview - (2025年10月)
gemini-2.5-flash-preview-09-2025
Turn-Taking Modes
话轮转换模式
| Mode | Behavior | Best For |
|---|---|---|
| Eager | Responds quickly | Fast-paced support, quick orders |
| Normal | Balanced (default) | General customer service |
| Patient | Waits longer | Information collection, therapy |
json
{ "conversation_config": { "turn": { "mode": "patient" } } }| 模式 | 行为特点 | 最佳适用场景 |
|---|---|---|
| Eager(主动响应) | 快速回复 | 快节奏客服、快速下单场景 |
| Normal(常规模式) | 平衡响应(默认) | 通用客户服务 |
| Patient(耐心等待) | 等待更长时间 | 信息收集、心理咨询场景 |
json
{ "conversation_config": { "turn": { "mode": "patient" } } }Workflows & Agent Management (2025)
工作流与Agent管理(2025年)
Workflow Features:
- Subagent Nodes - Override prompt, voice, turn-taking per node
- Tool Nodes - Guarantee tool execution
- Edges - Conditional routing with (determinism, Oct 2025)
edge_order
json
{
"workflow": {
"nodes": [
{ "id": "node_1", "type": "subagent", "config": { "system_prompt": "...", "turn_eagerness": "patient" } },
{ "id": "node_2", "type": "tool", "tool_name": "transfer_to_human" }
],
"edges": [{ "from": "node_1", "to": "node_2", "condition": "escalation", "edge_order": 1 }]
}
}Agent Management (2025):
- Agent Archiving - field (Oct 2025)
archived: true - Agent Duplication - Clone existing agents
- Service Account API Keys - Management endpoints (Jul 2025)
工作流特性:
- 子代理节点 - 可按节点覆盖提示词、音色、话轮转换规则
- 工具节点 - 确保工具执行
- 路由边 - 支持的条件路由(确定性,2025年10月)
edge_order
json
{
"workflow": {
"nodes": [
{ "id": "node_1", "type": "subagent", "config": { "system_prompt": "...", "turn_eagerness": "patient" } },
{ "id": "node_2", "type": "tool", "tool_name": "transfer_to_human" }
],
"edges": [{ "from": "node_1", "to": "node_2", "condition": "escalation", "edge_order": 1 }]
}
}Agent管理(2025年):
- Agent归档 - 支持字段(2025年10月)
archived: true - Agent复制 - 克隆现有代理
- 服务账户API密钥 - 管理端点(2025年7月)
Dynamic Variables
动态变量
Use syntax in prompts, messages, and tool parameters.
{{var_name}}System Variables:
- ,
{{system__agent_id}}{{system__conversation_id}} - ,
{{system__caller_id}}(telephony){{system__called_number}} - ,
{{system__call_duration_secs}}{{system__time_utc}} - (Twilio only)
{{system__call_sid}}
Custom Variables:
typescript
await client.conversations.create({
agent_id: "agent_123",
dynamic_variables: { user_name: "John", account_tier: "premium" }
});Secret Variables: (headers only, never sent to LLM)
{{secret__api_key}}⚠️ Error: Missing variables cause "Missing required dynamic variables" - always provide all referenced variables.
在提示词、消息和工具参数中使用语法。
{{var_name}}系统变量:
- 、
{{system__agent_id}}{{system__conversation_id}} - 、
{{system__caller_id}}(电话场景){{system__called_number}} - 、
{{system__call_duration_secs}}{{system__time_utc}} - (仅Twilio)
{{system__call_sid}}
自定义变量:
typescript
await client.conversations.create({
agent_id: "agent_123",
dynamic_variables: { user_name: "John", account_tier: "premium" }
});保密变量: (仅在请求头中使用,绝不会发送给LLM)
{{secret__api_key}}⚠️ 错误提示: 缺失变量会导致"Missing required dynamic variables"错误 - 务必提供所有引用的变量。
3. Voice & Language Features
3. 音色与语言特性
Multi-Voice, Pronunciation & Speed
多音色、发音与语速
Multi-Voice - Switch voices dynamically (adds ~200ms latency per switch):
json
{ "prompt": "When speaking as customer, use voice_id 'voice_abc'. As agent, use 'voice_def'." }Pronunciation Dictionary - IPA, CMU, word substitutions (Turbo v2/v2.5 only):
json
{
"pronunciation_dictionary": [
{ "word": "API", "pronunciation": "ey-pee-ay", "format": "cmu" },
{ "word": "AI", "substitution": "artificial intelligence" }
]
}PATCH Support (Aug 2025) - Update dictionaries without replacement
Speed Control - 0.7x-1.2x (use 0.9x-1.1x for natural sound):
json
{ "voice_settings": { "speed": 1.0 } }Voice Cloning Best Practices:
- Clean audio (no noise, music, pops)
- Consistent microphone distance
- 1-2 minutes of audio
- Use language-matched voices (English voices fail on non-English)
多音色切换 - 动态切换音色(每次切换增加约200ms延迟):
json
{ "prompt": "当以客户身份说话时,使用voice_id 'voice_abc';以代理身份说话时,使用'voice_def'。" }发音词典 - 支持IPA、CMU音标、词汇替换(仅Turbo v2/v2.5支持):
json
{
"pronunciation_dictionary": [
{ "word": "API", "pronunciation": "ey-pee-ay", "format": "cmu" },
{ "word": "AI", "substitution": "人工智能" }
]
}PATCH支持(2025年8月) - 无需替换即可更新词典
语速控制 - 支持0.7x-1.2x(推荐0.9x-1.1x以保证自然度):
json
{ "voice_settings": { "speed": 1.0 } }音色克隆最佳实践:
- 使用无噪音、无音乐的干净音频
- 保持麦克风距离一致
- 音频时长1-2分钟
- 使用与目标语言匹配的音色(英文音色无法处理非英文内容)
Language Configuration
语言配置
32+ Languages with automatic detection and in-conversation switching.
Multi-Language Presets:
json
{
"language_presets": [
{ "language": "en", "voice_id": "en_voice", "first_message": "Hello!" },
{ "language": "es", "voice_id": "es_voice", "first_message": "¡Hola!" }
]
}支持32+种语言,可自动检测并在对话中切换语言。
多语言预设:
json
{
"language_presets": [
{ "language": "en", "voice_id": "en_voice", "first_message": "Hello!" },
{ "language": "es", "voice_id": "es_voice", "first_message": "¡Hola!" }
]
}4. Knowledge Base & RAG
4. 知识库与RAG
Enable agents to access large knowledge bases without loading entire documents into context.
Workflow:
- Upload documents (PDF, TXT, DOCX)
- Compute RAG index (vector embeddings)
- Agent retrieves relevant chunks during conversation
Configuration:
json
{
"agent": { "prompt": { "knowledge_base": ["doc_id_1", "doc_id_2"] } },
"knowledge_base_config": {
"max_chunks": 5,
"vector_distance_threshold": 0.8
}
}API Upload:
typescript
const doc = await client.knowledgeBase.upload({ file: fs.createReadStream('docs.pdf'), name: 'Docs' });
await client.knowledgeBase.computeRagIndex({ document_id: doc.id, embedding_model: 'e5_mistral_7b' });⚠️ Gotchas: RAG adds ~500ms latency. Check index status before use - indexing can take minutes.
让代理无需加载整个文档即可访问大型知识库。
工作流程:
- 上传文档(PDF、TXT、DOCX格式)
- 计算RAG索引(向量嵌入)
- 代理在对话过程中检索相关文本片段
配置示例:
json
{
"agent": { "prompt": { "knowledge_base": ["doc_id_1", "doc_id_2"] } },
"knowledge_base_config": {
"max_chunks": 5,
"vector_distance_threshold": 0.8
}
}API上传方式:
typescript
const doc = await client.knowledgeBase.upload({ file: fs.createReadStream('docs.pdf'), name: 'Docs' });
await client.knowledgeBase.computeRagIndex({ document_id: doc.id, embedding_model: 'e5_mistral_7b' });⚠️ 注意事项: RAG会增加约500ms延迟。使用前请检查索引状态 - 索引构建可能需要数分钟。
5. Tools (4 Types)
5. 工具(4种类型)
⚠️ BREAKING CHANGE: prompt.tools Deprecated (July 2025)
⚠️ 重大变更:prompt.tools已废弃(2025年7月)
The legacy array was removed on July 23, 2025. All agent configurations must use the new format.
prompt.toolsMigration Timeline:
- July 14, 2025: Legacy format still accepted
- July 15, 2025: GET responses stop including field
tools - July 23, 2025: POST/PATCH reject (active now)
prompt.tools
Old Format (no longer works):
typescript
{
agent: {
prompt: {
tools: [{ name: "get_weather", url: "...", method: "GET" }]
}
}
}New Format (required):
typescript
{
agent: {
prompt: {
tool_ids: ["tool_abc123"], // Client/server tools
built_in_tools: ["end_call"] // System tools (new field)
}
}
}Error if both used: "A request must include either prompt.tool_ids or the legacy prompt.tools array — never both"
Note: All tools from legacy format were auto-migrated to standalone tool records.
Source: Official Migration Guide
旧版数组已于2025年7月23日移除。所有Agent配置必须使用新格式。
prompt.tools迁移时间线:
- 2025年7月14日:仍接受旧版格式
- 2025年7月15日:GET响应不再包含字段
tools - 2025年7月23日:POST/PATCH请求将拒绝字段(当前已生效)
prompt.tools
旧格式(已无法使用):
typescript
{
agent: {
prompt: {
tools: [{ name: "get_weather", url: "...", method: "GET" }]
}
}
}新格式(必填):
typescript
{
agent: {
prompt: {
tool_ids: ["tool_abc123"], // 客户端/服务器端工具
built_in_tools: ["end_call"] // 系统工具(新增字段)
}
}
}同时使用两种格式的错误提示: "A request must include either prompt.tool_ids or the legacy prompt.tools array — never both"
说明: 旧格式中的所有工具已自动迁移为独立的工具记录。
来源: 官方迁移指南
A. Client Tools (Browser/Mobile)
A. 客户端工具(浏览器/移动端)
Execute in browser or mobile app. Tool names case-sensitive.
typescript
clientTools: {
updateCart: {
description: "Update shopping cart",
parameters: z.object({ item: z.string(), quantity: z.number() }),
handler: async ({ item, quantity }) => {
// Client-side logic
return { success: true };
}
}
}在浏览器或移动应用中执行。工具名称区分大小写。
typescript
clientTools: {
updateCart: {
description: "更新购物车",
parameters: z.object({ item: z.string(), quantity: z.number() }),
handler: async ({ item, quantity }) => {
// 客户端逻辑
return { success: true };
}
}
}B. Server Tools (Webhooks)
B. 服务器端工具(Webhook)
HTTP requests to external APIs. PUT support added Apr 2025.
json
{
"name": "get_weather",
"url": "https://api.weather.com/{{user_id}}",
"method": "GET",
"headers": { "Authorization": "Bearer {{secret__api_key}}" },
"parameters": { "type": "object", "properties": { "city": { "type": "string" } } }
}⚠️ Secret variables only in headers (not URL/body)
2025 Features:
- transfer-to-human system tool (Apr 2025)
- tool_latency_secs tracking (Apr 2025)
⚠️ Historical Issue (Fixed Feb 2025):
Tool calling was broken with due to an OpenAI API change. This was fixed in SDK v2.25.0+ (Feb 17, 2025). If using older SDK versions, upgrade to avoid silent tool execution failures on that model.
gpt-4o-miniSource: Changelog Feb 17, 2025
向外部API发送HTTP请求。2025年4月新增PUT支持。
json
{
"name": "get_weather",
"url": "https://api.weather.com/{{user_id}}",
"method": "GET",
"headers": { "Authorization": "Bearer {{secret__api_key}}" },
"parameters": { "type": "object", "properties": { "city": { "type": "string" } } }
}⚠️ 保密变量仅可在请求头中使用(不可在URL或请求体中)
2025年新增特性:
- 系统工具(2025年4月)
transfer-to-human - 延迟跟踪(2025年4月)
tool_latency_secs
⚠️ 历史问题(2025年2月已修复):
由于OpenAI API变更,的工具调用功能曾出现故障。该问题已在SDK v2.25.0+(2025年2月17日)中修复。若使用旧版SDK,请升级以避免该模型下的工具执行静默失败。
gpt-4o-mini来源: 2025年2月17日更新日志
C. MCP Tools (Model Context Protocol)
C. MCP工具(模型上下文协议)
Connect to MCP servers for databases, IDEs, data sources.
Configuration: Dashboard → Add Custom MCP Server → Configure SSE/HTTP endpoint
Approval Modes: Always Ask | Fine-Grained | No Approval
2025 Updates:
- disable_interruptions flag (Oct 2025) - Prevents interruption during tool execution
- Tools Management Interface (Jun 2025)
⚠️ Limitations: SSE/HTTP only. Not available for Zero Retention or HIPAA.
连接MCP服务器以访问数据库、IDE、数据源。
配置方式: 控制台 → 添加自定义MCP服务器 → 配置SSE/HTTP端点
审批模式: 始终询问 | 细粒度控制 | 无需审批
2025年更新:
- 标志(2025年10月)- 工具执行过程中禁止打断
disable_interruptions - 工具管理界面(2025年6月)
⚠️ 限制: 仅支持SSE/HTTP。零保留或HIPAA合规模式下不可用。
D. System Tools
D. 系统工具
Built-in conversation control (no external APIs):
- ,
end_call,detect_languagetransfer_agent - (telephony)
transfer_to_number - ,
dtmf_playpad(telephony)voicemail_detection
2025: flag for telephony integration
use_out_of_band_dtmf内置对话控制工具(无需外部API):
- 、
end_call、detect_languagetransfer_agent - (电话场景)
transfer_to_number - 、
dtmf_playpad(电话场景)voicemail_detection
2025年新增: 电话集成中的标志
use_out_of_band_dtmf6. SDK Integration
6. SDK集成
useConversation Hook (React/React Native)
useConversation钩子(React/React Native)
typescript
const { startConversation, stopConversation, status, isSpeaking } = useConversation({
agentId: 'your-agent-id',
signedUrl: '/api/auth', // OR apiKey: process.env.NEXT_PUBLIC_ELEVENLABS_API_KEY
clientTools: { /* ... */ },
onEvent: (event) => { /* transcript, agent_response, tool_call, agent_tool_request (Oct 2025) */ },
onConnect/onDisconnect/onError,
serverLocation: 'us' // 'eu-residency' | 'in-residency' | 'global'
});2025 Events:
- - Streaming responses (Oct 2025)
agent_chat_response_part - - Tool interaction tracking (Oct 2025)
agent_tool_request
typescript
const { startConversation, stopConversation, status, isSpeaking } = useConversation({
agentId: 'your-agent-id',
signedUrl: '/api/auth', // 或 apiKey: process.env.NEXT_PUBLIC_ELEVENLABS_API_KEY
clientTools: { /* ... */ },
onEvent: (event) => { /* 处理转录文本、agent响应、工具调用、agent_tool_request(2025年10月新增)等事件 */ },
onConnect/onDisconnect/onError,
serverLocation: 'us' // 可选值:'eu-residency' | 'in-residency' | 'global'
});2025年新增事件:
- - 流式响应(2025年10月)
agent_chat_response_part - - 工具交互跟踪(2025年10月)
agent_tool_request
Connection Types: WebRTC vs WebSocket
连接类型:WebRTC vs WebSocket
| Feature | WebSocket | WebRTC (Jul 2025 rollout) |
|---|---|---|
| Auth | | |
| Audio | Configurable (16k/24k/48k) | PCM_48000 (hardcoded) |
| Latency | Standard | Lower |
| Best For | Flexibility | Low-latency |
⚠️ WebRTC: Hardcoded PCM_48000, limited device switching
| 特性 | WebSocket | WebRTC(2025年7月推出) |
|---|---|---|
| 认证方式 | | |
| 音频格式 | 可配置(16k/24k/48k) | PCM_48000(硬编码) |
| 延迟 | 标准延迟 | 更低延迟 |
| 最佳适用场景 | 灵活性需求 | 低延迟需求 |
⚠️ WebRTC注意事项: 硬编码PCM_48000格式,设备切换受限
Platforms
支持平台
- React:
@elevenlabs/react@0.12.3 - JavaScript: -
@elevenlabs/client@0.12.2new Conversation({...}) - React Native: - Expo SDK 47+, iOS/macOS (custom build required, no Expo Go)
@elevenlabs/react-native@0.5.7 - Swift: iOS 14.0+, macOS 11.0+, Swift 5.9+
- Embeddable Widget:
<script src="https://elevenlabs.io/convai-widget/index.js"></script> - Widget Packages (Dec 2025):
- - For embedding in existing apps
@elevenlabs/convai-widget-embed@0.5.5 - - Core widget functionality
@elevenlabs/convai-widget-core@0.5.5
- React:
@elevenlabs/react@0.12.3 - JavaScript:- 使用
@elevenlabs/client@0.12.2new Conversation({...}) - React Native:- 支持Expo SDK 47+,iOS/macOS(需自定义构建,不支持Expo Go)
@elevenlabs/react-native@0.5.7 - Swift:iOS 14.0+、macOS 11.0+、Swift 5.9+
- 可嵌入组件:
<script src="https://elevenlabs.io/convai-widget/index.js"></script> - 组件包(2025年12月):
- - 用于嵌入现有应用
@elevenlabs/convai-widget-embed@0.5.5 - - 核心组件功能
@elevenlabs/convai-widget-core@0.5.5
Scribe (Real-Time Speech-to-Text - Beta 2025)
Scribe(实时语音转文本 - 2025年测试版)
Real-time transcription with word-level timestamps. Single-use tokens, not API keys.
typescript
const { connect, startRecording, stopRecording, transcript, partialTranscript } = useScribe({
token: async () => (await fetch('/api/scribe/token')).json().then(d => d.token),
commitStrategy: 'vad', // 'vad' (auto on silence) | 'manual' (explicit .commit())
sampleRate: 16000, // 16000 or 24000
onPartialTranscript/onFinalTranscript/onError
});Events: PARTIAL_TRANSCRIPT, FINAL_TRANSCRIPT_WITH_TIMESTAMPS, SESSION_STARTED, ERROR
⚠️ Closed Beta - requires sales contact. For agents, use Agents Platform instead (LLM + TTS + two-way interaction).
⚠️ Webhook Mode Issue:
Using with causes SDK parsing errors. The API returns only for webhook mode, but the SDK expects the full transcription schema.
speechToText.convert()webhook: true{ request_id }Error Message:
ParseError: response: Missing required key "language_code"; Missing required key "text"; ...Workaround - Use direct fetch API instead of SDK:
typescript
const formData = new FormData();
formData.append('file', audioFile);
formData.append('model_id', 'scribe_v1');
formData.append('webhook', 'true');
formData.append('webhook_id', webhookId);
const response = await fetch('https://api.elevenlabs.io/v1/speech-to-text', {
method: 'POST',
headers: { 'xi-api-key': apiKey },
body: formData,
});
const result = await response.json(); // { request_id: 'xxx' }
// Actual transcription delivered to webhook endpointSource: GitHub Issue #232 (confirmed by maintainer)
带词级时间戳的实时转录功能。仅支持一次性令牌,不支持API密钥。
typescript
const { connect, startRecording, stopRecording, transcript, partialTranscript } = useScribe({
token: async () => (await fetch('/api/scribe/token')).json().then(d => d.token),
commitStrategy: 'vad', // 可选值:'vad'(静默时自动提交) | 'manual'(显式调用.commit())
sampleRate: 16000, // 可选值:16000或24000
onPartialTranscript/onFinalTranscript/onError
});事件类型: PARTIAL_TRANSCRIPT、FINAL_TRANSCRIPT_WITH_TIMESTAMPS、SESSION_STARTED、ERROR
⚠️ 封闭测试版 - 需要联系销售团队获取权限。若开发代理,请使用Agents平台(集成LLM + TTS + 双向交互)。
⚠️ Webhook模式问题:
使用并设置会导致SDK解析错误。API在Webhook模式下仅返回,但SDK期望完整的转录结构。
speechToText.convert()webhook: true{ request_id }错误提示:
ParseError: response: Missing required key "language_code"; Missing required key "text"; ...替代方案 - 使用直接fetch API而非SDK:
typescript
const formData = new FormData();
formData.append('file', audioFile);
formData.append('model_id', 'scribe_v1');
formData.append('webhook', 'true');
formData.append('webhook_id', webhookId);
const response = await fetch('https://api.elevenlabs.io/v1/speech-to-text', {
method: 'POST',
headers: { 'xi-api-key': apiKey },
body: formData,
});
const result = await response.json(); // { request_id: 'xxx' }
// 实际转录结果将发送至Webhook端点来源: GitHub Issue #232(已由维护者确认)
7. Testing & Evaluation
7. 测试与评估
🆕 Agent Testing Framework (Aug 2025)
🆕 Agent测试框架(2025年8月)
Comprehensive automated testing with 9 new API endpoints for creating, managing, and executing tests.
Test Types:
- Scenario Testing - LLM-based evaluation against success criteria
- Tool Call Testing - Verify correct tool usage and parameters
- Load Testing - High-concurrency capacity testing
CLI Workflow:
bash
undefined全面的自动化测试框架,包含9个新API端点用于创建、管理和执行测试。
测试类型:
- 场景测试 - 基于LLM的成功标准评估
- 工具调用测试 - 验证工具的正确使用与参数传递
- 负载测试 - 高并发容量测试
CLI工作流:
bash
undefinedCreate test
创建测试
elevenlabs tests add "Refund Test" --template basic-llm
elevenlabs tests add "退款测试" --template basic-llm
Configure in test_configs/refund-test.json
在test_configs/refund-test.json中配置
{
"name": "Refund Test",
"scenario": "Customer requests refund",
"success_criteria": ["Agent acknowledges empathetically", "Verifies order details"],
"expected_tool_call": { "tool_name": "lookup_order", "parameters": { "order_id": "..." } }
}
{
"name": "退款测试",
"scenario": "客户申请退款",
"success_criteria": ["Agent需共情地确认请求", "验证订单详情"],
"expected_tool_call": { "tool_name": "lookup_order", "parameters": { "order_id": "..." } }
}
Deploy and execute
部署并执行测试
elevenlabs tests push
elevenlabs agents test "Support Agent"
**9 New API Endpoints (Aug 2025):**
1. `POST /v1/convai/tests` - Create test
2. `GET /v1/convai/tests/:id` - Retrieve test
3. `PATCH /v1/convai/tests/:id` - Update test
4. `DELETE /v1/convai/tests/:id` - Delete test
5. `POST /v1/convai/tests/:id/execute` - Execute test
6. `GET /v1/convai/test-invocations` - List invocations (pagination, agent filtering)
7. `POST /v1/convai/test-invocations/:id/resubmit` - Resubmit failed test
8. `GET /v1/convai/test-results/:id` - Get results
9. `GET /v1/convai/test-results/:id/debug` - Detailed debugging info
**Test Invocation Listing (Oct 2025):**
```typescript
const invocations = await client.convai.testInvocations.list({
agent_id: 'agent_123', // Filter by agent
page_size: 30, // Default 30, max 100
cursor: 'next_page_cursor' // Pagination
});
// Returns: test run counts, pass/fail stats, titlesProgrammatic Testing:
typescript
const simulation = await client.agents.simulate({
agent_id: 'agent_123',
scenario: 'Refund request',
user_messages: ["I want a refund", "Order #12345"],
success_criteria: ["Acknowledges request", "Verifies order"]
});
console.log('Passed:', simulation.passed);Agent Tracking (Oct 2025): Tests now include association for better organization
agent_idelevenlabs tests push
elevenlabs agents test "Support Agent"
**9个新API端点(2025年8月):**
1. `POST /v1/convai/tests` - 创建测试
2. `GET /v1/convai/tests/:id` - 获取测试详情
3. `PATCH /v1/convai/tests/:id` - 更新测试
4. `DELETE /v1/convai/tests/:id` - 删除测试
5. `POST /v1/convai/tests/:id/execute` - 执行测试
6. `GET /v1/convai/test-invocations` - 列出测试执行记录(分页、按Agent筛选)
7. `POST /v1/convai/test-invocations/:id/resubmit` - 重新提交失败的测试
8. `GET /v1/convai/test-results/:id` - 获取测试结果
9. `GET /v1/convai/test-results/:id/debug` - 获取详细调试信息
**测试执行记录列表(2025年10月):**
```typescript
const invocations = await client.convai.testInvocations.list({
agent_id: 'agent_123', // 按Agent筛选
page_size: 30, // 默认30,最大100
cursor: 'next_page_cursor' // 分页游标
});
// 返回结果:测试运行次数、通过/失败统计、测试标题编程式测试:
typescript
const simulation = await client.agents.simulate({
agent_id: 'agent_123',
scenario: '退款请求',
user_messages: ["我要退款", "订单号#12345"],
success_criteria: ["确认请求", "验证订单"]
});
console.log('是否通过:', simulation.passed);Agent跟踪(2025年10月): 测试现在包含关联字段,便于组织管理
agent_id8. Analytics & Monitoring
8. 分析与监控
2025 Features:
- Custom Dashboard Charts (Apr 2025) - Display evaluation criteria metrics over time
- Call History Filtering (Apr 2025) - parameter
call_start_before_unix - Multi-Voice History - Separate conversation history by voice
- LLM Cost Tracking - Per agent/conversation costs with (hour/day/week/month)
aggregation_interval - Tool Latency (Apr 2025) - tracking
tool_latency_secs - Usage Metrics - minutes_used, request_count, ttfb_avg, ttfb_p95
Conversation Analysis: Success evaluation (LLM-based), data collection fields, post-call webhooks
Access: Dashboard → Analytics | Post-call Webhooks | API
2025年新增特性:
- 自定义控制台图表(2025年4月)- 随时间展示评估标准指标
- 通话历史筛选(2025年4月)- 参数
call_start_before_unix - 多音色历史记录 - 按音色分离对话历史
- LLM成本跟踪 - 按Agent/对话统计成本,支持(小时/天/周/月)
aggregation_interval - 工具延迟跟踪(2025年4月)- 指标
tool_latency_secs - 使用指标 - minutes_used、request_count、ttfb_avg、ttfb_p95
对话分析: 基于LLM的成功评估、数据收集字段、通话后Webhook
访问方式: 控制台 → 分析 | 通话后Webhook | API
9. Privacy & Compliance
9. 隐私与合规
Data Retention: 2 years default (GDPR). Configure:
{ "transcripts": { "retention_days": 730 }, "audio": { "retention_days": 2190 } }Encryption: TLS 1.3 (transit), AES-256 (rest)
Regional:
serverLocation: 'eu-residency' | 'us' | 'global' | 'in-residency'Zero Retention Mode: Immediate deletion (no history, analytics, webhooks, or MCP)
Compliance: GDPR (1-2 years), HIPAA (6 years), SOC 2 (automatic encryption)
数据保留: 默认2年(符合GDPR)。可配置:
{ "transcripts": { "retention_days": 730 }, "audio": { "retention_days": 2190 } }加密: 传输过程使用TLS 1.3,静态存储使用AES-256
区域部署:
serverLocation: 'eu-residency' | 'us' | 'global' | 'in-residency'零保留模式: 立即删除数据(无历史记录、分析、Webhook或MCP功能)
合规认证: GDPR(1-2年)、HIPAA(6年)、SOC 2(自动加密)
10. Cost Optimization
10. 成本优化
LLM Caching: Up to 90% savings on repeated inputs.
{ "caching": { "enabled": true, "ttl_seconds": 3600 } }Model Swapping: GPT-5.1, GPT-4o/mini, Claude Sonnet 4.5, Gemini 3 Pro/2.5 Flash (2025 models)
Burst Pricing: 3x concurrency limit at 2x cost.
{ "burst_pricing_enabled": true }LLM缓存: 重复输入可节省高达90%的成本。配置:
{ "caching": { "enabled": true, "ttl_seconds": 3600 } }模型切换: 支持GPT-5.1、GPT-4o/mini、Claude Sonnet 4.5、Gemini 3 Pro/2.5 Flash(2025年模型)
突发定价: 以2倍成本获得3倍并发限制。配置:
{ "burst_pricing_enabled": true }11. Advanced Features
11. 高级特性
2025 Platform Updates:
- Azure OpenAI (Jul 2025) - Custom LLM with Azure-hosted models (requires API version field)
- Genesys Output Variables (Jul 2025) - Enhanced call analytics
- LLMReasoningEffort "none" (Oct 2025) - Control model reasoning behavior
- Streaming Voice Previews (Jul 2025) - Real-time voice generation
- pcm_48000 audio format (Apr 2025) - New output format support
Events: , , , , (streaming, Oct 2025), (Oct 2025),
audiotranscriptagent_responsetool_callagent_chat_response_partagent_tool_requestconversation_stateCustom Models: Bring your own LLM (OpenAI-compatible endpoints).
{ "llm_config": { "custom": { "endpoint": "...", "api_key": "{{secret__key}}" } } }Post-Call Webhooks: HMAC verification required. Return 200 or auto-disable after 10 failures. Payload includes conversation_id, transcript, analysis.
Chat Mode: Text-only (no ASR/TTS). . Saves ~200ms + costs.
{ "chat_mode": true }Telephony: SIP (sip-static.rtc.elevenlabs.io), Twilio native, Vonage, RingCentral. 2025: Twilio keypad fix (Jul), SIP TLS remote_domains validation (Oct)
2025年平台更新:
- Azure OpenAI(2025年7月)- 支持Azure托管的自定义LLM(需API版本字段)
- Genesys输出变量(2025年7月)- 增强的通话分析
- LLMReasoningEffort "none"(2025年10月)- 控制模型推理行为
- 流式音色预览(2025年7月)- 实时音色生成
- pcm_48000音频格式(2025年4月)- 新增输出格式支持
事件类型: 、、、、(流式,2025年10月)、(2025年10月)、
audiotranscriptagent_responsetool_callagent_chat_response_partagent_tool_requestconversation_state自定义模型: 接入自有LLM(兼容OpenAI的端点)。配置:
{ "llm_config": { "custom": { "endpoint": "...", "api_key": "{{secret__key}}" } } }通话后Webhook: 要求HMAC验证。返回200响应,否则10次失败后自动禁用。负载包含conversation_id、转录文本、分析结果。
聊天模式: 纯文本模式(无ASR/TTS)。配置:。可节省约200ms延迟及成本。
{ "chat_mode": true }电话集成: SIP(sip-static.rtc.elevenlabs.io)、原生Twilio、Vonage、RingCentral。2025年更新: Twilio键盘修复(7月)、SIP TLS remote_domains验证(10月)
12. CLI & DevOps ("Agents as Code")
12. CLI与DevOps("代理即代码")
Installation & Auth:
bash
npm install -g @elevenlabs/agents-cli@0.6.1
elevenlabs auth login
elevenlabs auth residency eu-residency # 'in-residency' | 'global'
export ELEVENLABS_API_KEY=your-api-key # For CI/CDProject Structure: , , + , ,
agents.jsontools.jsontests.jsonagent_configs/tool_configs/test_configs/Key Commands:
bash
elevenlabs agents init
elevenlabs agents add "Bot" --template customer-service
elevenlabs agents push --env prod --dry-run # Preview
elevenlabs agents push --env prod # Deploy
elevenlabs agents pull # Import existing
elevenlabs agents test "Bot" # 2025: Enhanced testing
elevenlabs tools add-webhook "Weather" --config-path tool_configs/weather.json
elevenlabs tools push
elevenlabs tests add "Test" --template basic-llm
elevenlabs tests pushMulti-Environment: Create , , for overrides
agent.dev.jsonagent.staging.jsonagent.prod.jsonCI/CD: GitHub Actions with validation before deploy
--dry-run.gitignore: , ,
.env.elevenlabs/*.secret.json安装与认证:
bash
npm install -g @elevenlabs/agents-cli@0.6.1
elevenlabs auth login
elevenlabs auth residency eu-residency // 可选值:'in-residency' | 'global'
export ELEVENLABS_API_KEY=your-api-key // 用于CI/CD环境项目结构: 、、 + 、、
agents.jsontools.jsontests.jsonagent_configs/tool_configs/test_configs/核心命令:
bash
elevenlabs agents init
elevenlabs agents add "Bot" --template customer-service
elevenlabs agents push --env prod --dry-run // 预览部署变更
elevenlabs agents push --env prod // 部署到生产环境
elevenlabs agents pull // 导入现有代理配置
elevenlabs agents test "Bot" // 2025年:增强测试功能
elevenlabs tools add-webhook "天气查询" --config-path tool_configs/weather.json
elevenlabs tools push
elevenlabs tests add "测试用例" --template basic-llm
elevenlabs tests push多环境支持: 创建、、用于环境覆盖配置
agent.dev.jsonagent.staging.jsonagent.prod.jsonCI/CD集成: GitHub Actions中使用验证后再部署
--dry-run.gitignore配置: 、、
.env.elevenlabs/*.secret.json13. Common Errors & Solutions (27 Documented)
13. 常见错误与解决方案(27种已记录)
Error 1: Missing Required Dynamic Variables
错误1:缺失必填动态变量
Cause: Variables referenced in prompts not provided at conversation start
Solution: Provide all variables in
dynamic_variables: { user_name: "John", ... }原因: 提示词中引用的变量未在对话启动时提供
解决方案: 在中提供所有变量
dynamic_variables: { user_name: "John", ... }Error 2: Case-Sensitive Tool Names
错误2:工具名称大小写不匹配
Cause: Tool name mismatch (case-sensitive)
Solution: Ensure matches exactly
tool_ids: ["orderLookup"]name: "orderLookup"原因: 工具名称大小写不一致(区分大小写)
解决方案: 确保与完全匹配
tool_ids: ["orderLookup"]name: "orderLookup"Error 3: Webhook Authentication Failures
错误3:Webhook认证失败
Cause: Incorrect HMAC signature, not returning 200, or 10+ failures
Solution: Verify and return 200
⚠️ Header Name: Use (NOT - no X- prefix!)
hmac = crypto.createHmac('sha256', SECRET).update(payload).digest('hex')ElevenLabs-SignatureX-ElevenLabs-Signature原因: HMAC签名错误、未返回200响应或失败次数超过10次
解决方案: 验证,并确保返回200响应
⚠️ 注意请求头名称: 使用(不要加X-前缀,即不要使用!)
hmac = crypto.createHmac('sha256', SECRET).update(payload).digest('hex')ElevenLabs-SignatureX-ElevenLabs-SignatureError 4: Voice Consistency Issues
错误4:音色一致性问题
Cause: Background noise, inconsistent mic distance, extreme volumes in training
Solution: Use clean audio, consistent distance, avoid extremes
原因: 背景噪音、麦克风距离不一致、训练音频音量极端
解决方案: 使用干净音频、保持麦克风距离一致、避免音量极端
Error 5: Wrong Language Voice
错误5:音色与语言不匹配
Cause: English-trained voice for non-English language
Solution: Use language-matched voices:
{ "language": "es", "voice_id": "spanish_voice" }原因: 使用英文训练的音色处理非英文内容
解决方案: 使用与语言匹配的音色:
{ "language": "es", "voice_id": "spanish_voice" }Error 6: Restricted API Keys Not Supported (CLI)
错误6:CLI不支持受限API密钥
Cause: CLI doesn't support restricted API keys
Solution: Use unrestricted API key for CLI
原因: CLI工具不支持受限API密钥
解决方案: 为CLI使用无限制API密钥
Error 7: Agent Configuration Push Conflicts
错误7:Agent配置推送冲突
Cause: Hash-based change detection missed modification
Solution: + + push
elevenlabs agents init --overrideelevenlabs agents pull原因: 基于哈希的变更检测未捕获修改
解决方案: 执行 + 后再推送
elevenlabs agents init --overrideelevenlabs agents pullError 8: Tool Parameter Schema Mismatch
错误8:工具参数 schema 不匹配
Cause: Schema doesn't match usage
Solution: Add clear descriptions:
"description": "Order ID (format: ORD-12345)"原因: Schema定义与实际使用不匹配
解决方案: 添加清晰的描述:
"description": "订单ID(格式:ORD-12345)"Error 9: RAG Index Not Ready
错误9:RAG索引未就绪
Cause: Index still computing (takes minutes)
Solution: Check before using
index.status === 'ready'原因: 索引仍在构建中(需数分钟)
解决方案: 使用前检查
index.status === 'ready'Error 10: WebSocket Protocol Error (1002)
错误10:WebSocket协议错误(1002)
Cause: Network instability, incompatible browser, or firewall issues
Symptoms:
Error receiving message: received 1002 (protocol error)
Error sending user audio chunk: received 1002 (protocol error)
WebSocket is already in CLOSING or CLOSED stateConnection cycles: Disconnected → Connected → Disconnected rapidly
Solution:
- Use WebRTC instead of WebSocket for better stability:
connectionType: 'webrtc' - Implement reconnection logic with exponential backoff
- Check network stability and firewall rules (port restrictions)
- Test on different networks/browsers to isolate the issue
Source: GitHub Issue #134
原因: 网络不稳定、浏览器不兼容或防火墙限制
症状:
Error receiving message: received 1002 (protocol error)
Error sending user audio chunk: received 1002 (protocol error)
WebSocket is already in CLOSING or CLOSED state连接循环:断开 → 连接 → 快速断开
解决方案:
- 使用WebRTC替代WebSocket以提升稳定性:
connectionType: 'webrtc' - 实现带指数退避的重连逻辑
- 检查网络稳定性与防火墙规则(端口限制)
- 在不同网络/浏览器中测试以隔离问题
Error 11: 401 Unauthorized in Production
错误11:生产环境中401未授权
Cause: Agent visibility or API key config
Solution: Check visibility (public/private), verify API key in prod, check allowlist
原因: Agent可见性或API密钥配置问题
解决方案: 检查可见性(公开/私有)、验证生产环境API密钥、检查白名单配置
Error 12: Allowlist Connection Errors
错误12:白名单连接错误
Cause: Allowlist enabled but using shared link, OR localhost validation bug
Symptoms:
Host is not supported
Host is not valid or supported
Host is not in insights whitelist
WebSocket is already in CLOSING or CLOSED stateSolution:
- Configure allowlist domains in dashboard or disable for testing
- Localhost workaround: Use instead of
127.0.0.1:3000localhost:3000
⚠️ Localhost Validation Bug:
The dashboard has inconsistent validation for localhost URLs:
- ❌ → Rejected (should be valid)
localhost:3000 - ❌ → Rejected (protocol not allowed)
http://localhost:3000 - ❌ → Rejected (paths not allowed)
localhost:3000/voice-chat - ✅ → Accepted (invalid but accepted!)
www.localhost:3000 - ✅ → Accepted (use this for local dev)
127.0.0.1:3000
Source: GitHub Issue #320
原因: 已启用白名单但使用共享链接,或本地主机验证bug
症状:
Host is not supported
Host is not valid or supported
Host is not in insights whitelist
WebSocket is already in CLOSING or CLOSED state解决方案:
- 在控制台配置白名单域名,或测试时禁用白名单
- 本地主机替代方案: 使用而非
127.0.0.1:3000localhost:3000
⚠️ 本地主机验证bug:
控制台对本地主机URL的验证存在不一致:
- ❌ → 被拒绝(本应有效)
localhost:3000 - ❌ → 被拒绝(协议不允许)
http://localhost:3000 - ❌ → 被拒绝(不允许路径)
localhost:3000/voice-chat - ✅ → 被接受(无效但被允许!)
www.localhost:3000 - ✅ → 被接受(本地开发请使用此地址)
127.0.0.1:3000
Error 13: Workflow Infinite Loops
错误13:工作流无限循环
Cause: Edge conditions creating loops
Solution: Add max iteration limits, test all paths, explicit exit conditions
原因: 路由边条件导致循环
解决方案: 添加最大迭代限制、测试所有路径、设置明确的退出条件
Error 14: Burst Pricing Not Enabled
错误14:未启用突发定价
Cause: Burst not enabled in settings
Solution:
{ "call_limits": { "burst_pricing_enabled": true } }原因: 设置中未启用突发定价
解决方案: 配置
{ "call_limits": { "burst_pricing_enabled": true } }Error 15: MCP Server Timeout
错误15:MCP服务器超时
Cause: MCP server slow/unreachable
Solution: Check URL accessible, verify transport (SSE/HTTP), check auth, monitor logs
原因: MCP服务器缓慢或不可达
解决方案: 检查URL是否可访问、验证传输方式(SSE/HTTP)、检查认证、监控日志
Error 16: First Message Cutoff on Android
错误16:Android端第一条消息被截断
Cause: Android needs time to switch audio mode
Solution: (3s for audio routing)
connectionDelay: { android: 3_000, ios: 0 }原因: Android需要时间切换音频模式
解决方案: 配置(音频路由需3秒延迟)
connectionDelay: { android: 3_000, ios: 0 }Error 17: CSP (Content Security Policy) Violations
错误17:CSP(内容安全策略)违规
Cause: Strict CSP blocks URLs. SDK uses Audio Worklets loaded as blobs
Solution: Self-host worklets:
blob:cp node_modules/@elevenlabs/client/dist/worklets/*.js public/elevenlabs/- Configure:
workletPaths: { 'rawAudioProcessor': '/elevenlabs/rawAudioProcessor.worklet.js', 'audioConcatProcessor': '/elevenlabs/audioConcatProcessor.worklet.js' } - Update CSP: Gotcha: Update worklets when upgrading
script-src 'self' https://elevenlabs.io; worker-src 'self';@elevenlabs/client
原因: 严格的CSP阻止 URL。SDK使用以blob形式加载的Audio Worklets
解决方案: 自行托管worklets:
blob:cp node_modules/@elevenlabs/client/dist/worklets/*.js public/elevenlabs/- 配置:
workletPaths: { 'rawAudioProcessor': '/elevenlabs/rawAudioProcessor.worklet.js', 'audioConcatProcessor': '/elevenlabs/audioConcatProcessor.worklet.js' } - 更新CSP:注意事项: 升级
script-src 'self' https://elevenlabs.io; worker-src 'self';时需同步更新worklets@elevenlabs/client
Error 18: Webhook Payload - Null Message on Tool Calls
错误18:Webhook负载 - 工具调用时消息为Null
Cause: Schema expects but ElevenLabs sends when agent makes tool calls
Solution: Use for message field in Zod schemas
message: stringnullz.string().nullable()typescript
// ❌ Fails on tool call turns:
message: z.string()
// ✅ Correct:
message: z.string().nullable()Real payload example:
json
{ "role": "agent", "message": null, "tool_calls": [{ "tool_name": "my_tool", ... }] }原因: Schema期望但ElevenLabs在Agent调用工具时发送
解决方案: 在Zod schema中为message字段使用
message: stringnullz.string().nullable()typescript
// ❌ 工具调用回合会失败:
message: z.string()
// ✅ 正确配置:
message: z.string().nullable()实际负载示例:
json
{ "role": "agent", "message": null, "tool_calls": [{ "tool_name": "my_tool", ... }] }Error 19: Webhook Payload - call_successful is String, Not Boolean
错误19:Webhook负载 - call_successful为字符串而非布尔值
Cause: Schema expects but ElevenLabs sends or strings
Solution: Accept both types and convert for database storage
call_successful: boolean"success""failure"typescript
// Schema:
call_successful: z.union([z.boolean(), z.string()]).optional()
// Conversion helper:
function parseCallSuccessful(value: unknown): boolean | undefined {
if (value === undefined || value === null) return undefined
if (typeof value === 'boolean') return value
if (typeof value === 'string') return value.toLowerCase() === 'success'
return undefined
}原因: Schema期望但ElevenLabs发送或字符串
解决方案: 接受两种类型并转换后存储到数据库
call_successful: boolean"success""failure"typescript
// Schema定义:
call_successful: z.union([z.boolean(), z.string()]).optional()
// 转换工具函数:
function parseCallSuccessful(value: unknown): boolean | undefined {
if (value === undefined || value === null) return undefined
if (typeof value === 'boolean') return value
if (typeof value === 'string') return value.toLowerCase() === 'success'
return undefined
}Error 20: Webhook Schema Validation Fails Silently
错误20:Webhook Schema验证静默失败
Cause: Real ElevenLabs payloads have many undocumented fields that strict schemas reject
Undocumented fields in transcript turns:
- ,
agent_metadata,multivoice_message,llm_overriderag_retrieval_info - ,
llm_usage,interrupted,original_messageSolution: Add all assource_mediumwith.optional()for fields you don't process Debugging tip: Use https://webhook.site to capture real payloads, then test schema locallyz.any()
原因: 实际ElevenLabs负载包含许多未文档化的字段,严格的Schema会拒绝这些字段
转录回合中的未文档化字段:
- 、
agent_metadata、multivoice_message、llm_overriderag_retrieval_info - 、
llm_usage、interrupted、original_message解决方案: 为不需要处理的字段添加source_medium并使用.optional()类型 调试技巧: 使用https://webhook.site捕获实际负载,然后在本地测试Schemaz.any()
Error 21: Webhook Cost Field is Credits, NOT USD
错误21:Webhook成本字段为积分而非美元
Cause: contains ElevenLabs credits, not USD dollars. Displaying this directly shows wildly wrong values (e.g., "$78.0000" when actual cost is ~$0.003)
Solution: Extract actual USD from instead
metadata.costmetadata.charging.llm_pricetypescript
// ❌ Wrong - displays credits as dollars:
cost: metadata?.cost // Returns 78 (credits)
// ✅ Correct - actual USD cost:
const charging = metadata?.charging as any
cost: charging?.llm_price ?? null // Returns 0.0036 (USD)Real payload structure:
json
{
"metadata": {
"cost": 78, // ← CREDITS, not dollars!
"charging": {
"llm_price": 0.0036188999999999995, // ← Actual USD cost
"llm_charge": 18, // LLM credits
"call_charge": 60, // Audio credits
"tier": "pro"
}
}
}Note: only covers LLM costs. Audio costs may require separate calculation based on your plan.
llm_price原因: 包含ElevenLabs积分,而非美元。直接显示会导致数值严重错误(例如,实际成本约$0.003,但显示为"$78.0000")
解决方案: 从中提取实际美元成本
metadata.costmetadata.charging.llm_pricetypescript
// ❌ 错误 - 将积分显示为美元:
cost: metadata?.cost // 返回78(积分)
// ✅ 正确 - 实际美元成本:
const charging = metadata?.charging as any
cost: charging?.llm_price ?? null // 返回0.0036(美元)实际负载结构:
json
{
"metadata": {
"cost": 78, // ← 积分,而非美元!
"charging": {
"llm_price": 0.0036188999999999995, // ← 实际美元成本
"llm_charge": 18, // LLM积分消耗
"call_charge": 60, // 音频积分消耗
"tier": "pro"
}
}
}注意: 仅覆盖LLM成本。音频成本可能需要根据您的套餐单独计算。
llm_priceError 22: User Context Available But Not Extracted
错误22:用户上下文可用但未提取
Cause: Webhook contains authenticated user info from widget but code doesn't extract it
Solution: Extract from
dynamic_variablesconversation_initiation_client_datatypescript
const dynamicVars = data.conversation_initiation_client_data?.dynamic_variables
const callerName = dynamicVars?.user_name || null
const callerEmail = dynamicVars?.user_email || null
const currentPage = dynamicVars?.current_page || nullPayload example:
json
{
"conversation_initiation_client_data": {
"dynamic_variables": {
"user_name": "Jeremy Dawes",
"user_email": "jeremy@jezweb.net",
"current_page": "/dashboard/calls"
}
}
}原因: Webhook包含组件传递的已认证用户信息,但代码未提取
解决方案: 从中提取
conversation_initiation_client_datadynamic_variablestypescript
const dynamicVars = data.conversation_initiation_client_data?.dynamic_variables
const callerName = dynamicVars?.user_name || null
const callerEmail = dynamicVars?.user_email || null
const currentPage = dynamicVars?.current_page || null负载示例:
json
{
"conversation_initiation_client_data": {
"dynamic_variables": {
"user_name": "Jeremy Dawes",
"user_email": "jeremy@jezweb.net",
"current_page": "/dashboard/calls"
}
}
}Error 23: Data Collection Results Available But Not Displayed
错误23:数据收集结果可用但未展示
Cause: ElevenLabs agents can collect structured data during calls (configured in agent settings). This data is stored in but often not parsed/displayed in UI.
Solution: Parse the JSON and display collected fields with their values and rationales
analysis.data_collection_resultstypescript
const dataCollectionResults = analysis?.dataCollectionResults
? JSON.parse(analysis.dataCollectionResults)
: null
// Display each collected field:
Object.entries(dataCollectionResults).forEach(([key, data]) => {
console.log(`${key}: ${data.value} (${data.rationale})`)
})Payload example:
json
{
"data_collection_results": {
"customer_name": { "value": "John Smith", "rationale": "Customer stated their name" },
"intent": { "value": "billing_inquiry", "rationale": "Asking about invoice" },
"callback_number": { "value": "+61400123456", "rationale": "Provided for callback" }
}
}原因: ElevenLabs Agent可在通话过程中收集结构化数据(在Agent设置中配置)。这些数据存储在中,但通常未被解析/展示在UI中。
解决方案: 解析JSON并显示收集到的字段及其值和依据
analysis.data_collection_resultstypescript
const dataCollectionResults = analysis?.dataCollectionResults
? JSON.parse(analysis.dataCollectionResults)
: null
// 显示每个收集到的字段:
Object.entries(dataCollectionResults).forEach(([key, data]) => {
console.log(`${key}: ${data.value} (${data.rationale})`)
})负载示例:
json
{
"data_collection_results": {
"customer_name": { "value": "John Smith", "rationale": "客户已告知姓名" },
"intent": { "value": "billing_inquiry", "rationale": "询问发票相关问题" },
"callback_number": { "value": "+61400123456", "rationale": "客户提供的回拨号码" }
}
}Error 24: Evaluation Criteria Results Available But Not Displayed
错误24:评估标准结果可用但未展示
Cause: Custom success criteria (configured in agent) produce results in but often not parsed/displayed
Solution: Parse and show pass/fail status with rationales
analysis.evaluation_criteria_resultstypescript
const evaluationResults = analysis?.evaluationCriteriaResults
? JSON.parse(analysis.evaluationCriteriaResults)
: null
Object.entries(evaluationResults).forEach(([key, data]) => {
const passed = data.result === 'success' || data.result === true
console.log(`${key}: ${passed ? 'PASS' : 'FAIL'} - ${data.rationale}`)
})Payload example:
json
{
"evaluation_criteria_results": {
"verified_identity": { "result": "success", "rationale": "Customer verified DOB" },
"resolved_issue": { "result": "failure", "rationale": "Escalated to human" }
}
}原因: 自定义成功标准(在Agent中配置)的结果存储在中,但通常未被解析/展示
解决方案: 解析并显示通过/失败状态及依据
analysis.evaluation_criteria_resultstypescript
const evaluationResults = analysis?.evaluationCriteriaResults
? JSON.parse(analysis.evaluationCriteriaResults)
: null
Object.entries(evaluationResults).forEach(([key, data]) => {
const passed = data.result === 'success' || data.result === true
console.log(`${key}: ${passed ? '通过' : '失败'} - ${data.rationale}`)
})负载示例:
json
{
"evaluation_criteria_results": {
"verified_identity": { "result": "success", "rationale": "客户已验证出生日期" },
"resolved_issue": { "result": "failure", "rationale": "已转人工处理" }
}
}Error 25: Feedback Rating Available But Not Extracted
错误25:反馈评分可用但未提取
Cause: User can provide thumbs up/down feedback. Stored in but not extracted
Solution: Extract and store the rating (1 = thumbs up, -1 = thumbs down)
metadata.feedback.thumb_ratingtypescript
const feedback = metadata?.feedback as any
const feedbackRating = feedback?.thumb_rating ?? null // 1, -1, or null
// Also available:
const likes = feedback?.likes // Array of things user liked
const dislikes = feedback?.dislikes // Array of things user dislikedPayload example:
json
{
"metadata": {
"feedback": {
"thumb_rating": 1,
"likes": ["helpful", "natural"],
"dislikes": []
}
}
}原因: 用户可提供点赞/点踩反馈。存储在中但未被提取
解决方案: 提取并存储评分(1=点赞,-1=点踩)
metadata.feedback.thumb_ratingtypescript
const feedback = metadata?.feedback as any
const feedbackRating = feedback?.thumb_rating ?? null // 1、-1或null
// 还可获取:
const likes = feedback?.likes // 用户喜欢的点列表
const dislikes = feedback?.dislikes // 用户不喜欢的点列表负载示例:
json
{
"metadata": {
"feedback": {
"thumb_rating": 1,
"likes": ["有帮助", "自然"],
"dislikes": []
}
}
}Error 26: Per-Turn Metadata Not Extracted (interrupted, source_medium, rag_retrieval_info)
错误26:每回合元数据未提取(interrupted、source_medium、rag_retrieval_info)
Cause: Each transcript turn has valuable metadata that's often ignored
Solution: Store these fields per message for analytics and debugging
typescript
const turnAny = turn as any
const messageData = {
// ... existing fields
interrupted: turnAny.interrupted ?? null, // Was turn cut off by user?
sourceMedium: turnAny.source_medium ?? null, // Channel: web, phone, etc.
originalMessage: turnAny.original_message ?? null, // Pre-processed message
ragRetrievalInfo: turnAny.rag_retrieval_info // What knowledge was retrieved
? JSON.stringify(turnAny.rag_retrieval_info)
: null,
}Use cases:
- → User spoke over agent (UX insight)
interrupted: true - → Analytics by channel
source_medium - → Debug/improve knowledge base retrieval
rag_retrieval_info
原因: 每个转录回合包含有价值的元数据,但常被忽略
解决方案: 为每条消息存储这些字段以用于分析和调试
typescript
const turnAny = turn as any
const messageData = {
// ... 现有字段
interrupted: turnAny.interrupted ?? null, // 回合是否被用户打断?
sourceMedium: turnAny.source_medium ?? null, // 渠道:网页、电话等
originalMessage: turnAny.original_message ?? null, // 预处理前的消息
ragRetrievalInfo: turnAny.rag_retrieval_info // 检索到的知识库内容
? JSON.stringify(turnAny.rag_retrieval_info)
: null,
}使用场景:
- → 用户打断了Agent说话(用户体验洞察)
interrupted: true - → 按渠道分析
source_medium - → 调试/优化知识库检索
rag_retrieval_info
Error 27: Upcoming Audio Flags (August 2025)
错误27:即将到来的音频标志(2025年8月)
Cause: Three new boolean fields coming in August 2025 webhooks that may break schemas
Solution: Add these fields to schemas now (as optional) to be ready
typescript
// In webhook payload (coming August 15, 2025):
has_audio: boolean // Was full audio recorded?
has_user_audio: boolean // Was user audio captured?
has_response_audio: boolean // Was agent audio captured?
// Schema (future-proof):
const schema = z.object({
// ... existing fields
has_audio: z.boolean().optional(),
has_user_audio: z.boolean().optional(),
has_response_audio: z.boolean().optional(),
})Note: These match the existing fields in the GET Conversation API response
原因: 2025年8月Webhook将新增三个布尔字段,可能破坏现有Schema
解决方案: 现在就将这些字段添加到Schema中(设为可选)以做好准备
typescript
// Webhook负载(2025年8月15日起):
has_audio: boolean // 是否录制了完整音频?
has_user_audio: boolean // 是否捕获了用户音频?
has_response_audio: boolean // 是否捕获了Agent音频?
// Schema(面向未来):
const schema = z.object({
// ... 现有字段
has_audio: z.boolean().optional(),
has_user_audio: z.boolean().optional(),
has_response_audio: z.boolean().optional(),
})注意: 这些字段与GET Conversation API响应中的现有字段一致
Error 28: Tool Parsing Fails When Tool Not Found
错误28:工具不存在时工具解析失败
Cause: Calling when conversation contains tool_results where the tool was deleted/not found
Error Message:
conversations.get(id)Error: response -> transcript -> [11] -> tool_results -> [0] -> type:
Expected string. Received null.;
response -> transcript -> [11] -> tool_results -> [0] -> type:
[Variant 1] Expected "system". Received null.;
response -> transcript -> [11] -> tool_results -> [0] -> type:
[Variant 2] Expected "workflow". Received null.Solution:
- SDK fix needed - SDK should handle null tool_results.type gracefully
- Workaround for users:
- Ensure all referenced tools exist before deleting them
- Wrap in try-catch until SDK is fixed
conversation.get()
typescripttry { const conversation = await client.conversationalAi.conversations.get(id); } catch (error) { console.error('Tool parsing error - conversation may reference deleted tools'); }
Source: GitHub Issue #268
原因: 当对话包含已删除/不存在的工具的tool_results时,调用会失败
错误提示:
conversations.get(id)Error: response -> transcript -> [11] -> tool_results -> [0] -> type:
Expected string. Received null.;
response -> transcript -> [11] -> tool_results -> [0] -> type:
[Variant 1] Expected "system". Received null.;
response -> transcript -> [11] -> tool_results -> [0] -> type:
[Variant 2] Expected "workflow". Received null.解决方案:
- 需要SDK修复 - SDK应优雅处理null的tool_results.type
- 用户替代方案:
- 删除工具前确保所有引用该工具的对话已处理
- 在SDK修复前将包裹在try-catch中
conversation.get()
typescripttry { const conversation = await client.conversationalAi.conversations.get(id); } catch (error) { console.error('工具解析错误 - 对话可能引用了已删除的工具'); }
Error 29: SDK Parameter Naming Confusion (snake_case vs camelCase)
错误29:SDK参数命名混淆(蛇形命名 vs 驼峰式)
Cause: Using snake_case parameters (from API/Python SDK docs) in JS SDK, which expects camelCase
Symptoms: Parameters silently ignored, wrong model/voice used, no error messages
Common Mistakes:
typescript
// ❌ WRONG - parameter ignored:
convert(voiceId, { model_id: "eleven_v3" })
// ✅ CORRECT:
convert(voiceId, { modelId: "eleven_v3" })Solution: Always use camelCase for JS SDK parameters. Check TypeScript types if unsure.
Affected Parameters: , , , , and all API parameters
model_idvoice_idoutput_formatvoice_settingsSource: GitHub Issue #300
原因: 在JS SDK中使用API/Python SDK文档中的蛇形命名参数,但JS SDK期望驼峰式
症状: 参数被静默忽略、使用错误的模型/音色、无错误提示
常见错误:
typescript
// ❌ 错误 - 参数被忽略:
convert(voiceId, { model_id: "eleven_v3" })
// ✅ 正确:
convert(voiceId, { modelId: "eleven_v3" })解决方案: JS SDK参数始终使用驼峰式。不确定时请检查TypeScript类型定义。
受影响的参数: 、、、及所有API参数
model_idvoice_idoutput_formatvoice_settingsError 30: Webhook Mode ParseError with speechToText.convert()
错误30:speechToText.convert()的Webhook模式ParseError
Cause: SDK expects full transcription response but webhook mode returns only
Error Message:
{ request_id }ParseError: Missing required key "language_code"; Missing required key "text"; ...Solution: Use direct fetch API instead of SDK for webhook mode:
typescript
const formData = new FormData();
formData.append('file', audioFile);
formData.append('model_id', 'scribe_v1');
formData.append('webhook', 'true');
formData.append('webhook_id', webhookId);
const response = await fetch('https://api.elevenlabs.io/v1/speech-to-text', {
method: 'POST',
headers: { 'xi-api-key': apiKey },
body: formData,
});
const result = await response.json(); // { request_id: 'xxx' }Source: GitHub Issue #232
原因: SDK期望完整的转录响应,但Webhook模式仅返回
错误提示:
{ request_id }ParseError: Missing required key "language_code"; Missing required key "text"; ...解决方案: Webhook模式下使用直接fetch API而非SDK:
typescript
const formData = new FormData();
formData.append('file', audioFile);
formData.append('model_id', 'scribe_v1');
formData.append('webhook', 'true');
formData.append('webhook_id', webhookId);
const response = await fetch('https://api.elevenlabs.io/v1/speech-to-text', {
method: 'POST',
headers: { 'xi-api-key': apiKey },
body: formData,
});
const result = await response.json(); // { request_id: 'xxx' }Error 31: Package Not Compatible with Browser/Web
错误31:包与浏览器/网页环境不兼容
Cause: Using in browser/client environments (depends on Node.js )
Error Message:
@elevenlabs/elevenlabs-jschild_processModule not found: Can't resolve 'child_process'Affected Frameworks:
- Next.js client components
- Vite browser builds
- Electron renderer process
- Tauri webview
Solution:
- For browser/web: Use or
@elevenlabs/clientinstead@elevenlabs/react - For full API access in browser: Create proxy server endpoint using , call from browser
elevenlabs-js - For Electron/Tauri: Use in main process only, not renderer
elevenlabs-js
Note: This is by design - is server-only
elevenlabs-jsSource: GitHub Issue #293
原因: 在浏览器/客户端环境中使用(依赖Node.js的)
错误提示:
@elevenlabs/elevenlabs-jschild_processModule not found: Can't resolve 'child_process'受影响的框架:
- Next.js客户端组件
- Vite浏览器构建
- Electron渲染进程
- Tauri网页视图
解决方案:
- 浏览器/网页环境: 使用或
@elevenlabs/client替代@elevenlabs/react - 浏览器中需要完整API访问: 使用创建代理服务器端点,从浏览器调用该代理
elevenlabs-js - Electron/Tauri: 仅在主进程中使用,不要在渲染进程中使用
elevenlabs-js
说明: 这是设计使然 - 仅用于服务器端
elevenlabs-jsError 32: prompt.tools Deprecated - POST/PATCH Rejected
错误32:prompt.tools已废弃 - POST/PATCH请求被拒绝
Cause: Using legacy array field after July 23, 2025 cutoff
Error Message:
prompt.toolsA request must include either prompt.tool_ids or the legacy prompt.tools array — never bothSolution: Migrate to new format:
typescript
// ❌ Old (rejected):
{ agent: { prompt: { tools: [...] } } }
// ✅ New (required):
{
agent: {
prompt: {
tool_ids: ["tool_abc123"], // Client/server tools
built_in_tools: ["end_call"] // System tools
}
}
}Note: All legacy tools were auto-migrated to standalone records. Just update your configuration references.
Source: Official Migration Guide
原因: 2025年7月23日截止日期后仍使用旧版数组字段
错误提示:
prompt.toolsA request must include either prompt.tool_ids or the legacy prompt.tools array — never both解决方案: 迁移到新格式:
typescript
// ❌ 旧格式(已被拒绝):
{ agent: { prompt: { tools: [...] } } }
// ✅ 新格式(必填):
{
agent: {
prompt: {
tool_ids: ["tool_abc123"], // 客户端/服务器端工具
built_in_tools: ["end_call"] // 系统工具
}
}
}说明: 所有旧版工具已自动迁移为独立记录。只需更新配置引用即可。
来源: 官方迁移指南
Error 33: GPT-4o Mini Tool Calling Broken (Fixed Feb 2025)
错误33:GPT-4o Mini工具调用故障(2025年2月已修复)
Cause: OpenAI API breaking change affected tool execution (historical issue)
Symptoms: Tools silently fail to execute, no error messages
Solution: Upgrade to SDK v2.25.0+ (released Feb 17, 2025). If using older SDK versions, upgrade or avoid for tool-based workflows.
gpt-4o-minigpt-4o-miniSource: Changelog Feb 17, 2025
原因: OpenAI API的重大变更影响了的工具执行(历史问题)
症状: 工具执行静默失败,无错误提示
解决方案: 升级到SDK v2.25.0+(2025年2月17日发布)。若使用旧版SDK,请升级或避免在工具型工作流中使用。
gpt-4o-minigpt-4o-mini来源: 2025年2月17日更新日志
Error 34: Scribe Audio Format Parameter Not Transmitted (Fixed v2.32.0)
错误34:Scribe音频格式参数未传输(v2.32.0已修复)
Cause: WebSocket URI wasn't including parameter even when specified (historical issue)
Solution: Upgrade to or later (released Jan 19, 2026)
audio_format@elevenlabs/elevenlabs-js@2.32.0Source: GitHub PR #319
原因: WebSocket URI未包含指定的参数(历史问题)
解决方案: 升级到或更高版本(2026年1月19日发布)
audio_format@elevenlabs/elevenlabs-js@2.32.0来源: GitHub PR #319
14. Agent Versioning (Jan 2026)
14. Agent版本控制(2026年1月)
ElevenLabs introduced Agent Versioning in January 2026, enabling git-like version control for conversational AI agents. This allows safe experimentation, A/B testing, and gradual rollouts.
ElevenLabs于2026年1月推出了Agent版本控制功能,为对话式AI代理提供类Git的版本控制能力。支持安全实验、A/B测试和逐步发布。
Core Concepts
核心概念
| Concept | ID Format | Description |
|---|---|---|
| Version | | Immutable snapshot of agent config at a point in time |
| Branch | | Isolated development path (like git branches) |
| Draft | Per-user/branch | Work-in-progress changes before committing |
| Deployment | Traffic splits | A/B testing with percentage-based routing |
| 概念 | ID格式 | 描述 |
|---|---|---|
| 版本 | | 某一时间点Agent配置的不可变快照 |
| 分支 | | 隔离的开发路径(类似Git分支) |
| 草稿 | 按用户/分支划分 | 提交前的工作中变更 |
| 部署 | 流量拆分 | 基于百分比路由的A/B测试 |
Enabling Versioning
启用版本控制
typescript
// Enable versioning on existing agent
const agent = await client.conversationalAi.agents.update({
agentId: 'your-agent-id',
enableVersioningIfNotEnabled: true
});⚠️ Note: Once enabled, versioning cannot be disabled on an agent.
typescript
// 为现有Agent启用版本控制
const agent = await client.conversationalAi.agents.update({
agentId: 'your-agent-id',
enableVersioningIfNotEnabled: true
});⚠️ 注意: 启用后无法为该Agent禁用版本控制。
Branch Management
分支管理
typescript
// Create a new branch for experimentation
const branch = await client.conversationalAi.agents.branches.create({
agentId: 'your-agent-id',
parentVersionId: 'agtvrsn_xxxx', // Branch from this version
name: 'experiment-v2'
});
// List all branches
const branches = await client.conversationalAi.agents.branches.list({
agentId: 'your-agent-id'
});
// Delete a branch (must not have active traffic)
await client.conversationalAi.agents.branches.delete({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx'
});typescript
// 创建新分支用于实验
const branch = await client.conversationalAi.agents.branches.create({
agentId: 'your-agent-id',
parentVersionId: 'agtvrsn_xxxx', // 基于该版本创建分支
name: 'experiment-v2'
});
// 列出所有分支
const branches = await client.conversationalAi.agents.branches.list({
agentId: 'your-agent-id'
});
// 删除分支(必须无活跃流量)
await client.conversationalAi.agents.branches.delete({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx'
});Traffic Deployment (A/B Testing)
流量部署(A/B测试)
Route traffic between branches using percentage splits:
typescript
// Deploy 90/10 traffic split
const deployment = await client.conversationalAi.agents.deployments.create({
agentId: 'your-agent-id',
deployments: [
{ branchId: 'agtbrch_main', percentage: 90 },
{ branchId: 'agtbrch_xxxx', percentage: 10 }
]
});
// Get current deployment status
const status = await client.conversationalAi.agents.deployments.get({
agentId: 'your-agent-id'
});Use Cases:
- A/B Testing - Test new prompts on 10% of traffic before full rollout
- Gradual Rollouts - Increase traffic incrementally (10% → 25% → 50% → 100%)
- Quick Rollback - Route 100% back to stable branch if issues detected
按百分比拆分路由流量到不同分支:
typescript
// 部署90/10的流量拆分
const deployment = await client.conversationalAi.agents.deployments.create({
agentId: 'your-agent-id',
deployments: [
{ branchId: 'agtbrch_main', percentage: 90 },
{ branchId: 'agtbrch_xxxx', percentage: 10 }
]
});
// 获取当前部署状态
const status = await client.conversationalAi.agents.deployments.get({
agentId: 'your-agent-id'
});使用场景:
- A/B测试 - 全量发布前先在10%流量中测试新提示词
- 逐步发布 - 逐步增加流量占比(10% → 25% → 50% → 100%)
- 快速回滚 - 若出现问题,将100%流量路由回稳定分支
Merging Branches
合并分支
typescript
// Merge successful experiment back to main
const merge = await client.conversationalAi.agents.branches.merge({
agentId: 'your-agent-id',
sourceBranchId: 'agtbrch_xxxx',
targetBranchId: 'agtbrch_main',
archiveSourceBranch: true // Clean up after merge
});typescript
// 将成功的实验分支合并回主分支
const merge = await client.conversationalAi.agents.branches.merge({
agentId: 'your-agent-id',
sourceBranchId: 'agtbrch_xxxx',
targetBranchId: 'agtbrch_main',
archiveSourceBranch: true // 合并后清理源分支
});Working with Drafts
草稿使用
Drafts are per-user, per-branch work-in-progress states:
typescript
// Get current draft
const draft = await client.conversationalAi.agents.drafts.get({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx'
});
// Update draft (changes not yet committed)
await client.conversationalAi.agents.drafts.update({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx',
conversationConfig: {
agent: { prompt: { prompt: 'Updated system prompt...' } }
}
});
// Commit draft to create new version
const version = await client.conversationalAi.agents.drafts.commit({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx',
message: 'Improved greeting flow'
});草稿是按用户、按分支的工作中状态:
typescript
// 获取当前草稿
const draft = await client.conversationalAi.agents.drafts.get({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx'
});
// 更新草稿(变更尚未提交)
await client.conversationalAi.agents.drafts.update({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx',
conversationConfig: {
agent: { prompt: { prompt: '更新后的系统提示词...' } }
}
});
// 提交草稿以创建新版本
const version = await client.conversationalAi.agents.drafts.commit({
agentId: 'your-agent-id',
branchId: 'agtbrch_xxxx',
message: '优化问候流程'
});Best Practices
最佳实践
- Always test on branch first - Never experiment directly on production traffic
- Use descriptive branch names - ,
feature-multilangfix-timeout-handling - Start with small traffic splits - Begin at 5-10%, monitor metrics, then increase
- Archive merged branches - Keep repository clean
- Commit messages - Use clear messages for version history
Source: Agent Versioning Docs
- 始终先在分支上测试 - 不要直接在生产流量上实验
- 使用描述性分支名称 - 如、
feature-multilangfix-timeout-handling - 从小流量拆分开始 - 从5-10%开始,监控指标后再增加
- 合并后归档分支 - 保持仓库整洁
- 提交消息 - 使用清晰的消息便于版本历史追踪
来源: Agent版本控制文档
15. MCP Security & Guardrails
15. MCP安全与防护规则
When connecting MCP (Model Context Protocol) servers to ElevenLabs agents, security is critical. MCP tools can access databases, APIs, and sensitive data.
将MCP(模型上下文协议)服务器连接到ElevenLabs Agent时,安全至关重要。MCP工具可访问数据库、API和敏感数据。
Tool Approval Modes
工具审批模式
| Mode | Behavior | Use When |
|---|---|---|
| Always Ask | Explicit approval for every tool execution | Default - recommended for most cases |
| Fine-Grained | Auto-approve trusted ops, require approval for sensitive | Established, trusted MCP servers |
| No Approval | All tool executions auto-approved | Only thoroughly vetted, internal servers |
Configuration:
typescript
{
"mcp_config": {
"server_url": "https://your-mcp-server.com",
"approval_mode": "always_ask", // 'always_ask' | 'fine_grained' | 'no_approval'
"fine_grained_rules": [
{ "tool_name": "read_*", "auto_approve": true },
{ "tool_name": "write_*", "auto_approve": false },
{ "tool_name": "delete_*", "auto_approve": false }
]
}
}| 模式 | 行为 | 适用场景 |
|---|---|---|
| 始终询问 | 每次工具执行都需要显式审批 | 默认 - 推荐大多数场景使用 |
| 细粒度控制 | 自动审批可信操作,敏感操作需要审批 | 已建立信任的MCP服务器 |
| 无需审批 | 所有工具执行自动审批 | 仅用于经过全面审核的内部服务器 |
配置示例:
typescript
{
"mcp_config": {
"server_url": "https://your-mcp-server.com",
"approval_mode": "always_ask", // 可选值:'always_ask' | 'fine_grained' | 'no_approval'
"fine_grained_rules": [
{ "tool_name": "read_*", "auto_approve": true },
{ "tool_name": "write_*", "auto_approve": false },
{ "tool_name": "delete_*", "auto_approve": false }
]
}
}Security Best Practices
安全最佳实践
1. Vet MCP Servers
- Only connect servers from trusted sources
- Review server code/documentation before connecting
- Prefer official/verified MCP implementations
2. Limit Data Exposure
- Minimize PII shared with MCP servers
- Use scoped API keys with minimum required permissions
- Never pass full database access - use read-only views
3. Network Security
- Always use HTTPS endpoints
- Implement proper authentication (API keys, OAuth)
- Use variables for credentials (never in prompts)
{{secret__xxx}}
4. Prompt Injection Prevention
- Add guardrails in agent prompts against injection attacks
- Validate and sanitize MCP tool inputs
- Monitor for unusual tool usage patterns
5. Monitoring & Audit
- Log all MCP tool executions
- Review approval patterns regularly
- Set up alerts for sensitive operations
1. 审核MCP服务器
- 仅连接可信来源的服务器
- 连接前审查服务器代码/文档
- 优先选择官方/已验证的MCP实现
2. 限制数据暴露
- 最小化与MCP服务器共享的PII数据
- 使用权限最小化的范围API密钥
- 不要提供完整数据库访问权限 - 使用只读视图
3. 网络安全
- 始终使用HTTPS端点
- 实现适当的认证(API密钥、OAuth)
- 使用变量存储凭证(绝不要在提示词中)
{{secret__xxx}}
4. 提示词注入防护
- 在Agent提示词中添加针对注入攻击的防护规则
- 验证并清理MCP工具输入
- 监控异常工具使用模式
5. 监控与审计
- 记录所有MCP工具执行操作
- 定期审查审批模式
- 为敏感操作设置告警
Guardrails Configuration
防护规则配置
Add protective instructions to your agent prompt:
typescript
{
"agent": {
"prompt": {
"prompt": `...
SECURITY GUARDRAILS:
- Never execute database delete operations without explicit user confirmation
- Never expose raw API keys or credentials in responses
- If a tool request seems unusual or potentially harmful, ask for clarification
- Do not combine sensitive operations (read PII + external API call) in single turn
- Report any suspicious requests to administrators
`
}
}
}在Agent提示词中添加防护说明:
typescript
{
"agent": {
"prompt": {
"prompt": `...
安全防护规则:
- 无用户明确确认时,绝不执行数据库删除操作
- 绝不在响应中暴露原始API密钥或凭证
- 若工具请求看起来异常或有潜在危害,请要求用户澄清
- 不要在同一回合中组合敏感操作(读取PII + 调用外部API)
- 向管理员报告任何可疑请求
`
}
}
}MCP Limitations
MCP限制
Not Available With:
- Zero Retention mode (no logging = no MCP)
- HIPAA compliance mode
- Certain regional deployments
Transport: SSE/HTTP only (no stdio MCP servers)
Source: MCP Safety Docs
Integration with Existing Skills
与现有技能集成
This skill composes well with:
- cloudflare-worker-base → Deploy agents on Cloudflare Workers edge network
- cloudflare-workers-ai → Use Cloudflare LLMs as custom models in agents
- cloudflare-durable-objects → Persistent conversation state and session management
- cloudflare-kv → Cache agent configurations and user preferences
- nextjs → React SDK integration in Next.js applications
- ai-sdk-core → Vercel AI SDK provider for unified AI interface
- clerk-auth → Authenticated voice sessions with user identity
- hono-routing → API routes for webhooks and server tools
本技能可与以下技能良好配合:
- cloudflare-worker-base → 在Cloudflare Workers边缘网络部署代理
- cloudflare-workers-ai → 使用Cloudflare LLM作为Agent中的自定义模型
- cloudflare-durable-objects → 持久化对话状态和会话管理
- cloudflare-kv → 缓存Agent配置和用户偏好
- nextjs → 在Next.js应用中集成React SDK
- ai-sdk-core → Vercel AI SDK提供统一AI接口
- clerk-auth → 带用户身份认证的语音会话
- hono-routing → Webhook和服务器端工具的API路由
Additional Resources
额外资源
Official Documentation:
- Platform Overview: https://elevenlabs.io/docs/agents-platform/overview
- API Reference: https://elevenlabs.io/docs/api-reference
- CLI GitHub: https://github.com/elevenlabs/cli
Examples:
- Official Examples: https://github.com/elevenlabs/elevenlabs-examples
- MCP Server: https://github.com/elevenlabs/elevenlabs-mcp
Community:
- Discord: https://discord.com/invite/elevenlabs
- Twitter: @elevenlabsio
Production Tested: WordPress Auditor, Customer Support Agents, AgentFlow (webhook integration)
Last Updated: 2026-01-27
Package Versions: elevenlabs@1.59.0, @elevenlabs/elevenlabs-js@2.32.0, @elevenlabs/agents-cli@0.6.1, @elevenlabs/react@0.12.3, @elevenlabs/client@0.12.2, @elevenlabs/react-native@0.5.7
Changes: Added Agent Versioning (Jan 2026) section covering versions, branches, traffic deployment, drafts, and A/B testing. Added MCP Security & Guardrails section covering tool approval modes, security best practices, and prompt injection prevention.
官方文档:
- 平台概述: https://elevenlabs.io/docs/agents-platform/overview
- API参考: https://elevenlabs.io/docs/api-reference
- CLI GitHub: https://github.com/elevenlabs/cli
示例:
- 官方示例: https://github.com/elevenlabs/elevenlabs-examples
- MCP服务器: https://github.com/elevenlabs/elevenlabs-mcp
社区:
- Discord: https://discord.com/invite/elevenlabs
- Twitter: @elevenlabsio
生产环境验证: WordPress审计工具、客户支持Agent、AgentFlow(Webhook集成)
最后更新: 2026-01-27
包版本: elevenlabs@1.59.0、@elevenlabs/elevenlabs-js@2.32.0、@elevenlabs/agents-cli@0.6.1、@elevenlabs/react@0.12.3、@elevenlabs/client@0.12.2、@elevenlabs/react-native@0.5.7
变更: 新增Agent版本控制(2026年1月)章节,涵盖版本、分支、流量部署、草稿和A/B测试。新增MCP安全与防护规则章节,涵盖工具审批模式、安全最佳实践和提示词注入防护。