elevenlabs-agents

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

ElevenLabs Agents Platform

ElevenLabs Agents Platform

Overview

概述

ElevenLabs Agents Platform is a comprehensive solution for building production-ready conversational AI voice agents. The platform coordinates four core components:
  1. ASR (Automatic Speech Recognition) - Converts speech to text (32+ languages, sub-second latency)
  2. LLM (Large Language Model) - Reasoning and response generation (GPT, Claude, Gemini, custom models)
  3. TTS (Text-to-Speech) - Converts text to speech (5000+ voices, 31 languages, low latency)
  4. Turn-Taking Model - Proprietary model that handles conversation timing and interruptions
ElevenLabs Agents Platform是构建可投入生产的对话式AI语音代理的综合解决方案。该平台协调四大核心组件:
  1. ASR(Automatic Speech Recognition) - 将语音转换为文本(支持32+种语言,亚秒级延迟)
  2. LLM(Large Language Model) - 推理与响应生成(支持GPT、Claude、Gemini及自定义模型)
  3. TTS(Text-to-Speech) - 将文本转换为语音(5000+种音色,31种语言,低延迟)
  4. Turn-Taking Model - 专有模型,处理对话时序与打断逻辑

🚨 Package Migration (August 2025)

🚨 包迁移通知(2025年8月)

DEPRECATED (Do not use):
@11labs/react
and
@11labs/client
Current packages:
bash
bun add @elevenlabs/react@0.11.0        # React SDK
bun add @elevenlabs/client@0.11.0       # JavaScript SDK
bun add @elevenlabs/react-native@0.5.2  # React Native SDK
bun add @elevenlabs/elevenlabs-js@2.25.0 # Base SDK
bun add -g @elevenlabs/agents-cli@0.2.0  # CLI
If migrating, uninstall old packages first:
npm uninstall @11labs/react @11labs/client

已弃用(请勿使用):
@11labs/react
@11labs/client
当前可用包:
bash
bun add @elevenlabs/react@0.11.0        # React SDK
bun add @elevenlabs/client@0.11.0       # JavaScript SDK
bun add @elevenlabs/react-native@0.5.2  # React Native SDK
bun add @elevenlabs/elevenlabs-js@2.25.0 # Base SDK
bun add -g @elevenlabs/agents-cli@0.2.0  # CLI
迁移时请先卸载旧包:
npm uninstall @11labs/react @11labs/client

Quick Start

快速开始

Path A: React SDK (Embedded Voice Chat)

路径A:React SDK(嵌入式语音聊天)

For building voice chat interfaces in React applications.
Installation:
bash
bun add @elevenlabs/react zod
Basic Example:
typescript
import { useConversation } from '@elevenlabs/react';
import { z } from 'zod';

export default function VoiceChat() {
  const { startConversation, stopConversation, status } = useConversation({
    // Authentication (choose one)
    agentId: 'your-agent-id',  // Public agent (no key needed)
    // apiKey: process.env.NEXT_PUBLIC_ELEVENLABS_API_KEY,  // Private (dev only)
    // signedUrl: '/api/elevenlabs/auth',  // Signed URL (production)

    // Client-side tools
    clientTools: {
      updateCart: {
        description: "Update the shopping cart",
        parameters: z.object({
          item: z.string(),
          quantity: z.number()
        }),
        handler: async ({ item, quantity }) => {
          console.log('Updating cart:', item, quantity);
          return { success: true };
        }
      }
    },

    // Event handlers
    onEvent: (event) => {
      if (event.type === 'transcript') console.log('User:', event.data.text);
      if (event.type === 'agent_response') console.log('Agent:', event.data.text);
    },

    // Regional compliance
    serverLocation: 'us' // 'us' | 'global' | 'eu-residency' | 'in-residency'
  });

  return (
    <div>
      <button onClick={startConversation}>Start Conversation</button>
      <button onClick={stopConversation}>Stop</button>
      <p>Status: {status}</p>
    </div>
  );
}
Complete template: See
templates/basic-react-agent.tsx
适用于在React应用中构建语音聊天界面。
安装:
bash
bun add @elevenlabs/react zod
基础示例:
typescript
import { useConversation } from '@elevenlabs/react';
import { z } from 'zod';

export default function VoiceChat() {
  const { startConversation, stopConversation, status } = useConversation({
    // 认证方式(三选一)
    agentId: 'your-agent-id',  // 公开代理(无需密钥)
    // apiKey: process.env.NEXT_PUBLIC_ELEVENLABS_API_KEY,  // 私有代理(仅开发环境使用)
    // signedUrl: '/api/elevenlabs/auth',  // 签名URL(生产环境)

    // 客户端工具
    clientTools: {
      updateCart: {
        description: "更新购物车",
        parameters: z.object({
          item: z.string(),
          quantity: z.number()
        }),
        handler: async ({ item, quantity }) => {
          console.log('更新购物车:', item, quantity);
          return { success: true };
        }
      }
    },

    // 事件处理器
    onEvent: (event) => {
      if (event.type === 'transcript') console.log('用户:', event.data.text);
      if (event.type === 'agent_response') console.log('代理:', event.data.text);
    },

    // 区域合规配置
    serverLocation: 'us' // 'us' | 'global' | 'eu-residency' | 'in-residency'
  });

  return (
    <div>
      <button onClick={startConversation}>开始对话</button>
      <button onClick={stopConversation}>停止</button>
      <p>状态: {status}</p>
    </div>
  );
}
完整模板: 查看
templates/basic-react-agent.tsx

Path B: CLI ("Agents as Code")

路径B:CLI("代码化管理代理")

For managing agents via code with version control and CI/CD. Load
references/cli-commands.md
when using CLI workflows.
Quick workflow:
bash
bun add -g @elevenlabs/agents-cli
elevenlabs auth login
elevenlabs agents init
elevenlabs agents add "Support Agent" --template customer-service
适用于通过代码管理代理,支持版本控制与CI/CD流程。使用CLI工作流时请查阅
references/cli-commands.md
快速工作流:
bash
bun add -g @elevenlabs/agents-cli
elevenlabs auth login
elevenlabs agents init
elevenlabs agents add "Support Agent" --template customer-service

Edit agent_configs/support-agent.json

编辑 agent_configs/support-agent.json

elevenlabs agents push --env dev elevenlabs agents test "Support Agent"

**See:** `references/cli-commands.md` for complete CLI reference and workflows.
elevenlabs agents push --env dev elevenlabs agents test "Support Agent"

**参考:** 完整CLI命令与工作流请查看 `references/cli-commands.md`。

Path C: API (Programmatic Agent Management)

路径C:API(程序化代理管理)

For creating agents dynamically (multi-tenant, SaaS platforms). Load
references/api-reference.md
when using the API directly.
Quick example:
typescript
import { ElevenLabsClient } from 'elevenlabs';

const client = new ElevenLabsClient({
  apiKey: process.env.ELEVENLABS_API_KEY
});

const agent = await client.agents.create({
  name: 'Support Bot',
  conversation_config: {
    agent: {
      prompt: { prompt: "You are a helpful support agent.", llm: "gpt-4o" },
      first_message: "Hello! How can I help you today?"
    },
    tts: { model_id: "eleven_turbo_v2_5", voice_id: "your-voice-id" }
  }
});
See:
references/api-reference.md
for complete API reference.

适用于动态创建代理(多租户、SaaS平台场景)。直接使用API时请查阅
references/api-reference.md
快速示例:
typescript
import { ElevenLabsClient } from 'elevenlabs';

const client = new ElevenLabsClient({
  apiKey: process.env.ELEVENLABS_API_KEY
});

const agent = await client.agents.create({
  name: 'Support Bot',
  conversation_config: {
    agent: {
      prompt: { prompt: "你是一位乐于助人的客服代理。", llm: "gpt-4o" },
      first_message: "你好!请问我能为你提供什么帮助?"
    },
    tts: { model_id: "eleven_turbo_v2_5", voice_id: "your-voice-id" }
  }
});
参考: 完整API文档请查看
references/api-reference.md

Agent Configuration

代理配置

System Prompt Framework

系统提示词框架

ElevenLabs recommends a 6-component prompt structure: Personality (identity/role), Environment (communication context), Tone (formality/speech patterns), Goal (objectives/success criteria), Guardrails (boundaries/ethics), and Tools (available functions).
Example structure:
Personality: You are Alex, a friendly customer support specialist at TechCorp.
Environment: Phone communication, voice-only, potential background noise.
Tone: Professional yet warm. Use contractions. Keep responses to 2-3 sentences.
Goal: Resolve issues on first call. Success = customer confirms resolution.
Guardrails: Never give medical/legal advice. Escalate if customer becomes abusive.
Tools: lookup_order(id), transfer_to_supervisor(), send_password_reset(email)
Complete guide: Load
references/system-prompt-guide.md
when configuring agent prompts or improving conversation quality.
ElevenLabs推荐采用6组件提示词结构:Personality(人设/角色)Environment(沟通场景)Tone(语气)Goal(目标)Guardrails(规则边界)Tools(可用工具)
示例结构:
Personality: 你是Alex,TechCorp公司的友好客服专员。
Environment: 电话沟通,仅语音交互,可能存在背景噪音。
Tone: 专业且亲切,使用缩略语,回复控制在2-3句话。
Goal: 首次通话解决问题,成功标志为用户确认问题已解决。
Guardrails: 绝不提供医疗/法律建议,若用户出现辱骂行为则转接上级。
Tools: lookup_order(id), transfer_to_supervisor(), send_password_reset(email)
完整指南: 配置代理提示词或优化对话质量时,请查阅
references/system-prompt-guide.md

Turn-Taking Modes

话轮转换模式

ModeBehaviorBest For
EagerResponds quickly, jumps in earlyFast-paced support, quick orders
NormalBalanced, waits for natural breaksGeneral customer service (default)
PatientWaits longer for detailed responsesInformation collection, tutoring
Configuration:
"turn": { "mode": "patient" }
in
conversation_config

模式行为适用场景
Eager(主动)快速响应,提前插话快节奏支持、快速下单场景
Normal(标准)平衡模式,等待自然停顿通用客服场景(默认)
Patient(耐心)等待更长时间以获取详细回复信息收集、辅导场景
配置方式:在
conversation_config
中设置
"turn": { "mode": "patient" }

Core Features Summary

核心功能汇总

Voice & Language

语音与语言

  • Multi-Voice: Dynamic voice switching (adds ~200ms latency per switch)
  • Pronunciation Dictionary: IPA, CMU, or word substitutions for custom pronunciation
  • Speed Control: 0.7x - 1.2x (1.0x = normal)
  • Languages: 32+ languages with auto-detection and multi-language presets
  • 多音色切换: 动态切换音色(每次切换增加约200ms延迟)
  • 发音词典: 支持IPA、CMU音标或自定义词替换实现精准发音
  • 语速控制: 0.7x - 1.2x(1.0x为正常语速)
  • 多语言支持: 32+种语言,支持自动检测与多语言预设

Knowledge Base (RAG)

知识库(RAG)

Upload documents (PDF/TXT/DOCX) for semantic search during conversations. Agent retrieves relevant chunks automatically. Adds ~500ms latency per query. Documents must be indexed before use.
See:
references/api-reference.md
for knowledge base API and configuration.
上传文档(PDF/TXT/DOCX)进行对话时的语义检索。代理会自动获取相关内容片段,每次查询增加约500ms延迟。文档需先完成索引才能使用。
参考: 知识库API与配置请查阅
references/api-reference.md

Tools (4 Types)

工具类型(4种)

  1. Client Tools - Execute in browser (UI updates, navigation, local storage)
  2. Server Tools (Webhooks) - Execute on your backend (database, payments, CRM)
  3. MCP Tools - Connect to Model Context Protocol servers (enterprise APIs)
  4. System Tools - Built-in platform tools (end_conversation, transfer_call, mute_microphone, press_digit)
See:
references/tool-examples.md
when implementing tools. Load
references/api-reference.md
for webhook tool creation API.

  1. Client Tools(客户端工具) - 在浏览器中执行(UI更新、导航、本地存储操作)
  2. Server Tools(Webhooks) - 在你的后端执行(数据库操作、支付、CRM集成)
  3. MCP Tools - 连接Model Context Protocol服务器(企业级API)
  4. System Tools(系统工具) - 平台内置工具(end_conversation、transfer_call、mute_microphone、press_digit)
参考: 实现工具时请查阅
references/tool-examples.md
;Webhook工具创建API请查阅
references/api-reference.md

Top 3 Critical Errors

三大常见关键错误

Error 1: Package Deprecation (@11labs/*)

错误1:包弃用(@11labs/*)

Symptom: Import errors, "module not found"
Solution:
bash
npm uninstall @11labs/react @11labs/client
bun add @elevenlabs/react@0.11.0 @elevenlabs/client@0.11.0
症状: 导入错误、"module not found"
解决方案:
bash
npm uninstall @11labs/react @11labs/client
bun add @elevenlabs/react@0.11.0 @elevenlabs/client@0.11.0

Update imports:

更新导入语句:

import { useConversation } from '@elevenlabs/react'; // Not @11labs/react
undefined
import { useConversation } from '@elevenlabs/react'; // 不再是 @11labs/react
undefined

Error 2: Android Audio Cutoff (First Message)

错误2:Android音频中断(首次消息)

Symptom: First agent message cuts off on Android only (iOS/web work fine)
Solution:
typescript
const { startConversation } = useConversation({
  agentId: 'your-agent-id',
  connectionDelay: {
    android: 3_000,  // 3 seconds for Android audio mode switch
    ios: 0,
    default: 0
  }
});
症状: 仅Android平台出现代理首次消息中断(iOS/web正常)
解决方案:
typescript
const { startConversation } = useConversation({
  agentId: 'your-agent-id',
  connectionDelay: {
    android: 3_000,  // Android音频模式切换需3秒
    ios: 0,
    default: 0
  }
});

Error 3: CSP (Content Security Policy) Violations

错误3:CSP(内容安全策略)违规

Symptom: "Refused to load the script..." errors, CSP blocks blob URLs
Solution - Self-host worklet files:
bash
cp node_modules/@elevenlabs/client/dist/worklets/*.js public/elevenlabs/
typescript
const { startConversation } = useConversation({
  agentId: 'your-agent-id',
  workletPaths: {
    'rawAudioProcessor': '/elevenlabs/rawAudioProcessor.worklet.js',
    'audioConcatProcessor': '/elevenlabs/audioConcatProcessor.worklet.js',
  }
});
See all 17 errors: Load
references/error-catalog.md
when troubleshooting errors, debugging webhook failures, RAG issues, or platform-specific problems.

症状: "Refused to load the script..." 错误,CSP阻止blob URL
解决方案 - 自托管worklet文件:
bash
cp node_modules/@elevenlabs/client/dist/worklets/*.js public/elevenlabs/
typescript
const { startConversation } = useConversation({
  agentId: 'your-agent-id',
  workletPaths: {
    'rawAudioProcessor': '/elevenlabs/rawAudioProcessor.worklet.js',
    'audioConcatProcessor': '/elevenlabs/audioConcatProcessor.worklet.js',
  }
});
查看全部17种错误: 排查错误、调试Webhook失败、RAG问题或平台特定问题时,请查阅
references/error-catalog.md

When to Load References

何时查阅参考文档

Load specific reference files based on your current task:
根据当前任务选择对应的参考文档:

references/api-reference.md

references/api-reference.md

Load when: Creating agents via API, managing agents programmatically, building multi-tenant systems, implementing knowledge base, creating webhook tools, handling API errors, or understanding API rate limits.
适用场景:通过API创建代理、程序化管理代理、构建多租户系统、实现知识库、创建Webhook工具、处理API错误、了解API速率限制。

references/cli-commands.md

references/cli-commands.md

Load when: Using CLI for agent management, setting up CI/CD pipelines, managing multi-environment deployments (dev/staging/prod), version controlling agent configs, or troubleshooting CLI issues.
适用场景:使用CLI管理代理、搭建CI/CD流水线、管理多环境部署(dev/staging/prod)、版本控制代理配置、排查CLI问题。

references/compliance-guide.md

references/compliance-guide.md

Load when: Implementing GDPR compliance, handling HIPAA requirements (healthcare), configuring SOC 2 controls, setting data retention policies, implementing regional data residency (EU/IN), or handling PCI DSS (payments).
适用场景:实现GDPR合规、满足HIPAA要求(医疗场景)、配置SOC 2控制、设置数据保留策略、实现区域数据驻留(EU/IN)、处理PCI DSS(支付场景)。

references/cost-optimization.md

references/cost-optimization.md

Load when: Optimizing LLM costs, implementing caching strategies, choosing between models (GPT/Claude/Gemini), handling traffic spikes (burst pricing), reducing token usage, or setting budget limits.
适用场景:优化LLM成本、实现缓存策略、选择模型(GPT/Claude/Gemini)、处理流量峰值(阶梯定价)、减少Token使用、设置预算限制。

references/error-catalog.md

references/error-catalog.md

Load when: Troubleshooting errors, debugging webhook failures, fixing voice consistency issues, resolving authentication problems, handling RAG index issues, or diagnosing platform-specific bugs.
适用场景:排查错误、调试Webhook失败、修复语音一致性问题、解决认证问题、处理RAG索引问题、诊断平台特定Bug。

references/system-prompt-guide.md

references/system-prompt-guide.md

Load when: Writing agent system prompts, improving conversation quality, implementing 6-component framework, iterating on prompts, testing prompt effectiveness, or defining agent personality/goals/guardrails.
适用场景:编写代理系统提示词、提升对话质量、实现6组件框架、迭代优化提示词、测试提示词效果、定义代理人设/目标/规则边界。

references/testing-guide.md

references/testing-guide.md

Load when: Setting up automated tests, implementing scenario testing, load testing agents, converting real conversations to tests, integrating with CI/CD, or debugging failed tests.
适用场景:搭建自动化测试、实现场景测试、代理负载测试、将真实对话转换为测试用例、集成CI/CD、调试测试失败问题。

references/tool-examples.md

references/tool-examples.md

Load when: Implementing client tools (browser functions), creating server tools (webhooks), connecting MCP servers, using system tools, or debugging tool execution issues.
适用场景:实现客户端工具(浏览器功能)、创建服务器工具(Webhooks)、连接MCP服务器、使用系统工具、调试工具执行问题。

references/workflow-examples.md

references/workflow-examples.md

Load when: Building multi-agent workflows, implementing call routing, creating escalation flows, designing multi-language support, or debugging workflow transitions.

适用场景:构建多代理工作流、实现呼叫路由、创建升级流程、设计多语言支持、调试工作流转场。

SDK Integrations

SDK集成

React SDK (
@elevenlabs/react
)

React SDK (
@elevenlabs/react
)

Primary hook:
useConversation()
. Features: client tools, event streaming, connection management, regional compliance. See Quick Start Path A above.
核心Hook:
useConversation()
。功能包括:客户端工具、事件流、连接管理、区域合规配置。详见上方快速开始路径A。

Other SDKs

其他SDK

  • JavaScript SDK (
    @elevenlabs/client
    ) - Vanilla JS/Node.js
  • React Native SDK (
    @elevenlabs/react-native
    ) - Mobile apps (Expo)
  • Swift SDK - iOS/macOS native apps
  • Widget - Embeddable web component (no code)
Template: See
templates/basic-react-agent.tsx
for complete React implementation.

  • JavaScript SDK (
    @elevenlabs/client
    ) - 原生JS/Node.js环境
  • React Native SDK (
    @elevenlabs/react-native
    ) - 移动应用(Expo)
  • Swift SDK - iOS/macOS原生应用
  • Widget - 可嵌入的Web组件(无需代码)
模板: 完整React实现请查看
templates/basic-react-agent.tsx

Additional Platform Features

平台附加功能

Testing & Evaluation: Scenario testing (LLM-based), tool call testing, load testing, simulation API. See
references/testing-guide.md
.
Analytics: Conversation analysis, success evaluation, data collection, analytics dashboard.
Privacy & Compliance: Data retention, encryption, zero retention mode, GDPR/HIPAA/SOC 2. See
references/compliance-guide.md
.
Cost Optimization: LLM caching (90% savings), model swapping, burst pricing. See
references/cost-optimization.md
.
Advanced: Custom models (bring your own LLM), post-call webhooks, chat mode (text-only), telephony (Twilio/SIP), workflows (multi-agent routing).

测试与评估: 场景测试(基于LLM)、工具调用测试、负载测试、模拟API。详见
references/testing-guide.md
分析: 对话分析、成功率评估、数据收集、分析仪表盘。
隐私与合规: 数据保留、加密、零保留模式、GDPR/HIPAA/SOC 2合规。详见
references/compliance-guide.md
成本优化: LLM缓存(最高节省90%成本)、模型切换、阶梯定价。详见
references/cost-optimization.md
高级功能: 自定义模型(接入自有LLM)、通话后Webhook、纯文本聊天模式、电话集成(Twilio/SIP)、工作流(多代理路由)。

Integration with Other Skills

与其他技能集成

Composes with:
  • cloudflare-worker-base → Deploy agents on edge network
  • cloudflare-workers-ai → Use Cloudflare LLMs as custom models
  • cloudflare-durable-objects → Persistent conversation state
  • nextjs → React SDK integration in Next.js
  • ai-sdk-core → Vercel AI SDK provider
  • clerk-auth → Authenticated voice sessions
  • hono-routing → API routes for webhooks

可与以下技能组合使用:
  • cloudflare-worker-base → 在边缘网络部署代理
  • cloudflare-workers-ai → 使用Cloudflare LLMs作为自定义模型
  • cloudflare-durable-objects → 持久化对话状态
  • nextjs → 在Next.js中集成React SDK
  • ai-sdk-core → Vercel AI SDK提供商
  • clerk-auth → 认证语音会话
  • hono-routing → Webhook API路由

Secure Installation

安全安装

When installing AI agent SDK packages, follow supply chain security best practices:
  • Block post-install scripts
    npm config set ignore-scripts true
    (or Bun: disabled by default)
  • Cooldown period — Wait 7 days for new package versions to be vetted by the community
  • Audit before installing — Run
    socket package score npm <pkg>
    or use
    socket npm install <pkg>
    to check packages
Load the
dependency-upgrade
skill for full security configuration including Socket CLI integration, cooldown setup, lockfile validation, and CI enforcement.
安装AI代理SDK包时,请遵循供应链安全最佳实践:
  • 阻止安装后脚本
    npm config set ignore-scripts true
    (Bun默认已禁用)
  • 冷却期 — 等待7天,让社区验证新包版本的安全性
  • 安装前审计 — 运行
    socket package score npm <pkg>
    或使用
    socket npm install <pkg>
    检查包安全性
如需完整安全配置(包括Socket CLI集成、冷却期设置、锁文件验证、CI强制执行),请加载
dependency-upgrade
技能。

Resources

资源

Templates (
templates/
):
  • basic-react-agent.tsx
    - React SDK with client tools and events
  • basic-cli-agent.json
    - CLI agent with 6-component prompt
References (
references/
):
  • api-reference.md
    - Complete API reference
  • cli-commands.md
    - CLI commands and workflows
  • compliance-guide.md
    - GDPR/HIPAA/SOC 2 compliance
  • cost-optimization.md
    - LLM caching and cost reduction
  • error-catalog.md
    - All 17 errors with solutions
  • system-prompt-guide.md
    - 6-component framework guide
  • testing-guide.md
    - Testing and evaluation
  • tool-examples.md
    - Client/server/MCP tool examples
  • workflow-examples.md
    - Multi-agent workflow patterns
Official Documentation:

Production Tested: WordPress Auditor, Customer Support Agents Last Updated: 2025-11-21 Package Versions: Verified current (see metadata)
模板(
templates/
):
  • basic-react-agent.tsx
    - 集成客户端工具与事件的React SDK示例
  • basic-cli-agent.json
    - 采用6组件提示词的CLI代理示例
参考文档(
references/
):
  • api-reference.md
    - 完整API参考
  • cli-commands.md
    - CLI命令与工作流
  • compliance-guide.md
    - GDPR/HIPAA/SOC 2合规指南
  • cost-optimization.md
    - LLM缓存与成本优化
  • error-catalog.md
    - 全部17种错误及解决方案
  • system-prompt-guide.md
    - 6组件框架指南
  • testing-guide.md
    - 测试与评估指南
  • tool-examples.md
    - 客户端/服务器/MCP工具示例
  • workflow-examples.md
    - 多代理工作流模式
官方文档:

生产验证: WordPress审计工具、客服代理 最后更新: 2025-11-21 包版本: 已验证为当前最新版本(详见元数据)