tavus-cvi-interactions

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Tavus CVI Interactions Protocol

Tavus CVI 交互协议

Control live conversations programmatically via WebRTC data channel.

通过WebRTC数据通道以编程方式控制实时对话。

Setup: Daily.js Client

配置：Daily.js 客户端

html

<script src="https://unpkg.com/@daily-co/daily-js"></script>
<script>
const call = window.Daily.createFrame();

// Listen for events from CVI
call.on('app-message', (event) => {
  console.log('CVI event:', event.data);
});

// Join the conversation
call.join({ url: 'YOUR_CONVERSATION_URL' });

// Send interaction
function send(interaction) {
  call.sendAppMessage(interaction, '*');
}
</script>

html

<script src="https://unpkg.com/@daily-co/daily-js"></script>
<script>
const call = window.Daily.createFrame();

// Listen for events from CVI
call.on('app-message', (event) => {
  console.log('CVI event:', event.data);
});

// Join the conversation
call.join({ url: 'YOUR_CONVERSATION_URL' });

// Send interaction
function send(interaction) {
  call.sendAppMessage(interaction, '*');
}
</script>

Interactions You Can Send

可发送的交互指令

Echo: Make Replica Speak Text

Echo：让副本朗读文本

Bypass LLM, replica speaks exactly what you provide:

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.echo",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "modality": "text",
    "text": "Hello! Let me tell you about our product."
  }
});

For streaming audio (base64):

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.echo",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "modality": "audio",
    "audio": "BASE64_ENCODED_AUDIO",
    "sample_rate": 24000,
    "inference_id": "unique-id",
    "done": "true"
  }
});

绕过LLM，副本将完全朗读你提供的内容：

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.echo",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "modality": "text",
    "text": "Hello! Let me tell you about our product."
  }
});

对于流式音频（base64格式）：

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.echo",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "modality": "audio",
    "audio": "BASE64_ENCODED_AUDIO",
    "sample_rate": 24000,
    "inference_id": "unique-id",
    "done": "true"
  }
});

Respond: Inject User Input

Respond：注入用户输入

Treat text as if user spoke it (goes through LLM):

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.respond",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "text": "What are your pricing plans?"
  }
});

将文本视为用户所说内容（会经过LLM处理）：

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.respond",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "text": "What are your pricing plans?"
  }
});

Interrupt: Stop Replica Speaking

Interrupt：停止副本朗读

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.interrupt",
  "conversation_id": "YOUR_CONVERSATION_ID"
});

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.interrupt",
  "conversation_id": "YOUR_CONVERSATION_ID"
});

Overwrite Context

Overwrite Context：覆盖上下文

Replace the entire conversational context:

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.overwrite_context",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "context": "User is now asking about enterprise features."
  }
});

替换整个对话上下文：

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.overwrite_context",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "context": "User is now asking about enterprise features."
  }
});

Append Context

Append Context：追加上下文

Add to existing context without replacing:

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.append_context",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "context": "User mentioned they have a team of 50 people."
  }
});

在现有上下文基础上添加内容，而非替换：

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.append_context",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "context": "User mentioned they have a team of 50 people."
  }
});

Adjust Sensitivity

Adjust Sensitivity：调整敏感度

Change turn-taking sensitivity mid-conversation:

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.sensitivity",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "participant_pause_sensitivity": "high",
    "participant_interrupt_sensitivity": "low"
  }
});

Values:

low

medium

high

在对话中途调整话轮转换敏感度：

javascript

send({
  "message_type": "conversation",
  "event_type": "conversation.sensitivity",
  "conversation_id": "YOUR_CONVERSATION_ID",
  "properties": {
    "participant_pause_sensitivity": "high",
    "participant_interrupt_sensitivity": "low"
  }
});

可选值：

low

medium

high

Events You Receive

可接收的事件

Utterance (What Was Said)

Utterance（对话内容）

json

{
  "event_type": "conversation.utterance",
  "properties": {
    "role": "user",
    "content": "Tell me about your product"
  }
}

Role:

user

replica

json

{
  "event_type": "conversation.utterance",
  "properties": {
    "role": "user",
    "content": "Tell me about your product"
  }
}

角色：

user

或

replica

Replica Started/Stopped Speaking

Replica Started/Stopped Speaking（副本开始/停止朗读）

json

{
  "event_type": "conversation.replica.started_speaking",
  "properties": {
    "inference_id": "inf-123"
  }
}

json

{
  "event_type": "conversation.replica.stopped_speaking",
  "properties": {
    "inference_id": "inf-123",
    "duration": 4.5
  }
}

json

{
  "event_type": "conversation.replica.started_speaking",
  "properties": {
    "inference_id": "inf-123"
  }
}

json

{
  "event_type": "conversation.replica.stopped_speaking",
  "properties": {
    "inference_id": "inf-123",
    "duration": 4.5
  }
}

User Started/Stopped Speaking

User Started/Stopped Speaking（用户开始/停止说话）

json

{
  "event_type": "conversation.user.started_speaking"
}

json

{
  "event_type": "conversation.user.started_speaking"
}

Tool Call (Function Calling)

Tool Call（工具调用）

When LLM invokes a tool:

json

{
  "event_type": "conversation.tool_call",
  "properties": {
    "tool_name": "get_weather",
    "arguments": {
      "location": "San Francisco, CA"
    },
    "inference_id": "inf-123"
  }
}

Handle it, then respond with echo or respond interaction.

当LLM调用工具时：

json

{
  "event_type": "conversation.tool_call",
  "properties": {
    "tool_name": "get_weather",
    "arguments": {
      "location": "San Francisco, CA"
    },
    "inference_id": "inf-123"
  }
}

处理该调用后，通过echo或respond交互进行回复。

Perception Analysis

Perception Analysis（感知分析）

When Raven analyzes the user:

json

{
  "event_type": "conversation.perception_analysis",
  "properties": {
    "analysis": "User appears engaged, smiling, looking at camera"
  }
}

当Raven分析用户时：

json

{
  "event_type": "conversation.perception_analysis",
  "properties": {
    "analysis": "User appears engaged, smiling, looking at camera"
  }
}

Replica Interrupted

Replica Interrupted（副本被中断）

Fired when replica was interrupted:

json

{
  "event_type": "conversation.replica.interrupted",
  "properties": {
    "inference_id": "inf-123"
  }
}

当副本被中断时触发：

json

{
  "event_type": "conversation.replica.interrupted",
  "properties": {
    "inference_id": "inf-123"
  }
}

Python Client (Daily-Python)

Python Client（Daily-Python）

python

from daily import Daily, CallClient

Daily.init()
client = CallClient()

def on_app_message(message, sender):
    print(f"Received: {message}")

client.set_user_name("bot")
client.join(meeting_url, completion=on_join)

python

from daily import Daily, CallClient

Daily.init()
client = CallClient()

def on_app_message(message, sender):
    print(f"Received: {message}")

client.set_user_name("bot")
client.join(meeting_url, completion=on_join)

Send interaction

client.send_app_message({ "message_type": "conversation", "event_type": "conversation.echo", "conversation_id": "xxx", "properties": {"text": "Hello!"} })

undefined

client.send_app_message({ "message_type": "conversation", "event_type": "conversation.echo", "conversation_id": "xxx", "properties": {"text": "Hello!"} })

undefined

Common Patterns

常见模式

Echo Mode with Manual Control

手动控制的Echo模式

Create persona with
```
pipeline_mode: "echo"
```
Join conversation with Daily client
Send
```
conversation.echo
```
events to control speech
Send
```
conversation.interrupt
```
to stop
Listen for events to track state

创建 persona 时设置
```
pipeline_mode: "echo"
```
使用Daily客户端加入对话
发送
```
conversation.echo
```
事件控制朗读内容
发送
```
conversation.interrupt
```
停止朗读
监听事件以跟踪状态

Hybrid: LLM + Manual Injection

混合模式：LLM + 手动注入

Use
```
pipeline_mode: "full"
```
for normal conversation
Inject context with
```
conversation.append_context
```
Override with
```
conversation.echo
```
when needed
Use
```
conversation.interrupt
```
+
```
conversation.echo
```
for immediate takeover

使用
```
pipeline_mode: "full"
```
进行正常对话
通过
```
conversation.append_context
```
注入上下文
需要时使用
```
conversation.echo
```
覆盖内容
结合
```
conversation.interrupt
```
+
```
conversation.echo
```
实现即时接管