nemotron-voice-agent-builder

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Create Voice Agent

创建语音 Agent

Creates a working NVIDIA voice agent or updates an existing project:

Cascaded: ASR transcribes, a text LLM answers, TTS speaks.
Omni: one audio-in LLM replaces ASR and the text LLM. TTS still speaks.

创建一个可正常工作的 NVIDIA 语音 Agent，或更新现有项目：

Cascaded：ASR 负责转录，文本 LLM 负责回答，TTS 负责播报。
Omni：一个音频输入 LLM 取代 ASR 和文本 LLM，TTS 仍负责播报。

When to Use This Skill

何时使用此 Skill

Use this skill when the user wants to build, scaffold, configure, refine, or fix an NVIDIA voice agent — a real-time speech pipeline with audio input and spoken output — on Pipecat or LiveKit. This covers Cascaded (ASR → LLM → TTS) and Omni (audio-in LLM + TTS) pipelines, speech customization (ASR word boosting, TTS pronunciation), multilingual routing, and cloud or local deployment (NIM, vLLM, NeMo-Speech.cpp) on workstations, DGX Spark, or Jetson Thor.

Do not use this skill for text-only chatbots or RAG, standalone batch speech-to-text transcription, generic Docker or infrastructure help, or unrelated CUDA or model work that has no voice-agent pipeline.

当用户希望使用 Pipecat 或 LiveKit 构建、脚手架搭建、配置、优化或修复 NVIDIA 语音 Agent——一种具备音频输入和语音输出的实时语音流水线——时，使用此 skill。涵盖 Cascaded（ASR → LLM → TTS）和 Omni（音频输入 LLM + TTS）流水线、语音定制（ASR 词权重提升、TTS 发音）、多语言路由，以及在工作站、DGX Spark 或 Jetson Thor 上进行云端或本地部署（NIM、vLLM、NeMo-Speech.cpp）。

不要将此 skill 用于纯文本聊天机器人或 RAG、独立的批量语音转文本、与语音 Agent 流水线无关的通用 Docker 或基础设施帮助，以及其他不相关的 CUDA 或模型工作。

Workflow

工作流

Classify the starting state before following the phases:

For an empty project, follow all phases in order.
For a working existing project, read
```
references/operations/iterate.md
```
first, and then route only the references needed for the requested change.
For a broken existing project, read
```
references/operations/troubleshoot.md
```
first. After restoring the baseline, continue with
```
references/operations/iterate.md
```
.

For an empty project, follow these phases in order:

Phase	Read
1. Intake	`references/intake.md` resolves pipeline and framework first, then routes preflight, models, language, and domain files
2. Build	`references/platforms/readiness.md` if self-hosted → exact profile or quantization variant in the routed model file → `references/output-contract.md` → routed deployment path → `references/operations/observability.md` → selected framework files
3. Hand over	`references/operations/run.md` gates the client on the generated `scripts/smoke.sh` , then `references/operations/troubleshoot.md` if the spoken exchange fails
4. Iterate	`references/operations/iterate.md` for changes to a working agent

Wait for approval before writing files.

references/preflight.md

§4 probes the host and selects the deployment path. The routed platform file owns that path from there.

Also read

references/networking/remote-webrtc.md

when Pipecat WebRTC crosses hosts or networks.

在遵循各阶段之前，先对初始状态进行分类：

对于空项目，按顺序执行所有阶段。
对于可正常运行的现有项目，先读取
```
references/operations/iterate.md
```
，然后仅路由完成所请求变更所需的参考文件。
对于无法正常运行的现有项目，先读取
```
references/operations/troubleshoot.md
```
。恢复基线后，继续执行
```
references/operations/iterate.md
```
。

对于空项目，按顺序执行以下阶段：

阶段	读取内容
1. 需求接入	`references/intake.md` 首先确定流水线和框架，然后路由预检、模型、语言和领域文件
2. 构建	`references/platforms/readiness.md` （如果是自托管）→ 路由到的模型文件中的确切 profile 或量化变体 → `references/output-contract.md` → 路由后的部署路径 → `references/operations/observability.md` → 所选框架文件
3. 交付	使用生成的 `scripts/smoke.sh` 对客户端进行验收；如果语音交互失败，则执行 `references/operations/troubleshoot.md`
4. 迭代	对可正常工作的 Agent 进行更改时，执行 `references/operations/iterate.md`

写入文件前，等待用户批准。

references/preflight.md

的 §4 会探测主机并选择部署路径。此后，部署路径由路由到的平台文件负责确定。

当 Pipecat WebRTC 跨主机或跨网络时，还需读取

references/networking/remote-webrtc.md

。

Rules

规则

Resolve every model id, image tag, profile, and serve flag from current NVIDIA documentation at build time. Never from memory.
Read every routed file before generating code, and keep the language and behavior locks approved in intake.
Handover requires a successful spoken exchange. A running process is not a working voice agent.

构建时，必须根据当前的 NVIDIA 文档确定每个 model id、image tag、profile 和 serve flag，绝不能凭记忆判断。
生成代码前，读取每个路由到的文件，并保持需求接入阶段已批准的语言和行为约束不变。
交付必须以成功完成语音交互为前提。进程正在运行并不代表语音 Agent 可以正常工作。