livekit-simulations

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

⚠️ Simulations are in private beta (not yet generally available).
No docs/MCP coverage yet. For the
lk agent simulate
command surface, use
lk agent simulate --help
and the LiveKit Cloud dashboard rather than
lk docs
/ MCP until simulations are documented.
Recent SDK required. Running simulations needs the 1.6 line of
livekit-agents
. Confirm the installed version rather than assuming.
Limited availability / auth. Creating runs needs the project enabled for simulations and a current
lk cloud auth
session. (Generating scenarios — the main job of this skill — needs neither; it's fully local.)

⚠️ 模拟功能处于私有测试阶段（尚未全面可用）。
暂无文档/MCP支持。 对于
lk agent simulate
命令，请使用
lk agent simulate --help
和LiveKit Cloud控制台，而非
lk docs
/ MCP，直至模拟功能完成文档编写。
需要最新版SDK。 运行模拟需要
livekit-agents
的1.6版本系列。请确认已安装的版本，不要默认假设版本符合要求。
可用性/权限受限。 创建运行实例需要项目已启用模拟功能，且当前有有效的
lk cloud auth
会话。（生成场景——此技能的核心功能——无需上述条件；完全在本地完成。）

Generating Simulation Scenarios

生成模拟测试场景

The most valuable thing you can do with simulations is generate good test scenarios for the user's agent — grounded in the agent's actual code and in what the user wants stress-tested — then run them. You do this locally: you read the code with your normal tools (nothing is uploaded), and you (the coding agent) are the model that does the generation, so no extra API keys or services are needed.

A scenario = a simulated user's persona + goals (

instructions

) and the pass criteria (

agent_expectations

). A simulation plays each scenario against the agent over text and an LLM judge scores it. Your job is to produce a high-quality, diverse, on-target set of scenarios and write them to a YAML scenarios file the CLI can run.

使用模拟功能最有价值的做法是为用户的Agent生成优质测试场景——基于Agent的实际代码以及用户想要压力测试的内容——然后运行这些场景。此过程完全在本地完成：你可以用常规工具读取代码（无任何内容上传），而你（编码Agent）作为生成模型，无需额外的API密钥或服务。

一个场景 = 模拟用户的角色 + 目标（

instructions

）以及通过标准（

agent_expectations

）。模拟会将每个场景通过文本与Agent交互，并由LLM评判器进行评分。你的任务是生成一组高质量、多样化且精准匹配需求的场景，并将其写入CLI可运行的YAML场景文件。

What makes this better than autopilot

为何此功能优于自动模式

A naive "just generate some tests" misses the point. Three things make this skill worth using:

It reads the agent's real code — so scenarios respect what the agent can actually do and where it blocks (especially constraints/unavailable items), instead of guessing from the name.
It is steered by the user. The user knows what they're worried about. Always capture that intent and thread it through. This is the headline — see
```
references/user-guidance.md
```
.
It guarantees coverage of every risk. Left alone, generation drifts to plausible happy-path calls and silently skips the hard cases — withholding a required field, supplying an invalid value, an empty lookup, and the guardrail/abuse surface (out-of-scope, harmful, professional-advice, sensitive-data, prompt-extraction). This skill turns the agent's constraints into an explicit risk checklist and requires at least one scenario per item — see
```
references/analyzing-the-agent.md
```
and
```
references/writing-scenarios.md
```
.

简单的“生成一些测试”没有抓住核心。此技能的优势体现在三点：

读取Agent的真实代码——因此场景会符合Agent的实际能力范围以及限制（尤其是约束条件/不可用项），而非仅凭名称猜测。
由用户主导。用户清楚他们担心的点。务必捕捉用户的意图并贯穿始终。这是核心要点——详见
```
references/user-guidance.md
```
。
确保覆盖所有风险。如果放任自动生成，场景会偏向看似合理的正常流程调用，而忽略那些复杂场景——比如遗漏必填字段、提供无效值、空查询，以及防护机制/滥用场景（超出范围、有害内容、专业建议、敏感数据、提示提取）。此技能会将Agent的约束条件转化为明确的风险检查清单，要求每个条目至少对应一个场景——详见
```
references/analyzing-the-agent.md
```
和
```
references/writing-scenarios.md
```
。

The flow

流程

Describe the agent + build the risk checklist — read its code locally and write a test-oriented description (Identity / Capabilities / Constraints) to
```
description.md
```
, and an explicit risk checklist to
```
risks.yaml
```
(one entry per must-test constraint/guardrail, each with an
```
id
```
and
```
category
```
). Follow
```
references/analyzing-the-agent.md
```
. Never upload the code.
Get the user's test focus — if they didn't say what to probe, ask. Apply it per
```
references/user-guidance.md
```
(append a
```
# Test Focus
```
to
```
description.md
```
, and bias authoring). Focus is additive — it deepens chosen risks but never drops the per-risk coverage floor. If they truly have no preference, generate broad and say so.
Author the scenarios — at least one per risk — write a diverse set of ~10 scenarios grounded in
```
description.md
```
and the focus, generating the persona / mood / situation variety from your own judgment (this version ships no attribute libraries). Guarantee coverage: every
```
risks.yaml
```
item gets ≥1 dedicated scenario, written with the shape that actually exercises it, and tagged with
```
covers: [<risk id>, …]
```
. Follow
```
references/writing-scenarios.md
```
(schema, the "Party A talks to the agent" rules, no prior state, no real PII, outcome-based expectations, the adversarial-shape taxonomy, the coverage check, don't write bad tests). Write them to
```
authored.yaml
```
. Add any user-pinned must-tests here too.

Assemble the config (coverage-enforced) —

python scripts/build_scenarios.py assemble --in authored.yaml --agent-description-file description.md --risks risks.yaml --strict --out scenarios.yaml

(validates the schema, fails if any risk is uncovered, and emits the YAML scenarios file

lk agent simulate --scenarios

loads). Fix gaps and re-run until it passes.

Run it —
```
lk agent simulate --scenarios scenarios.yaml
```
(confirm exact flags with
```
--help
```
; needs the SDK/auth noted in the beta block). Show the user the results and offer to re-roll, re-focus, or add scenarios.

Reuse saved

scenarios.yaml

files as a regression suite — re-run them after prompt/model/tool changes.

描述Agent + 构建风险检查清单——在本地读取其代码，将面向测试的描述（身份/能力/约束条件）写入
```
description.md
```
，并将明确的风险检查清单写入
```
risks.yaml
```
（每个必须测试的约束/防护机制对应一个条目，包含
```
id
```
和
```
category
```
）。遵循
```
references/analyzing-the-agent.md
```
的指引。绝不上传代码。
获取用户的测试重点——如果用户未说明要探查的内容，请询问。按照
```
references/user-guidance.md
```
的要求应用（在
```
description.md
```
中添加
```
# Test Focus
```
部分，并调整场景编写的偏向）。测试重点是附加性的——它会深化选定风险的测试，但绝不会降低每个风险的最低覆盖标准。如果用户确实没有偏好，则生成全面的场景并告知用户。
编写场景——每个风险至少对应一个——基于
```
description.md
```
和测试重点生成约10个多样化场景，根据你自身的判断生成角色/情绪/场景的多样性（当前版本未附带属性库）。确保覆盖所有风险：
```
risks.yaml
```
中的每个条目都要有≥1个专门场景，场景的设计要能实际触发该风险，并标记
```
covers: [<risk id>, …]
```
。遵循
```
references/writing-scenarios.md
```
的指引（schema、“甲方与Agent对话”规则、无前置状态、无真实个人身份信息、基于结果的预期、对抗性场景分类、覆盖检查、不编写无效测试）。将场景写入
```
authored.yaml
```
。用户指定的必须测试项也添加至此。

组装配置（强制覆盖检查）——

python scripts/build_scenarios.py assemble --in authored.yaml --agent-description-file description.md --risks risks.yaml --strict --out scenarios.yaml

（验证schema，如果有任何风险未覆盖则失败，并生成

lk agent simulate --scenarios

可加载的YAML场景文件

scenarios.yaml

）。修复缺口后重新运行，直至通过。

运行模拟——
```
lk agent simulate --scenarios scenarios.yaml
```
（请使用
```
--help
```
确认确切参数；需要测试阶段说明中的SDK/权限）。向用户展示结果，并提供重新生成、调整重点或添加场景的选项。

将保存的

scenarios.yaml

文件作为回归测试套件重复使用——在提示词/模型/工具变更后重新运行。

Principles

原则

Never upload the user's code. Reading it locally is the point; it's their IP.
The user's intent is the differentiator — incorporate it every time; don't silently autopilot.
Ground every scenario in the description, especially Constraints — a scenario the agent can't possibly satisfy (or a guardrail it should refuse) must have expectations that reflect that.
The script is deterministic glue; you are the generator. Let
```
build_scenarios.py
```
handle assembly + the coverage check; you do the reading, the judgement, the diversity, and the authoring.

绝不上传用户的代码。本地读取代码是核心；这是用户的知识产权。
用户的意图是关键——每次都要纳入用户意图；不要静默启用自动模式。
每个场景都要基于描述，尤其是约束条件——Agent不可能满足的场景（或应该拒绝的防护场景）必须有符合实际的预期。
脚本是确定性的工具；你是生成者。让
```
build_scenarios.py
```
处理组装和覆盖检查；你负责读取代码、判断、生成多样性内容以及编写场景。

Verify, don't invent (freeze-forever)

验证而非臆造（永久固定）

This skill is the method (no bundled libraries — you supply diversity yourself). The exact

lk agent simulate

flags, the CI wait/fail flag, the minimum SDK version, and the dashboard come from live sources because they change — use

lk agent simulate --help

and (post-beta)

lk docs

/ the LiveKit MCP server. A wrong flag wastes a run; look it up rather than guessing.

此技能是一套方法（无捆绑库——你自行提供多样性内容）。

lk agent simulate

的确切参数、CI等待/失败参数、最低SDK版本以及控制台信息来自实时源，因为这些内容会变化——请使用

lk agent simulate --help

以及（测试阶段后）

lk docs

/ LiveKit MCP服务器。错误的参数会浪费运行时间；请查询而非猜测。

After running: acting on results (secondary)

运行后：根据结果采取行动（次要步骤）

Once a run completes, read the per-scenario pass/fail, the run summary, and the transcripts of failures. Fix the agent where a failure is real (and re-run); recognize when a failure is actually a bad scenario and fix the scenario instead. Keep this lightweight — modern models are already good at the fix step; the durable value of this skill is the scenarios you generate and keep.

运行完成后，查看每个场景的通过/失败情况、运行摘要以及失败场景的对话记录。如果失败是真实问题，则修复Agent并重新运行；如果失败是场景本身的问题，则修复场景。保持流程轻量化——现代模型已能很好地完成修复步骤；此技能的持久价值在于你生成并保存的场景。