talking-avatar-skill

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Talking Avatar

会说话的头像

Create one focused voice conversation with a fixed character portrait whose mouth poses follow the actual remote audio. Accept either a photograph or a character description, generate a canonical portrait plus small mouth sprites, and build the app around OpenAI Realtime.

创建一个专注于单一语音对话的应用，使用固定角色肖像，其嘴型姿态跟随实际的远程音频。接受照片或角色描述作为输入，生成一张标准肖像和小型嘴型精灵图，并围绕 OpenAI Realtime 构建应用。

Coordinate the required skills

协调所需的 skill

Use
```
imagegen
```
for every canonical portrait or mouth-frame generation/edit. Read its full instructions before generating.
Use
```
openai-docs
```
and, when applicable,
```
openai-platform-api-key
```
to verify the current Realtime WebRTC endpoint, session schema, model, voices, and credential flow. Never rely on this skill's model examples as current truth.
Use the available frontend design skill for interface work.

.openai/hosting.json

exists, follow

sites-building

and

sites-hosting

through deployment.

Do not add backgrounds, wardrobes, role modes, or multiple characters unless the user explicitly asks. The default product is one avatar and one conversation.

对每张标准肖像或嘴型帧的生成或编辑，都使用
```
imagegen
```
。生成前阅读其完整说明。
使用
```
openai-docs
```
，并在适用时使用
```
openai-platform-api-key
```
，验证当前的 Realtime WebRTC 端点、session schema、模型、音色和凭据流程。绝不要将本 skill 中的模型示例视为当前事实。
界面工作使用可用的 frontend design skill。

如果存在

.openai/hosting.json

，部署时遵循

sites-building

和

sites-hosting

的要求。

除非用户明确要求，否则不要添加背景、服装、角色模式或多个角色。默认产品是一个头像和一段对话。

Choose the input path

选择输入方式

Photograph supplied: inspect it with
```
view_image
```
, treat it as an identity reference, and preserve face shape, skin tone, hair, glasses, facial hair, and distinguishing features.
Character described: normalize the description into a concrete visual spec. Ask only for a missing detail that materially changes identity, such as photorealistic versus illustrated. Otherwise proceed with tasteful defaults.
Establish the character name, concise persona, app name, language, and preferred voice. Infer reasonable defaults when safe.

Read references/image-pipeline.md before creating or editing the avatar assets.

提供了照片： 使用
```
view_image
```
检查照片，将其作为身份参考，并保留脸型、肤色、头发、眼镜、胡须和显著特征。
描述了角色： 将描述规范化为具体的视觉规格。仅当缺失的细节会实质改变身份时才询问，例如写实还是插画。否则采用得体的默认值。
确定角色名称、简洁人设、应用名称、语言和偏好的音色。在安全的情况下推断合理默认值。

在创建或编辑头像资源之前，读取 references/image-pipeline.md。

Build the asset set

构建资源集

Generate one canonical, front-facing, evenly lit, closed-mouth portrait. Lock its canvas, crop, head position, gaze, pose, clothing, background, and lighting.
From that canonical image, generate three separate identity-preserving edits:
- ```
soft
```
  : lips slightly parted;
- ```
round
```
  : a natural rounded vowel;
- ```
open
```
  : a natural medium-open vowel.
Repeat the invariants in every edit prompt. Change only the immediate lips and inner mouth. Reject frames with moved eyes, jaw, glasses, head, beard, clothing, or lighting.
Crop the smallest practical identical rectangle from all three speech frames. Keep enough surrounding skin for feathering, but do not ship full-frame speech images.

Save exactly these project assets by default:

```
public/avatar/avatar-base.jpg
```
```
public/avatar/mouth-soft.png
```
```
public/avatar/mouth-round.png
```
```
public/avatar/mouth-open.png
```

Run

scripts/validate_avatar_assets.py --dir <project>/public/avatar

and fix all failures.

Never animate the complete portrait. Only replace the mouth patch.

生成一张正面、光照均匀、闭嘴的标准肖像。锁定其画布、裁剪、头部位置、视线、姿势、服装、背景和光照。
基于该标准图像，生成三张独立的身份保持型编辑图：
- ```
soft
```
  ：嘴唇略微张开；
- ```
round
```
  ：自然圆润的元音口型；
- ```
open
```
  ：自然的中等张开元音口型。
在每个编辑提示词中重复这些不变量。仅改变紧邻嘴唇和口腔内部。拒绝使用眼睛、下颌、眼镜、头部、胡须、服装或光照发生变化的帧。
从三个语音帧中裁剪出尺寸完全一致且尽可能小的实用矩形。保留足够的周围皮肤用于羽化，但不要交付整帧语音图像。

默认将项目资源精确保存为：

```
public/avatar/avatar-base.jpg
```
```
public/avatar/mouth-soft.png
```
```
public/avatar/mouth-round.png
```
```
public/avatar/mouth-open.png
```

运行

scripts/validate_avatar_assets.py --dir <project>/public/avatar

并修复所有失败项。

绝不要对完整肖像制作动画。只替换嘴型补丁。

Build or integrate the app

构建或集成应用

Prefer an existing working stack. For a new project, use Next.js or vinext because the user's API key must cross a server route; use plain Vite only when a safe backend already exists.

For a compatible new Next/vinext project, scaffold the proven core:

bash

python3 <skill-dir>/scripts/scaffold_app.py \
  --target <project-dir> \
  --character-name "Mira" \
  --app-name "Mira Live" \
  --persona "a concise, curious field researcher" \
  --model "<verified-realtime-model>" \
  --transcription-model "<verified-transcription-model>" \
  --voice "<verified-realtime-voice>"

The target must already contain the framework configuration and dependencies. Install

framer-motion

and

@phosphor-icons/react

when missing. Use

--force

only for a newly initialized project after inspecting the files it will replace. For an existing product, copy or adapt individual files from

assets/starter/

instead of overwriting its architecture.

Read references/app-contract.md before changing the Realtime route, API-key flow, or event handling.

优先使用现有可工作的技术栈。对于新项目，使用 Next.js 或 vinext，因为用户的 API 密钥必须通过服务器路由传递；仅当已有安全后端时才使用纯 Vite。

对于兼容的新 Next/vinext 项目，基于已验证的核心搭建项目：

bash

python3 <skill-dir>/scripts/scaffold_app.py \
  --target <project-dir> \
  --character-name "Mira" \
  --app-name "Mira Live" \
  --persona "a concise, curious field researcher" \
  --model "<verified-realtime-model>" \
  --transcription-model "<verified-transcription-model>" \
  --voice "<verified-realtime-voice>"

目标目录必须已包含框架配置和依赖。缺少

framer-motion

和

@phosphor-icons/react

时进行安装。仅在检查将要被覆盖的文件后，且项目刚初始化完成时，才对新项目使用

--force

。对于现有产品，从

assets/starter/

复制或改造单个文件，而不是覆盖其架构。

在修改 Realtime 路由、API 密钥流程或事件处理之前，读取 references/app-contract.md。

Implement natural lip sync

实现自然的口型同步

Read references/realtime-lipsync.md and preserve its behavioral contract:

Analyze the remote output
```
MediaStream
```
, not microphone input and not Realtime event cadence.
Sample audio every animation frame without updating React state.
Update the visible mouth pose no faster than about every 96 ms.
Smooth attack and decay, normalize against a rolling peak, and close after short silence.
Move through adjacent poses (
```
closed → soft → round → open
```
) instead of jumping.
Set a DOM
```
data-mouth
```
attribute and let CSS reveal exactly one patch.
Keep the portrait pixels completely static.

Do not use a canned infinite mouth animation.

response.output_audio.delta

identifies speaking state but does not provide lip timing.

读取 references/realtime-lipsync.md，并保持其行为约定：

分析远程输出
```
MediaStream
```
，而不是麦克风输入或 Realtime 事件节奏。
每个动画帧都采样音频，但不更新 React state。
可见嘴型姿态的更新间隔不得快于约每 96 毫秒一次。
平滑起音和衰减，基于滚动峰值进行归一化，并在短暂静音后闭合。
依次切换相邻姿态（
```
closed → soft → round → open
```
），不要直接跳转。
设置 DOM
```
data-mouth
```
属性，并让 CSS 仅显示一个补丁。
保持肖像像素完全静止。

不要使用预制的无限循环嘴型动画。

response.output_audio.delta

可用于判断说话状态，但不提供嘴型时序。

Preserve the BYOK security boundary

保持 BYOK 安全边界

Ask the user to enter the OpenAI API key inside the app.
Keep the key in component memory only; never use local storage, session storage, IndexedDB, analytics, logs, or source files.
Send it once to the same-origin session route over HTTPS, clear client state after negotiation, and return
```
Cache-Control: no-store
```
.
Never expose a long-lived key directly to the Realtime WebRTC peer connection.
Map upstream authentication and rate-limit failures to short user-facing errors without echoing response bodies or secrets.

要求用户在应用内输入 OpenAI API 密钥。
密钥只保存在组件内存中；绝不使用 local storage、session storage、IndexedDB、分析工具、日志或源文件。
通过 HTTPS 将其一次性发送到同源 session 路由，协商完成后清除客户端状态，并返回
```
Cache-Control: no-store
```
。
绝不要将长期有效的密钥直接暴露给 Realtime WebRTC 对等连接。
将上游身份验证和速率限制失败映射为简短的用户可见错误，且不得回显响应正文或机密信息。

Validate before handoff

交付前验证

Run the production build.
Run lint and the starter regression test when available.
Confirm the source contains an
```
AnalyserNode
```
,
```
requestAnimationFrame
```
, a 90–110 ms pose interval, adjacent-pose stepping, and no CSS
```
@keyframes mouth-*
```
loop.
Confirm all mouth patches have identical dimensions and align over the canonical mouth at desktop and mobile sizes.
Confirm only the mouth changes during speech; blinking, head bobbing, pose drift, and full-frame swaps are failures.
Confirm mic mute, speaker mute, interruption, transcript deltas, typed messages, disconnect cleanup, audio-context cleanup, empty state, connection errors, and invalid-key errors.
If the user requests visual browser QA, test a live voice turn and tune timing from observed speech. Otherwise do not claim that browser-level lip sync was visually verified.
Deploy when requested or when the active hosting skill requires it, then return the live URL.

运行生产构建。
运行 lint 和 starter 回归测试（如可用）。
确认源代码包含
```
AnalyserNode
```
、
```
requestAnimationFrame
```
、90–110 ms 的姿态更新间隔、相邻姿态切换，并且不存在 CSS
```
@keyframes mouth-*
```
循环。
确认所有嘴型补丁尺寸一致，并在桌面端和移动端尺寸下与标准肖像中的嘴部对齐。
确认说话期间只有嘴部发生变化；眨眼、头部晃动、姿态漂移和整帧切换均视为失败。
确认麦克风静音、扬声器静音、打断、转录增量、文字消息、断开连接清理、AudioContext 清理、空状态、连接错误和无效密钥错误。
如果用户要求进行可视化浏览器 QA，请测试一次实时语音交互，并根据观察到的语音调整时序。否则不要声称已在浏览器中通过视觉验证口型同步。
按要求或根据当前启用的 hosting skill 要求进行部署，然后返回线上 URL。

Deliverables

交付内容

Return the runnable app, the final avatar asset paths, the generation prompt set, whether photo or description mode was used, the Realtime model/voice verified for that run, validation results, and the deployed URL when applicable.

交付可运行的应用、最终头像资源路径、生成提示词集、使用的照片或描述模式、此次运行所验证的 Realtime 模型/音色、验证结果，以及适用时的部署 URL。