xilo-opus-video

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

xilo-opus-video：用代码做视频

xilo-opus-video: Making Videos with Code

Opus 自己不输出视频。这个 skill 让它写一个会自己动的网页（或 Manim 场景），网页暴露

render(t)

，由脚本一帧一帧截图，再用 FFmpeg 合成 MP4。声音也用代码合成或接语音模型。

一句话提示词做视频是在抽卡：出来什么样、为什么这样、下次能不能再做出来，都不可控。这个 skill 的做法是先把规划摆到用户面前，确认了再动手：三个方案 → 每个方案一张关键帧或几秒预览 → 用户确认 → 写成

brief.md

→ 按 brief 逐镜头制作、逐镜头自检 → 交付。

Opus does not generate video output on its own. This skill enables it to write a self-animating web page (or a Manim scene) that exposes

render(t)

. A script captures screenshots frame by frame, and FFmpeg assembles them into an MP4. Audio can also be synthesized in code or supplied by a speech model.

Creating a video from a one-line prompt is like rolling the dice: the result, the reason behind it, and whether it can be reproduced next time are all unpredictable. This skill takes a different approach: present the plan to the user first and begin only after confirmation: three options → one key frame or a few-second preview for each option → user confirmation → write

brief.md

→ produce and self-check each shot according to the brief → deliver the final result.

工作目录

Working Directory

在当前目录下建

opus-video/<slug>/

，所有东西放在里面：

opus-video/<slug>/
  source/        用户给的原材料、参考视频抽帧、分析笔记
  previews/      三个方案的预览（plan-a.png / plan-b.mp4 ...）
  brief.md       用户确认后的完整方案，也是可以复用的提示词
  index.html     成片的画面代码（或 Remotion / HyperFrames / Manim 工程）
  audio/         合成的配乐、音效、配音
  qa/            自检用的静帧和抽帧图
  out/final.mp4  成片

Create

opus-video/<slug>/

in the current directory and put everything inside it:

opus-video/<slug>/
  source/        用户给的原材料、参考视频抽帧、分析笔记
  previews/      三个方案的预览（plan-a.png / plan-b.mp4 ...）
  brief.md       用户确认后的完整方案，也是可以复用的提示词
  index.html     成片的画面代码（或 Remotion / HyperFrames / Manim 工程）
  audio/         合成的配乐、音效、配音
  qa/            自检用的静帧和抽帧图
  out/final.mp4  成片

第 0 步：检查环境

Step 0: Check the Environment

运行一次

python3 scripts/check_env.py

（路径相对本 skill 目录）。它会检查 python3、ffmpeg、Playwright 以及可用的浏览器，缺什么会告诉你怎么装。缺依赖时先告诉用户要装什么，得到同意再装。

Run

python3 scripts/check_env.py

once (the path is relative to this skill’s directory). It checks for python3, ffmpeg, Playwright and available browsers, and tells you how to install anything that is missing. If a dependency is missing, first tell the user what needs to be installed and install it only with their consent.

第 1 步：问清楚要做什么

Step 1: Clarify What to Create

先问用户想做什么，接受三种原材料，给了哪种就用哪种，可以混着给：

想法：一句话主题也行，越具体越好
原内容：文章、口播稿、产品网址、产品代码库、数据表
参考视频：本地文件或链接

用户给了现成的音乐时，先运行

python3 scripts/audio_tools.py beats <音乐文件> > source/beats.json

测出节拍、重拍和鼓点，方案和分镜都按这张节拍表来排。

只追问会改变方案的信息，一次最多问 3 个：时长、画幅（16:9 / 9:16 / 1:1）、要不要配音、有没有指定的品牌色或字体。用户没说的用合理默认值（30 秒、1920×1080、30fps、代码合成配乐、不配音），在方案里写明，让用户改。

处理原材料：

原内容：提炼出核心信息、受众和必须出现的事实。事实只能来自原材料，不确定的就不写，不要替用户编功能、数字、年份。
参考视频：链接需要下载时，先告诉用户文件来源并征得同意再下载。然后运行
```
python3 scripts/analyze_video.py <视频> --out source/ref
```
（
```
--out
```
是文件名前缀，会生成
```
source/ref-sheet.png
```
、
```
source/ref-info.txt
```
），它会输出时长、分辨率、帧率、镜头切换时间点和一张抽帧拼图。看拼图，写下参考视频的镜头结构、节奏、配色、字体和用到的代码类型（对照 references/code-stack.md）。参考视频只学结构和手法，不照搬别人的素材和文案。

First ask the user what they want to create. Accept the following three types of source material, use whichever ones are provided, and allow them to be combined:

Idea: A one-line topic is enough; the more specific, the better.
Source material: An article, narration script, product URL, product codebase or dataset.
Reference video: A local file or link.

If the user provides existing music, first run

python3 scripts/audio_tools.py beats <music file> > source/beats.json

to detect beats, downbeats and drum hits. Base the options and storyboard on this beat sheet.

Only ask about information that would change the approach, with no more than three questions at a time: duration, aspect ratio (16:9 / 9:16 / 1:1), whether voiceover is needed, and whether there are specified brand colors or fonts. Use sensible defaults for anything the user does not specify (30 seconds, 1920×1080, 30fps, code-synthesized soundtrack, no voiceover), state them in the options, and invite the user to make changes.

Process the source material as follows:

Source material: Extract the core message, audience and facts that must appear. Facts may come only from the source material; do not include uncertain claims, and do not invent features, numbers or years on the user’s behalf.
Reference video: If a link must be downloaded, first tell the user where the file will come from and obtain consent before downloading it. Then run
```
python3 scripts/analyze_video.py <video> --out source/ref
```
(
```
--out
```
is the output filename prefix; this generates
```
source/ref-sheet.png
```
and
```
source/ref-info.txt
```
). It reports the duration, resolution, frame rate, shot-change timestamps and a contact sheet of sampled frames. Review the contact sheet and document the reference video’s shot structure, pacing, color palette, typography and code types used, comparing them with references/code-stack.md. Learn only from its structure and techniques; do not copy another video’s assets or written content.

第 2 步：给三个方案

Step 2: Present Three Options

读 references/code-stack.md 选技术组合，按 references/plan-format.md 的格式写三个方案。

三个方案要有明显差别，差别在风格、叙事结构或技术组合上，不能只是换个配色。常见的拉开方式：一个稳妥（最贴近原材料和参考），一个风格化（换一种画面语言），一个结构上大胆（换一种讲法，比如用一个贯穿全片的主角或者一个不切镜头的长镜头）。

每个方案都要写画面概括（逐镜头的一两句话）、技术组合、声音、预计渲染耗时和风险。

Read references/code-stack.md to select the technology stack, then write three options in the format specified by references/plan-format.md.

The three options must be clearly differentiated in style, narrative structure or technology stack; changing only the color palette is not enough. Common approaches include one conservative option that stays closest to the source material and reference, one stylized option that uses a different visual language, and one structurally bold option that changes the storytelling approach—for example, through a protagonist that appears throughout the video or a continuous long take without cuts.

Each option must include a visual summary (one or two sentences per shot), technology stack, audio approach, estimated rendering time and risks.

第 3 步：给每个方案做预览

Step 3: Create a Preview for Each Option

预览是为了让用户在花大量时间渲染之前，先看到画面长什么样。预览页面也是

render(t)

页面，动手前先读 references/craft-rules.md 的页面骨架和字体两节，后面正式制作时可以直接在预览代码上接着做。

每个方案的预览页只搭一个镜头，时间用这个镜头自己的时间（从 0 开始），不用搭整条时间轴。

默认每个方案做 一张关键帧：只搭这个方案最有代表性的那一个镜头，用
```
python3 scripts/render.py previews/plan-a.html previews/plan-a.png --still 2.0 --size 1920x1080
```
。

方案的卖点在运动本身时（界面变形、节拍卡点、转场），改做 2–4 秒的小样：

python3 scripts/render.py previews/plan-b.html previews/plan-b.mp4 --size <方案画幅> --fps 30 --duration 3

。

预览做出来后自己先看一遍图，明显的问题（字溢出、元素挤在一起、画面太空）先修，再给用户。
方案里的「预计渲染耗时」用预览来估：渲染脚本会打印用时，单帧耗时 × 总帧数 × 子帧数，再乘 1.3 左右留余量。

把三个方案和预览路径一起给用户，请他选一个，或者说要怎么混合、修改。用户要改就回到第 2 或第 3 步，直到确认。

The preview lets users see how each option will look before spending substantial time on rendering. Preview pages use the same

render(t)

model. Before starting, read the page skeleton and typography sections of references/craft-rules.md; this code can later be extended for the final production.

Build only one shot in each preview page, using that shot’s own local time starting from 0. There is no need to build the entire timeline.

By default, create one key frame for each option: build only the option’s most representative shot, using
```
python3 scripts/render.py previews/plan-a.html previews/plan-a.png --still 2.0 --size 1920x1080
```
.
If motion itself is the option’s main selling point, such as interface morphing, beat-synced action or transitions, create a 2–4 second sample instead:
```
python3 scripts/render.py previews/plan-b.html previews/plan-b.mp4 --size <plan aspect ratio> --fps 30 --duration 3
```
.
Review every preview yourself before sharing it. Fix obvious problems such as overflowing text, crowded elements or an overly empty frame.
Estimate each option’s “rendering time” from its preview: the rendering script prints the elapsed time, so multiply the per-frame time by the total number of frames and subframes, then multiply by approximately 1.3 to leave a margin.

Present all three options and their preview paths, and ask the user to choose one or explain how to combine or modify them. If the user requests changes, return to Step 2 or Step 3 and repeat until they confirm an option.

第 4 步：写 brief.md

Step 4: Write brief.md

用户确认后，按 references/brief-template.md 写

brief.md

。它就是这支视频完整的结构化提示词，分

<role>

<inputs>

<direction>

<structure>

<build>

<gotchas>

<start>

七块，已经确认的内容全部填实。

这份文件有两个用处：接下来的制作严格按它来；用户以后想复用、换主题、或者拿到别的工具里跑，直接用这份。写完告诉用户文件位置。

After the user confirms an option, write

brief.md

according to references/brief-template.md. It is the video’s complete structured prompt, divided into the seven sections

<role>

<inputs>

<direction>

<structure>

<build>

<gotchas>

and

<start>

. Fill in all confirmed information.

This file has two purposes: it governs the production that follows, and it serves as a reusable prompt for future videos, different topics or other tools. Tell the user where the file is located.

第 5 步：制作

Step 5: Production

制作规则在 references/craft-rules.md，动手前把整份读完。最要紧的几条：

画面只由时间决定。
```
window.render(t)
```
画出第 t 秒；不用
```
requestAnimationFrame
```
、
```
setTimeout
```
、CSS transition，随机数带固定种子，帧和帧之间不保存状态。这样任何一帧都能单独重渲，改一个镜头不会影响别的镜头。
一个镜头一个镜头做。每做完一个镜头，用
```
render.py --still
```
渲 3 张静帧，自己看：文字有没有溢出、元素有没有重叠、主体够不够大、字停留够不够久。修好再做下一个。30 秒以上的片子，所有镜头搭完后先渲一版 960×540、不加运动模糊的低清小样，节奏和转场对了再精修，精修前别花时间在细节上。
声音和画面用同一张时间表。先定节拍网格或旁白时间轴，镜头切换和关键动作落在上面；音效放在画面动作发生的那一帧。配音整段合成，不要一句一句拼。
镜头要动。推、拉、跟随，让主体在画面里足够大；brief 没写镜头的时候，Opus 默认会固定机位，画面容易显得空。

用户点名框架时按框架来（Remotion / HyperFrames / Manim），没点名默认单个 HTML 文件，具体选择见 code-stack.md。

Read the complete references/craft-rules.md before starting. The most important rules are:

Visuals are determined solely by time.
```
window.render(t)
```
draws the frame at second t. Do not use
```
requestAnimationFrame
```
,
```
setTimeout
```
or CSS transitions. Use a fixed seed for random values, and do not preserve state between frames. This makes every frame independently re-renderable and prevents a change to one shot from affecting another.
Build one shot at a time. After completing each shot, render three stills with
```
render.py --still
```
and inspect them: check for text overflow, overlapping elements, whether the subject is large enough and whether the text remains on screen long enough. Fix the shot before moving on. For videos longer than 30 seconds, first render a 960×540 rough cut without motion blur after all shots are built. Refine only after the pacing and transitions are correct; do not spend time on details before that point.
Use the same timeline for audio and visuals. Establish the beat grid or narration timeline first, then align shot changes and key actions to it. Place sound effects on the exact frame where the corresponding action occurs. Synthesize the voiceover as one continuous piece rather than assembling it sentence by sentence.
Shots should move. Push in, pull out and follow the action while keeping the subject large enough in the frame. If the brief does not specify camera movement, Opus defaults to a locked-off camera, which can make the image feel empty.

Use the framework requested by the user (Remotion / HyperFrames / Manim). If no framework is specified, default to a single HTML file; see code-stack.md for the available choices.

第 6 步：渲染和验收

Step 6: Rendering and Acceptance

渲染：

python3 scripts/render.py index.html out/video.mp4 --size 1920x1080 --fps 30 --duration 30 --audio audio/mix.wav

，要运动模糊加

--subframes 4

（渲染时间也会变成 4 倍，先告诉用户）。

声音：音效用

python3 scripts/audio_tools.py sfx timeline.json audio/sfx.wav

按时间点合成，混好后

audio_tools.py normalize

到 -14 LUFS，再用

audio_tools.py check audio/mix.wav --out qa/audio

看响度和波形图。

验收：
```
python3 scripts/analyze_video.py out/final.mp4 --out qa/final
```
出抽帧图，再加一张手机尺寸的（
```
--cell-width 360
```
）。按 craft-rules.md 的「自检打分」给自己打分，列出最严重的 3 个问题和时间点，改完只重渲那几秒，重复到每项 8 分以上，再对照验收清单核对。把每一轮的分数和问题记在
```
qa/review_log.md
```
里，交付时一起给用户。
有问题就回到对应镜头修，只重渲那一段也可以（
```
--start
```
/
```
--duration
```
），最后再整片合成。

Render with

python3 scripts/render.py index.html out/video.mp4 --size 1920x1080 --fps 30 --duration 30 --audio audio/mix.wav

. To add motion blur, use

--subframes 4

; this also makes rendering four times slower, so tell the user first.

For sound effects, synthesize them at the appropriate timestamps with
```
python3 scripts/audio_tools.py sfx timeline.json audio/sfx.wav
```
. After mixing, normalize with
```
audio_tools.py normalize
```
to -14 LUFS, then use
```
audio_tools.py check audio/mix.wav --out qa/audio
```
to inspect the loudness and waveform.
For acceptance, run
```
python3 scripts/analyze_video.py out/final.mp4 --out qa/final
```
to generate a contact sheet, then generate one at mobile size as well (
```
--cell-width 360
```
). Follow the “Self-Check Scoring” section of craft-rules.md, list the three most serious problems and their timestamps, fix them, and re-render only the affected seconds. Repeat until every item scores above 8, then check the result against the acceptance checklist. Record every round’s scores and issues in
```
qa/review_log.md
```
and provide it with the final delivery.
If there is a problem, return to the corresponding shot and fix it. It is also possible to re-render only that segment with
```
--start
```
/
```
--duration
```
, then combine the final video after all fixes are complete.

交付

Delivery

告诉用户：成片路径、

brief.md

路径、工程目录，以及一段简短说明，包括这次的关键创作决定、已知的不足（比如某个镜头可以更好）、渲染用时。事实类内容里有不确定的地方，单独列出来让用户核对。

Tell the user the paths to the final video,

brief.md

and the project directory. Include a short explanation covering the key creative decisions, known limitations (for example, a shot that could be improved) and rendering time. If any factual content is uncertain, list it separately so the user can verify it.

遇到这些情况

Special Cases

用户想要真人或写实角色：代码画不出来。可以建议用户先用视频模型生成纯绿幕背景的人物片段，交给本 skill 抠像，再用代码搭场景、大字和动效，做法见 code-stack.md 的「视频模型角色 + 代码场景」。
渲染太慢：先降分辨率或帧率出小样，确认后再出正式版；运动模糊和 WebGL 后期是最耗时的两项。
浏览器里 WebGL 不可用：
```
check_env.py
```
会提示；先渲一帧确认着色器真的生效了，再渲整片。
用户只想要提示词：跳过第 5、6 步，把
```
brief.md
```
交给用户即可。

The user wants live-action or photorealistic characters: Code cannot draw these. Suggest that the user first generate character footage with a video model using a plain green-screen background, then let the skill key out the background and build the scene, large typography and motion effects in code. See the “Video-Model Characters + Code-Built Scenes” section of code-stack.md.
Rendering is too slow: First reduce the resolution or frame rate to create a sample, and produce the final version only after confirmation. Motion blur and WebGL post-processing are usually the two most expensive operations.
WebGL is unavailable in the browser:
```
check_env.py
```
will report this. Render one frame first to confirm that the shader is actually working, then render the full video.
The user only wants a prompt: Skip Steps 5 and 6 and provide
```
brief.md
```
to the user.