xilo-opus-video: Making Videos with Code

Opus does not generate video output on its own. This skill enables it to write a self-animating web page (or a Manim scene) that exposes

render(t)

. A script captures screenshots frame by frame, and FFmpeg assembles them into an MP4. Audio can also be synthesized in code or supplied by a speech model.

Creating a video from a one-line prompt is like rolling the dice: the result, the reason behind it, and whether it can be reproduced next time are all unpredictable. This skill takes a different approach: present the plan to the user first and begin only after confirmation: three options → one key frame or a few-second preview for each option → user confirmation → write

brief.md

→ produce and self-check each shot according to the brief → deliver the final result.

Working Directory

Create

opus-video/<slug>/

in the current directory and put everything inside it:

opus-video/<slug>/
  source/        用户给的原材料、参考视频抽帧、分析笔记
  previews/      三个方案的预览（plan-a.png / plan-b.mp4 ...）
  brief.md       用户确认后的完整方案，也是可以复用的提示词
  index.html     成片的画面代码（或 Remotion / HyperFrames / Manim 工程）
  audio/         合成的配乐、音效、配音
  qa/            自检用的静帧和抽帧图
  out/final.mp4  成片

Step 0: Check the Environment

Run

python3 scripts/check_env.py

once (the path is relative to this skill’s directory). It checks for python3, ffmpeg, Playwright and available browsers, and tells you how to install anything that is missing. If a dependency is missing, first tell the user what needs to be installed and install it only with their consent.

Step 1: Clarify What to Create

First ask the user what they want to create. Accept the following three types of source material, use whichever ones are provided, and allow them to be combined:

Idea: A one-line topic is enough; the more specific, the better.
Source material: An article, narration script, product URL, product codebase or dataset.
Reference video: A local file or link.

If the user provides existing music, first run

python3 scripts/audio_tools.py beats <music file> > source/beats.json

to detect beats, downbeats and drum hits. Base the options and storyboard on this beat sheet.

Only ask about information that would change the approach, with no more than three questions at a time: duration, aspect ratio (16:9 / 9:16 / 1:1), whether voiceover is needed, and whether there are specified brand colors or fonts. Use sensible defaults for anything the user does not specify (30 seconds, 1920×1080, 30fps, code-synthesized soundtrack, no voiceover), state them in the options, and invite the user to make changes.

Process the source material as follows:

Source material: Extract the core message, audience and facts that must appear. Facts may come only from the source material; do not include uncertain claims, and do not invent features, numbers or years on the user’s behalf.
Reference video: If a link must be downloaded, first tell the user where the file will come from and obtain consent before downloading it. Then run
```
python3 scripts/analyze_video.py <video> --out source/ref
```
(
```
--out
```
is the output filename prefix; this generates
```
source/ref-sheet.png
```
and
```
source/ref-info.txt
```
). It reports the duration, resolution, frame rate, shot-change timestamps and a contact sheet of sampled frames. Review the contact sheet and document the reference video’s shot structure, pacing, color palette, typography and code types used, comparing them with references/code-stack.md. Learn only from its structure and techniques; do not copy another video’s assets or written content.

Step 2: Present Three Options

Read references/code-stack.md to select the technology stack, then write three options in the format specified by references/plan-format.md.

The three options must be clearly differentiated in style, narrative structure or technology stack; changing only the color palette is not enough. Common approaches include one conservative option that stays closest to the source material and reference, one stylized option that uses a different visual language, and one structurally bold option that changes the storytelling approach—for example, through a protagonist that appears throughout the video or a continuous long take without cuts.

Each option must include a visual summary (one or two sentences per shot), technology stack, audio approach, estimated rendering time and risks.

Step 3: Create a Preview for Each Option

The preview lets users see how each option will look before spending substantial time on rendering. Preview pages use the same

render(t)

model. Before starting, read the page skeleton and typography sections of references/craft-rules.md; this code can later be extended for the final production.

Build only one shot in each preview page, using that shot’s own local time starting from 0. There is no need to build the entire timeline.

By default, create one key frame for each option: build only the option’s most representative shot, using
```
python3 scripts/render.py previews/plan-a.html previews/plan-a.png --still 2.0 --size 1920x1080
```
.
If motion itself is the option’s main selling point, such as interface morphing, beat-synced action or transitions, create a 2–4 second sample instead:
```
python3 scripts/render.py previews/plan-b.html previews/plan-b.mp4 --size <plan aspect ratio> --fps 30 --duration 3
```
.
Review every preview yourself before sharing it. Fix obvious problems such as overflowing text, crowded elements or an overly empty frame.
Estimate each option’s “rendering time” from its preview: the rendering script prints the elapsed time, so multiply the per-frame time by the total number of frames and subframes, then multiply by approximately 1.3 to leave a margin.

Present all three options and their preview paths, and ask the user to choose one or explain how to combine or modify them. If the user requests changes, return to Step 2 or Step 3 and repeat until they confirm an option.

Step 4: Write brief.md

After the user confirms an option, write

brief.md

according to references/brief-template.md. It is the video’s complete structured prompt, divided into the seven sections

<role>

<inputs>

<direction>

<structure>

<build>

<gotchas>

and

<start>

. Fill in all confirmed information.

This file has two purposes: it governs the production that follows, and it serves as a reusable prompt for future videos, different topics or other tools. Tell the user where the file is located.

Step 5: Production

Read the complete references/craft-rules.md before starting. The most important rules are:

Visuals are determined solely by time.
```
window.render(t)
```
draws the frame at second t. Do not use
```
requestAnimationFrame
```
,
```
setTimeout
```
or CSS transitions. Use a fixed seed for random values, and do not preserve state between frames. This makes every frame independently re-renderable and prevents a change to one shot from affecting another.
Build one shot at a time. After completing each shot, render three stills with
```
render.py --still
```
and inspect them: check for text overflow, overlapping elements, whether the subject is large enough and whether the text remains on screen long enough. Fix the shot before moving on. For videos longer than 30 seconds, first render a 960×540 rough cut without motion blur after all shots are built. Refine only after the pacing and transitions are correct; do not spend time on details before that point.
Use the same timeline for audio and visuals. Establish the beat grid or narration timeline first, then align shot changes and key actions to it. Place sound effects on the exact frame where the corresponding action occurs. Synthesize the voiceover as one continuous piece rather than assembling it sentence by sentence.
Shots should move. Push in, pull out and follow the action while keeping the subject large enough in the frame. If the brief does not specify camera movement, Opus defaults to a locked-off camera, which can make the image feel empty.

Use the framework requested by the user (Remotion / HyperFrames / Manim). If no framework is specified, default to a single HTML file; see code-stack.md for the available choices.

Step 6: Rendering and Acceptance

Render with

python3 scripts/render.py index.html out/video.mp4 --size 1920x1080 --fps 30 --duration 30 --audio audio/mix.wav

. To add motion blur, use

--subframes 4

; this also makes rendering four times slower, so tell the user first.

For sound effects, synthesize them at the appropriate timestamps with
```
python3 scripts/audio_tools.py sfx timeline.json audio/sfx.wav
```
. After mixing, normalize with
```
audio_tools.py normalize
```
to -14 LUFS, then use
```
audio_tools.py check audio/mix.wav --out qa/audio
```
to inspect the loudness and waveform.
For acceptance, run
```
python3 scripts/analyze_video.py out/final.mp4 --out qa/final
```
to generate a contact sheet, then generate one at mobile size as well (
```
--cell-width 360
```
). Follow the “Self-Check Scoring” section of craft-rules.md, list the three most serious problems and their timestamps, fix them, and re-render only the affected seconds. Repeat until every item scores above 8, then check the result against the acceptance checklist. Record every round’s scores and issues in
```
qa/review_log.md
```
and provide it with the final delivery.
If there is a problem, return to the corresponding shot and fix it. It is also possible to re-render only that segment with
```
--start
```
/
```
--duration
```
, then combine the final video after all fixes are complete.

Delivery

Tell the user the paths to the final video,

brief.md

and the project directory. Include a short explanation covering the key creative decisions, known limitations (for example, a shot that could be improved) and rendering time. If any factual content is uncertain, list it separately so the user can verify it.

Special Cases

The user wants live-action or photorealistic characters: Code cannot draw these. Suggest that the user first generate character footage with a video model using a plain green-screen background, then let the skill key out the background and build the scene, large typography and motion effects in code. See the “Video-Model Characters + Code-Built Scenes” section of code-stack.md.
Rendering is too slow: First reduce the resolution or frame rate to create a sample, and produce the final version only after confirmation. Motion blur and WebGL post-processing are usually the two most expensive operations.
WebGL is unavailable in the browser:
```
check_env.py
```
will report this. Render one frame first to confirm that the shader is actually working, then render the full video.
The user only wants a prompt: Skip Steps 5 and 6 and provide
```
brief.md
```
to the user.

xilo-opus-video

NPX Install

Tags

SKILL.md Content (Chinese)