gitframes-render
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
Chinesegitframes-render
gitframes-render
Setup
设置
Import everything from the npm package. If the project doesn't have gitframes set up yet, follow the skill first (install, project scaffold, first render).
gitframesgitframes从 npm 包中导入全部内容。如果项目尚未配置 gitframes,请先按照 skill 的流程操作(安装、项目脚手架、首次渲染)。
gitframesgitframesOverview
概述
gitframesBecause video rendering is resource-intensive and native GPU pipelines run asynchronously in worker pools, gitframes workflows enforce a strict verification ladder: inspect inexpensive intermediate artifacts first before committing to a full video render.
gitframes由于视频渲染非常消耗资源,而且原生 GPU 流水线会在 worker pool 中异步运行,gitframes 工作流强制执行严格的验证阶梯:在提交完整视频渲染之前,先检查成本较低的中间产物。
The 5-Step Verification Ladder
5 步验证阶梯
Always proceed through the ladder in order. Catching a layout or timing bug at step 1 or 2 takes seconds; catching it after a 30-second 4K video render wastes minutes.
始终按顺序执行验证阶梯。在步骤 1 或 2 中发现布局或时序问题只需几秒钟;等到 30 秒的 4K 视频渲染完成后才发现问题,则会浪费数分钟。
Step 1: Static Typecheck & Schema Validation
步骤 1:静态类型检查与 Schema 验证
Before touching the GPU, verify TypeScript types and parse the composition against the strict runtime schema:
typescript
import { CompositorProgramSchema } from "gitframes";
// 1. Compile spec
const spec = comp.toSpec();
// 2. Validate against strict schema
const parsed = CompositorProgramSchema.safeParse(spec);
if (!parsed.success) {
console.error("Schema validation failed:", parsed.error.format());
process.exit(1);
}Typecheck the project:
bash
npx tsc --noEmit -p .在调用 GPU 之前,验证 TypeScript 类型,并根据严格的运行时 schema 解析合成内容:
typescript
import { CompositorProgramSchema } from "gitframes";
// 1. Compile spec
const spec = comp.toSpec();
// 2. Validate against strict schema
const parsed = CompositorProgramSchema.safeParse(spec);
if (!parsed.success) {
console.error("Schema validation failed:", parsed.error.format());
process.exit(1);
}对项目执行类型检查:
bash
npx tsc --noEmit -p .Step 2: Single Frame Inspection (renderFrame
)
renderFrame步骤 2:单帧检查(renderFrame
)
renderFrameRender isolated keyframe points (e.g. entry, peak action, exit) directly to PNG files on disk to inspect alignment, typography, colors, and opacity:
typescript
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer } from "gitframes";
const renderer = new HeadlessMediaRenderer();
// Render frame 45 (1.5s at 30fps)
const pngBuffer = await comp.renderFrame({
frame: 45,
renderer,
});
await fs.writeFile("output/frame-45.png", pngBuffer);将独立的关键帧位置(例如入场、动作峰值、退场)直接渲染到磁盘上的 PNG 文件中,以检查对齐方式、排版、颜色和不透明度:
typescript
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer } from "gitframes";
const renderer = new HeadlessMediaRenderer();
// Render frame 45 (1.5s at 30fps)
const pngBuffer = await comp.renderFrame({
frame: 45,
renderer,
});
await fs.writeFile("output/frame-45.png", pngBuffer);Step 3: Motion & Layout Stability Contact Sheets (renderFrameGrid
)
renderFrameGrid步骤 3:动态与布局稳定性帧网格(renderFrameGrid
)
renderFrameGridTo verify easing curves, cut points, kinetic sweeps, or text wrapping without encoding a full video, generate a sequential frame grid image:
typescript
import fs from "node:fs/promises";
import { renderFrameGrid } from "gitframes";
// Samples frames across the intro sweep (frames 0 to 60)
const gridPng = await comp.renderFrameGrid({
frames: [0, 6, 12, 18, 24, 30, 36, 42, 48, 54, 60],
columns: 4,
cellWidth: 480, // Downscaled thumbnail size for rapid visual scanning
showLabels: true, // frame number and timestamp under each cell
});
await fs.writeFile("output/intro-grid.png", gridPng);Contact sheets immediately reveal:
- Motion hitches and non-smooth easing deceleration.
- Text clipping or wrapping jumps across keyframe boundaries.
- Bad layer z-indexing or missing cut transitions.
在不对完整视频进行编码的情况下验证缓动曲线、剪切点、动态扫掠或文本换行,可生成顺序帧网格图像:
typescript
import fs from "node:fs/promises";
import { renderFrameGrid } from "gitframes";
// Samples frames across the intro sweep (frames 0 to 60)
const gridPng = await comp.renderFrameGrid({
frames: [0, 6, 12, 18, 24, 30, 36, 42, 48, 54, 60],
columns: 4,
cellWidth: 480, // Downscaled thumbnail size for rapid visual scanning
showLabels: true, // frame number and timestamp under each cell
});
await fs.writeFile("output/intro-grid.png", gridPng);帧网格可以立即显示:
- 动作卡顿和缓动减速不平滑。
- 文本被裁切,或跨关键帧边界时换行发生跳变。
- 图层 z-index 排序错误或缺少转场。
Step 4: Programmatic Conformance Probes in Vitest
步骤 4:Vitest 中的程序化一致性探针
Write automated headless vitest tests that assert pixel colors, transparency, or layout bounding boxes:
typescript
import { describe, it, expect } from "vitest";
import { HeadlessMediaRenderer } from "gitframes";
import { buildFilm } from "../film.js";
describe("Hero Scene Conformance", () => {
it("renders non-black title text on frame 30", async () => {
const comp = await buildFilm();
const renderer = new HeadlessMediaRenderer();
const png = await comp.renderFrame({ frame: 30, renderer });
expect(png.length).toBeGreaterThan(1000);
// Use skia-canvas or pixel probes to assert color at target coordinates
});
});编写自动化的无头 vitest 测试,断言像素颜色、透明度或布局边界框:
typescript
import { describe, it, expect } from "vitest";
import { HeadlessMediaRenderer } from "gitframes";
import { buildFilm } from "../film.js";
describe("Hero Scene Conformance", () => {
it("renders non-black title text on frame 30", async () => {
const comp = await buildFilm();
const renderer = new HeadlessMediaRenderer();
const png = await comp.renderFrame({ frame: 30, renderer });
expect(png.length).toBeGreaterThan(1000);
// Use skia-canvas or pixel probes to assert color at target coordinates
});
});Step 4.5: Show the User Before Exporting (preview
)
preview步骤 4.5:导出前向用户展示(preview
)
previewWhen a person is in the loop, let them watch the composition before you spend time on the final render. serves a localhost player with the soundtrack, a timeline with the waveform, and frame stepping. The page loads the composition's own code and renders it with WebGPU in the browser, so playback runs at or near full frame rate; the server only serves that code, the project's files, and the soundtrack mixed by the export engine. Point it at the module that builds the composition:
startPreviewtypescript
import { startPreview } from "gitframes";
const session = await startPreview(
{ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },
{ title: "Launch film" },
);
console.log(`Preview at ${session.url}`);
await session.closed; // keep the process alive while the user watches- The export is a function returning the (it may be async), or a
Compositionitself. The browser calls it, so build-time Node code in it must stay simple:Composition,node:pathand synchronousfileURLToPath(import.meta.url)reads (JSON, text) work through shims; writes are skipped. Assets referenced by absolute path load from the server, limited to the project that holds the entry (its nearestfsor.git); passpackage.jsonwith one or more paths for assets kept outside it (e.g. a shared fonts folder).root - Run it in the background. The preview serves until it is stopped. Start the script as a background command with the longest timeout your harness allows, tell the user the URL, and ask for feedback in the chat. Feedback comes from the user, not from the page.
- One tab for the whole loop. The port is fixed per project directory. Running the preview again (after you've made changes) takes over from the one already running, and a tab left open reconnects and reloads into the new version by itself. Don't ask the user to reopen anything. When the user closes the tab, the preview stops by itself about 5 s later (a reload doesn't stop it). If the preview was stopped (tab closed, time limit, or you killed it), just start it again.
- Show it where the user is. only serves the page and returns the URL. If your app has a built-in browser (Claude Code's browser pane, Codex's), open the URL there. Otherwise give the user the link, or pass
startPreviewto open their default browser (skipped when an open tab reconnects on its own). The page needs WebGPU (current Chrome, Edge or Safari).open: true - Audio is the same mix muxes (
renderVideoskips it). Playback uses the soundtrack as its clock; when a frame takes long (a video seek, a heavy shot), the clock and the sound hold until the picture catches up.audio: false
在有人参与审阅的流程中,应先让对方观看合成结果,再投入时间进行最终渲染。 会启动一个 localhost 播放器,其中包含音轨、波形时间线和逐帧步进功能。页面会加载合成内容自身的代码,并在浏览器中使用 WebGPU 渲染,因此播放速度可达到或接近全帧率;服务器只提供该代码、项目文件,以及由导出引擎混合完成的音轨。请将其指向构建合成内容的模块:
startPreviewtypescript
import { startPreview } from "gitframes";
const session = await startPreview(
{ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" },
{ title: "Launch film" },
);
console.log(`Preview at ${session.url}`);
await session.closed; // keep the process alive while the user watches- 是一个返回
export的函数(可以是异步函数),也可以直接是Composition。浏览器会调用它,因此其中的构建时 Node 代码必须保持简单:通过 shim 支持Composition、node:path和同步fileURLToPath(import.meta.url)读取(JSON、文本);写入操作会被跳过。通过绝对路径引用的资源会从服务器加载,范围仅限于入口文件所在的项目(以最近的fs或.git为界);对于存放在该项目之外的资源(例如共享字体文件夹),请通过package.json传入一个或多个路径。root - 在后台运行。 预览服务会持续运行,直到被停止。将脚本作为后台命令启动,并使用执行框架允许的最长超时时间,然后把 URL 告知用户,并请其在聊天中提供反馈。反馈来自用户,而不是页面。
- 整个循环只使用一个标签页。 端口按项目目录固定。再次运行预览(修改内容后)会接管已运行的实例,而保持打开的标签页会自动重连并重新加载到新版本。不要要求用户重新打开任何内容。当用户关闭标签页时,预览会在约 5 秒后自动停止(刷新页面不会使其停止)。如果预览已停止(标签页关闭、达到时限或进程被终止),只需重新启动即可。
- 在用户所在位置展示。 只负责提供页面并返回 URL。如果你的应用内置浏览器(Claude Code 的 browser pane、Codex 的),请在其中打开 URL。否则向用户提供链接,或传入
startPreview以打开默认浏览器(已有标签页自动重连时将跳过此操作)。页面需要 WebGPU(当前版本的 Chrome、Edge 或 Safari)。open: true
音频使用与 混流的相同混音( 会跳过音频)。播放以音轨作为时钟;当某一帧耗时较长(视频跳转或复杂镜头)时,时钟和声音会保持,直到画面跟上。
renderVideoaudio: falseStep 5: Full Video Export (renderVideo
)
renderVideo步骤 5:完整视频导出(renderVideo
)
renderVideoOnce static frames and motion grids are verified, encode the final MP4/WebM video:
typescript
import path from "node:path";
const result = await comp.renderVideo({
outputPath: "output/final-video.mp4",
quality: "high", // "draft" | "medium" | "high" | "lossless"
codec: "avc", // "avc" (H.264) | "hevc" (H.265) | "vp9" | "av1"
audioCodec: "aac",
concurrency: 4, // Parallel frame encoding threads
});
// Always invoke cleanup to release temporary frame cache and file handles
await result.cleanup?.();Pass to have the engine check the output while it renders (see Built-in Video QA below) and return the findings as .
qa: trueresult.qa静态帧和动态帧网格验证完成后,编码最终 MP4/WebM 视频:
typescript
import path from "node:path";
const result = await comp.renderVideo({
outputPath: "output/final-video.mp4",
quality: "high", // "draft" | "medium" | "high" | "lossless"
codec: "avc", // "avc" (H.264) | "hevc" (H.265) | "vp9" | "av1"
audioCodec: "aac",
concurrency: 4, // Parallel frame encoding threads
});
// Always invoke cleanup to release temporary frame cache and file handles
await result.cleanup?.();传入 可让引擎在渲染过程中检查输出(见下文的内置视频 QA),并通过 返回检查结果。
qa: trueresult.qaProduction Render Script Pattern
生产环境渲染脚本模式
Production projects structure their to support both quick frame probes and full exports from the CLI:
src/render.tstypescript
// src/render.ts
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer, startPreview } from "gitframes";
import { buildFilm } from "./film.js";
const OUT = path.resolve(import.meta.dirname, "../output");
const [mode = "video", ...args] = process.argv.slice(2);
if (mode === "preview") {
// The browser runs film.ts itself; no need to build the film here.
const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });
console.log(`Preview at ${session.url}`);
await session.closed; // serves until its tab closes or the next preview replaces it
process.exit(0);
}
const comp = await buildFilm();
await fs.mkdir(OUT, { recursive: true });
if (mode === "frames") {
const renderer = new HeadlessMediaRenderer();
await fs.mkdir(path.join(OUT, "frames"), { recursive: true });
for (const f of args.map(Number)) {
const png = await comp.renderFrame({ frame: f, renderer });
const file = path.join(OUT, "frames", `f${String(f).padStart(4, "0")}.png`);
await fs.writeFile(file, png);
console.log(`Rendered frame: ${file}`);
}
} else {
const started = Date.now();
const result = await comp.renderVideo({
outputPath: path.join(OUT, "output.mp4"),
quality: "high",
});
await result.cleanup?.();
console.log(`Exported video: ${result.filePath} in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}
// CRITICAL: Always explicitly exit process to prevent native WebGPU/mediabunny thread hangs
process.exit(0);CLI Usage:
bash
undefined生产项目会组织 的结构,使其能够通过 CLI 同时支持快速帧探测和完整导出:
src/render.tstypescript
// src/render.ts
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer, startPreview } from "gitframes";
import { buildFilm } from "./film.js";
const OUT = path.resolve(import.meta.dirname, "../output");
const [mode = "video", ...args] = process.argv.slice(2);
if (mode === "preview") {
// The browser runs film.ts itself; no need to build the film here.
const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });
console.log(`Preview at ${session.url}`);
await session.closed; // serves until its tab closes or the next preview replaces it
process.exit(0);
}
const comp = await buildFilm();
await fs.mkdir(OUT, { recursive: true });
if (mode === "frames") {
const renderer = new HeadlessMediaRenderer();
await fs.mkdir(path.join(OUT, "frames"), { recursive: true });
for (const f of args.map(Number)) {
const png = await comp.renderFrame({ frame: f, renderer });
const file = path.join(OUT, "frames", `f${String(f).padStart(4, "0")}.png`);
await fs.writeFile(file, png);
console.log(`Rendered frame: ${file}`);
}
} else {
const started = Date.now();
const result = await comp.renderVideo({
outputPath: path.join(OUT, "output.mp4"),
quality: "high",
});
await result.cleanup?.();
console.log(`Exported video: ${result.filePath} in ${((Date.now() - started) / 1000).toFixed(1)}s`);
}
// CRITICAL: Always explicitly exit process to prevent native WebGPU/mediabunny thread hangs
process.exit(0);CLI 用法:
bash
undefinedRender specific keyframes for instant review:
Render specific keyframes for instant review:
tsx src/render.ts frames 0 18 36 72
tsx src/render.ts frames 0 18 36 72
Live player for the user (run in the background; re-run after changes and the open tab reloads):
Live player for the user (run in the background; re-run after changes and the open tab reloads):
tsx src/render.ts preview
tsx src/render.ts preview
Render full master video:
Render full master video:
tsx src/render.ts
---tsx src/render.ts
---Built-in Video QA (renderVideo({ qa })
)
renderVideo({ qa })内置视频 QA(renderVideo({ qa })
)
renderVideo({ qa })The engine measures the video while it renders: every frame as it is read back for the encoder, and the audio as it is mixed. No second decode pass, and the numbers describe exactly what was encoded.
typescript
import { formatQaReport } from "gitframes";
const result = await comp.renderVideo({
outputPath: "output/final-video.mp4",
qa: { targetLufs: -14 }, // or `qa: true` for defaults (no loudness target)
});
console.log(formatQaReport(result.qa!));
if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loopresult.qa | Severity | Meaning |
|---|---|---|
| error | Fewer frames encoded than the composition has |
| warning | ≥ |
| warning | Samples at or beyond full scale |
| warning | Integrated loudness more than |
| warning | Keyframes past a layer's clip window never play; has |
| info | Picture unchanged for ≥ |
| info | ≥ |
| info | The whole track is silent |
passedqa.audio.integratedLufsqa.audio.peakDbfs引擎在渲染过程中对视频进行测量:每帧在回读至编码器时进行测量,音频则在混合时进行测量。无需第二次解码过程,所得数据准确描述实际编码的内容。
typescript
import { formatQaReport } from "gitframes";
const result = await comp.renderVideo({
outputPath: "output/final-video.mp4",
qa: { targetLufs: -14 }, // or `qa: true` for defaults (no loudness target)
});
console.log(formatQaReport(result.qa!));
if (!result.qa!.passed) process.exitCode = 1; // fail CI / the agent loopresult.qa | 严重程度 | 含义 |
|---|---|---|
| 错误 | 编码的帧数少于合成内容所含的帧数 |
| 警告 | 黑帧持续时间 ≥ |
| 警告 | 采样值达到或超过满量程 |
| 警告 | 整体响度与 |
| 警告 | 位于图层剪辑窗口之后的关键帧从未播放;包含 |
| 信息 | 画面在 ≥ |
| 信息 | 低于 |
| 信息 | 整条音轨均为静音 |
只要出现任何错误或警告, 就会为 false。 是 ITU-R BS.1770 响度(流媒体平台通常会标准化到约 -14 LUFS); 是采样峰值。
passedqa.audio.integratedLufsqa.audio.peakDbfsInspecting a Finished File with ffmpeg
使用 ffmpeg 检查已完成文件
For files rendered elsewhere, or to compare against a previous render, inspect with /:
ffmpegffprobebash
undefined对于在其他地方渲染的文件,或需要与之前的渲染结果进行比较时,可使用 / 检查:
ffmpegffprobebash
undefined1. Check video and audio stream properties
1. Check video and audio stream properties
ffprobe -v error -show_entries stream=codec_name,width,height,duration,r_frame_rate output/final-video.mp4
ffprobe -v error -show_entries stream=codec_name,width,height,duration,r_frame_rate output/final-video.mp4
2. Detect unexpected black frames or dead transitions
2. Detect unexpected black frames or dead transitions
ffmpeg -i output/final-video.mp4 -vf "blackdetect=d=0.5:pix_th=0.10" -f null -
ffmpeg -i output/final-video.mp4 -vf "blackdetect=d=0.5:pix_th=0.10" -f null -
3. Verify audio loudness levels (prevent clipping / silence)
3. Verify audio loudness levels (prevent clipping / silence)
ffmpeg -i output/final-video.mp4 -af "volumedetect" -f null -
---ffmpeg -i output/final-video.mp4 -af "volumedetect" -f null -
---Common Rendering Failure Modes
常见渲染故障模式
| Symptom | Root Cause | Solution |
|---|---|---|
| Entire frame renders black | Missing font registration or | Ensure every font used in |
| Media layer is invisible / blank | Bad input handle binding or zero opacity | Check that the media path exists and |
| Audio cuts out early | Missing | Explicitly assign |
Vision cutout / mask is empty on a single | Vision reads the previous rendered frame; frame 0 has none | Render two or more consecutive frames with the same |
| Ghosted double of the subject in a vision cutout | Frames rendered out of order (frame grid, or jumping from frame 3 to 48), so the cutout uses pixels from another frame | Verify vision layers with sequential frames or the exported MP4, not |
| Script hangs after rendering | Background worker thread pool held open | Always include |
| Text looks misaligned or truncated | Layout node box smaller than text string | Set explicit wrapping |
| Colors appear washed out / dark edges | Unpremultiplied alpha blending | The engine operates with premultiplied alpha. Ensure custom shaders use |
| 症状 | 根本原因 | 解决方案 |
|---|---|---|
| 整帧渲染为黑色 | 未注册字体,或 uniform 中出现 | 确保 |
| 媒体图层不可见或为空白 | 输入句柄绑定错误或 opacity 为零 | 检查媒体路径是否存在,并确认 |
| 音频过早中断 | 音频图层缺少 | 在 |
单次 | Vision 会读取上一个已渲染帧;第 0 帧没有上一帧 | 使用同一个 |
| vision cutout 中的主体出现重影 | 帧未按顺序渲染(使用帧网格,或从第 3 帧直接跳到第 48 帧),导致 cutout 使用了其他帧的像素 | 使用连续帧或导出的 MP4 验证 vision 图层,不要使用 |
| 渲染后脚本挂起 | 后台 worker thread pool 未关闭 | 始终在渲染脚本末尾加入 |
| 文本显示错位或被截断 | 布局节点框小于文本字符串 | 设置明确的换行 |
| 颜色显得褪色或边缘过暗 | 使用了非预乘 alpha 混合 | 引擎采用预乘 alpha。确保自定义 shader 使用 |