gitframes-effects
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
Chinesegitframes-effects
gitframes-effects
Setup
设置
Import everything from the npm package. If the project doesn't have gitframes set up yet, follow the skill first (install, project scaffold, first render).
gitframesgitframes从 npm package 导入所有内容。如果项目尚未配置 gitframes,请先按照 skill 的说明操作(安装、项目脚手架、首次渲染)。
gitframesgitframesOverview
概览
gitframesEffects in gitframes are strongly typed subclasses whose uniform properties can be driven by static values, time-varying keyframes, reactive instances, or audio extraction pipelines.
EffectSignalSee references/effects-catalog.md for the generated prop-level catalog, defaults, and type bounds.
gitframesgitframes 中的效果是强类型 子类,其 uniform 属性可由静态值、随时间变化的关键帧、响应式 实例或音频提取管线驱动。
EffectSignal有关自动生成的属性级目录、默认值和类型边界,请参阅 references/effects-catalog.md。
Attaching Effects
附加效果
There are 4 main attachment modes in the engine:
引擎中主要有 4 种效果附加模式:
1. Layer-Level Effect Pipeline
1. 图层级效果管线
Attach one or more effects to an individual media, box, or text layer:
typescript
import { Layer, Curves, Vignette, Blur } from "gitframes";
const videoLayer = Layer.video("assets/footage.mp4", { fit: "cover" })
.withEffect(new Curves({
master: [{ x: 0, y: 0.05 }, { x: 0.5, y: 0.52 }, { x: 1, y: 0.95 }]
}))
.withEffect(new Vignette({ strength: 0.4, radius: 0.85, softness: 0.6 }))
.withEffect(new Blur({ strength: 8, blurType: "Gaussian" }));将一个或多个效果附加到单个媒体、方框或文本图层:
typescript
import { Layer, Curves, Vignette, Blur } from "gitframes";
const videoLayer = Layer.video("assets/footage.mp4", { fit: "cover" })
.withEffect(new Curves({
master: [{ x: 0, y: 0.05 }, { x: 0.5, y: 0.52 }, { x: 1, y: 0.95 }]
}))
.withEffect(new Vignette({ strength: 0.4, radius: 0.85, softness: 0.6 }))
.withEffect(new Blur({ strength: 8, blurType: "Gaussian" }));2. Whole-Composition Master Emulsion
2. 全合成主胶片效果
Apply an overarching post-processing grade or film emulsion across all rendered layers:
typescript
import { Composition, FilmGrain, ColorBalance } from "gitframes";
const comp = new Composition({ width: 1920, height: 1080, fps: 30 });
// Shared organic 35mm grain applied across titles, background video, and vector graphics
comp.apply(new FilmGrain({
strength: 0.06,
size: 1.5,
monochrome: true,
animated: true,
speed: 1.0,
}));
comp.apply(new ColorBalance({
shadows: { cyanRed: 4, yellowBlue: -6 },
highlights: { cyanRed: 2, yellowBlue: -4 },
preserveLuminosity: true,
}));对所有已渲染图层应用统一的后期调色或胶片效果:
typescript
import { Composition, FilmGrain, ColorBalance } from "gitframes";
const comp = new Composition({ width: 1920, height: 1080, fps: 30 });
// Shared organic 35mm grain applied across titles, background video, and vector graphics
comp.apply(new FilmGrain({
strength: 0.06,
size: 1.5,
monochrome: true,
animated: true,
speed: 1.0,
}));
comp.apply(new ColorBalance({
shadows: { cyanRed: 4, yellowBlue: -6 },
highlights: { cyanRed: 2, yellowBlue: -4 },
preserveLuminosity: true,
}));3. Spatial & Tracked Sections (comp.section
/ Layer.section
)
comp.sectionLayer.section3. 空间区域与跟踪区域(comp.section
/ Layer.section
)
comp.sectionLayer.sectionIsolate a sub-region of a plate or canvas to apply localized effects:
typescript
// Box blur on a tracked object or explicit bounding rectangle
const blurredPlate = comp.section({
source: "assets/interview.mp4",
x: 400,
y: 200,
width: 300,
height: 300,
borderRadius: 150,
effects: [new Blur({ strength: 25, blurType: "Gaussian" })],
});隔离底板或画布中的子区域,以应用局部效果:
typescript
// Box blur on a tracked object or explicit bounding rectangle
const blurredPlate = comp.section({
source: "assets/interview.mp4",
x: 400,
y: 200,
width: 300,
height: 300,
borderRadius: 150,
effects: [new Blur({ strength: 25, blurType: "Gaussian" })],
});4. Fluid Fluent Media Chains (Media
)
Media4. 灵活的媒体处理链(Media
)
MediaProcess clips outside a layout tree before mounting or exporting:
typescript
import { Media, ApplyLUT, Crop } from "gitframes";
const clip = Media.video("assets/raw_log.mp4")
.apply(new ApplyLUT({ lutUrl: "assets/luts/cinematic.cube", intensity: 1.0 }))
.apply(new Crop({ cropType: "rectangle", topPercentage: 10, heightPercentage: 80 }));
const pngBuffer = await clip.renderFrame({ atMs: 1500 });在挂载或导出之前,在布局树之外处理片段:
typescript
import { Media, ApplyLUT, Crop } from "gitframes";
const clip = Media.video("assets/raw_log.mp4")
.apply(new ApplyLUT({ lutUrl: "assets/luts/cinematic.cube", intensity: 1.0 }))
.apply(new Crop({ cropType: "rectangle", topPercentage: 10, heightPercentage: 80 }));
const pngBuffer = await clip.renderFrame({ atMs: 1500 });Effect Categories
效果类别
1. Color Grading & Tonal Dynamics
1. 调色与色调动态
- : Spline-interpolated tone curves for master, red, green, blue, plus hue/saturation curves (
Curves,hueVsHue,hueVsSat,lumVsSat).satVsSat - : Shadows, midtones, and highlights split color adjustments with
ColorBalance.preserveLuminosity - : Master and per-channel input/output black/white points and gamma.
Levels - : CMYK-style photographic selective adjustments targeting reds, yellows, greens, cyans, blues, magentas, whites, neutrals, and blacks.
SelectiveColor - : Multi-stop gradient remapping (
GradientMap).stops: [{ position: 0, color: "#000" }, { position: 1, color: "#FFF" }] - : Separate shadow lifting and highlight recovery with radius and tonal width controls.
ShadowsHighlights - : 1D and 3D
ApplyLUTcolor lookup table applicator..cube - : Real-time HSL/contrast controls:
Modulate,hue,brightness,contrast,exposure,saturation.sepia
- :用于主通道以及红、绿、蓝通道的样条插值色调曲线,并支持色相/饱和度曲线(
Curves、hueVsHue、hueVsSat、lumVsSat)。satVsSat - :分别调整阴影、中间调和高光,支持
ColorBalance。preserveLuminosity - :控制主通道及各通道输入/输出的黑白点和 gamma。
Levels - :以 CMYK 风格进行摄影选择性调整,可针对红、黄、绿、青、蓝、洋红、白、中性和黑色。
SelectiveColor - :多色标渐变重映射(
GradientMap)。stops: [{ position: 0, color: "#000" }, { position: 1, color: "#FFF" }] - :分别提升阴影和恢复高光,并提供半径与色调宽度控制。
ShadowsHighlights - :应用 1D 和 3D
ApplyLUT色彩查找表。.cube - :实时 HSL/对比度控制:
Modulate、hue、brightness、contrast、exposure、saturation。sepia
2. Optical, Texture & Cinematic Emulsions
2. 光学、纹理与电影胶片效果
- : GPU-synthesized film grain with grain sizing, shadow/highlight masking, and temporal animation speed.
FilmGrain - : Elliptical or circular lens falloff with customizable center, roundness, and softness.
Vignette - : High-performance multi-mode blur:
Blur,"Gaussian","Box","Median","Motion","Bilateral","Edge-preserving","Radial". Supports partial blur regions and tracked object bounding boxes."Zoom" - : High-frequency edge sharpener with threshold suppression.
UnsharpMask - : Edge and frequency isolation with contrast boost.
HighPass - : Procedural print screening with
HalftoneScreen,"Circle","Diamond", or"Line"dots in"Square"or"Monochrome"angles."CMYK" - : Seamless UV wrapping with
TileOffset,"wrap","clamp", or"mirror"boundaries."transparent" - : Directional shutter simulation with custom shutter angles and velocity clamps.
MotionBlur - : Multi-frame optical flow blending to eliminate flickering in generative AI clips or high-speed footage.
TemporalDeflicker
- :由 GPU 生成的胶片颗粒,支持颗粒尺寸、阴影/高光遮罩以及时间动画速度。
FilmGrain - :椭圆或圆形镜头衰减,可自定义中心、圆度和柔和度。
Vignette - :高性能多模式模糊:
Blur、"Gaussian"、"Box"、"Median"、"Motion"、"Bilateral"、"Edge-preserving"、"Radial"。支持局部模糊区域和跟踪对象边界框。"Zoom" - :高频边缘锐化器,并支持阈值抑制。
UnsharpMask - :隔离边缘和频率,同时增强对比度。
HighPass - :程序化印刷网点效果,可使用
HalftoneScreen、"Circle"、"Diamond"或"Line"圆点,以及"Square"或"Monochrome"角度。"CMYK" - :无缝 UV 环绕,支持
TileOffset、"wrap"、"clamp"或"mirror"边界。"transparent" - :快门方向模拟,支持自定义快门角度和速度限制。
MotionBlur - :通过多帧光流混合消除生成式 AI 片段或高速素材中的闪烁。
TemporalDeflicker
3. 3D Spatial Lighting & Materials
3. 3D 空间光照与材质
- : Screen-space normal map relighting with point, spot, and directional lights, specular highlights, roughness, and metallic uniforms.
Relight3D - : Screen-space ambient occlusion generating contact shadows from depth buffers.
SSAO - : Physically-based transmission, index of refraction (IOR), optical dispersion, and Fresnel reflections.
PBRGlass - : Camera lens simulation with aperture size, focal length, focus plane distance, and circle-of-confusion (CoC) blur.
DepthOfField
- :屏幕空间法线贴图重新照明,支持点光源、聚光灯和平行光,以及高光、粗糙度和金属度 uniform。
Relight3D - :屏幕空间环境光遮蔽,可从深度缓冲生成接触阴影。
SSAO - :基于物理的透射、折射率(IOR)、光学色散和 Fresnel 反射。
PBRGlass - :相机镜头模拟,支持光圈大小、焦距、焦平面距离和弥散圆(CoC)模糊。
DepthOfField
4. Matte, Keying & Edge Operations
4. 抠像、抠色与边缘处理
- : Studio greenscreen/bluescreen keyer with spill suppression and smoothness controls.
ColorKey - : Rectangular, circular, or arbitrary polygon path cropping with corner rounding.
Crop
- :演播室绿幕/蓝幕抠色器,支持溢色抑制和平滑度控制。
ColorKey - :使用矩形、圆形或任意多边形路径裁切,并可设置圆角。
Crop
On-Device Vision
设备端视觉
gitframes$GITFRAMES_MODELS_DIR~/.cache/gitframes/models| Option | Model | Gives you |
|---|---|---|
| RTMDet-Ins | COCO-80 boxes, tracked over time → |
| RTMDet-Ins (same pass) | Soft per-instance masks → |
| RTMO | 17 COCO keypoints per person → |
| Selfie Segmenter | Fast person alpha → |
variant: "t" | "s" | "m""s"confidenceclasses: ["person", ...]typescript
// Whole composition: returns the reactive bundle
const vision = comp.withVision({ enableSegmentation: true, enablePose: true });
// One layer: renders through a node mode
Layer.video("assets/dancer.mp4").withVision({ mode: "matte", enableSegmentation: true });Node modes: , , , , , , . For / / , (default; any COCO class, overlapping parts such as a dress merged into the subject) or (people only, fastest).
passthroughmaskmattecropskeletonboxestrackingmaskmattecropmatteSource: "instance""selfie"gitframes$GITFRAMES_MODELS_DIR~/.cache/gitframes/models| 选项 | 模型 | 提供内容 |
|---|---|---|
| RTMDet-Ins | COCO-80 边界框,并随时间跟踪 → |
| RTMDet-Ins(同一推理过程) | 软性逐实例掩码 → |
| RTMO | 每个人 17 个 COCO 关键点 → |
| Selfie Segmenter | 快速人物 alpha → |
variant: "t" | "s" | "m""s"confidenceclasses: ["person", ...]typescript
// Whole composition: returns the reactive bundle
const vision = comp.withVision({ enableSegmentation: true, enablePose: true });
// One layer: renders through a node mode
Layer.video("assets/dancer.mp4").withVision({ mode: "matte", enableSegmentation: true });节点模式包括:、、、、、、。对于 / / ,可使用 (默认值;适用于任意 COCO 类别,与主体重叠的部件(例如连衣裙)会并入主体)或 (仅限人物,速度最快)。
passthroughmaskmattecropskeletonboxestrackingmaskmattecropmatteSource: "instance""selfie"Choosing settings
设置选择
Measured per 1280–2048 px frame on CPU ():
onnxruntime-node| Task | | | |
|---|---|---|---|
| detect / segment (one shared pass) | 150–300 ms | 200–340 ms | 420–700 ms |
| pose | ~50 ms | ~120 ms | ~280 ms |
| matte (Selfie Segmenter) | ~20 ms | ~20 ms | ~20 ms |
- Start with . Use
"s"for hero shots with fast motion blur or busy backgrounds;"m"mainly saves time on pose."t" - Enabling both and
enableDetectioncosts one inference, not two.enableSegmentation - Selfie matte only for close framing. It is tuned for a person filling much of the frame: it misses distant figures and can report a "person" on close-ups with nobody in them. Keep for anything else.
matteSource: "instance" - Raise (e.g.
confidence) on abstract or stylized footage — at the default0.5the model will put loose labels ("teddy bear", "donut") on smoke, eyes and planets.0.3 - Restrict when you only care about one thing; it also stops the subject from switching to another object when the person leaves frame.
classes
以下数据是在 CPU()上按 1280–2048 px 帧测得:
onnxruntime-node| 任务 | | | |
|---|---|---|---|
| 检测/分割(共享一次推理) | 150–300 ms | 200–340 ms | 420–700 ms |
| 姿态 | ~50 ms | ~120 ms | ~280 ms |
| 抠像(Selfie Segmenter) | ~20 ms | ~20 ms | ~20 ms |
- 先使用 。对于包含快速运动、运动模糊或繁忙背景的重点镜头,使用
"s";"m"主要用于缩短姿态处理时间。"t" - 同时启用 和
enableDetection只会执行一次推理,而不是两次。enableSegmentation - 仅在近距离取景时使用 Selfie 抠像。 它针对占据画面大部分的人物进行了调校:会漏掉远处人物,也可能在无人出现的特写镜头中报告“person”。其他情况请保持 。
matteSource: "instance" - 对抽象或风格化素材提高 (例如
confidence)——在默认0.5下,模型可能会给烟雾、眼睛和行星随意标注类别(“teddy bear”“donut”)。0.3 - 如果只关心一个目标,请限制 ;这样也能防止人物离开画面后主体切换到其他对象。
classes
How it behaves at render time
渲染时的行为
- One-frame delay. Vision reads the layer's previous rendered frame, so the very first frame has no results (a cutout renders transparent) and masks trail the plate by one frame. This is invisible at normal playback speed.
- Render vision frames in order. does this for you. For stills, render at least two consecutive frames with the same renderer and keep the last one. Jumping straight to a later frame (or using a frame grid) cuts out pixels from whatever frame was rendered before it, which shows up as a ghosted double of the subject.
renderVideo - Subject choice. /
mask/matteuse the largest person; with no person, the most confident instance. Instances overlapping the subject and no more than twice its size are merged in (a dress, a held instrument) — large containers around it (a tunnel, a window frame) are not.crop - Each vision layer tracks objects independently; track ids from two layers are unrelated.
- 延迟一帧。 视觉模型会读取图层上一个已渲染帧,因此第一帧没有结果(抠图会渲染为透明),掩码也会比底板晚一帧。在正常播放速度下,这一延迟不可见。
- 按顺序渲染视觉帧。 会自动完成。对于静态图,应使用同一个 renderer 至少连续渲染两帧,并保留最后一帧。直接跳转到后续帧(或使用帧网格),会切掉此前渲染帧中的像素,从而表现为主体出现重影。
renderVideo - 主体选择。 /
mask/matte使用画面中最大的人物;如果没有人物,则使用置信度最高的实例。与主体重叠且尺寸不超过主体两倍的实例会被合并进来(例如连衣裙、手持乐器),但主体周围的大型容器(隧道、窗框)不会被合并。crop - 每个视觉图层都会独立跟踪对象;两个图层中的 track id 互不相关。
Troubleshooting
故障排除
| Symptom | Fix |
|---|---|
| Part of a fast-moving garment drops out of the cutout | |
| Cutout is the wrong object | Set |
| Faint halo around the cutout on dark backgrounds | Lower |
| Renders offline / in CI | Pre-download with |
| Check the models themselves | |
| 症状 | 修复方案 |
|---|---|
| 快速移动的服装有一部分从抠图中丢失 | 使用 |
| 抠出了错误对象 | 设置 |
| 深色背景上抠图周围有淡淡光晕 | 降低 |
| 在离线环境或 CI 中渲染 | 使用 |
| 检查模型本身 | 在全新的模型目录中执行 |
1. Subject Sandwich ("Text Behind Subject")
1. 主体夹层(“文字置于主体后方”)
Cuts out the foreground subject from footage and sandwiches typography or graphics directly behind them:
typescript
comp.addSubjectSandwich({
source: "assets/dancer.mp4",
behind: [
Layer.text("HEADLINE BEHIND", {
fontSize: 120,
fill: "#FF5A1F",
fontWeight: 900
})
],
feather: 4,
fit: "cover"
});从素材中抠出前景主体,并将文字或图形直接置于其后方:
typescript
comp.addSubjectSandwich({
source: "assets/dancer.mp4",
behind: [
Layer.text("HEADLINE BEHIND", {
fontSize: 120,
fill: "#FF5A1F",
fontWeight: 900
})
],
feather: 4,
fit: "cover"
});2. Smart Re-Framing (16:9 to 9:16 Auto-Crop)
2. 智能重构图(16:9 转 9:16 自动裁剪)
Smoothly reframes landscape video into vertical shorts by following a tracked subject with virtual camera damping:
typescript
const vision = comp.withVision({ enableDetection: true });
comp.addSmartFraming({
source: "assets/action.mp4",
target: vision.objects.primary,
targetAspect: 9 / 16,
damping: 0.15,
leadHeadroom: 0.1
});通过虚拟摄像机阻尼跟随被跟踪主体,将横屏视频平滑重构图为竖屏短视频:
typescript
const vision = comp.withVision({ enableDetection: true });
comp.addSmartFraming({
source: "assets/action.mp4",
target: vision.objects.primary,
targetAspect: 9 / 16,
damping: 0.15,
leadHeadroom: 0.1
});3. Subject Outline Glow & Neon Pulse
3. 主体轮廓辉光与霓虹脉冲
Strokes the segmented subject boundary with an audio-reactive contour glow:
typescript
const vision = comp.withVision({ enableSegmentation: true });
comp.addSubjectOutline(vision.segmentation.subject, {
source: "assets/character.mp4",
color: "#FF5A1F",
width: 6,
blur: 12
});使用音频响应式轮廓辉光描边分割后的主体边界:
typescript
const vision = comp.withVision({ enableSegmentation: true });
comp.addSubjectOutline(vision.segmentation.subject, {
source: "assets/character.mp4",
color: "#FF5A1F",
width: 6,
blur: 12
});4. Tracked Region Blur
4. 跟踪区域模糊
Blurs faces, license plates, or any detected class by following a live track:
typescript
const vision = comp.withVision({ classes: ["person"] });
layer.blurRegion(vision.objects.byCategory("person"), { strength: 30 });通过跟随实时跟踪结果,对人脸、车牌或任意检测类别进行模糊处理:
typescript
const vision = comp.withVision({ classes: ["person"] });
layer.blurRegion(vision.objects.byCategory("person"), { strength: 30 });5. Analyze Before Authoring
5. 创作前分析
One-shot, ffmpeg-free report of what is in a clip (tracks, classes, mask coverage):
typescript
const report = await comp.analyzeVisionSequence("assets/street.mp4", {
tasks: ["detect", "pose"],
categories: ["person"]
});
report.tracks; // [{ trackId, category, frames: [start, end], centerPath, ... }]无需 ffmpeg 的一次性报告,用于分析片段中的内容(跟踪、类别、掩码覆盖率):
typescript
const report = await comp.analyzeVisionSequence("assets/street.mp4", {
tasks: ["detect", "pose"],
categories: ["person"]
});
report.tracks; // [{ trackId, category, frames: [start, end], centerPath, ... }]Reactive Signal Modulation
响应式 Signal 调制
Uniforms on any accept reactive instances instead of static values. When the signal changes, the GPU uniform buffer updates automatically without rebuilding the pipeline:
EffectSignaltypescript
import { Layer, Blur, sineSignal, computed } from "gitframes";
// Oscillating breath blur
const blurSignal = sineSignal({
frequencyHz: 0.5,
min: 0,
max: 20
});
const layer = Layer.image("assets/bg.jpg")
.withEffect(new Blur({
strength: blurSignal,
blurType: "Gaussian"
}));任何 上的 uniform 都可以接受响应式 实例,而非静态值。信号发生变化时,GPU uniform buffer 会自动更新,无需重新构建管线:
EffectSignaltypescript
import { Layer, Blur, sineSignal, computed } from "gitframes";
// Oscillating breath blur
const blurSignal = sineSignal({
frequencyHz: 0.5,
min: 0,
max: 20
});
const layer = Layer.image("assets/bg.jpg")
.withEffect(new Blur({
strength: blurSignal,
blurType: "Gaussian"
}));Best Practices:
最佳实践:
- Emulsion Restraint: Keep (
FilmGrain) and0.04–0.08(Vignette) subtle. Over-filtering looks amateurish.0.2–0.45 - Order Matters: Place tonal grading (,
Curves) before lens optics (ColorBalance,Blur), and apply grain (DepthOfField) as the outermost layer.FilmGrain - No Redundant Shaders: For static image exports, set on
animated: falseto preserve deterministic static renders.FilmGrain
- 胶片效果克制: 将 (
FilmGrain)和0.04–0.08(Vignette)保持细微。过度滤镜化会显得业余。0.2–0.45 - 顺序很重要: 将色调分级(、
Curves)放在镜头光学效果(ColorBalance、Blur)之前,并将颗粒(DepthOfField)作为最外层应用。FilmGrain - 不使用冗余 shader: 导出静态图像时,将 的
FilmGrain设为animated,以保留确定性的静态渲染结果。false