framefields-effects
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
Chineseframefields-effects
framefields-effects
Setup
安装配置
Import everything from the npm package. If the project doesn't have framefields set up yet, follow the skill first (install, project scaffold, first render).
framefieldsframefields从 npm 包导入所有内容。如果项目尚未配置 framefields,请先按照 skill 操作(安装、项目脚手架、首次渲染)。
framefieldsframefieldsOverview
概览
framefieldsEffects in framefields are strongly typed subclasses whose uniform properties can be driven by static values, time-varying keyframes, reactive instances, or audio extraction pipelines.
EffectSignalSee references/effects-catalog.md for the generated prop-level catalog, defaults, and type bounds.
framefieldsframefields 中的效果是强类型 子类,其 uniform 属性可由静态值、随时间变化的关键帧、响应式 实例或音频提取管线驱动。
EffectSignal请参阅 references/effects-catalog.md,了解生成的属性级目录、默认值和类型边界。
Attaching Effects
挂载效果
There are 4 main attachment modes in the engine:
引擎中主要有 4 种挂载模式:
1. Layer-Level Effect Pipeline
1. 图层级效果管线
Attach one or more effects to an individual media, box, or text layer:
typescript
import { Layer, Curves, Vignette, Blur } from "framefields";
const videoLayer = Layer.video("assets/footage.mp4", { fit: "cover" })
.withEffect(new Curves({
master: [{ x: 0, y: 0.05 }, { x: 0.5, y: 0.52 }, { x: 1, y: 0.95 }]
}))
.withEffect(new Vignette({ strength: 0.4, radius: 0.85, softness: 0.6 }))
.withEffect(new Blur({ strength: 8, blurType: "Gaussian" }));将一个或多个效果挂载到单个媒体、框体或文本图层:
typescript
import { Layer, Curves, Vignette, Blur } from "framefields";
const videoLayer = Layer.video("assets/footage.mp4", { fit: "cover" })
.withEffect(new Curves({
master: [{ x: 0, y: 0.05 }, { x: 0.5, y: 0.52 }, { x: 1, y: 0.95 }]
}))
.withEffect(new Vignette({ strength: 0.4, radius: 0.85, softness: 0.6 }))
.withEffect(new Blur({ strength: 8, blurType: "Gaussian" }));2. Whole-Composition Master Emulsion
2. 整体合成级胶片效果
Apply an overarching post-processing grade or film emulsion across all rendered layers:
typescript
import { Composition, FilmGrain, ColorBalance } from "framefields";
const comp = new Composition({ width: 1920, height: 1080, fps: 30 });
// Shared organic 35mm grain applied across titles, background video, and vector graphics
comp.apply(new FilmGrain({
strength: 0.06,
size: 1.5,
monochrome: true,
animated: true,
speed: 1.0,
}));
comp.apply(new ColorBalance({
shadows: { cyanRed: 4, yellowBlue: -6 },
highlights: { cyanRed: 2, yellowBlue: -4 },
preserveLuminosity: true,
}));为所有已渲染图层应用统一的后期分级或胶片效果:
typescript
import { Composition, FilmGrain, ColorBalance } from "framefields";
const comp = new Composition({ width: 1920, height: 1080, fps: 30 });
// Shared organic 35mm grain applied across titles, background video, and vector graphics
comp.apply(new FilmGrain({
strength: 0.06,
size: 1.5,
monochrome: true,
animated: true,
speed: 1.0,
}));
comp.apply(new ColorBalance({
shadows: { cyanRed: 4, yellowBlue: -6 },
highlights: { cyanRed: 2, yellowBlue: -4 },
preserveLuminosity: true,
}));3. Spatial & Tracked Sections (comp.section
/ Layer.section
)
comp.sectionLayer.section3. 空间与跟踪区域(comp.section
/ Layer.section
)
comp.sectionLayer.sectionIsolate a sub-region of a plate or canvas to apply localized effects:
typescript
// Box blur on a tracked object or explicit bounding rectangle
const blurredPlate = comp.section({
source: "assets/interview.mp4",
x: 400,
y: 200,
width: 300,
height: 300,
borderRadius: 150,
effects: [new Blur({ strength: 25, blurType: "Gaussian" })],
});将底板或画布的某个子区域隔离出来,以应用局部效果:
typescript
// Box blur on a tracked object or explicit bounding rectangle
const blurredPlate = comp.section({
source: "assets/interview.mp4",
x: 400,
y: 200,
width: 300,
height: 300,
borderRadius: 150,
effects: [new Blur({ strength: 25, blurType: "Gaussian" })],
});4. Fluid Fluent Media Chains (Media
)
Media4. 灵活的流媒体链(Media
)
MediaProcess clips outside a layout tree before mounting or exporting:
typescript
import { Media, ApplyLUT, Crop } from "framefields";
const clip = Media.video("assets/raw_log.mp4")
.apply(new ApplyLUT({ lutUrl: "assets/luts/cinematic.cube", intensity: 1.0 }))
.apply(new Crop({ cropType: "rectangle", topPercentage: 10, heightPercentage: 80 }));
const pngBuffer = await clip.renderFrame({ atMs: 1500 });在挂载或导出之前,在布局树之外处理片段:
typescript
import { Media, ApplyLUT, Crop } from "framefields";
const clip = Media.video("assets/raw_log.mp4")
.apply(new ApplyLUT({ lutUrl: "assets/luts/cinematic.cube", intensity: 1.0 }))
.apply(new Crop({ cropType: "rectangle", topPercentage: 10, heightPercentage: 80 }));
const pngBuffer = await clip.renderFrame({ atMs: 1500 });Effect Categories
效果类别
1. Color Grading & Tonal Dynamics
1. 色彩分级与色调动态
- : Spline-interpolated tone curves for master, red, green, blue, plus hue/saturation curves (
Curves,hueVsHue,hueVsSat,lumVsSat).satVsSat - : Shadows, midtones, and highlights split color adjustments with
ColorBalance.preserveLuminosity - : Master and per-channel input/output black/white points and gamma.
Levels - : CMYK-style photographic selective adjustments targeting reds, yellows, greens, cyans, blues, magentas, whites, neutrals, and blacks.
SelectiveColor - : Multi-stop gradient remapping (
GradientMap).stops: [{ position: 0, color: "#000" }, { position: 1, color: "#FFF" }] - : Separate shadow lifting and highlight recovery with radius and tonal width controls.
ShadowsHighlights - : 1D and 3D
ApplyLUTcolor lookup table applicator..cube - : Real-time HSL/contrast controls:
Modulate,hue,brightness,contrast,exposure,saturation.sepia
- :用于主通道以及红、绿、蓝通道的样条插值色调曲线,并支持色相/饱和度曲线(
Curves、hueVsHue、hueVsSat、lumVsSat)。satVsSat - :使用
ColorBalance分别调整阴影、中间调和高光的颜色。preserveLuminosity - :控制主通道及各通道输入/输出的黑白点和 gamma。
Levels - :CMYK 风格的摄影选择性调整,可针对红色、黄色、绿色、青色、蓝色、洋红色、白色、中性色和黑色进行调整。
SelectiveColor - :多色标渐变重映射(
GradientMap)。stops: [{ position: 0, color: "#000" }, { position: 1, color: "#FFF" }] - :分别提亮阴影和恢复高光,支持半径和色调宽度控制。
ShadowsHighlights - :1D 和 3D
ApplyLUT色彩查找表应用器。.cube - :实时 HSL/对比度控制,包括
Modulate、hue、brightness、contrast、exposure和saturation。sepia
2. Optical, Texture & Cinematic Emulsions
2. 光学、纹理与电影胶片效果
- : GPU-synthesized film grain with grain sizing, shadow/highlight masking, and temporal animation speed.
FilmGrain - : Elliptical or circular lens falloff with customizable center, roundness, and softness.
Vignette - : High-performance multi-mode blur:
Blur,"Gaussian","Box","Median","Motion","Bilateral","Edge-preserving","Radial". Supports partial blur regions and tracked object bounding boxes."Zoom" - : High-frequency edge sharpener with threshold suppression.
UnsharpMask - : Edge and frequency isolation with contrast boost.
HighPass - : Procedural print screening with
HalftoneScreen,"Circle","Diamond", or"Line"dots in"Square"or"Monochrome"angles."CMYK" - : Seamless UV wrapping with
TileOffset,"wrap","clamp", or"mirror"boundaries."transparent" - : Directional shutter simulation with custom shutter angles and velocity clamps.
MotionBlur - : Multi-frame optical flow blending to eliminate flickering in generative AI clips or high-speed footage.
TemporalDeflicker
- :GPU 合成胶片颗粒,支持颗粒尺寸、阴影/高光遮罩和时间动画速度。
FilmGrain - :椭圆形或圆形镜头衰减,可自定义中心、圆度和柔和度。
Vignette - :高性能多模式模糊:
Blur、"Gaussian"、"Box"、"Median"、"Motion"、"Bilateral"、"Edge-preserving"、"Radial"。支持局部模糊区域和跟踪对象边界框。"Zoom" - :带阈值抑制的高频边缘锐化器。
UnsharpMask - :分离边缘和频率,并增强对比度。
HighPass - :程序化印刷网屏,支持
HalftoneScreen、"Circle"、"Diamond"或"Line"网点,以及"Square"或"Monochrome"角度。"CMYK" - :无缝 UV 环绕,支持
TileOffset、"wrap"、"clamp"或"mirror"边界。"transparent" - :带自定义快门角度和速度限制的方向快门模拟。
MotionBlur - :多帧光流混合,用于消除生成式 AI 片段或高速素材中的闪烁。
TemporalDeflicker
3. 3D Spatial Lighting & Materials
3. 3D 空间光照与材质
- : Screen-space normal map relighting with point, spot, and directional lights, specular highlights, roughness, and metallic uniforms.
Relight3D - : Screen-space ambient occlusion generating contact shadows from depth buffers.
SSAO - : Physically-based transmission, index of refraction (IOR), optical dispersion, and Fresnel reflections.
PBRGlass - : Camera lens simulation with aperture size, focal length, focus plane distance, and circle-of-confusion (CoC) blur.
DepthOfField
- :屏幕空间法线贴图重新照明,支持点光源、聚光灯和平行光,以及镜面高光、粗糙度和金属度 uniform。
Relight3D - :屏幕空间环境光遮蔽,利用深度缓冲区生成接触阴影。
SSAO - :基于物理的透射、折射率(IOR)、光学色散和 Fresnel 反射。
PBRGlass - :相机镜头模拟,支持光圈大小、焦距、焦点平面距离和弥散圆(CoC)模糊。
DepthOfField
4. Matte, Keying & Edge Operations
4. 抠像、键控与边缘操作
- : Studio greenscreen/bluescreen keyer with spill suppression and smoothness controls.
ColorKey - : Rectangular, circular, or arbitrary polygon path cropping with corner rounding.
Crop
- :演播室绿幕/蓝幕键控器,支持溢色抑制和平滑度控制。
ColorKey - :矩形、圆形或任意多边形路径裁剪,并支持圆角。
Crop
On-Device Vision
端侧视觉
framefields$FRAMEFIELDS_MODELS_DIR~/.cache/framefields/models| Option | Model | Gives you |
|---|---|---|
| RTMDet-Ins | COCO-80 boxes, tracked over time → |
| RTMDet-Ins (same pass) | Soft per-instance masks → |
| RTMO | 17 COCO keypoints per person → |
| Selfie Segmenter | Fast person alpha → |
variant: "t" | "s" | "m""s"confidenceclasses: ["person", ...]typescript
// Whole composition: returns the reactive bundle
const vision = comp.withVision({ enableSegmentation: true, enablePose: true });
// One layer: renders through a node mode
Layer.video("assets/dancer.mp4").withVision({ mode: "matte", enableSegmentation: true });Node modes: , , , , , , . For / / , (default; any COCO class, overlapping parts such as a dress merged into the subject) or (people only, fastest).
passthroughmaskmattecropskeletonboxestrackingmaskmattecropmatteSource: "instance""selfie"framefields$FRAMEFIELDS_MODELS_DIR~/.cache/framefields/models| 选项 | 模型 | 输出 |
|---|---|---|
| RTMDet-Ins | COCO-80 边界框,并随时间跟踪 → |
| RTMDet-Ins(同一轮推理) | 软性实例遮罩 → |
| RTMO | 每个人的 17 个 COCO 关键点 → |
| Selfie Segmenter | 快速人物 alpha → |
variant: "t" | "s" | "m""s"confidenceclasses: ["person", ...]typescript
// Whole composition: returns the reactive bundle
const vision = comp.withVision({ enableSegmentation: true, enablePose: true });
// One layer: renders through a node mode
Layer.video("assets/dancer.mp4").withVision({ mode: "matte", enableSegmentation: true });节点模式:、、、、、、。对于 / / ,可使用 (默认值;适用于任意 COCO 类别,与主体重叠的部位(例如连衣裙)会合并到主体中)或 (仅限人物,速度最快)。
passthroughmaskmattecropskeletonboxestrackingmaskmattecropmatteSource: "instance""selfie"Choosing settings
设置选择
Measured per 1280–2048 px frame on CPU ():
onnxruntime-node| Task | | | |
|---|---|---|---|
| detect / segment (one shared pass) | 150–300 ms | 200–340 ms | 420–700 ms |
| pose | ~50 ms | ~120 ms | ~280 ms |
| matte (Selfie Segmenter) | ~20 ms | ~20 ms | ~20 ms |
- Start with . Use
"s"for hero shots with fast motion blur or busy backgrounds;"m"mainly saves time on pose."t" - Enabling both and
enableDetectioncosts one inference, not two.enableSegmentation - Selfie matte only for close framing. It is tuned for a person filling much of the frame: it misses distant figures and can report a "person" on close-ups with nobody in them. Keep for anything else.
matteSource: "instance" - Raise (e.g.
confidence) on abstract or stylized footage — at the default0.5the model will put loose labels ("teddy bear", "donut") on smoke, eyes and planets.0.3 - Restrict when you only care about one thing; it also stops the subject from switching to another object when the person leaves frame.
classes
在 CPU()上针对 1280–2048 px 帧测得:
onnxruntime-node| 任务 | | | |
|---|---|---|---|
| 检测 / 分割(共享一次推理) | 150–300 ms | 200–340 ms | 420–700 ms |
| 姿态 | ~50 ms | ~120 ms | ~280 ms |
| 抠像(Selfie Segmenter) | ~20 ms | ~20 ms | ~20 ms |
- 首先使用 。对于带有快速运动模糊或复杂背景的重点镜头,使用
"s";"m"主要用于缩短姿态推理时间。"t" - 同时启用 和
enableDetection只会进行一次推理,而不是两次。enableSegmentation - Selfie 抠像仅适用于近景。 它针对占据画面大部分区域的人物进行了调校:会漏掉远处人物,也可能在画面中无人时仍报告 。其他情况应保持使用
"person"。matteSource: "instance" - 对于抽象或风格化素材,应提高 (例如
confidence);在默认值为0.5时,模型可能会将烟雾、眼睛和行星错误地标记为宽泛类别("teddy bear"、"donut")。0.3 - 如果只关心一个目标,应限制 ;这样也能防止人物离开画面后,主体切换到其他对象。
classes
How it behaves at render time
渲染时的行为
- One-frame delay. Vision reads the layer's previous rendered frame, so the very first frame has no results (a cutout renders transparent) and masks trail the plate by one frame. This is invisible at normal playback speed.
- Render vision frames in order. does this for you. For stills, render at least two consecutive frames with the same renderer and keep the last one. Jumping straight to a later frame (or using a frame grid) cuts out pixels from whatever frame was rendered before it, which shows up as a ghosted double of the subject.
renderVideo - Subject choice. /
mask/matteuse the largest person; with no person, the most confident instance. Instances overlapping the subject and no more than twice its size are merged in (a dress, a held instrument) — large containers around it (a tunnel, a window frame) are not.crop - Each vision layer tracks objects independently; track ids from two layers are unrelated.
- 一帧延迟。 视觉模块读取图层上一张已渲染帧,因此第一帧没有结果(抠像会渲染为透明),遮罩会比底板晚一帧。在正常播放速度下不可见。
- 按顺序渲染视觉帧。 会自动处理。对于静态图像,使用同一个 renderer 至少连续渲染两帧,并保留最后一帧。直接跳转到后续帧(或使用帧网格)会抠除上一个已渲染帧中的像素,表现为主体的重影双像。
renderVideo - 主体选择。 /
mask/matte使用画面中最大的人物;如果没有人物,则使用置信度最高的实例。与主体重叠且尺寸不超过其两倍的实例会被合并进来(连衣裙、手持乐器等);大型容器(隧道、窗框等)不会。crop - 每个视觉图层都会独立跟踪对象;两个图层中的 track id 互不相关。
Troubleshooting
故障排查
| Symptom | Fix |
|---|---|
| Part of a fast-moving garment drops out of the cutout | |
| Cutout is the wrong object | Set |
| Faint halo around the cutout on dark backgrounds | Lower |
| Renders offline / in CI | Pre-download with |
| Check the models themselves | |
| 症状 | 修复方案 |
|---|---|
| 快速移动服装的一部分从抠像结果中消失 | 使用 |
| 抠出了错误的对象 | 设置 |
| 深色背景上的抠像边缘出现淡淡光晕 | 降低 |
| 在离线环境或 CI 中无法渲染 | 使用 |
| 检查模型本身 | 在全新的模型目录中执行 |
1. Subject Sandwich ("Text Behind Subject")
1. 主体夹层(“文字置于主体后方”)
Cuts out the foreground subject from footage and sandwiches typography or graphics directly behind them:
typescript
comp.addSubjectSandwich({
source: "assets/dancer.mp4",
behind: [
Layer.text("HEADLINE BEHIND", {
fontSize: 120,
fill: "#FF5A1F",
fontWeight: 900
})
],
feather: 4,
fit: "cover"
});从素材中抠出前景主体,并将文字或图形直接置于其后方:
typescript
comp.addSubjectSandwich({
source: "assets/dancer.mp4",
behind: [
Layer.text("HEADLINE BEHIND", {
fontSize: 120,
fill: "#FF5A1F",
fontWeight: 900
})
],
feather: 4,
fit: "cover"
});2. Smart Re-Framing (16:9 to 9:16 Auto-Crop)
2. 智能重构图(16:9 转 9:16 自动裁剪)
Smoothly reframes landscape video into vertical shorts by following a tracked subject with virtual camera damping:
typescript
const vision = comp.withVision({ enableDetection: true });
comp.addSmartFraming({
source: "assets/action.mp4",
target: vision.objects.primary,
targetAspect: 9 / 16,
damping: 0.15,
leadHeadroom: 0.1
});通过带虚拟相机阻尼的跟踪主体,平滑地将横屏视频重构为竖屏短视频:
typescript
const vision = comp.withVision({ enableDetection: true });
comp.addSmartFraming({
source: "assets/action.mp4",
target: vision.objects.primary,
targetAspect: 9 / 16,
damping: 0.15,
leadHeadroom: 0.1
});3. Subject Outline Glow & Neon Pulse
3. 主体轮廓辉光与霓虹脉冲
Strokes the segmented subject boundary with an audio-reactive contour glow:
typescript
const vision = comp.withVision({ enableSegmentation: true });
comp.addSubjectOutline(vision.segmentation.subject, {
source: "assets/character.mp4",
color: "#FF5A1F",
width: 6,
blur: 12
});为分割出的主体边界描边,并添加音频响应式轮廓辉光:
typescript
const vision = comp.withVision({ enableSegmentation: true });
comp.addSubjectOutline(vision.segmentation.subject, {
source: "assets/character.mp4",
color: "#FF5A1F",
width: 6,
blur: 12
});4. Tracked Region Blur
4. 跟踪区域模糊
Blurs faces, license plates, or any detected class by following a live track:
typescript
const vision = comp.withVision({ classes: ["person"] });
layer.blurRegion(vision.objects.byCategory("person"), { strength: 30 });通过实时跟踪来模糊人脸、车牌或任何检测到的类别:
typescript
const vision = comp.withVision({ classes: ["person"] });
layer.blurRegion(vision.objects.byCategory("person"), { strength: 30 });5. Analyze Before Authoring
5. 创作前分析
One-shot, ffmpeg-free report of what is in a clip (tracks, classes, mask coverage):
typescript
const report = await comp.analyzeVisionSequence("assets/street.mp4", {
tasks: ["detect", "pose"],
categories: ["person"]
});
report.tracks; // [{ trackId, category, frames: [start, end], centerPath, ... }]一次性生成无需 ffmpeg 的报告,用于分析片段中的内容(跟踪、类别、遮罩覆盖率):
typescript
const report = await comp.analyzeVisionSequence("assets/street.mp4", {
tasks: ["detect", "pose"],
categories: ["person"]
});
report.tracks; // [{ trackId, category, frames: [start, end], centerPath, ... }]Reactive Signal Modulation
响应式 Signal 调制
Uniforms on any accept reactive instances instead of static values. When the signal changes, the GPU uniform buffer updates automatically without rebuilding the pipeline:
EffectSignaltypescript
import { Layer, Blur, sineSignal, computed } from "framefields";
// Oscillating breath blur
const blurSignal = sineSignal({
frequencyHz: 0.5,
min: 0,
max: 20
});
const layer = Layer.image("assets/bg.jpg")
.withEffect(new Blur({
strength: blurSignal,
blurType: "Gaussian"
}));任意 的 uniform 都可以接受响应式 实例,而非静态值。信号发生变化后,GPU uniform buffer 会自动更新,无需重建管线:
EffectSignaltypescript
import { Layer, Blur, sineSignal, computed } from "framefields";
// Oscillating breath blur
const blurSignal = sineSignal({
frequencyHz: 0.5,
min: 0,
max: 20
});
const layer = Layer.image("assets/bg.jpg")
.withEffect(new Blur({
strength: blurSignal,
blurType: "Gaussian"
}));Best Practices:
最佳实践:
- Emulsion Restraint: Keep (
FilmGrain) and0.04–0.08(Vignette) subtle. Over-filtering looks amateurish.0.2–0.45 - Order Matters: Place tonal grading (,
Curves) before lens optics (ColorBalance,Blur), and apply grain (DepthOfField) as the outermost layer.FilmGrain - No Redundant Shaders: For static image exports, set on
animated: falseto preserve deterministic static renders.FilmGrain
- 胶片效果克制: 将 (
FilmGrain)和0.04–0.08(Vignette)保持得细微。过度滤镜处理会显得业余。0.2–0.45 - 顺序很重要: 将色调分级(、
Curves)放在镜头光学效果(ColorBalance、Blur)之前,并将颗粒(DepthOfField)作为最外层效果。FilmGrain - 避免冗余 shader: 对于静态图片导出,将 的
FilmGrain设置为animated,以保留确定性的静态渲染结果。false