taste-application

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Taste Application

风格应用

The second half of the pipeline. taste-distillation measures references into a style pack; this generates against that pack and cuts the result.
流水线的后半部分。风格蒸馏(taste-distillation) 将参考视频测量生成风格包;本模块则基于该风格包生成内容并剪辑成片。

Execution and Delivery Contract

执行与交付约定

The original implementation ships here in
scripts/
; no separate
ito-video
checkout is required. Install
scripts/requirements.txt
for local processing. Install
scripts/requirements-live.txt
only for provider execution. Live uploads, submissions and downloads require explicit
TASTE_FORGE_ALLOW_LIVE=1
in addition to credentials; set it only for the user's authorized run.
--dry-run
remains credential-free and produces labelled placeholders. An ambiguous provider timeout is not retried as a new paid job. Inspect the provider request before deciding whether another submission is warranted.
Use existing completed takes without any provider calls:
bash
python scripts/pipeline.py --genre example --root stylepacks \
  --takes media/take-a.mp4 media/take-b.mp4 --duration 12 --fps 30 \
  --out out/review-v1.mp4
--duration
is a best-effort cadence target, not an exact runtime. Complete shots may produce a shorter or longer edit; the assembler does not duplicate clips or add padding to meet the target. The output manifest retains actual
duration
and adds
duration_contract
with requested and actual seconds, shortfall, overrun, and the
cadence_target
policy. Differences of at least one output frame are warned explicitly. No exact-duration mode is provided; when an exact runtime is required, inspect the receipt and revise or reject the cut before delivery.
The pipeline keeps each run's graded shot files because the editable FCPXML and EDL reference them. It refuses output collisions and reports timeline export failures. A completed render is not a saved editor project or creative approval. Preserve source assets and versioned project checkpoints before and after live edits; record the saved path and digest separately from the in-memory timeline receipt.
Clone-ready Fal graphs and their offline input compiler are documented in
workflows/README.md
. Compile brief and style direction before submission; do not assume a disconnected schema field affects a model prompt. Validate current endpoint fields against the actual provider before a paid run.
For Blender, use the full textured source GLB; retopology is a separately named derivative and never replaces that source.
blender_prop.py
supports explicit
--width 1920 --height 1080 --fps 30 --receipt receipt.json
and preserves pack-derived rim lighting and material textures. Saving a scene is distinct from rendering it; inspect the receipt's state and packed images.
For Resolve overlays, use
taste.resolve.apply_placements
with injected objects from an explicitly selected, versioned target. Set
source_end_mode
to the convention verified on that host. Do not assume an inclusive source end across Resolve versions. The adapter allocates overlapping effects above preserved tracks and checks every placement immediately and again after all appends. Composite integers must match the installed API; for the verified Studio 21 host, Screen is 5, not the historical builder's incorrect 22. The receipt proves in-memory placement only. Save the project, verify its checkpoint, then inspect the exact rendered output before reporting delivery.
原始实现已包含在
scripts/
目录中,无需单独检出
ito-video
仓库。本地处理请安装
scripts/requirements.txt
中的依赖。仅在服务端执行时安装
scripts/requirements-live.txt
。实时上传、提交和下载除了凭证外,还需要显式设置
TASTE_FORGE_ALLOW_LIVE=1
;请仅在用户授权运行时设置该变量。
--dry-run
模式无需凭证,会生成带标签的占位符。对于原因不明的服务提供商超时,不要作为新的付费任务重试。请先检查服务请求内容,再决定是否需要重新提交。
无需调用任何服务,直接使用已完成的拍摄片段:
bash
python scripts/pipeline.py --genre example --root stylepacks \
  --takes media/take-a.mp4 media/take-b.mp4 --duration 12 --fps 30 \
  --out out/review-v1.mp4
--duration
是尽力而为的节奏目标,而非精确的运行时长。完整镜头生成的剪辑可能更短或更长;组装器不会为了达到目标而复制片段或添加填充。输出清单会保留实际的
duration
,并新增
duration_contract
字段,包含请求时长、实际时长、不足时长、超时时长以及
cadence_target
策略。如果差异至少达到一帧,会显式发出警告。本工具不提供精确时长模式;如果需要精确的运行时长,请在交付前检查回执,修改或拒绝该剪辑。
流水线会保留每次运行的调色后镜头文件,因为可编辑的 FCPXML 和 EDL 会引用这些文件。它会拒绝输出冲突并报告时间线导出失败。完成渲染并不等同于保存了编辑器项目或获得了创意批准。在实时编辑前后,请保留源素材和带版本的项目检查点;将保存路径和摘要与内存中的时间线回执分开记录。
可直接克隆的 Fal 计算图及其离线输入编译器已在
workflows/README.md
中说明。提交前请编译好简介和风格方向;不要假设不相关的 schema 字段会影响模型提示词。在付费运行前,请对照实际的服务提供商验证当前端点的字段。
对于 Blender,请使用带完整纹理的源 GLB 文件;重拓扑是单独命名的衍生文件,永远不会替换源文件。
blender_prop.py
支持显式设置
--width 1920 --height 1080 --fps 30 --receipt receipt.json
,并保留从风格包派生的轮廓光和材质纹理。保存场景与渲染场景是两回事;请检查回执的状态和打包的图像。
对于 Resolve 叠加效果,请使用
taste.resolve.apply_placements
函数,注入来自显式选择的带版本目标的对象。将
source_end_mode
设置为在该主机上验证过的约定值。不要假设所有 Resolve 版本的源结束位置都是包含式的。适配器会在保留轨道上方分配重叠效果,并立即检查每个放置位置,在所有追加操作完成后再次检查。合成模式的整数值必须与安装的 API 匹配;对于已验证的 Studio 21 主机,Screen 模式的值为5,而非旧版构建器错误使用的22。回执仅证明内存中的放置操作成功。请保存项目,验证其检查点,然后在报告交付前检查实际渲染输出。

When to Activate

适用场景

  • "make a video in this style" / "apply the pack" / "supplement this footage"
  • Assembling generated clips into something with real edit rhythm
  • Minting 3D props from a look and getting them back into the video
  • Verifying that a finished piece actually matches its reference
  • "用这种风格制作视频" / "应用风格包" / "补充该素材"
  • 将生成的片段组装成具有真实剪辑节奏的作品
  • 根据外观生成3D道具并整合到视频中
  • 验证成片是否确实与参考视频匹配

Division of Labour

分工

The model supplies content, motion, framing and lighting structure. The pack supplies colour and rhythm. This is measured, not stylistic preference — see taste-distillation for the numbers. Practical consequences:
  • The generation prompt contains zero colour language. Add "Colour: none. Render neutral. Grading is applied afterwards."
  • Keep two separate flags:
    --brief
    (what HAPPENS: subject, action, place) and
    --style-steer
    (how it LOOKS). Merging them leaks style words into the scene ("teal" becomes a teal object) and subject words into the grade.
  • The model responds to local, checkable rules far better than global ones. "Backgrounds pure black and unlit; subjects blowing toward white" works; "extreme contrast" does not.
模型提供内容、运动、构图和光照结构。风格包提供色彩和节奏。 这是实测得出的,而非风格偏好——具体数据请参考风格蒸馏(taste-distillation)模块。实际影响如下:
  • 生成提示词中完全不包含色彩描述。请添加 "Colour: none. Render neutral. Grading is applied afterwards."(中文释义:色彩:无。渲染为中性色调。后续再进行调色。)
  • 请保持两个独立的标志参数:
    --brief
    (描述发生的内容:主体、动作、地点)和
    --style-steer
    (描述呈现的效果)。将两者合并会导致风格词汇泄露到场景中(比如 "teal/青绿色" 会变成一个青色的物体),而主体词汇会泄露到调色中。
  • 模型对局部的、可验证的规则的响应远好于全局规则。例如 "Backgrounds pure black and unlit; subjects blowing toward white"(背景为纯黑色且无光照;主体向白色过曝)这样的描述有效;"extreme contrast"(高对比度)则无效。

Generate TAKES, Not Shots

生成拍摄片段(TAKES),而非单个镜头

The obvious reading of "match the cadence" is one generation per shot. It is economically absurd. A reference averaging 0.78s/shot against an endpoint with a 4-second floor turns a 10s piece into 12 calls, 48 generated seconds for 10 used (21% efficiency), and twelve unrelated clips stitched into what should read as continuous.
Editors roll a longer take and cut inside it. Grouping shots into ~5s takes: 3 calls, 13 generated seconds, 77% efficiency, and consecutive shots that actually belong to each other because they came from the same generation.
Tell the model what shape you want, or it renders a slow locked-off push and six cuts inside it read as a stutter:
"Filmed as ONE continuous take with no hard cuts inside it. It will be cut into 6 pieces of roughly 0.8s in the edit, so the framing, subject and light must keep changing throughout — any 0.8s window has to stand alone as its own shot."
对"匹配节奏"的直观理解是每个镜头生成一次。这在成本上是极不合理的。如果参考视频平均每个镜头0.78秒,而服务端点的最低生成时长为4秒,那么一个10秒的作品需要12次调用,生成48秒内容,仅使用10秒(效率21%),而且12个不相关的片段拼接在一起,本该是连续的内容会显得割裂。
剪辑师会拍摄更长的片段,然后在其中剪辑。将镜头分组为约5秒的拍摄片段:3次调用,生成13秒内容,效率77%,而且连续的镜头实际上是同源的,因为它们来自同一次生成。
告诉模型你想要的片段形态,否则它会渲染一个缓慢的固定推镜,在里面剪6刀会显得卡顿:
"Filmed as ONE continuous take with no hard cuts inside it. It will be cut into 6 pieces of roughly 0.8s in the edit, so the framing, subject and light must keep changing throughout — any 0.8s window has to stand alone as its own shot." (中文释义:作为一个连续的拍摄片段拍摄,内部无硬切。剪辑时会被剪成约0.8秒的6段,因此构图、主体和光线必须全程持续变化——任何0.8秒的窗口都必须能作为独立镜头存在。)

Cutting Rules

剪辑规则

  • Re-encode, never stream-copy. Stream copy only cuts on keyframes, which at 0.78s mean shot length rounds every boundary to the nearest GOP — destroying the exact thing the pipeline exists to preserve.
  • When supplementing existing footage, use its own shot boundaries. Slicing a base video into contiguous pieces and playing them in order just reassembles the original: every "cut" lands mid-shot and is invisible. Measured, a 20-shot assembly registered only 12 detected cuts. Detect real boundaries and take every Nth so consecutive picks are guaranteed discontinuous. That moved a cut measurement from 1.29s to 0.83s against a 0.78s target.
  • Sample shot lengths from the reference's distribution, not from its mean, so the cut inherits rhythm variance instead of flattening to even clips.
  • 重新编码,切勿使用流复制。 流复制只能在关键帧处剪辑,对于0.78秒的镜头来说,意味着每个边界的镜头长度都会四舍五入到最近的GOP——这会破坏流水线本要保留的精确节奏。
  • 补充现有素材时,使用素材自身的镜头边界。 将基础视频切成连续片段再按顺序播放,只是重新组装了原视频:每个"剪辑点"都落在镜头中间,完全看不出来。经实测,一个20个镜头的组装视频仅检测到12个剪辑点。请检测真实的镜头边界,然后每隔N个取一个,确保连续选取的片段是不连续的。这使得剪辑点的实测间隔从1.29秒降至0.83秒,接近0.78秒的目标。
  • 从参考视频的分布中采样镜头长度,而非取平均值,这样剪辑会继承节奏的差异性,而不是变成均匀的片段。

Grading Rules

调色规则

  • Direct measurement beats a baked LUT when you have the clip: measure it, match its L* CDF, apply zone chroma.
    grade_clip_direct
    reached MAE 1.58 and contrast 34.0 against 34.7.
  • Anchor, do not CDF-match, when the clip's histogram is unlike the reference's. Forcing a 68%-black generated clip onto a busy reference histogram lifted the entire background out of black: background preservation fell to 26.0% (CDF) versus 69.0% (anchor), while MAE and contrast both still looked excellent. Default to anchored tone.
  • Batch size matters. Grading 48 frames at once OOM-killed the process; 6 is safe.
  • 当你有素材片段时,直接测量优于预烘焙LUT:测量素材,匹配其L* CDF,应用分区色度。
    grade_clip_direct
    函数的MAE达到1.58,对比度达到34.0(参考值为34.7)。
  • 当素材的直方图与参考视频差异较大时,使用锚定法,而非CDF匹配。 将黑色占比68%的生成片段强制匹配内容复杂的参考直方图,会导致整个背景的黑位被抬升:背景保留率从69.0%(锚定法)降至26.0%(CDF匹配法),尽管MAE和对比度看起来仍然很好。默认使用锚定色调法。
  • 批量大小很重要。 一次调色48帧会导致进程因OOM(内存不足)被终止;6帧是安全的。

Recovered fal Platform Behavior

已探明的fal平台行为

These observations and endpoint examples came from the recovered workflow. Recheck current endpoint metadata; they are not guarantees about every future provider version. The bundled graph templates record the separately verified workflow inputs, including explicit generated-audio control.
These cost real time to discover. Check them before designing a graph.
LimitDetail
No 3D renderer at allfal has
image-to-3d
,
text-to-3d
,
3d-to-3d
and nothing else. Every
3d-to-3d
endpoint emits another mesh. There is no
3d-to-image
/
3d-to-video
category, so a minted GLB cannot re-enter a fal video graph. Render locally, then use
fal-ai/ffmpeg-api/images-to-video
.
compose cannot overlay
fal-ai/ffmpeg-api/compose
rejects a second track with "Multiple video tracks are not supported" — and it counts an
image
track as a video track. It sequences one video track only. Composite locally with ffmpeg.
compose keyframes are millisecondsNothing in the response says so. A run submitted in seconds is accepted and returns a video that is 1000x too short.
extract-frame offers first/middle/last onlyNo arbitrary timestamp. Use the three as three distinct conditioning images.
No loops, no string concat in the DAGPer-shot fan-out has to be authored node by node, or kept local.
Kling 3.0 has no reference-to-videoThe v3 line is text/image/motion-control only; reference-to-video lives on the
o3
line:
fal-ai/kling-video/o3/pro/reference-to-video
.
Prefixes are not uniform
bytedance/*
,
tripo3d/*
,
meshy/*
,
minimax/*
,
openai/*
carry no
fal-ai/
prefix.
kling-video
,
veo3.1
,
flux-*
,
hunyuan-3d
,
ffmpeg-api
do.
这些观察结果和端点示例来自已还原的工作流。请重新检查当前端点的元数据;它们不能保证适用于未来所有的服务提供商版本。附带的计算图模板记录了单独验证过的工作流输入,包括显式的生成音频控制。
这些都是花了大量时间才发现的。设计计算图之前请务必检查。
限制详情
完全没有3D渲染器fal 仅提供
image-to-3d
、
text-to-3d
、
3d-to-3d
接口,别无其他。每个
3d-to-3d
端点都输出另一个网格。不存在
3d-to-image
/
3d-to-video
类别,因此生成的 GLB 无法重新进入 fal 视频计算图。请在本地渲染,然后使用
fal-ai/ffmpeg-api/images-to-video
。
compose 接口不支持叠加
fal-ai/ffmpeg-api/compose
会拒绝第二个轨道,并提示 "Multiple video tracks are not supported"(不支持多个视频轨道)——而且它会将
image
轨道算作视频轨道。它仅能对单个视频轨道进行排序。请使用 ffmpeg 在本地合成。
compose 关键帧单位为毫秒响应中完全没有说明这一点。以秒为单位提交的任务会被接受,但返回的视频时长会短1000倍。
extract-frame 仅支持首帧/中间帧/末帧不支持任意时间戳。请将这三帧作为三个不同的条件图像使用。
DAG 中不支持循环和字符串拼接每个镜头的扇出必须逐个节点编写,或者放在本地处理。
Kling 3.0 没有 reference-to-video 接口v3 系列仅支持文本/图像/运动控制;reference-to-video 功能属于
o3
系列:
fal-ai/kling-video/o3/pro/reference-to-video
。
前缀不统一
bytedance/*
、
tripo3d/*
、
meshy/*
、
minimax/*
、
openai/*
没有
fal-ai/
前缀。而
kling-video
、
veo3.1
、
flux-*
、
hunyuan-3d
、
ffmpeg-api
有。

Endpoint picks

端点选择

SlotBestValue alternative
reference→video
bytedance/seedance-2.5/reference-to-video
(~$0.473/s @720p)
fal-ai/kling-video/o3/pro/reference-to-video
(~$0.112/s)
image→3D
fal-ai/hunyuan-3d/v3.1/pro/image-to-3d
($0.375, up to 8 views)
tripo3d/h3.1/image-to-3d
($0.20)
text→3D
fal-ai/hunyuan-3d/v3.1/pro/text-to-3d
tripo3d/h3.1/text-to-3d
retopology
fal-ai/hunyuan-3d/v3.1/smart-topology
($0.75)
tripo3d/tripo/remesh
(~75x cheaper)
part split
fal-ai/hunyuan-3d/v3.1/part
(FBX only)
tripo3d/tripo/segment
text→image
fal-ai/nano-banana-pro
($0.15 flat)
fal-ai/flux-2-pro
($0.03/MP)
Seedance is ~4x Kling o3's price for the same 5 seconds. It earns that on multi-reference fidelity (up to 50 mixed image/video/audio refs) and does not earn it when conditioning on a single still.
Model IDs and prices drift. Verify against fal.ai/models before promising any of them.
用途最佳选择高性价比替代
参考转视频
bytedance/seedance-2.5/reference-to-video
(720p下约$0.473/秒)
fal-ai/kling-video/o3/pro/reference-to-video
(约$0.112/秒)
图像转3D
fal-ai/hunyuan-3d/v3.1/pro/image-to-3d
($0.375,最多8视角)
tripo3d/h3.1/image-to-3d
($0.20)
文本转3D
fal-ai/hunyuan-3d/v3.1/pro/text-to-3d
tripo3d/h3.1/text-to-3d
重拓扑
fal-ai/hunyuan-3d/v3.1/smart-topology
($0.75)
tripo3d/tripo/remesh
(便宜约75倍)
部件拆分
fal-ai/hunyuan-3d/v3.1/part
(仅支持FBX)
tripo3d/tripo/segment
文本转图像
fal-ai/nano-banana-pro
(固定费用$0.15)
fal-ai/flux-2-pro
($0.03/百万像素)
同样5秒的内容,Seedance的价格约为Kling o3的4倍。它的优势在于多参考保真度(最多支持50个混合图像/视频/音频参考),如果仅使用单张静态图像作为条件,就不值这个价了。
模型ID和价格会变动。在承诺使用任何模型之前,请先在 fal.ai/models 上验证。

The 3D Branch

3D分支

bash
python mint3d.py --genre <name> --from-stills 4 --retopo --render
python mint3d.py --genre <name> --prompt "a cracked chrome visor" --render
  • Generate a clean plate first; do not lift from reference stills. The endpoint's stated input requirement is simple background, single object, object >50% of frame. Reference reels are the opposite of that — collages, wide shots, several subjects, burnt-in graphics — and they produce sculpted noise. Text → single-object plate → mesh costs ~$0.15 extra and is the difference between a usable mesh and a discarded one.
  • Multi-view is named per-angle fields, not a list.
    input_image_url
    (front, required), then
    back_image_url
    ,
    left_image_url
    ,
    right_image_url
    ,
    left_front_image_url
    ,
    right_front_image_url
    ,
    top_image_url
    ,
    bottom_image_url
    . There is no
    input_image_urls
    and no
    multi_view
    flag — inventing them degrades every mint to single-view while appearing to work. A wrong angle label is worse than omitting the view, because the model trusts it.
  • Address the GLB by key, not by position. The response carries a
    thumbnail
    PNG and a
    model_urls
    block alongside
    model_glb
    ; taking the first URL works only until the keys reorder, and a preview PNG downloads fine — nothing fails until Blender refuses to open it.
  • Request PBR maps. Without them the mesh lights like painted cardboard.
  • Retopologise if anyone will edit or rig it. Generated meshes are dense and chaotic.
  • Render locally to close the loop. Once a turntable is frames, it is footage, and every downstream stage already handles footage — grade it, cut it, screen it as an element, or upload it as a conditioning reference. Use Blender when a binary is on PATH; keep a dependency-light software rasteriser as the default, because a headless GL context is the single most common thing missing from a container and a renderer that only works on a workstation is not part of a pipeline.
bash
python mint3d.py --genre <name> --from-stills 4 --retopo --render
python mint3d.py --genre <name> --prompt "a cracked chrome visor" --render
  • 先生成干净的素材板,不要直接从参考静帧中提取。 端点声明的输入要求是简单背景、单个物体、物体占画面比例>50%。参考视频则完全相反——拼贴画、广角镜头、多个主体、内嵌图形——生成的结果会是雕刻状的噪点。文本→单物体素材板→网格的流程仅多花约$0.15,但能决定生成的网格是可用的还是废品。
  • 多视角使用按角度命名的字段,而非列表。
    input_image_url
    (正面,必填),然后是
    back_image_url
    、
    left_image_url
    、
    right_image_url
    、
    left_front_image_url
    、
    right_front_image_url
    、
    top_image_url
    、
    bottom_image_url
    。不存在
    input_image_urls
    字段,也没有
    multi_view
    标志——自行编造这些字段会导致所有生成都降为单视角,表面上却看起来正常。角度标签错误比省略该视角更糟糕,因为模型会信任标签。
  • 通过键名获取GLB文件,而非按位置。 响应中包含
    thumbnail
    PNG图、
    model_urls
    块以及
    model_glb
    ;直接取第一个URL的方法只有在键名顺序不变时才有效,而且预览PNG可以正常下载——直到Blender无法打开文件时才会发现问题。
  • 请求PBR贴图。 没有PBR贴图的话,网格的光照效果会像涂了颜料的硬纸板。
  • 如果需要编辑或绑定骨骼,请进行重拓扑。 生成的网格既密集又混乱。
  • 在本地渲染以形成闭环。 一旦转盘动画渲染为帧序列,它就变成了素材,下游的所有阶段都可以处理——调色、剪辑、作为元素叠加,或者作为条件参考上传。如果PATH中有Blender二进制文件就用Blender;默认使用依赖较少的软件光栅化器,因为无头GL环境是容器中最常缺失的东西,而只能在工作站上运行的渲染器不属于流水线的一部分。

Verify, Then Believe

先验证,再采信

Every other stage claims a result. Check it, and check distribution shape, not just moments:
  • background
    — share of frame below L*10 vs the pack's figure. Compare to the reference, not to the source clip: a generated source at 68% black is blacker than any reference in a 24–55% band, so "preserve the source's blacks" demands the wrong thing and equally excuses a lifted grade.
  • chroma_mae
    — per-zone a*/b* error, using the median (matching how the pack's targets were measured; a mean here compares a skew-sensitive statistic to a robust one and reports a definition mismatch as an error).
  • contrast
    /
    black_point
    /
    white_point
  • banding
    — empty L* histogram bins between occupied ones. Counting total empty bins does not work: a legitimately dark clip has empty highlight bins.
  • cadence
    — detected mean shot length vs the reference's.
Watch the units trap: OpenCV changes Lab convention with dtype. On float32, L* is 0–100 and a*/b* are signed; on uint8, L* is 0–255 and a*/b* are biased +128. Mixing them reports chroma errors in the hundreds.
其他每个阶段都会声称得到了结果。请进行验证,而且要检查分布形态,而不仅仅是矩量:
  • background
    (背景)—— L*值低于10的画面占比,与风格包的数值对比。请和参考视频对比,不要和源素材对比:生成的源素材黑色占比为68%,比24-55%区间内的任何参考视频都更黑,因此"保留源素材的黑位"是错误的要求,也会为调色时黑位抬升找借口。
  • chroma_mae
    (色度平均绝对误差)—— 每个分区的a*/b*误差,使用中位数计算(与风格包目标的测量方式一致;在这里使用均值会把对偏度敏感的统计量和稳健统计量对比,将定义不匹配报告为误差)。
  • contrast
    (对比度)/
    black_point
    (黑点)/
    white_point
    (白点)
  • banding
    (色带)—— 已占用的L*直方图区间之间的空区间。统计总空区间数是无效的:正常的暗调片段在高光区域本来就有空区间。
  • cadence
    (节奏)—— 检测到的平均镜头长度与参考视频的对比。
注意单位陷阱:OpenCV会根据dtype改变Lab颜色空间的约定。在float32类型下,L*范围是0-100,a*/b*是有符号的;在uint8类型下,L*范围是0-255,a*/b*有+128的偏移。混用两者会导致色度误差达到数百。

Full Chain

完整流程

bash
python pipeline.py --genre <name> --refs a.mov b.mov \
  --brief "what happens" --duration 12 \
  --base-video existing.mp4 --out out/FINAL.mp4
mint → distill → (mint3d) → apply → forge → verify. Stages 1–3 are cached, so iterating on briefs never re-measures anything.
--dry-run
stubs every network call: the plan, prompts, track layout and manifest all still get exercised.
bash
python pipeline.py --genre <name> --refs a.mov b.mov \
  --brief "what happens" --duration 12 \
  --base-video existing.mp4 --out out/FINAL.mp4
生成→蒸馏→(3D生成)→应用→合成→验证。前3个阶段有缓存,因此迭代修改简介时不会重新进行任何测量。
--dry-run
模式会桩化所有网络调用:计划、提示词、轨道布局和清单仍然会被执行。

Anti-Patterns

反模式

Don'tWhy
One generation per shot21% efficiency, 12 unrelated clips
Put colour in the promptMeasured not to work; pushes away from the neutral base the LUT wants
Stream-copy the cutsKeyframe-only boundaries destroy sub-second rhythm
Cut a base video contiguouslyReassembles the original; every cut invisible
Trust compose to overlayIt cannot; it rejects the second track
Send seconds to composeSilently 1000x too short
Ship on MAE aloneAdd the background-share check
错误做法原因
每个镜头生成一次效率仅21%,12个片段互不相关
在提示词中加入色彩描述实测无效;会偏离LUT所需的中性基底
使用流复制进行剪辑仅关键帧处的边界会破坏亚秒级节奏
连续切割基础视频只是重组了原视频;所有剪辑点都不可见
依赖compose接口做叠加它做不到;会拒绝第二个轨道
向compose接口传入秒为单位的数值会悄无声息地短1000倍
仅靠MAE就交付需增加背景占比检查

Borrowed Footage Carries the Capture App's UI

借用素材会带有录制应用的UI

The single worst defect found in a delivered cut: the finished video shipped someone else's like button, view counter and comment bubble, because the base footage was a screen recording and nothing cropped them out.
content_mask
does not solve this and its bounding box makes it worse.
Temporal variance keeps interface chrome, because chrome animates - the heart pulses, the counter ticks - so the mask marks it as moving content. Measured on three references, the mask bbox kept 100% of the width every time while the interface sat plainly in the right-hand margin.
The separating signal is the temporal median, not the variance. Footage moves, so the median of many frames averages into mush with almost no edge energy; chrome sits at fixed coordinates, so its edges survive intact. Sobel energy on the median frame lights up on chrome and goes quiet on content - measured, a right-hand column read 0.23 against an interior background of 0.03 on one reference and 0.31 against 0.15 on another.
Two implementation details that cost a cycle each:
  • Trim past the innermost outlier in each outer band, not inward from the edge. Walking in while the current line is hot stops immediately, because the outermost lines are letterbox - flat black, zero edge energy - and the chrome sits inside that at 90-95% of width. The naive version trimmed 1% of frame while the like button stayed in shot.
  • Scale to cover, not pad, when portrait source lands in a landscape cut. Padding 9:16 (narrower still after the UI crop) into 16:9 left ~60% of frame as black bars, one shot was nearly an empty rectangle, and it poisoned the background metric because bars are pure black. Covering loses the sides, which is the right trade for centre-framed material.
在交付的剪辑中发现的最严重缺陷:成片里带有别人的点赞按钮、观看计数器和评论气泡,因为基础素材是录屏,没有把这些元素裁掉。
content_mask
无法解决这个问题,它的边界框反而会让情况更糟。
时间方差会保留界面元素,因为界面元素会动——爱心会跳动,计数器会变化——因此蒙版会将其标记为运动内容。在三个参考视频上实测,蒙版的边界框每次都保留了100%的宽度,而界面元素明明就在右侧边栏。
区分两者的信号是时间中位数,而非方差。素材内容是运动的,因此多帧的中位数平均后会变得模糊,几乎没有边缘能量;界面元素位于固定坐标,因此边缘会完整保留。中位数帧的Sobel能量在界面元素处很高,在内容处则很低——实测显示,在一个参考视频中,右侧列的数值为0.23,而内部背景为0.03;另一个参考视频中分别为0.31和0.15。
两个各耗费了一个周期才发现的实现细节:
  • 从每个外带的最内侧异常值开始向内裁剪,而非从边缘向内裁剪。 当当前行有高能量时就向内走的方法会立即停止,因为最外层的行是信箱模式的黑边——纯黑,边缘能量为零——而界面元素就在黑边内侧,占宽度的90-95%。简单版本只裁剪了1%的画面,点赞按钮还留在镜头里。
  • 当竖屏素材放入横屏剪辑时,使用缩放填充(cover),而非加黑边(pad)。 将9:16的视频(裁剪UI后会更窄)填充到16:9的画面中,会导致约60%的画面是黑边,有一个镜头几乎变成了空矩形,而且黑边是纯黑色,会破坏背景指标。缩放填充会损失两侧内容,但对于主体居中的素材来说是正确的权衡。

Overlay Plates Are Elements, Not Washes

叠加素材板是元素,而非全屏叠层

A glow plate is ~4% covered by construction. Composite it at frame size and you get a small bright dot parked mid-shot - it reads as a sticker, and it was visible in a delivered cut as an unexplained coloured blob.
  • Tighten each plate to its alpha bounding box first. That raises coverage from ~4% to 15-35% and hands size control to the caller instead of inheriting whatever fraction of the source frame the element happened to occupy.
  • Choose wash vs element by coverage. Diffuse plates (<10% after tightening) work stretched full-frame at low opacity; concentrated ones want to be scaled to 35-70% of frame width and placed.
  • Vary placement, scale and rotation per shot from a seeded RNG, so the cut stays reproducible but no two stamped shots share a mark. One plate in one spot every Nth shot reads as a watermark.
  • Resolve element geometry in Python, not in ffmpeg expressions:
    pad()
    rejects a negative offset and cannot pad below its input size, so an oversized or off-frame element kills the whole filtergraph.
一个发光板的默认覆盖率约为4%。按帧大小合成的话,你会得到一个停在镜头中间的小亮点——看起来像贴纸,在一个交付的剪辑中,它就像一个莫名其妙的彩色斑点。
  • 先将每个素材板裁剪到其alpha边界框。 这会将覆盖率从约4%提升到15-35%,并将大小控制权交给调用者,而不是继承元素在源帧中所占的任意比例。
  • 根据覆盖率选择是作为全屏叠层还是独立元素使用。 漫射板(裁剪后覆盖率<10%)适合拉伸到全屏,以低不透明度叠加;集中的元素则适合缩放到画面宽度的35-70%,然后放置在合适位置。
  • 每个镜头的位置、缩放和旋转都要变化,使用带种子的随机数生成器,这样剪辑是可复现的,但不会有两个叠加的镜头有相同的标记。如果每隔N个镜头就把同一个板放在同一个位置,看起来就像水印。
  • 请在Python中计算Resolve元素的几何参数,不要用ffmpeg表达式:
    pad()
    函数拒绝负偏移,也不能在输入尺寸之外填充,因此过大或超出画面的元素会导致整个滤镜图失效。

Downstream Handoff (Resolve / Blender)

下游交付(Resolve / Blender)

Always ship an editable timeline beside the mp4. The flattened video is a viewing copy and the one thing a colourist cannot work with — every cut is baked in and the shots are no longer separable.
forge.py
writes FCPXML 1.9 and a CMX3600 EDL referencing the individual graded shot files, so the piece lands as a timeline that can be re-cut and re-graded.
Validate the export, do not assume it: check that asset-clip offsets equal the running sum of prior durations (no gaps), that every
ref
resolves to a declared asset, that every
media-rep src
exists on disk, and that the total matches the mp4. A timeline that imports but drifts is worse than one that fails loudly.
Two things that silently destroy the work:
  • Project frame rate must be set before import. Resolve locks timeline fps on first timeline creation and conforms the cadence silently. At a sub-second mean shot length that conform is visible.
  • The delivered shots are already graded.
    look.cube
    is a normalising LUT for new material and for matching — applying it to the supplied shots double-grades them. Node 1, nothing before it, corrections after.
Ship a handoff doc with the measured targets in it (
scripts/HANDOFF-TEMPLATE.md
is a filled example): the zone chroma table, the cadence distribution including rhythm variance — an editor who matches the mean but not the variance produces something that reads completely differently — and an explicit "do not" list.
For Blender,
scripts/blender_prop.py
derives its lighting from the pack: black world with transparent film (so renders composite with no keying), key plus rim (a key alone lets the silhouette die against black), rim colour converted from the pack's peak-chroma zone via Lab→linear sRGB so the prop picks up the same cast the footage is graded to. View transform Standard, not AgX/Filmic, and no grading in Blender — AgX applies its own tone curve before the LUT ever sees the pixels, and grading twice compounds.
除了mp4文件外,务必同时交付可编辑的时间线。 扁平化的视频只是观看用的副本,是调色师完全无法处理的——所有剪辑都已固化,镜头无法再分离。
forge.py
会生成FCPXML 1.9和CMX3600 EDL文件,引用单独的调色后镜头文件,因此作品会以时间线形式交付,可以重新剪辑和重新调色。
请验证导出结果,不要想当然:检查素材片段的偏移量等于之前所有片段时长的总和(无间隙),每个
ref
都能解析到已声明的素材,每个
media-rep src
都在磁盘上存在,总时长与mp4文件一致。一个能导入但时长漂移的时间线,比一个直接报错的更糟糕。
两个会悄无声息毁掉工作的问题:
  • 必须在导入前设置项目帧率。 Resolve在首次创建时间线时会锁定时间线帧率,并悄无声息地适配节奏。在平均镜头长度不足1秒的情况下,这种适配是肉眼可见的。
  • 交付的镜头已经过调色。
    look.cube
    是用于新素材和匹配的标准化LUT——将其应用到提供的镜头上会导致重复调色。节点1之前不要加任何东西,校正放在后面。
交付时请附带一份包含实测目标的交接文档(
scripts/HANDOFF-TEMPLATE.md
是一个填好的示例):分区色度表、包含节奏方差的节奏分布——如果剪辑师只匹配了平均时长而没匹配方差,最终作品的观感会完全不同——以及一份明确的"禁止操作"清单。
对于Blender,
scripts/blender_prop.py
会从风格包中派生光照:黑色世界背景+透明胶片(这样渲染结果无需抠像就能合成),主光加轮廓光(只有主光的话,剪影在黑色背景中会消失),轮廓光的颜色从风格包的峰值色度区通过Lab→线性sRGB转换而来,这样道具会带有与素材调色后相同的色偏。视图变换使用Standard,不要用AgX/Filmic,且不要在Blender中调色——AgX会在LUT处理像素之前应用自己的色调曲线,两次调色会导致效果叠加。

Bundled Code

附带代码

scripts/
in this skill is a working implementation, not pseudocode. It has no project-specific assumptions: point it at any reference videos and it produces a pack.
bash
pip install -r scripts/requirements.txt
export FAL_KEY=...            # only needed for the stages that call fal
Every network call is stubbed under
TASTE_FORGE_DRY_RUN=1
or
--dry-run
, so the plan, prompts, track layout and manifest can be inspected without spending.
本技能中的
scripts/
目录是可运行的实现,而非伪代码。它没有项目特定的假设:指向任何参考视频,它都能生成风格包。
bash
pip install -r scripts/requirements.txt
export FAL_KEY=...            # only needed for the stages that call fal
在
TASTE_FORGE_DRY_RUN=1
或
--dry-run
模式下,所有网络调用都会被桩化,因此无需花费成本就能检查计划、提示词、轨道布局和清单。