Back to Details

humanoid-motion

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

humanoid-motion — a generated two-view take in, a gated clip on the contract out

humanoid-motion — 双视角 take 输入，按 contract 输出经过 gate 的片段

A clip here starts as one video in which a front and a side view of the character perform the move together, and ends as an action on the humanoid contract skeleton (
unity-rigged-character
→ 
reference/humanoid-contract.md
), so every contract character plays it unchanged. The front half solves the motion straight onto the character's own rig; the back half cleans, gates and ships it.
此处的片段始于一个视频，其中角色的正面和侧面视角同步完成同一动作，最终生成 humanoid contract 骨架（
unity-rigged-character
→ 
reference/humanoid-contract.md
）上的 action，使所有 contract 角色均可原样播放。前半部分直接将动作解算到角色自身的 rig；后半部分负责清理、gate 验证并交付。

Start here — symptom → where to look

从这里开始——症状 → 查看位置

SymptomGo to
"Turn this move into an animation"; two-camera mocap; which source (video or a pack)Which source; 
reference/two-camera.md
, then 
reference/pipeline.md
The halves of the H3 take play out of step, colours swap, a limb vanishesreroll the seed (
reference/two-camera.md
, Limits)
reader.val_mean_px
fails; train the reader
reference/two-camera.md
(Reader)
two_view_agreement
fails
reference/two-camera.md
(Gates): the views were animated differently; reroll
"The solve pops": 
pops
in the solve; 
pops_in_source
reference/pipeline.md
§4
Jitter on a clean overlayraise 
solve.py --acc-weight
(
reference/two-camera.md
, Fit)
"Feet slide", "lock the feet"; no plant found
twocam.py bake
; 
reference/pipeline.md
§2 (
--rig-info
)
reach_m
or 
contact_height_m
fails
reference/pipeline.md
§4
Where to cut the walk loop; the loop pops at the seam
reference/pipeline.md
§2
A clip without travel for a controller
blender-retarget
(in place)
Fingers flat or wrong
reference/hands.md
Can this clip be sold in the EU?
reference/pipeline.md
§3; 
art-asset-registry
→ 
reference/licences.md
The solve is raw material, never the clip. Each stage has a gate that can fail, and a take that fails is re-rolled, re-filmed or trimmed, not hand-repaired. Nothing here retimes a clip or paints a pose.
症状查看位置
“把这个动作转换为动画”；双摄像头 mocap；应使用哪种源素材（视频还是素材包）选择源素材；
reference/two-camera.md
，然后阅读 
reference/pipeline.md
H3 take 的前后两部分播放不同步、颜色互换或某个肢体消失重新生成 seed（
reference/two-camera.md
，Limits）
reader.val_mean_px
失败；训练 reader
reference/two-camera.md
（Reader）
two_view_agreement
失败
reference/two-camera.md
（Gates）：两个视图采用了不同的动画；请重新生成
“解算结果出现跳变”：解算中存在 
pops
；
pops_in_source
reference/pipeline.md
§4
干净叠加画面上出现抖动提高 
solve.py --acc-weight
（
reference/two-camera.md
，Fit）
“脚部滑动”“锁定双脚”；未找到 plant（足部落地点）
twocam.py bake
；
reference/pipeline.md
§2（
--rig-info
）
reach_m
或 
contact_height_m
失败
reference/pipeline.md
§4
在哪里剪切行走循环；循环在接缝处出现跳变
reference/pipeline.md
§2
为 controller 提供不带位移的片段
blender-retarget
（原地）
手指平直或姿势错误
reference/hands.md
此片段能否在欧盟销售？
reference/pipeline.md
§3；
art-asset-registry
→ 
reference/licences.md
解算结果只是原始素材，绝不是最终片段。 每个阶段都有一个可能失败的 gate；失败的 take 应重新生成、重新拍摄或裁剪，而不是手工修复。这里没有任何步骤会重新定时片段或绘制 pose。

Which source

选择源素材

The moveSourceRead
Any move for a contract-rigged character: walk, run, idle, a strike, a leap no actor can do, a Q-version bodytwo cameras: front + side reference → local H3 video → per-character reader → two-view triangulation → fitted to the rig, no retarget
reference/two-camera.md
A clip pack already has ita pack, retargeted (in place or with translation)
blender-retarget
动作源素材阅读
contract rig 角色的任意动作：行走、跑步、待机、一次攻击、演员无法完成的跳跃、Q 版体型双摄像头：正面 + 侧面参考 → 本地 H3 视频 → 角色专属 reader → 双视图三角测量 → 拟合至 rig，无需 retarget
reference/two-camera.md
动画片段包中已有对应动作片段包，经过 retarget（原地或带位移）
blender-retarget

The front half — 
scripts/twocam/twocam.py

前半部分——
scripts/twocam/twocam.py

One cross-platform runner, one subcommand per stage, files under 
runs/<character>/
. Blender comes from 
--blender
or the 
BLENDER
environment variable (default 
blender
on the PATH). Exit codes: 0 pass, 2 a gate failed (the runner carries on, so the overlay, the fit and the GLB are there to look at), 1 refused input or an error (it stops).
#SubcommandOnce perWritesLook at before the next
1
prepare
character
rig/rig_info.json
, 
rig/twocam.blend
(the painted copy)		the solved bones and feet it lists
2
ref
character
ref/ref.png
, the first frame
ref.inside_frame
, 
ref.on_ground
3
film
move
videos/<move>/s<seed>/h3.mp4
, 
film.json
(its licence)		the H3 take itself: both halves in step, colours unswapped. A bad take is rerolled here, before an hour of training
4
dataset
character
dataset/
dataset.in_view
, 
dataset.written
5
train
character (CUDA)
reader.pt
: 20 epochs, or 12 with 
--init
for the next character on about 1,000 new samples
reader.val_mean_px
6
capture
move
clips/<name>/
: detect → 
solve.py --samples-out
(despike on) → 
motion_metrics samples.json --pops-min-move 0.03
→ 
apply_motion --glb
overlay.mp4
, 
two_view_agreement
, 
pops
; exit 2 on 
foot_slide_*
is expected, 
bake
locks the feet
7
bake
move
apply_motion --save keyed.blend
→ 
contact_bake.py
in Blender, 
--legs contract --rig-info rig_info.json
(its 
contract_names
find an Auto-Rig Pro rig's bones) 
--readings samples.json
→ 
motion_metrics
on the baked samples; 
--render
: 
baked.mp4
, front | side		slide, 
reach_m
, 
contact_height_m
, 
pops
and jitter (what the video has is discounted: 
pops_in_source
, 
jitter_over_source_mps2
)
The prompt for 
film
comes from 
make_prompt.py
(one action sentence per clip). Training does not run on a Mac; read and fit do. The stages, the frame, the reader, the fit and their gates: 
reference/two-camera.md
.
一个跨平台 runner，每个阶段对应一个子命令，文件存放在 
runs/<character>/
下。Blender 路径来自 
--blender
或 
BLENDER
环境变量（默认从 PATH 中查找 
blender
）。退出码： 0 表示通过；2 表示 gate 失败（runner 会继续执行，因此仍可查看叠加结果、拟合结果和 GLB）；1 表示输入被拒绝或发生错误（流程停止）。
#子命令执行范围输出下一步前检查
1
prepare
角色
rig/rig_info.json
、
rig/twocam.blend
（已绘制的副本）		列出的解算骨骼和脚部
2
ref
角色
ref/ref.png
、首帧
ref.inside_frame
、
ref.on_ground
3
film
动作
videos/<move>/<s<seed>/h3.mp4
、
film.json
（其 licence）		H3 take 本身：前后两部分同步、颜色未互换。错误的 take 应在此阶段重新生成，避免浪费数小时训练时间
4
dataset
角色
dataset/
dataset.in_view
、
dataset.written
5
train
角色（CUDA）
reader.pt
：训练 20 个 epoch；使用 
--init
为下一个角色训练时，基于约 1,000 个新样本训练 12 个 epoch
reader.val_mean_px
6
capture
动作
clips/<name>/
：检测 → 
solve.py --samples-out
（开启 despike）→ 
motion_metrics samples.json --pops-min-move 0.03
→ 
apply_motion --glb
overlay.mp4
、
two_view_agreement
、
pops
；出现 
foot_slide_*
时退出码 2 属于预期，后续 
bake
会锁定双脚
7
bake
动作
apply_motion --save keyed.blend
→ 在 Blender 中运行 
contact_bake.py
，参数包括 
--legs contract --rig-info rig_info.json
（通过 
contract_names
找到 Auto-Rig Pro rig 的骨骼）和 
--readings samples.json
→ 对烘焙后的样本运行 
motion_metrics
；使用 
--render
可输出 
baked.mp4
（front | side）		滑动、
reach_m
、
contact_height_m
、
pops
和抖动（视频本身已有的问题会被折减：
pops_in_source
、
jitter_over_source_mps2
）
film
使用的 prompt 来自 
make_prompt.py
（每个片段一句动作描述）。训练无法在 Mac 上运行，但读取和拟合可以。阶段、坐标系、reader、拟合及其 gate 的说明见：
reference/two-camera.md
。

The back half

后半部分

#StageRunGate
1Contact bake
twocam.py bake
, or 
blender -b keyed.blend -P contact_bake.py -- --armature RIG --action ACT --legs contract --rig-info rig_info.json --out baked.blend --samples-out S.npz
exit 0: slide p95 ≤ 1.5 cm, max ≤ 3 cm, 
reach_m
, 
contact_height_m
2Loop cut
motion_metrics.py S.npz --state walk --loop
→ 
loop.start/length
; set the action's range; 
contact_bake.py ... --sample-only --frame-range A B
on the cut
loop_period_found
, 
loop_periodicity
, 
loop_seam_ratio
3Gate
motion_metrics.py CUT.npz --state walk --pops-min-move 0.03
(no 
--loop
: one period, its seam gated in 2)		exit 0
4FBX
blender-character-rig
bcr_fbx
snapshot → export → reimport, 
rig_gate.py
rig_gate.py
exit 0
5Unity
unity-rigged-character
SharedSkeletonClipImporter.cs
, 
RootBonePath
= 
<rig>/Root/Hips
every binding resolves on every contract character
6Record
art-asset-registry
: the video with its 
licence
, then the clip with 
--link depends-on=<the video>
registry.py check
exit 0
Commands in full, the runner's walk worked through, and what each failure means: 
reference/pipeline.md
.
#阶段运行Gate
1接触烘焙
twocam.py bake
，或 
blender -b keyed.blend -P contact_bake.py -- --armature RIG --action ACT --legs contract --rig-info rig_info.json --out baked.blend --samples-out S.npz
退出码 0：滑动 p95 ≤ 1.5 cm、最大值 ≤ 3 cm、
reach_m
、
contact_height_m
2循环剪切
motion_metrics.py S.npz --state walk --loop
→ 
loop.start/length
；设置 action 的范围；在剪切后的区间运行 
contact_bake.py ... --sample-only --frame-range A B
loop_period_found
、
loop_periodicity
、
loop_seam_ratio
3Gate
motion_metrics.py CUT.npz --state walk --pops-min-move 0.03
（不使用 
--loop
：仅处理一个周期，其接缝在步骤 2 中 gate）		退出码 0
4FBX使用 
blender-character-rig
的 
bcr_fbx
创建快照 → 导出 → 重新导入，再运行 
rig_gate.py
rig_gate.py
退出码 0
5Unity
unity-rigged-character
的 
SharedSkeletonClipImporter.cs
，
RootBonePath
= 
<rig>/Root/Hips
所有 contract 角色上的每个绑定均可正确解析
6记录
art-asset-registry
：先记录带有 
licence
的视频，再使用 
--link depends-on=<the video>
记录片段
registry.py check
退出码 0
完整命令、runner 的实际执行流程以及每种失败的含义见：
reference/pipeline.md
。

Files

文件

PathRole
scripts/twocam/twocam.py
the runner: 
prepare
, 
ref
, 
film
, 
dataset
, 
train
, 
capture
, 
bake
; exit 0/2/1
scripts/twocam/
the stages it calls (layout, skeleton, prompt, film, reader, detect, solve, exam, Blender scripts): 
reference/two-camera.md
scripts/contact_bake.py
host (BVH) or Blender (action): plants from contact probabilities or the shared plant detector, sole heights from 
--rig-info
or a samples file's 
sole_offsets
(rig bone names through its 
contract_names
), 
--readings
carried into the samples, foot point locked per plant, analytic two-bone leg IK, slide before/after through 
motion_metrics
; 
--sample-only
measures; exit 0/2/1
scripts/bvh_io.py
BVH read/write/FK and Euler conversions for 
contact_bake.py
's host mode
reference/two-camera.md
the front half: the frame, the skeleton, the stages, the reader, the fit, the gates, the exam
reference/pipeline.md
the back half: commands, the ledger records, failures and fixes
reference/hands.md
the fixed hand poses a clip ships with
bash
S=<repo>/skills/humanoid-motion/scripts
python3 $S/bvh_io.py --selftest && python3 $S/contact_bake.py --selftest
blender -b --factory-startup -P $S/contact_bake.py -- --selftest-blender
contact_bake.py
imports 
motion_metrics.py
from 
skills/blender-retarget/scripts
(the one plant definition); keep the two skills side by side. It needs numpy only; the reader and the fit need torch (
reference/two-camera.md
).
路径作用
scripts/twocam/twocam.py
runner：
prepare
、
ref
、
film
、
dataset
、
train
、
capture
、
bake
；退出码 0/2/1
scripts/twocam/
runner 调用的各个阶段（布局、骨架、prompt、film、reader、detect、solve、exam、Blender 脚本）：
reference/two-camera.md
scripts/contact_bake.py
host（BVH）或 Blender（action）模式：根据接触概率或共享 plant 检测器生成 plant；从 
--rig-info
或 samples 文件中的 
sole_offsets
获取脚底高度（通过 
contract_names
映射 rig 骨骼名称）；将 
--readings
写入样本；按每次 plant 锁定足点；使用解析式双腿骨 IK；通过 
motion_metrics
测量烘焙前后的滑动；
--sample-only
仅执行测量；退出码 0/2/1
scripts/bvh_io.py
为 
contact_bake.py
的 host 模式提供 BVH 读写、FK 和 Euler 角转换
reference/two-camera.md
前半部分：坐标系、骨架、各阶段、reader、拟合、gates 和检查
reference/pipeline.md
后半部分：命令、记录条目、失败与修复方法
reference/hands.md
片段交付时使用的固定 hand pose
bash
S=<repo>/skills/humanoid-motion/scripts
python3 $S/bvh_io.py --selftest && python3 $S/contact_bake.py --selftest
blender -b --factory-startup -P $S/contact_bake.py -- --selftest-blender
contact_bake.py
从 
skills/blender-retarget/scripts
导入 
motion_metrics.py
（二者使用相同的 plant 定义）；请将这两个 skill 保持并列放置。它仅依赖 numpy；reader 和拟合需要 torch（
reference/two-camera.md
）。

Rules the route is built on

此流程所依据的规则

  • One skeleton. The solve is fitted onto the contract rig itself; nothing downstream knows how the motion was made.
  • Look at the take before training. 
    film
    comes before 
    dataset
    and 
    train
    : a take whose halves drift apart is rerolled for a few minutes of filming, not discovered after the reader.
  • Translation is part of the clip. The solve moves Hips; 
    Root
    stays at rest until root motion is split off downstream.
  • Plant once, measure everywhere. A plant is the shared definition (sole under 4 cm, under 0.15 m/s ground-relative, 3 frames) or a contact probability ≥ 0.5. The sole is the joint's height less its rest height above the ground (
    sole_offsets
    ), and the bake and the gate read the same plant.
  • A pop the video has is not the solve's. 
    motion_metrics
    reports a spike the two-view readings also show as 
    pops_in_source
    , not gated; only a spike the fit added counts in 
    pops
    . Both count one event per frame and joint, so a twist bone beside the shin is not a second pop. Jitter is gated over 1.2 x the readings' on the same frames; unread fingers take their hand's. 
    solve.py --despike
    replaces a one- or two-frame pop the fit added with a slerp of its neighbours and lists what it changed.
  • Hands are fixed. Two views do not resolve fingers; the clip ships with a hand pose layered on (
    reference/hands.md
    ).
  • The licence travels by link. The video carries 
    licence
    ; the clip links to it with 
    depends-on
    , so a local H3 video's exclusion (
    minimax-h3-local
    : US, EU, UK, KR) reaches every clip solved from it.
  • 单一骨架。 解算结果直接拟合到 contract rig 本身；下游无需了解动作的制作方式。
  • 训练前先检查 take。 
    film
    位于 
    dataset
    和 
    train
    之前：如果 take 的前后两部分逐渐错位，应花费几分钟重新拍摄，而不是等到 reader 训练完成后才发现问题。
  • 位移是片段的一部分。 解算会移动 Hips；在下游分离 root motion 之前，Root 保持静止。
  • 只生成一次 plant，并在所有位置统一测量。 plant 的统一定义为：脚底低于 4 cm、相对地面的速度低于 0.15 m/s，并持续 3 帧；或者接触概率 ≥ 0.5。脚底高度等于关节高度减去其静止时离地的高度（
    sole_offsets
    ）；bake 和 gate 读取同一份 plant 数据。
  • 视频中已有的跳变不属于解算问题。 如果双视图读数也将某个尖峰显示为 
    pops_in_source
    ，则 
    motion_metrics
    不会将其纳入 gate；只有拟合引入的尖峰才计入 
    pops
    。两者对每帧、每个关节都只计为一次事件，因此胫骨旁的扭转骨骼不会被计为第二次跳变。抖动 gate 的阈值为相同帧上读数抖动的 1.2 倍；无法读取手指时，使用手部的抖动值。
    solve.py --despike
    会用相邻姿态的 slerp 替换拟合引入的一帧或两帧跳变，并列出所做的修改。
  • 手部姿态固定。 两个视图无法解析手指；片段交付时会叠加固定 hand pose（
    reference/hands.md
    ）。
  • licence 通过链接传递。 视频携带 
    licence
    ；片段通过 
    depends-on
    链接到视频，因此本地 H3 视频的排除限制（
    minimax-h3-local
    ：美国、欧盟、英国、韩国）会传递到由该视频解算出的每个片段。

Scope — what this skill does NOT do

范围——此 skill 不做什么

  • Installing models or GPU environments: the user does it.
  • Rigging the character onto the contract: 
    character-autorig
    (Auto-Rig Pro), or 
    blender-character-rig
    by hand.
  • Retarget internals and pack clips (conjugation, 
    k
    , in place, the bake-off): 
    blender-retarget
    .
  • Generating video (the ComfyUI server, H3 templates, the queue): 
    comfyui-asset-generation
    .
  • Unity import, Avatars and masks: 
    unity-rigged-character
    ; runtime foot IK and blending: 
    unity-animation
    .
  • Text-to-motion: not in this release.
  • 2D sprite loops from video: 
    sprite-from-video
    .
  • 安装模型或 GPU 环境：由用户完成。
  • 将角色绑定到 contract：
    character-autorig
    （Auto-Rig Pro），或使用 
    blender-character-rig
    手动绑定。
  • Retarget 内部处理和片段包（conjugation、
    k
    、原地处理、bake-off）：
    blender-retarget
    。
  • 生成视频（ComfyUI server、H3 templates、队列）：
    comfyui-asset-generation
    。
  • Unity 导入、Avatar 和遮罩：
    unity-rigged-character
    ；运行时脚部 IK 和混合：
    unity-animation
    。
  • Text-to-motion：此版本不支持。
  • 从视频生成 2D sprite 循环：
    sprite-from-video
    。