edu-math-video

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

数学题讲解视频 (edu-math-video)

Math Problem Explainer Video (edu-math-video)

把一道数学题做成 16:9、1920×1080、带中文配音 + 中英双语字幕 + 手绘笔记本风格动画的 MP4。
流水线(每个视频一个文件夹):
script.json ──build_audio.py──► build/timeline.json + build/mix.wav + ../<name>.srt
     (旁白)     (GLM-TTS + 合成音乐)          │
anim.js + engine.js ◄─────────────────────────┘ (按旁白时间点驱动动画)
     └──node render.mjs video──► ../<name>.mp4
核心原则:画面跟着旁白走,而且图会"讲题"。 每句旁白,图上都有一个动作把这句推理演出来(线段滑过去重合、三角形叠上去、相机转到俯视、动点移动……),不是只多出一条线、一个标签。每个动画都用"第 k 句旁白开始的时刻"
S.at(k, f)
定时,绝不写死秒数。
本技能自带全部代码,不要从别的项目复制文件:
  • template/
    :可直接运行的完整示例项目(直角三角形斜边中线题)。
    engine.js
    是通用引擎(不改),
    anim.js
    /
    script.json
    /
    episode.json
    /
    problem.png
    是每道题都要重写的。
  • shared/pron.py
    +
    shared/pron.json
    :读音控制(GLM-TTS 没有 SSML/拼音输入)。
  • examples/cone-parallel/
    :立体几何完整示例(anim.js + storyboard.md + script.json):相机斜视↔俯视、侧面展开成扇形、内错角旋转、平面平移。做立体几何、圆、多问题时先读它。
  • scripts/
    :
    new_video.sh
    (建项目)、
    setup_check.sh
    (查环境)、
    make_problem_png.py
    (文字题→图片+高亮框)、
    problem_boxes.py
    (截图→高亮框坐标)、
    contact_sheet.py
    (截图拼图审查)。
下文
SKILL
= 本文件所在目录,
WS
= 用户当前所在的目录(你的工作目录,
$PWD
)
:视频文件夹建在这里,mp4/srt 也输出到这里。不要因为别的目录(包括本技能所在的项目)已经有
.env
、
node_modules
、
pron.json
就把 WS 换过去
,缺什么由
new_video.sh
和第 2 步补齐。
PROJ
= 本视频文件夹,
PY
=
setup_check.sh
报告的 python(macOS 上一般是
/usr/bin/python3
;PATH 里的
python3
可能没有 numpy)。
Turn a math problem into a 16:9, 1920×1080 MP4 with Chinese voice-over, bilingual Chinese-English subtitles, and hand-drawn notebook-style animation.
Pipeline (one folder per video):
script.json ──build_audio.py──► build/timeline.json + build/mix.wav + ../<name>.srt
     (旁白)     (GLM-TTS + 合成音乐)          │
anim.js + engine.js ◄─────────────────────────┘ (按旁白时间点驱动动画)
     └──node render.mjs video──► ../<name>.mp4
Core principle: the visuals follow the voice-over, and the graphics themselves “explain the problem.” For every narration line, an on-screen action performs the reasoning expressed by that line (a segment slides over until it overlaps, a triangle is overlaid, the camera rotates to a top-down view, a moving point travels, and so on); it is not enough to merely add one more line or label. Every animation is scheduled using “the moment narration line k begins,”
S.at(k, f)
, and never uses hard-coded seconds.
This skill includes all the code; do not copy files from another project:
  • template/
    : a complete, directly runnable example project for the problem about the median to the hypotenuse of a right triangle.
    engine.js
    is the general-purpose engine and must remain unchanged, while
    anim.js
    ,
    script.json
    ,
    episode.json
    , and
    problem.png
    must be rewritten for every problem.
  • shared/pron.py
    +
    shared/pron.json
    : pronunciation control, because GLM-TTS does not accept SSML or pinyin input.
  • examples/cone-parallel/
    : a complete solid-geometry example containing
    anim.js
    ,
    storyboard.md
    , and
    script.json
    : an angled camera view transitioning to a top-down view, a side unfolding into a sector, rotation to illustrate alternate interior angles, and translation across a plane. Read this first when working on solid geometry, circles, or multiple problems.
  • scripts/
    :
    new_video.sh
    (create a project),
    setup_check.sh
    (check the environment),
    make_problem_png.py
    (convert a text problem into an image with highlighted boxes),
    problem_boxes.py
    (extract highlighted-box coordinates from a screenshot), and
    contact_sheet.py
    (create a contact sheet for screenshot review).
Below,
SKILL
= the directory containing this file, and
WS
= the user's current directory (your working directory,
$PWD
)
. The video folder is created here, and the MP4 and SRT files are also output here. Do not switch to a different directory merely because another directory, including the project containing this skill, already has
.env
,
node_modules
, or
pron.json
.
Fill any gaps with
new_video.sh
and Step 2.
PROJ
= the folder for this video, and
PY
= the Python executable reported by
setup_check.sh
(on macOS this is generally
/usr/bin/python3
; the
python3
in PATH may not have numpy).

必须遵守的规则

Mandatory Rules

退出码不是 0 就是没完成。
--check
退出码 1(哪怕只剩一个 ⚠ 多音字)、
motion
退出码 1、截图里有问题,都不能写"只是警告、不影响"然后继续。修好再往下走。
  1. 严格按下面 10 步顺序做,每步的"通过标准"满足了才能进入下一步。 不要跳步,不要"最后一起检查"。
  2. 先讲清楚题目。 视频第一幕必须展示原题图片(截图或
    make_problem_png.py
    生成的图),旁白逐条读条件,每读一条就在图上框出那一条。
  3. tts
    字段只能是能念出来的中文。
    不能有阿拉伯数字、
    = + − × ÷ / ^ √ ∠ △ ∥ ° ( )
    等符号:
    AC=3
    写成
    AC等于三
    ,
    x²
    写成
    x的平方
    。字幕
    zh
    字段保留正常数学写法。
    --check
    会报错拦住。
  4. 读音必须固定。 调 TTS 之前
    --check
    必须显示
    未固定读音的生僻字/多音字: 0
    且
    script errors: 0
    。多音字的标注写在该字后面:
    长[cháng]
    、
    要[yào]求
    。不能写成
    中点[zhōng]
    (那会把"点"读成 zhōng)。
  5. API Key 只能由用户提供。 缺
    GLM_API_KEY
    时先问用户:配置 key(音质最好,推荐),还是用兜底引擎(edge-tts / macOS
    say
    ,见 reference/glm-tts-setup.md)。不要编造 key、模型名或接口地址,不要把 key 打印出来或写进项目文件夹。
  6. 所有画面内容在 y ≤ 860 以内(y≈9141044 是字幕框)。图形放左半边 x 60900,推导文字写在右边的横线板
    board()
    上。
  7. 先写分镜,再写动画。
    storyboard.md
    给每句旁白规划"指/动/留",按 reference/visual-design.md 的动作表选动作;
    node render.mjs motion
    必须通过。
  8. 先预览再花钱。 用
    --preview
    +
    stills auto
    检查版面,没问题再跑真正的 TTS。
  9. 看图验证。 每次渲染截图后都要打开
    build/sheet.png
    亲眼检查(用读图工具),不能只看命令有没有报错。汇报时写你实际看到了什么,不要凭印象全部打勾。
  10. 不要修改
    engine.js
    ,除非用户要求换风格;需要新图形就在
    anim.js
    里写函数。
  11. 不要把
    node_modules
    、
    package*.json
    复制进视频文件夹
    ,依赖在
    WS
    根目录装一次,所有视频共享。
An exit code other than 0 means the task is incomplete. A
--check
exit code of 1, even if only one ⚠ polyphonic-character warning remains, a
motion
exit code of 1, or any problem visible in the screenshots means the work is not finished. Do not claim that an issue is merely a warning and does not affect the result. Fix it before continuing.
  1. Strictly follow the 10 steps below in order. Each step's acceptance criteria must be satisfied before proceeding to the next step. Do not skip steps or defer all checks until the end.
  2. Explain the problem clearly first. The first scene of the video must display the original problem image, whether it is a screenshot or generated by
    make_problem_png.py
    . The voice-over must read each condition one at a time, and the corresponding condition must be boxed on the image as it is read.
  3. The
    tts
    field may contain only Chinese text that can actually be spoken.
    It must not contain Arabic numerals or symbols such as
    = + − × ÷ / ^ √ ∠ △ ∥ ° ( )
    . Write
    AC=3
    as
    AC等于三
    , and
    x²
    as
    x的平方
    . The subtitle's
    zh
    field retains normal mathematical notation.
    --check
    will report an error and block the workflow.
  4. Pronunciation must be deterministic. Before invoking TTS,
    --check
    must show
    未固定读音的生僻字/多音字: 0
    and
    script errors: 0
    . A polyphonic-character annotation must appear immediately after the relevant character:
    长[cháng]
    ,
    要[yào]求
    . Do not write
    中点[zhōng]
    , because that causes 点 to be pronounced as zhōng.
  5. Only the user may provide the API key. If
    GLM_API_KEY
    is missing, first ask the user whether they want to configure a key for the best audio quality, which is recommended, or use a fallback engine such as edge-tts or macOS
    say
    ; see reference/glm-tts-setup.md. Do not invent a key, model name, or endpoint. Do not print the key or write it into the project folder.
  6. All visual content must remain within y ≤ 860. The area around y ≈ 914–1044 is reserved for subtitles. Place graphics in the left half at x 60–900, and write derivations on the ruled board on the right using
    board()
    .
  7. Create the storyboard before the animation. In
    storyboard.md
    , plan a “point/move/retain” action for every narration line and select actions from the table in reference/visual-design.md.
    node render.mjs motion
    must pass.
  8. Preview before spending money. Use
    --preview
    together with
    stills auto
    to check the layout, and run the real TTS only after the preview is correct.
  9. Verify the rendered images visually. After every rendering pass, open
    build/sheet.png
    and inspect it with an image-reading tool. Do not rely only on whether the command reported an error. When reporting the results, describe what you actually saw rather than marking every item as passed based on assumptions.
  10. Do not modify
    engine.js
    unless the user requests a different visual style. Implement new graphics as functions in
    anim.js
    .
  11. Do not copy
    node_modules
    or
    package*.json
    into the video folder.
    Install dependencies once at the
    WS
    root and share them across videos.

11 个步骤

The 11 Steps

第 1 步:弄清题目,自己先把题解对

Step 1: Understand the Problem and Solve It Correctly First

  • 拿到题目文字(有截图就先读图)。先自己完整解一遍并验算,写下:已知条件清单、所求、关键思路(1~2 个)、每一步计算、最终答案、验算方法。
  • 答案不确定或题目看不清时,问用户,不要猜。
  • 通过标准: 你能用 3~6 个"幕"讲完,每一幕一句话说清目的;答案已验算。
  • Obtain the problem text; if a screenshot is provided, read it first. Solve the entire problem yourself and verify the result, then record the list of given conditions, the target, the key idea or ideas (1–2), every calculation, the final answer, and the verification method.
  • If the answer is uncertain or the problem is unreadable, ask the user rather than guessing.
  • Acceptance criteria: You can explain the problem in 3–6 “scenes,” with one sentence per scene explaining its purpose, and the answer has been verified.

第 2 步:建项目、查环境

Step 2: Create the Project and Check the Environment

bash
bash SKILL/scripts/new_video.sh "$PWD" <ascii_folder_name>     # WS 就是当前目录
bash SKILL/scripts/setup_check.sh WS/<ascii_folder_name>
  • new_video.sh
    会把
    pron.py
    /
    pron.json
    链接到技能自带的共享词表,并在找得到时把
    node_modules
    链接到已有安装(不复制);找不到才需要在
    WS
    下
    npm install
    。python 包用
    PY -m pip install --user numpy requests pypinyin pillow
    。
  • new_video.sh
    报
    ERROR: ... is where the skill is installed
    说明你把 WS 设成了技能所在的项目,改用当前目录。
  • 默认音色是
    chuichui
    (锤锤),用户可以在
    .env
    里用
    GLM_VOICE
    换。
  • 缺
    GLM_API_KEY
    :停下,按 reference/glm-tts-setup.md 引导用户配置(推荐放
    ~/.config/math-problem-video/.env
    ,所有目录通用),然后继续。不要去别的项目里找
    .env
    用,也不要因此换工作目录。
    用户没有 key 或不想配:
    TTS_ENGINE=auto
    (默认)会自动改用兜底引擎,
    setup_check.sh
    会显示用的是哪个;装 edge-tts 前先征得用户同意。
  • 通过标准:
    setup_check.sh
    输出
    ALL OK
    。
bash
bash SKILL/scripts/new_video.sh "$PWD" <ascii_folder_name>     # WS 就是当前目录
bash SKILL/scripts/setup_check.sh WS/<ascii_folder_name>
  • new_video.sh
    links
    pron.py
    and
    pron.json
    to the skill's shared pronunciation dictionary and links to an existing
    node_modules
    installation when one can be found instead of copying it. Run
    npm install
    under
    WS
    only if no suitable installation is found. Install the Python packages with
    PY -m pip install --user numpy requests pypinyin pillow
    .
  • If
    new_video.sh
    reports
    ERROR: ... is where the skill is installed
    , it means WS was set to the skill's own project. Use the current directory instead.
  • The default voice is
    chuichui
    (Chuichui). Users can change it with
    GLM_VOICE
    in
    .env
    .
  • If
    GLM_API_KEY
    is missing, stop and guide the user through the configuration described in reference/glm-tts-setup.md. The recommended location is
    ~/.config/math-problem-video/.env
    , which works across all directories. Then continue. Do not look for a
    .env
    file in another project, and do not change the working directory because the key is missing.
    If the user has no key or does not want to configure one,
    TTS_ENGINE=auto
    , which is the default, automatically switches to a fallback engine, and
    setup_check.sh
    reports which engine is in use. Ask for permission before installing edge-tts.
  • Acceptance criteria:
    setup_check.sh
    prints
    ALL OK
    .

第 3 步:测试 TTS(验证 key 和音色)

Step 3: Test TTS and Verify the Key and Voice

bash
cd PROJ && PY build_audio.py --say "你好,我们来看一道数学题。"
  • 成功会打印
    WROTE build/say_xxxx.wav
    和
    voice: ...
    (
    edge:
    /
    say:
    开头 = 兜底引擎)。401/403 = key 错或没余额,告诉用户,不要重试 10 次。
  • 第一次给这个用户做视频时,把这个 wav 路径告诉用户试听,确认音色(音色表见 glm-tts-setup.md)。
  • 通过标准: 生成了 wav。
bash
cd PROJ && PY build_audio.py --say "你好,我们来看一道数学题。"
  • Success prints
    WROTE build/say_xxxx.wav
    and
    voice: ...
    . A voice beginning with
    edge:
    or
    say:
    indicates a fallback engine. A 401/403 response means the key is invalid or has no remaining balance. Tell the user rather than retrying ten times.
  • When creating the first video for a user, provide the WAV path so the user can audition the voice. The voice list is available in glm-tts-setup.md.
  • Acceptance criteria: A WAV file is generated.

第 4 步:准备题目图片
problem.png
和高亮框

Step 4: Prepare
problem.png
and the Highlighted Boxes

二选一(细节见 reference/animation.md):
  • 只有文字:
    PY SKILL/scripts/make_problem_png.py --text "完整题目" --mark "条件1" "条件2" ... "所求" --out PROJ/problem.png
    ,它直接输出
    srcW/srcH/boxes
    。
  • 有截图:复制到
    PROJ/
    ,必要时
    problem_boxes.py crop
    裁掉无关部分并存为
    problem.png
    ;然后
    problem_boxes.py grid
    出坐标网格图,读出每个条件的像素框;再
    problem_boxes.py check
    画框确认。
  • 把尺寸和框填进
    anim.js
    顶部的
    PROBLEM
    。
  • 通过标准: 你打开了
    *.preview.png
    或
    *.check.png
    ,文字完整(没有 ☒/□ 方框),每个框都恰好罩住对应文字。
Choose one of the following methods; see reference/animation.md for details:
  • Text only:
    PY SKILL/scripts/make_problem_png.py --text "完整题目" --mark "条件1" "条件2" ... "所求" --out PROJ/problem.png
    . The script directly outputs
    srcW
    ,
    srcH
    , and
    boxes
    .
  • Screenshot provided: Copy it into
    PROJ/
    . If necessary, use
    problem_boxes.py crop
    to remove irrelevant content and save the result as
    problem.png
    . Then use
    problem_boxes.py grid
    to generate a coordinate-grid image and read the pixel box for each condition. Finally, use
    problem_boxes.py check
    to draw and verify the boxes.
  • Enter the dimensions and boxes at the top of
    anim.js
    in the
    PROBLEM
    object.
  • Acceptance criteria: You have opened the
    *.preview.png
    or
    *.check.png
    file, the text is complete with no ☒ or □ placeholder glyphs, and every box covers exactly the corresponding text.

第 5 步:写
episode.json
和
script.json

Step 5: Write
episode.json
and
script.json

按 reference/script-writing.md 写。要点:
  • episode.json
    :
    title
    (视频标题)、
    output_name
    (输出文件名,如
    斜边中线_Median_to_Hypotenuse
    )、
    pop_scenes
    (要加"啵"音效的幕)。
  • script.json
    :幕的数组,每幕
    {"scene": "英文id", "lines": [{"zh","tts","en"}, ...]}
    。第一幕
    intro
    读题,最后一幕
    outro
    回顾 + 报答案。
  • 每句旁白 ≤ 36 个汉字宽(否则字幕折两行),一句只讲一件事。
  • 通过标准: JSON 合法;每一幕的 id 都有计划好的画面。
Follow reference/script-writing.md. Key points:
  • episode.json
    :
    title
    (video title),
    output_name
    (output filename, such as
    斜边中线_Median_to_Hypotenuse
    ), and
    pop_scenes
    (scenes that should include the “pop” sound effect).
  • script.json
    : an array of scenes, where each scene has the form
    {"scene": "English ID", "lines": [{"zh","tts","en"}, ...]}
    . The first scene,
    intro
    , reads the problem, and the final scene,
    outro
    , reviews the solution and states the answer.
  • Each narration line must be no more than 36 Chinese-character widths wide; otherwise the subtitle wraps onto two lines. Each line should communicate only one idea.
  • Acceptance criteria: The JSON is valid, and every scene ID has a planned visual composition.

第 6 步:写分镜
storyboard.md

Step 6: Write the Storyboard
storyboard.md

先读 reference/visual-design.md,再按
template/storyboard.md
的格式给每句旁白写一行:
| 幕id 句号 | 旁白要点 | 指 | 动 | 留 | 板书 |
。
  • "指":这句提到的元素怎么点亮。"动":图上一个体现推理的动作,从动作表里选(相等→复制滑过去重合,全等→三角形叠上去,立体↔平面→相机转动,求长度→数字计数……)。"留":动作后留下的标记。
  • 除第一幕和最后一幕外,"动"不能空,也不能只写"出现/显示"。每幕至少一个连续运动。
  • 先想清楚:只看图、不看板子,观众能不能看懂这一步为什么成立?不能就换动作。
  • 先在 storyboard.md 顶部写图形清单:题目里所有的点、线段、平行/垂直/中点关系、已知长度,全部要画出来,位置按数据计算。
  • 通过标准: 每句都有一行;图形清单完整;你能说出每一幕里"最能让人看懂"的那个动作。
Read reference/visual-design.md first, then follow the format in
template/storyboard.md
and write one row per narration line:
| scene ID line number | narration point | point | move | retain | board notes |
.
  • “Point” describes how the element mentioned by the line is highlighted. “Move” describes one on-screen action that demonstrates the reasoning, selected from the action table (equal → copy and slide until it overlaps, congruent → overlay the triangles, solid ↔ planar → rotate the camera, find a length → count units, and so on). “Retain” describes the mark left after the action.
  • Except for the first and final scenes, “Move” must not be empty and must not merely say “appears” or “is displayed.” Every scene must contain at least one continuous motion.
  • First determine whether a viewer can understand why the step is valid by looking only at the graphics, without reading the board. If not, replace the action with a clearer one.
  • At the top of
    storyboard.md
    , write the graphics inventory: every point, segment, parallel/perpendicular/midpoint relationship, and given length in the problem must be drawn, with positions calculated from the data.
  • Acceptance criteria: Every narration line has a row, the graphics inventory is complete, and you can identify the action in each scene that makes that step easiest to understand.

第 7 步:读音与脚本检查,修到 0

Step 7: Check Pronunciation and the Script Until All Errors Are Resolved

bash
cd PROJ && PY build_audio.py --check ; cat build/pron_report.txt
  • ERROR
    行:按提示改
    script.json
    (通常是 tts 里有数字/符号,或 anim.js 缺
    SC.<id>
    ;第 8 步写完 anim.js 前后者会报错,属正常)。
  • ⚠ 多音/生僻
    行:按 reference/pronunciation.md 处理:该字后面加
    [拼音]
    ,或反复出现的词加进
    pron.json
    的
    words
    ,确认默认读音正确的词加进
    ok
    。
  • 通过标准:
    未固定读音的生僻字/多音字: 0
    且
    script errors: 0
    ,退出码 0。
bash
cd PROJ && PY build_audio.py --check ; cat build/pron_report.txt
  • For an
    ERROR
    line, follow the instructions to fix
    script.json
    . This is usually caused by digits or symbols in
    tts
    , or by a missing
    SC.<id>
    in
    anim.js
    . The latter is expected before Step 8 and is not an error at that point.
  • For a
    ⚠ 多音/生僻
    line, follow reference/pronunciation.md. Add
    [pinyin]
    after the character, or add recurring words to the
    words
    list in
    pron.json
    . Add words whose default pronunciation is already correct to
    ok
    .
  • Acceptance criteria:
    未固定读音的生僻字/多音字: 0
    ,
    script errors: 0
    , and the exit code is 0.

第 8 步:写
anim.js

Step 8: Write
anim.js

按 storyboard.md 逐句实现,API 见 reference/animation.md。先通读
template/anim.js
(示例)再动手,结构保持一致:
  • PROBLEM
    (第 4 步的数据)、
    TAGS
    (每幕左上角标签)、图形函数(如
    fig(st)
    )、
    SC.<id> = (lt, S) => {...}
    (每幕一个)。
  • 每个元素的出现时间 = 旁白提到它的时刻:
    S.P(S.at(k, f), 时长)
    。
  • 每句:
    glow(..., bump(lt, at(k)))
    点亮提到的元素 → 分镜里的动作(
    slideSeg
    、
    movePoly
    、
    camTween
    、动点、
    countTo
    …)→ 留下标记。
  • 右侧
    board()
    +
    bl(行号, 文字, S.at(k, f), lt)
    逐行写推导(与图同色),最后
    stamp()
    盖章给答案。
  • 通过标准:
    PY build_audio.py --check
    无 ERROR。
Implement the storyboard line by line, using the API in reference/animation.md. Read the complete example in
template/anim.js
before starting, and preserve the same structure:
  • PROBLEM
    (the data from Step 4),
    TAGS
    (the tag in the upper-left corner of each scene), graphics functions such as
    fig(st)
    , and
    SC.<id> = (lt, S) => {...}
    for each scene.
  • Each element's appearance time must match the moment the narration mentions it:
    S.P(S.at(k, f), duration)
    .
  • For every line: use
    glow(..., bump(lt, at(k)))
    to highlight the mentioned element, then perform the storyboard action, such as
    slideSeg
    ,
    movePoly
    ,
    camTween
    , moving a point, or
    countTo
    , and finally leave the resulting mark.
  • On the right, use
    board()
    and
    bl(line number, text, S.at(k, f), lt)
    to write the derivation line by line in the same color as the corresponding graphic. Finally, use
    stamp()
    to stamp the answer.
  • Acceptance criteria:
    PY build_audio.py --check
    reports no errors.

第 9 步:免费预览:动作检查 + 版面检查

Step 9: Perform a Free Preview and Check Motion and Layout

bash
cd PROJ && PY build_audio.py --preview && node render.mjs motion && node render.mjs stills auto && PY SKILL/scripts/contact_sheet.py .
  • motion
    在每句里取 6 帧,比较左半边图形区的变化:
    STATIC
    (没变化)必须修;每幕至少一句
    MOVE
    。不通过就回到分镜换更好的动作,不要只加一个无意义的晃动来凑数。
  • stills auto
    在每句旁白快结束时截一帧(此时这句该出现的东西都应在画面上)。命令输出
    FAILED: page error
    说明 anim.js 有 JS 错误,先修。
  • 打开
    build/sheet.png
    (多于 12 张时是
    sheet_1.png
    、
    sheet_2.png
    …)逐张检查,对照 reference/animation.md 的审查清单:没有重叠、没有出界、没有挡住字幕、每句提到的东西都画出来了、数值与解答一致。
  • 有问题 → 改 anim.js → 重跑本步。
  • 通过标准:
    motion check passed
    ;全部截图检查通过(在回复里逐幕说明你看到了什么)。
bash
cd PROJ && PY build_audio.py --preview && node render.mjs motion && node render.mjs stills auto && PY SKILL/scripts/contact_sheet.py .
  • motion
    takes six frames from every narration line and compares changes in the graphics area on the left. Every
    STATIC
    result, meaning no visual change, must be fixed. Every scene must contain at least one
    MOVE
    . If the check fails, return to the storyboard and choose a better action; do not add a meaningless shake merely to pass the check.
  • stills auto
    captures a frame near the end of each narration line, by which time everything introduced by that line should be visible. A
    FAILED: page error
    message indicates a JavaScript error in
    anim.js
    and must be fixed.
  • Open
    build/sheet.png
    and inspect it image by image. If there are more than 12 images, the files are named
    sheet_1.png
    ,
    sheet_2.png
    , and so on. Check each image against the reference/animation.md review checklist: there must be no overlaps, no out-of-bounds content, nothing blocking the subtitles, every mentioned element must be drawn, and all values must match the solution.
  • If there is a problem, edit
    anim.js
    and rerun this step.
  • Acceptance criteria: The output includes
    motion check passed
    , and every screenshot passes review. In the response, describe what was actually observed in each scene.

第 10 步:真正生成配音,自动听写核对,再检查一次画面

Step 10: Generate the Real Voice-over, Verify It with ASR, and Recheck the Visuals

bash
cd PROJ && PY build_audio.py && PY build_audio.py --asr && node render.mjs stills auto && PY SKILL/scripts/contact_sheet.py .
  • TTS 按文本缓存在
    build/tts/
    ,改了某句只会重新合成那一句。
  • --asr
    用智谱语音识别把每句配音转回文字,写入
    build/asr_report.txt
    (结果有缓存)。字母序列对不上的句子标 ✗,退出码 1:修改那句(见 pronunciation.md 的"字母"一节)后重跑这两条命令。报告里的汉字也要扫一遍,发现明显读错的字(不是同音字,也不是 ASR 把"三"写成"3"这种)就按第 7 步处理。
  • 真实时长与预览不同,重新看一遍 sheet。
  • 通过标准: 打印
    mix + srt written
    ;
    --asr
    显示
    字母读错的句子: 0
    (没有 GLM key 时显示
    ASR SKIPPED
    :交付时告诉用户字母读音没有机器核对,列出含点名的句子请用户重点听);截图检查通过。
bash
cd PROJ && PY build_audio.py && PY build_audio.py --asr && node render.mjs stills auto && PY SKILL/scripts/contact_sheet.py .
  • TTS output is cached in
    build/tts/
    , so only modified lines are synthesized again.
  • --asr
    uses Zhipu speech recognition to transcribe each narration line back into text and writes the results to
    build/asr_report.txt
    ; results are cached. A sentence whose letter sequence does not match is marked with ✗ and produces exit code 1. Modify that sentence according to the “letters” section of pronunciation.md, then rerun these two commands. Also review the Chinese characters in the report. If an obviously mispronounced character is found, and it is not merely a homophone or an ASR result such as converting 三 to 3, correct it according to Step 7.
  • The real duration may differ from the preview. Review the contact sheet again.
  • Acceptance criteria: The command prints
    mix + srt written
    ;
    --asr
    reports
    字母读错的句子: 0
    . Without a GLM key, it displays
    ASR SKIPPED
    ; in that case, tell the user during delivery that the letter pronunciation was not machine-verified and list the sentences containing names or other potentially ambiguous terms so the user can listen to them carefully.

第 11 步:渲染视频并交付

Step 11: Render and Deliver the Video

bash
cd PROJ && node render.mjs video 6      # 6 = 并行浏览器页数(不是帧率!帧率固定 30)
  • 输出
    WS/<output_name>.mp4
    和
    WS/<output_name>.srt
    (WS = 用户的当前目录)。2~3 分钟的视频约需几分钟。
  • 交付时告诉用户:mp4/srt 路径、时长、各幕内容;说明你无法听音频,请用户试听读音,特别列出你标注过读音的字。用户反馈读错时:加到
    pron.json
    或行内标注 → 再跑第 10、11 步(只会重合成改过的句子)。
bash
cd PROJ && node render.mjs video 6      # 6 = 并行浏览器页数(不是帧率!帧率固定 30)
  • This outputs
    WS/<output_name>.mp4
    and
    WS/<output_name>.srt
    , where
    WS
    is the user's current directory. A two- to three-minute video generally takes a few minutes to render.
  • During delivery, provide the MP4 and SRT paths, the duration, and a summary of every scene. Explain that you cannot listen to the audio and ask the user to audition the pronunciation. In particular, list every character whose pronunciation was manually annotated. If the user reports a mispronunciation, add it to
    pron.json
    or use an inline annotation, then rerun Steps 10 and 11. Only the modified lines will be synthesized again.

常见错误(真实发生过)

Common Mistakes That Have Actually Occurred

错误做法正确做法
当前目录没有
.env
/
node_modules
,就把项目建到有这些东西的别的目录(例如技能所在的项目)
WS 永远是用户当前目录;缺依赖由 new_video.sh 链接,缺 key 请用户配置
~/.config/math-problem-video/.env
从别的视频文件夹(化学 chem 等)复制代码用
new_video.sh
从本技能
template/
建项目
编造
GLM_MODEL=glm-4-voice
、
api.zhipuai.cn
等配置
模型只有
glm-tts
,地址已写在 build_audio.py;只需用户提供
GLM_API_KEY
,可选
GLM_VOICE
node render.mjs video 24
以为是 24fps
参数是并行数;预览版面用
stills auto
,不要反复渲染整段视频
中点[zhōng]
标注只作用于紧挨着的前一个字:
中[zhōng]点
(其实"中点"不需要标)
tts 里写
AC=3
、
60 − 7t
、
x²
AC等于三
、
六十减七t
、
x的平方
题目截图随便放在 (50,50) 400×400用
PROBLEM
+
problemCard()
自动适配到标题下方的卡片里
用绝对秒数定时
if (lt > 12.5)
S.at(k, f)
:第 k 句开始后 f 比例处
渲染完不看画面就交付每次都看
build/sheet.png
tts 里手动写
C、O
或
C,O
来分开字母
不用管:pron.py 自动把相邻字母拆成
C O
(实测逗号、顿号会多出停顿)
--check
还有 ⚠ 就调 TTS
修到 0;否则会花钱合成出错误读音
为了"保险"把常见词都加进
pron.json
或行内标注
只标会读错的字;替换成同音字会破坏断句(
直搅三搅形
),
pron.py
已经只在上下文会读错时才替换
一个分句二十多个字不加标点在换气处加逗号,分句 ≤ 18 字(
--check
会 warn),否则 TTS 自己在词中间停
题目图片里
x²
显示成
x☒
却没发现
审查时放大看文字;make_problem_png.py 已自动换字体
用编辑工具改 JSON 后出现弯引号
“
,解析失败
JSON 语法引号必须是英文
"
;见 troubleshooting.md
图一次画好,之后每句只多一条线/一个标签,解释全在板子上每句都有"指→动→留";相等就滑过去重合,立体↔俯视用
camTween
;
render.mjs motion
必须通过
需要俯视图时在旁边另画一张图同一个 3D 模型
camTween
转到 pitch = π/2
一句旁白塞三四个知识点一句一个点,画面同步出现一个元素
Incorrect approachCorrect approach
Create the project in another directory that already has
.env
or
node_modules
, such as the project containing the skill, simply because the current directory lacks them
WS is always the user's current directory. Use
new_video.sh
to link dependencies and ask the user to configure
~/.config/math-problem-video/.env
when a key is required
Copy code from another video folder, such as a chemistry projectCreate the project from this skill's
template/
with
new_video.sh
Invent settings such as
GLM_MODEL=glm-4-voice
or
api.zhipuai.cn
The only model is
glm-tts
, and the endpoint is already defined in build_audio.py. The user only needs to provide
GLM_API_KEY
;
GLM_VOICE
is optional
Assume
node render.mjs video 24
means 24 fps
The argument is the number of parallel processes. Use
stills auto
for layout previews, and do not repeatedly render the entire video
Write
中点[zhōng]
An annotation applies only to the immediately preceding character:
中[zhōng]点
. In fact, 中点 does not require an annotation
Write expressions such as
AC=3
,
60 − 7t
, or
x²
in
tts
Write
AC等于三
,
六十减七t
, and
x的平方
Place an arbitrary problem screenshot at (50,50) with a size of 400×400Use
PROBLEM
and
problemCard()
to fit it automatically into the card below the title
Schedule animation with an absolute timestamp such as
if (lt > 12.5)
Use
S.at(k, f)
, where
f
is a proportion of the time since narration line
k
began
Deliver the video without inspecting the rendered imagesInspect
build/sheet.png
after every rendering pass
Manually write
C、O
or
C,O
in
tts
to separate letters
This is unnecessary: pron.py automatically splits adjacent letters into
C O
. In testing, commas and enumeration commas introduce extra pauses
Invoke TTS while
--check
still contains a ⚠
Resolve it to zero; otherwise, the system may spend money synthesizing incorrect pronunciation
Add common words to
pron.json
or annotate them inline “just to be safe”
Annotate only characters that may be mispronounced. Replacing words with homophones damages phrasing, such as
直搅三搅形
; pron.py already replaces text only when the context would otherwise produce an incorrect pronunciation
Put more than about 20 characters in a clause without punctuationAdd commas at natural breathing points and keep clauses at 18 characters or fewer.
--check
will warn otherwise, and TTS may otherwise pause in the middle of a phrase
Fail to notice that
x²
in a problem image has become
x☒
Zoom in on the text during review. make_problem_png.py automatically switches fonts
Introduce curly quotation marks such as
“
into JSON after editing it, causing parsing to fail
JSON syntax requires straight English quotation marks (
"
); see troubleshooting.md
Draw everything at once and then add only one line or label per narration line, leaving all reasoning on the boardEvery line must contain point → move → retain. Slide equal elements until they overlap, use
camTween
for solid-to-top-view transitions, and ensure
render.mjs motion
passes
Draw a separate top-view diagram beside the original when a top view is neededRotate the same 3D model with
camTween
until pitch = π/2
Put three or four knowledge points into one narration lineUse one point per line and introduce one element in sync with the visuals

参考文件

Reference Files

  • reference/glm-tts-setup.md:智谱 GLM-TTS 注册、API Key、
    .env
    、音色、语速、报错处理。引导用户配置时读这个。
  • reference/script-writing.md:
    script.json
    格式、幕的设计、数学式子的口语写法对照表。
  • reference/pronunciation.md:读音控制原理、标注语法、
    pron.json
    、数学常见多音字。
  • reference/visual-design.md:讲解动画怎么设计:指→动→留→连、推理→动画动作表、立体转俯视、圆锥展开、分镜格式。写分镜前必读。
  • reference/animation.md:engine.js API、场景写法、版面坐标、几何/函数/立体/行程题画法、审查清单。
  • reference/troubleshooting.md:报错与处理。
  • reference/glm-tts-setup.md: Zhipu GLM-TTS registration, API key,
    .env
    , voices, speaking rate, and error handling. Read this when guiding the user through configuration.
  • reference/script-writing.md:
    script.json
    format, scene design, and a comparison table for converting mathematical notation into spoken language.
  • reference/pronunciation.md: pronunciation-control principles, annotation syntax,
    pron.json
    , and common mathematical polyphonic characters.
  • reference/visual-design.md: How to design instructional animation, including point → move → retain → connect, a table mapping reasoning to animation actions, solid-to-top-view transitions, cone unfolding, and the storyboard format. This file is required before creating a storyboard.
  • reference/animation.md: engine.js API, scene implementation, layout coordinates, drawing methods for geometry, functions, solid geometry, and motion/distance problems, and the review checklist.
  • reference/troubleshooting.md: errors and solutions.