caption-animation

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Caption Animation

字幕动画

Turn a transcript or voiceover into word-timed, scroll-stopping captions: each word pops in on the syllable, the active word highlights, and the text stays glued to the narration. Build the kind of captions that drive watch-time on TikTok, Reels, and Shorts — readable, on-beat, and inside the safe area.
将转录文本或旁白转换为带逐词时间轴、吸引用户停留的字幕:每个单词会在对应音节时弹出,当前词高亮显示,文本与旁白完全同步。制作出能提升TikTok、Reels和Shorts观看时长的字幕——易读、踩点精准且位于安全区域内。

When to use

使用场景

  • Word-by-word or karaoke captions on short-form vertical video (9:16).
  • Burning open captions onto a voiceover/narration track from a transcript.
  • Turning a Whisper/SRT transcript into animated text in Remotion or on the web.
  • Restyling auto-generated subtitles into a branded, "Hormozi-style" reveal.
  • 短视频竖屏(9:16)中的逐词字幕或卡拉OK字幕。
  • 将转录文本中的旁白/配音轨道转换为嵌入式开放字幕。
  • 在Remotion或网页中将Whisper/SRT转录文本转换为动画文字。
  • 将自动生成的字幕重新设计为品牌化的「Hormozi风格」展示效果。

The pipeline

流程管线

StageJobTool
TranscribeAudio → word-level timestampsWhisper (
@remotion/install-whisper-cpp
), AssemblyAI
Normalize
{ text, startMs, endMs }[]
tokens
@remotion/captions
Caption
type
PageGroup words into 1–4 word "pages"
createTikTokStyleCaptions()
AnimatePer-word pop + active-word highlightRemotion
spring()
/ CSS
PlaceSafe-area, readable typelayout rules below
ExportBurn-in (MP4) and/or sidecar SRT/VTTRemotion render / file emit
阶段任务工具
转录音频 → 逐词时间戳Whisper (
@remotion/install-whisper-cpp
), AssemblyAI
标准化
{ text, startMs, endMs }[]
令牌格式
@remotion/captions
Caption
类型
分页将单词分组为1–4词的「页面」
createTikTokStyleCaptions()
动画逐词弹出 + 当前词高亮Remotion
spring()
/ CSS
布局安全区域、易读字体下方布局规则
导出嵌入字幕(MP4)和/或外挂SRT/VTT字幕Remotion渲染 / 文件输出

Two non-negotiable rules

两条不可妥协的规则

  1. Word-level timing, not line-level. Karaoke reads as magic only when each word lands on the syllable. Always transcribe to word timestamps; never fake them by splitting a line evenly over its duration — drift is instantly visible.
  2. Readability beats style. Captions are read on a phone, in sunlight, muted. Bold sans-serif, heavy stroke or shadow, high contrast first; decoration second. A caption nobody can read is decoration, not a caption.
  1. 逐词时间轴,而非逐行。 卡拉OK字幕的「魔力」仅在于每个单词精准对应音节出现。始终转录到逐词时间戳;绝不要通过将一行文本平均拆分到其时长来伪造时间戳——时间偏移会立刻被察觉。
  2. 可读性优先于样式。 字幕是在手机上、阳光下、静音状态下阅读的。先保证粗体无衬线字体、粗描边或阴影、高对比度;装饰性效果放在其次。没人能读懂的字幕只是装饰,而非真正的字幕。

Word timing from a transcript

基于转录文本的逐词时间轴

Whisper emits word-level timestamps directly. Normalize every source into one flat token shape so the renderer never cares where the words came from:
ts
// Caption token — the one shape everything downstream consumes
type Token = { text: string; startMs: number; endMs: number };

// Parse an SRT cue block "00:00:01,200 --> 00:00:01,640" into ms
const toMs = (t: string) => {
  const [h, m, rest] = t.split(":");
  const [s, ms] = rest.split(",");
  return ((+h * 60 + +m) * 60 + +s) * 1000 + +ms;
};
If only sentence-level cues exist, re-transcribe for word timing — do not interpolate. Whisper's
medium.en
is ~2x faster than
large
and accurate enough for clean voice audio.
Whisper可直接输出逐词时间戳。将所有来源标准化为统一的令牌格式,这样渲染器无需关心单词的来源:
ts
// 字幕令牌——下游所有模块使用的统一格式
type Token = { text: string; startMs: number; endMs: number };

// 将SRT时间块 "00:00:01,200 --> 00:00:01,640" 转换为毫秒
const toMs = (t: string) => {
  const [h, m, rest] = t.split(":");
  const [s, ms] = rest.split(",");
  return ((+h * 60 + +m) * 60 + +s) * 1000 + +ms;
};
如果只有句子级别的时间提示,重新转录以获取逐词时间轴——不要插值。Whisper的
medium.en
模型速度比
large
快约2倍,对于清晰的语音音频足够准确。

Per-word pop-in (Remotion)

逐词弹出动画(Remotion)

Drive each word's scale and opacity off a spring anchored to its own
startMs
. The micro-overshoot is what makes it feel "alive".
tsx
import { useCurrentFrame, useVideoConfig, spring, interpolate } from "remotion";

const Word: React.FC<{ token: Token; active: boolean }> = ({ token, active }) => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();
  const enter = (token.startMs / 1000) * fps;            // word's own entrance frame
  const p = spring({ frame: frame - enter, fps, config: { damping: 12, mass: 0.6 } });
  const scale = interpolate(p, [0, 1], [0.6, 1]);        // pop from 60% → 100% (overshoots)
  return (
    <span style={{
      display: "inline-block",
      transform: `scale(${scale})`,
      opacity: interpolate(p, [0, 1], [0, 1]),
      color: active ? "#FFE45E" : "#FFFFFF",             // active-word highlight
      transition: "color 80ms linear",
    }}>{token.text}&nbsp;</span>
  );
};
Compute
active
by testing
frame
against each token's
[startMs, endMs]
. Keep the highlight a single accent color or a filled "box" behind the live word — never animate every word's color at once.
基于每个单词自身的
startMs
,通过弹簧动画驱动单词的缩放和透明度。微小的过冲效果会让字幕显得「生动」。
tsx
import { useCurrentFrame, useVideoConfig, spring, interpolate } from "remotion";

const Word: React.FC<{ token: Token; active: boolean }> = ({ token, active }) => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();
  const enter = (token.startMs / 1000) * fps;            // 单词的入场帧
  const p = spring({ frame: frame - enter, fps, config: { damping: 12, mass: 0.6 } });
  const scale = interpolate(p, [0, 1], [0.6, 1]);        // 从60%缩放至100%(带过冲)
  return (
    <span style={{
      display: "inline-block",
      transform: `scale(${scale})`,
      opacity: interpolate(p, [0, 1], [0, 1]),
      color: active ? "#FFE45E" : "#FFFFFF",             // 当前词高亮
      transition: "color 80ms linear",
    }}>{token.text}&nbsp;</span>
  );
};
通过检测
frame
是否在每个令牌的
[startMs, endMs]
范围内来计算
active
状态。高亮效果使用单一强调色或当前单词后的填充「背景框」——不要同时动画所有单词的颜色。

Pages, not walls of text

分页展示,避免文本堆砌

Show 1–4 words at a time. Use Remotion's
createTikTokStyleCaptions()
(or group manually) and tune
combineTokensWithinMilliseconds
: ~200–500ms for true word-by-word, ~1000–1500ms for short readable phrases.
ts
import { createTikTokStyleCaptions } from "@remotion/captions";
const { pages } = createTikTokStyleCaptions({
  captions, combineTokensWithinMilliseconds: 1200,       // ~2–3 words per page
});
每次显示1–4个单词。使用Remotion的
createTikTokStyleCaptions()
(或手动分组),并调整
combineTokensWithinMilliseconds
:逐词显示设置为200–500ms,短短语显示设置为1000–1500ms。
ts
import { createTikTokStyleCaptions } from "@remotion/captions";
const { pages } = createTikTokStyleCaptions({
  captions, combineTokensWithinMilliseconds: 1200,       // 每页约2–3个单词
});

Web/CSS pop (no Remotion)

网页/CSS弹出动画(无需Remotion)

For a DOM/live player, give each word its own
animation-delay
equal to
startMs
:
css
.word { display:inline-block; opacity:0; animation: pop .26s cubic-bezier(.34,1.56,.64,1) forwards; }
@keyframes pop { from { opacity:0; transform:translateY(.18em) scale(.7) } to { opacity:1; transform:none } }
.word.active { color:#FFE45E; }
js
words.forEach(w => { const el = mk(w.text); el.style.animationDelay = `${w.startMs}ms`; track.append(el); });
对于DOM/实时播放器,为每个单词设置等于
startMs
animation-delay
css
.word { display:inline-block; opacity:0; animation: pop .26s cubic-bezier(.34,1.56,.64,1) forwards; }
@keyframes pop { from { opacity:0; transform:translateY(.18em) scale(.7) } to { opacity:1; transform:none } }
.word.active { color:#FFE45E; }
js
words.forEach(w => { const el = mk(w.text); el.style.animationDelay = `${w.startMs}ms`; track.append(el); });

Readable type and safe-area placement (9:16, 1080×1920)

易读字体与安全区域布局(9:16,1080×1920)

PropertyValueWhy
FontBold/ExtraBold sans (Montserrat, Inter, Helvetica)Reads on small, busy screens
Size56–80px (≈8% of frame height), min 45pxLegible muted on a phone
Stroke2–6px solid black outlineSurvives any background
ShadowSoft drop shadow as backup to strokeSeparation on bright frames
CaseUppercase or sentence; high contrast fillPunch + scannability
Vertical posCenter band, ~62–70% downAbove the UI, below the action
Bottom safeKeep clear of bottom ~280px / 15%Avoids caption/CTA/audio UI
Side safeKeep within center 80% widthAvoids right-rail icons
Place captions in the center band — not pinned to the very bottom, where the platform UI (username, audio, buttons) lives. Pad text to a max of ~2 lines.
属性原因
字体粗体/特粗体无衬线字体(Montserrat, Inter, Helvetica)在小屏、复杂画面上清晰可读
字号56–80px(约为帧高度的8%),最小45px手机静音状态下仍清晰可见
描边2–6px 黑色实心描边在任何背景下都能清晰显示
阴影柔和投影作为描边的补充在亮色画面上与背景分离
大小写全大写或句子大小写;高对比度填充醒目且易于快速阅读
垂直位置中间区域,约为帧高度的62–70%处位于UI上方、画面主体下方
底部安全区避开底部约280px / 15%的区域避免与字幕/CTA/音频UI重叠
侧边安全区保持在中间80%的宽度范围内避开右侧操作栏图标
将字幕放在中间区域——不要固定在最底部,因为平台UI(用户名、音频、按钮)位于底部。文本最多限制为2行。

Sync to a voiceover / narration

与旁白/配音同步

The transcript already carries the timing of the exact audio. Lay that same audio under the composition and keep timestamps in the audio's timebase — captions and voice stay locked with zero manual nudging. If the VO is re-recorded, re-transcribe; never hand-shift offsets.
转录文本已包含对应音频的精确时间轴。将同一音频放在合成轨道下,并保持时间戳与音频时间基准一致——字幕与语音会完全锁定,无需手动调整偏移。如果旁白重新录制,重新转录;绝不要手动调整时间偏移。

Burn-in vs sidecar SRT/VTT

嵌入字幕 vs 外挂SRT/VTT字幕

Burn-in (open)Sidecar SRT/VTT (closed)
WherePixels in the MP4Separate
.srt
/
.vtt
file
Social (TikTok/Reels)Required — guaranteed, styleableOften ignored by the platform
Accessibility (ADA/WCAG)Does not satisfy aloneRequired (toggleable)
Best practiceBurn animated captions for socialAlso ship a sidecar for the web/SEO
Emit both: burn the animated captions for social, and write a plain SRT/VTT from the same tokens for accessible/SEO playback.
嵌入字幕(开放字幕)外挂SRT/VTT字幕(闭合字幕)
存在形式MP4中的像素内容独立的
.srt
/
.vtt
文件
社交平台(TikTok/Reels)必填——显示效果有保障且可自定义样式常被平台忽略
无障碍合规(ADA/WCAG)无法单独满足合规要求必填(可切换)
最佳实践为社交平台制作嵌入动画字幕同时提供外挂字幕用于网页/SEO
同时输出两种格式:为社交平台生成嵌入动画字幕,从同一令牌生成纯文本SRT/VTT字幕用于无障碍/SEO播放。

Output checklist

输出检查清单

  • Word-level timestamps (real, from transcription) — no even-split fakery.
  • Per-word pop with micro-overshoot; one accent color for the active word.
  • 1–4 words per page; never a wall of text.
  • Bold sans, 56–80px, 2–6px stroke + shadow, high contrast.
  • Center band, clear of bottom ~280px and side rails.
  • Audio laid under composition; timestamps in the audio timebase.
  • Ship burn-in for social and a sidecar SRT/VTT for accessibility.
  • 逐词时间戳(来自真实转录,而非平均拆分伪造)。
  • 带微小过冲的逐词弹出效果;当前词使用单一强调色高亮。
  • 每页1–4个单词;绝不要文本堆砌。
  • 粗体无衬线字体,56–80px字号,2–6px描边+阴影,高对比度。
  • 位于中间区域,避开底部约280px区域和侧边栏。
  • 音频置于合成轨道下;时间戳基于音频时间基准。
  • 为社交平台提供嵌入字幕,为无障碍需求提供外挂SRT/VTT字幕。

Deliver & verify (rendered stills → MP4)

交付与验证(渲染静帧 → MP4)

Packaged helper (
scripts/
): tile your stills with
scripts/contact-sheet.sh sheet.png f-hook.png f-mid.png f-end.png
, then assert the encode with
scripts/probe-mp4.sh out.mp4 [WxH] [fps]
. See
scripts/README.md
.
Captions ship as a Remotion composition (
<Composition>
+ zod
schema
+
defaultProps
) — all word motion frame-driven off
useCurrentFrame()
, never
Date.now()
/
Math.random()
/ timers. Deliverable =
out/*.mp4
(burned-in) + the project + the sidecar SRT/VTT. 9:16 vertical (1080×1920) is the default.
Verify loop — render stills → inspect → encode. Word timing is the thing that breaks; check it at exact frames before you spend an encode.
bash
undefined
打包工具
scripts/
):使用
scripts/contact-sheet.sh sheet.png f-hook.png f-mid.png f-end.png
将静帧拼接成预览图,然后使用
scripts/probe-mp4.sh out.mp4 [WxH] [fps]
验证编码。详见
scripts/README.md
字幕以Remotion合成组件(
<Composition>
+ zod
schema
+
defaultProps
)交付——所有单词动画均基于
useCurrentFrame()
驱动,绝不要使用
Date.now()
/
Math.random()
/ 定时器。交付物包括
out/*.mp4
(嵌入字幕)、项目文件以及外挂SRT/VTT字幕。默认格式为9:16竖屏(1080×1920)。
验证流程——渲染静帧 → 检查 → 编码。 逐词时间轴是最容易出错的部分;在编码前检查关键帧的时间准确性。
bash
undefined

Stills at start / a sampled active-word frame / end — WITH SHIPPED PROPS (real tokens + audio)

渲染起始帧、采样的当前词高亮帧、结束帧的静帧——使用真实属性(真实令牌 + 音频)

npx remotion still Captions out/f-start.png --frame=0 --props='{"captionsSrc":"vo.json"}' npx remotion still Captions out/f-mid.png --frame=90 --props='{"captionsSrc":"vo.json"}' npx remotion still Captions out/f-end.png --frame=N --props='{"captionsSrc":"vo.json"}' # N = durationInFrames-1
npx remotion still Captions out/f-start.png --frame=0 --props='{"captionsSrc":"vo.json"}' npx remotion still Captions out/f-mid.png --frame=90 --props='{"captionsSrc":"vo.json"}' npx remotion still Captions out/f-end.png --frame=N --props='{"captionsSrc":"vo.json"}' # N = durationInFrames-1

Inspect each PNG:

检查每张PNG:

- the word highlighted at frame 90 is the word whose [startMs,endMs] contains 90/fps (no drift)

- 第90帧高亮的单词,其 [startMs,endMs] 应包含 90/fps(无时间偏移)

- burn-in legible: bold sans, stroke+shadow holds, no clipping

- 嵌入字幕清晰可读:粗体无衬线字体,描边+阴影有效,无裁切

- 9:16 safe area: caption sits in the center band, clear of top ~12% and bottom ~20-35% (captions/CTA/audio UI) and the right action rail

- 9:16安全区域:字幕位于中间区域,避开顶部约12%、底部约20-35%(字幕/CTA/音频UI)和右侧操作栏

npx remotion render Captions out/captions.mp4 --props='{"captionsSrc":"vo.json"}' # encode once stills are right npx remotion render Captions out/demo.gif --codec=gif # README proof clip

Use `npx remotion compositions` to read `durationInFrames`/`fps` and pick the active-word + end frames.

**Before you finish:**
1. Stills render cleanly at frame 0, a mid active-word frame, and last — no missing font/audio.
2. The correct word is highlighted at the sampled frame (frame/fps lands inside its token); no even-split fakery.
3. Burn-in is legible (stroke+shadow) and the caption is fully inside the 9:16 safe area at every checked frame.
4. Frame-driven only — no `Date.now()` / `Math.random()` / timers.
5. Shipped props (real tokens, not just `defaultProps`) render correctly; MP4 + sidecar SRT/VTT emitted, GIF optional.
npx remotion render Captions out/captions.mp4 --props='{"captionsSrc":"vo.json"}' # 静帧检查无误后编码 npx remotion render Captions out/demo.gif --codec=gif # README演示动图

使用 `npx remotion compositions` 查看 `durationInFrames`/`fps`,选择当前词高亮帧和结束帧。

**完成前检查:**
1. 起始帧、中间当前词高亮帧、结束帧的静帧渲染正常,无字体/音频缺失。
2. 采样帧中高亮的单词正确(帧/fps落在对应令牌的时间范围内);无平均拆分伪造时间戳的情况。
3. 嵌入字幕清晰可读(描边+阴影有效),且在所有检查帧中完全位于9:16安全区域内。
4. 仅基于帧驱动——无 `Date.now()` / `Math.random()` / 定时器。
5. 使用真实属性(而非仅 `defaultProps`)可正确渲染;已输出MP4 + 外挂SRT/VTT字幕,GIF为可选。

Reference files

参考文件

  • references/word-timed-captions.md
    — end-to-end build: Whisper transcription and the
    Caption
    type, a full SRT parser, Remotion word-timed component with active-highlight, manual paging, an SRT/VTT emitter, per-platform safe-area maps, and a readable-type spec sheet.
  • references/word-timed-captions.md
    —— 端到端构建指南:Whisper转录与
    Caption
    类型、完整SRT解析器、带高亮的Remotion逐词时间轴组件、手动分页、SRT/VTT输出器、各平台安全区域映射,以及易读字体规格表。