audiogram

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Audiogram

音频可视化视频（Audiogram）

Turn an audio clip — a podcast highlight, a voiceover, a quote — into a short, captioned video that stops the scroll in a muted feed. The moving element (a waveform or equalizer bars) signals "there is sound here," the captions carry the message for the 80% who watch on mute, and cover art + title + a progress bar give it a finished, branded frame.

将一段音频片段——比如播客精彩片段、旁白、语录——制作成带字幕的短视频，在静音信息流中抓住用户注意力。动态元素（波形或均衡器条）传递「此处包含声音」的信号，字幕为80%静音观看的用户传递信息，封面图+标题+进度条则为视频打造完整的品牌化框架。

When to use

使用场景

A podcast soundbite or quote clip for Reels / TikTok / Shorts / feed posts.
A voiceover or narration that needs a visual so it can post as video.
Music or any audio where a reactive waveform/bars is the hero.

用于Reels/TikTok/Shorts/信息流帖子的播客片段或语录剪辑。
需要搭配视觉元素才能以视频形式发布的旁白或解说内容。
以响应式波形/音频条为核心视觉的音乐或任意音频内容。

The one rule that prevents 90% of bugs

避免90%错误的关键规则

Drive every bar height from the current frame, never from a real-time analyser loop. Live

AnalyserNode

requestAnimationFrame

reads "what is playing right now" — but a video renderer paints frames out of order and faster/slower than real time, so the wave desyncs or freezes. Instead, decode the whole file to amplitude samples once, then compute the displayed value as a pure function of the frame. In Remotion this is

useAudioData()

visualizeAudio()

; in plain canvas it is

decodeAudioData()

into a sample array you index by frame.

tsx

import { useAudioData, visualizeAudio } from "@remotion/media-utils";
import { useCurrentFrame, useVideoConfig } from "remotion";

const audioData = useAudioData(staticFile("episode.mp3"));
if (!audioData) return null; // still loading
const { fps } = useVideoConfig();
const bars = visualizeAudio({
  audioData,
  frame: useCurrentFrame(),
  fps,
  numberOfSamples: 32, // MUST be a power of two
}); // → Float array, length 32, each 0–1, low freq → high freq

所有音频条高度都由当前帧驱动，绝不要使用实时分析器循环。 实时

AnalyserNode

requestAnimationFrame

读取的是「当前正在播放的内容」——但视频渲染器的帧绘制顺序混乱，且速度可能快于或慢于实时，这会导致波形不同步或冻结。正确的做法是：先将整个文件解码为振幅样本，然后将显示值计算为仅依赖帧的纯函数。在Remotion中，这对应

useAudioData()

visualizeAudio()

；在原生Canvas中，则是通过

decodeAudioData()

将音频解码为样本数组，再按帧索引取值。

tsx

import { useAudioData, visualizeAudio } from "@remotion/media-utils";
import { useCurrentFrame, useVideoConfig } from "remotion";

const audioData = useAudioData(staticFile("episode.mp3"));
if (!audioData) return null; // 仍在加载中
const { fps } = useVideoConfig();
const bars = visualizeAudio({
  audioData,
  frame: useCurrentFrame(),
  fps,
  numberOfSamples: 32, // 必须是2的幂
}); // → 浮点数组，长度32，取值范围0–1，从低频到高频

Two ways to render the wave

两种波形渲染方式

Look	Data	API	Best for
Equalizer bars	frequency spectrum, values 0–1	`visualizeAudio()`	music, energy, "feed the bars"
Smooth oscilloscope wave	time-domain amplitude, −1…1	`visualizeAudioWaveform()`	voice, podcasts, a calm minimal look

visualizeAudio

returns lows on the left, highs on the right. For a centered equalizer, take the first N bars and mirror them around the middle so the bass sits in the center.

numberOfSamples

must be a power of two (16/32/64); use 16–32 for a chunky branded look, 64+ for a detailed spectrum. See

references/waveform-render.md

for full bars and oscilloscope components.

视觉效果	数据类型	API	适用场景
均衡器条	频率频谱，取值0–1	`visualizeAudio()`	音乐、充满活力的内容、「音频驱动条动效」场景
平滑示波器波形	时域振幅，取值−1…1	`visualizeAudioWaveform()`	人声、播客、简约平静的视觉风格

visualizeAudio

返回的结果中，左侧为低频，右侧为高频。若要实现居中的均衡器，可取前N个条并在中间镜像，让低音位于中心。

numberOfSamples

必须是2的幂（16/32/64）；16–32适合打造厚重的品牌风格，64及以上适合呈现详细频谱。完整的音频条和示波器组件可参考

references/waveform-render.md

。

Long files: don't load the whole episode

长文件处理：不要加载整集内容

useAudioData()

reads the entire file into memory — fine for a 30–90s clip, slow and memory-heavy for a full episode. For anything long, trim first or use

useWindowedAudioData()

, which fetches only the audio around the current frame via HTTP range requests.

tsx

import { useWindowedAudioData } from "@remotion/media-utils";
const { audioData } = useWindowedAudioData(
  staticFile("full-episode.mp3"),
  fps,
  /* windowInSeconds */ 10,
);

Best practice: trim to the soundbite (≤90s) before rendering. A tight clip is a better social asset and a lighter render.

useAudioData()

会将整个文件读入内存——对于30–90秒的片段来说没问题，但对于整集播客来说会很慢且占用大量内存。处理长文件时，应先裁剪片段，或使用

useWindowedAudioData()

，它会通过HTTP范围请求仅获取当前帧附近的音频内容。

tsx

import { useWindowedAudioData } from "@remotion/media-utils";
const { audioData } = useWindowedAudioData(
  staticFile("full-episode.mp3"),
  fps,
  /* windowInSeconds */ 10,
);

最佳实践：渲染前先裁剪为片段（≤90秒）。紧凑的片段是更好的社交资产，也能减轻渲染负担。

The layout

布局结构

A finished audiogram is five stacked layers. Keep them in fixed zones so one composition crops cleanly to every aspect.

Layer	Job	Notes
Background	brand color / subtle gradient / blurred cover	never busy enough to fight captions
Cover art + title	who/what this is	small square cover + episode/show title, top zone
Waveform / bars	the motion that signals audio	center band, the hero element
Captions	the message, for muted viewers	high contrast, one short phrase at a time
Progress bar	how far through the clip	thin bar driven by `frame / durationInFrames`

tsx

// progress bar — a pure function of frame, no audio needed
const { durationInFrames } = useVideoConfig();
const progress = useCurrentFrame() / durationInFrames; // 0 → 1
<div style={{ width: `${progress * 100}%`, height: 6, background: "#fff" }} />

一个完整的音频可视化视频包含五层堆叠结构。将它们放在固定区域，这样一个合成项目就能轻松适配各种画幅。

层级	作用	注意事项
背景	品牌色/柔和渐变/模糊封面	不要过于繁杂，避免干扰字幕
封面图+标题	展示内容主体	顶部区域放置小方形封面+剧集/节目标题
波形/音频条	传递音频存在的动态元素	中心区域，核心视觉元素
字幕	为静音用户传递信息	高对比度，每次显示短句
进度条	显示播放进度	由 `frame / durationInFrames` 驱动的细条

tsx

// 进度条——仅依赖帧的纯函数，无需音频
const { durationInFrames } = useVideoConfig();
const progress = useCurrentFrame() / durationInFrames; // 0 → 1
<div style={{ width: `${progress * 100}%`, height: 6, background: "#fff" }} />

Captions synced to speech

与语音同步的字幕

Captions are not optional — most feed video plays muted, and subtitled video holds attention far longer. Generate word/phrase timings with a transcription step (Whisper /

@remotion/captions

parseSrt

), then show one short phrase at a time, switching on its start frame. The mechanics of word-by-word reveal, active-word highlighting, and SRT/JSON timing belong to the caption-animation skill — use it for the caption layer rather than duplicating it here. This skill's job is to place that caption track inside the audiogram layout and keep it synced to the same audio clock the waveform uses.

字幕并非可选——大多数信息流视频都是静音播放，带字幕的视频能吸引用户更久的注意力。通过转录步骤（Whisper /

@remotion/captions

的

parseSrt

）生成单词/短语的时间轴，然后每次显示一个短句，在对应起始帧切换。逐词显示、高亮当前单词以及SRT/JSON时间轴的实现属于caption-animation技能——请使用该技能处理字幕层，无需重复开发。本技能的职责是将字幕轨道整合到音频可视化视频的布局中，并确保其与波形使用的音频时钟同步。

Social framing

社交平台画幅适配

Design once inside the center safe zone, then reframe — don't letterbox.

Aspect	Use	Resolution	Layout
1:1	feed posts	1080×1080	cover+title top, wave center, captions lower third
9:16	Reels / TikTok / Shorts	1080×1920	more vertical breathing room; keep captions clear of bottom 18% (UI)
16:9	YouTube, landing embed	1920×1080	wave wide and low, captions centered

Make the clip playable on mute and on sound: the waveform + captions must tell the whole story silently, while the actual audio track plays for anyone who taps. Always include the real audio with

<Audio src={...} />

so the export carries sound.

在中心安全区域设计一次，然后重新适配画幅——不要添加黑边。

画幅比例	适用场景	分辨率	布局建议
1:1	信息流帖子	1080×1080	封面+标题在顶部，波形在中心，字幕在下部三分之一区域
9:16	Reels/TikTok/Shorts	1080×1920	预留更多垂直空间；确保字幕避开底部18%的UI区域
16:9	YouTube、落地页嵌入	1920×1080	波形宽且靠下，字幕居中

确保视频静音和有声状态下都可观看：波形+字幕必须能在静音时完整传递信息，而实际音频轨道则为点击开启声音的用户播放。务必通过

<Audio src={...} />

添加真实音频，确保导出的视频包含声音。

Output checklist

输出检查清单

Bar/wave height is a pure function of
```
useCurrentFrame()
```
— no live analyser loop.
```
numberOfSamples
```
is a power of two; centered bars are mirrored around the middle.
Clip trimmed to ≤90s, or
```
useWindowedAudioData()
```
for long source.
Captions present, high-contrast, synced to the same audio clock (via caption-animation).
Cover art + title + progress bar render; progress =
```
frame / durationInFrames
```
.
Real audio muxed in (
```
<Audio>
```
); story still reads with sound off.
1:1 + 9:16 (and 16:9 if needed) from one composition, key content in the safe zone.

音频条/波形高度是
```
useCurrentFrame()
```
的纯函数——不使用实时分析器循环。
```
numberOfSamples
```
是2的幂；居中音频条在中间镜像。
片段裁剪至≤90秒，或对长源文件使用
```
useWindowedAudioData()
```
。
包含高对比度字幕，并与同一音频时钟同步（通过caption-animation技能）。
封面图+标题+进度条正常渲染；进度值=
```
frame / durationInFrames
```
。
嵌入真实音频（
```
<Audio>
```
）；静音状态下仍能传递完整内容。
基于一个合成项目生成1:1+9:16（必要时添加16:9）的视频，核心内容位于安全区域。

Deliver & verify (rendered stills → MP4)

交付与验证（渲染静帧→MP4）

Packaged helper (
scripts/
): tile your stills with
scripts/contact-sheet.sh sheet.png f-hook.png f-mid.png f-end.png
, then assert the encode with
scripts/probe-mp4.sh out.mp4 [WxH] [fps]
. See
scripts/README.md
.

This is HEAVY tier: the deliverable is a real MP4 (audio muxed in), and the waveform must be data-driven — its shape at any frame must match the audio at that time. Verify a few frames as stills before paying for the full encode.

Output contract:

A Remotion project with the audiogram registered (
```
<Composition>
```
+ zod
```
schema
```
+
```
defaultProps
```
), all motion frame-driven — no
```
Date.now()
```
/
```
Math.random()
```
/ timers, and no realtime analyser.
Bake the amplitude/waveform samples into props (or decode once via
```
useAudioData
```
) offline — never analyse audio at render time; headless render has no realtime audio clock.
Deliverable = the rendered
```
out/*.mp4
```
(audio muxed via
```
<Audio src={staticFile()}>
```
) plus the project, so the user can re-render with new audio/captions.

Verify loop — render stills → inspect → encode.

bash

undefined

打包工具（
scripts/
）：使用
scripts/contact-sheet.sh sheet.png f-hook.png f-mid.png f-end.png
将静帧拼接成预览图，然后使用
scripts/probe-mp4.sh out.mp4 [WxH] [fps]
验证编码。详见
scripts/README.md
。

这属于重量级交付：交付物是真实的MP4（嵌入音频），且波形必须是数据驱动的——任意帧的波形形状必须与该时刻的音频匹配。在进行完整编码前，先验证几帧静帧。

输出规范：

一个注册了音频可视化视频的Remotion项目（包含
```
<Composition>
```
+ zod
```
schema
```
+
```
defaultProps
```
），所有动效都由帧驱动——不使用
```
Date.now()
```
/
```
Math.random()
```
/ 计时器，且不使用实时分析器。
将振幅/波形样本离线烘焙到props中（或通过
```
useAudioData
```
解码一次）——绝不在渲染时分析音频；无头渲染没有实时音频时钟。
交付物=渲染后的
```
out/*.mp4
```
（通过
```
<Audio src={staticFile()}>
```
嵌入音频）加上项目文件，以便用户使用新的音频/字幕重新渲染。

验证流程：渲染静帧→检查→编码。

bash

undefined

1. Frame-exact stills at start / mid / end, WITH SHIPPED PROPS (not just defaultProps)

1. 使用实际交付的Props（不仅是defaultProps）渲染起始/中间/结束帧的精确静帧

npx remotion still Audiogram out/f-start.png --frame=0 --props=./props.json npx remotion still Audiogram out/f-mid.png --frame=N --props=./props.json npx remotion still Audiogram out/f-end.png --frame=N-1 --props=./props.json # last = durationInFrames-1

2. Inspect each PNG — fidelity (waveform/bar heights match the audio's amplitude at

2. 检查每张PNG的保真度（波形/音频条高度与该时刻音频振幅匹配；屏幕上显示正确的字幕短语并同步）以及是否存在瑕疵（封面图缺失、字幕超出画布/位于底部UI区域下方、音频条溢出）。

that exact time; the right caption phrase is on screen and synced) AND artifacts

3. 只有在静帧检查通过后，才编码完整视频（嵌入音频）：

(missing cover art, caption off-canvas / under the bottom UI band, bar overflow).

—

3. Only once the stills check out, encode the full video (audio muxed):

—

npx remotion render Audiogram out/audiogram.mp4 --props=./props.json


- The waveform is **data-driven** — verify amplitude + caption sync at a few frames *before* the full render, so a desync is caught once, not after a long encode.
- Use `npx remotion compositions` to read `durationInFrames`/`fps` and pick the end frame.
- README demo: `npx remotion render Audiogram out/demo.gif --codec=gif` (silent, but proves the motion).

**Before you finish:**
1. `npx remotion still` renders cleanly at frame 0, mid, and last — no errors, no missing cover/audio assets.
2. Bar/wave height at each checked frame matches the audio's amplitude there; the correct caption phrase is on screen and synced.
3. Frame-driven only — no realtime analyser / `Date.now()` / `Math.random()` / timers; samples baked offline.
4. Captions stay inside the safe zone (clear of the bottom ~18% UI band) at every checked frame; cover art + title + progress bar present.
5. Full MP4 encoded with real audio muxed in and plays; (optional) GIF rendered for the README.

npx remotion render Audiogram out/audiogram.mp4 --props=./props.json


- 波形是**数据驱动**的——在完整渲染前，先验证几帧的振幅+字幕同步，这样能及时发现不同步问题，避免长时间编码后才出错。
- 使用`npx remotion compositions`查看`durationInFrames`/`fps`并选择结束帧。
- README演示：`npx remotion render Audiogram out/demo.gif --codec=gif`（静音，但能展示动效）。

**完成前检查：**
1. `npx remotion still`能在起始帧、中间帧和最后一帧干净渲染——无错误，无缺失的封面/音频资源。
2. 每个检查帧的音频条/波形高度与该时刻音频振幅匹配；正确的字幕短语显示在屏幕上并同步。
3. 仅由帧驱动——无实时分析器/`Date.now()`/`Math.random()`/计时器；样本已离线烘焙。
4. 字幕在所有检查帧中都位于安全区域（避开底部约18%的UI区域）；封面图+标题+进度条均存在。
5. 完整MP4已编码并嵌入真实音频，可正常播放；（可选）为README渲染GIF演示。

Reference files

参考文件

```
references/waveform-render.md
```
— complete runnable components for both looks: a mirrored frequency-bar equalizer and a smooth oscilloscope wave (
```
visualizeAudio
```
/
```
visualizeAudioWaveform
```
), the full five-layer audiogram composition with cover art, title, progress bar and audio mux, a power-of-two/smoothing cheat sheet, and a dependency-free Web Audio + canvas variant that decodes to a per-frame sample array. Ends with build/attribution notes.

```
references/waveform-render.md
```
——两种视觉效果的完整可运行组件：镜像频率条均衡器和平滑示波器波形（
```
visualizeAudio
```
/
```
visualizeAudioWaveform
```
），包含封面图、标题、进度条和音频嵌入的完整五层音频可视化视频合成项目，2的幂/平滑处理速查表，以及无需依赖的Web Audio + Canvas实现（解码为逐帧样本数组）。结尾包含构建/署名说明。