audio-analysis

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

AudioAnalysis

Real-time audio signal analysis attached to any entity that already has an

AudioSource

AudioStream

, or

VideoPlayer

. The renderer analyzes the audio frame buffer and writes results back into the component each tick. Scenes read those results to drive visualizers, beat-reactive geometry, audio-driven lights, etc.

可附加到任何已经具有

AudioSource

、

AudioStream

或

VideoPlayer

的实体上的实时音频信号分析。renderer 对音频帧缓冲区进行分析，并在每个 tick 将结果回写到该组件中。场景读取这些结果，以驱动可视化器、节拍响应几何体、音频驱动的灯光等。

RULE: Requires an audio-emitting component on the same entity

规则：同一实体上必须存在音频输出组件

AudioAnalysis

does nothing on its own. The entity MUST also have one of:

AudioSource

AudioStream

, or

VideoPlayer

. The renderer taps that component's audio frame buffer to compute amplitude/bands. An entity with only

AudioAnalysis

produces no data.

AudioAnalysis

本身不会执行任何操作。该实体必须同时具备以下组件之一：

AudioSource

、

AudioStream

或

VideoPlayer

。renderer 会从该组件的音频帧缓冲区取样，以计算振幅/频段。只有

AudioAnalysis

的实体不会产生任何数据。

RULE: Audio must be playing for non-zero output

规则：音频必须正在播放才能产生非零输出

Values are derived from live audio frames. If the source is paused, muted, or not yet loaded,

amplitude

and all

bands[]

stay at

. There is no "ready" event — start your reactive systems unconditionally, they will simply animate toward

while silent.

这些值派生自实时音频帧。如果源已暂停、静音或尚未加载，

amplitude

和所有

bands[]

都会保持为

。不存在“ready”事件——请无条件启动你的响应系统；静音时，它们只会逐渐回落至

。

RULE: Only the Unity explorer implements this

规则：仅 Unity explorer 实现此功能

Bevy and the mobile Godot explorer ignore the component (no analysis written). Treat

AudioAnalysis

as a Unity-explorer-only enhancement and design fallbacks (e.g. a base scale that doesn't depend on

amplitude

) so the scene still looks reasonable elsewhere.

Bevy 和移动端 Godot explorer 会忽略该组件（不会写入分析结果）。将

AudioAnalysis

视为仅限 Unity explorer 的增强功能，并设计回退方案（例如采用不依赖

amplitude

的基础缩放），确保场景在其他环境中仍然呈现合理效果。

RULE: Read via

readIntoView

into a pre-allocated view

规则：通过

readIntoView

读取到预分配的 view

readIntoView

tryReadIntoView

write into a caller-owned

AudioAnalysisView = { amplitude: number, bands: number[] }

. Allocate the view once at scene init and reuse it every frame — do not

new

it inside the system. The

bands

array MUST be pre-sized to 8.

readIntoView

tryReadIntoView

会将数据写入由调用方持有的

AudioAnalysisView = { amplitude: number, bands: number[] }

。请在场景初始化时分配一次该 view，并在每帧重复使用——不要在 system 内

new

它。

bands

数组必须预先设定为 8 个元素。

RULE: Use

createAudioAnalysis

, not

create

规则：使用

createAudioAnalysis

，而不是

create

Use the helper

AudioAnalysis.createAudioAnalysis(entity, mode?, amplitudeGain?, bandsGain?)

. It pre-fills all required protobuf fields (8 bands + amplitude + mode) with safe defaults. Calling the raw

AudioAnalysis.create(entity, {...})

requires you to provide every band/amplitude field manually. Use

createOrReplaceAudioAnalysis

to overwrite an existing one without throwing.

请使用辅助函数

AudioAnalysis.createAudioAnalysis(entity, mode?, amplitudeGain?, bandsGain?)

。它会以安全默认值预填所有必需的 protobuf 字段（8 个频段 + 振幅 + 模式）。如果调用原始的

AudioAnalysis.create(entity, {...})

，则必须手动提供每个频段/振幅字段。请使用

createOrReplaceAudioAnalysis

覆盖现有组件，且不会抛出异常。

Import

typescript

import {
  AudioAnalysis,
  AudioAnalysisView,
  PBAudioAnalysisMode,
  AudioSource, // or AudioStream / VideoPlayer
  engine,
  Transform,
} from "@dcl/sdk/ecs";
import { Vector3 } from "@dcl/sdk/math";

AudioAnalysisView

is a TypeScript type alias (not a component), exported from

@dcl/sdk/ecs

typescript

import {
  AudioAnalysis,
  AudioAnalysisView,
  PBAudioAnalysisMode,
  AudioSource, // or AudioStream / VideoPlayer
  engine,
  Transform,
} from "@dcl/sdk/ecs";
import { Vector3 } from "@dcl/sdk/math";

AudioAnalysisView

是一个 TypeScript 类型别名（不是组件），从

@dcl/sdk/ecs

导出。

Component fields

组件字段

Output (filled by the renderer; read in your systems):

Field	Type	Range	Notes
`amplitude`	`number`	`0..~1` (mode-dep.)	Aggregate signal strength of the current audio frame.
`band0`	`number`	`0..~1` (mode-dep.)	Lowest frequency bin (sub-bass).
`band1..6`	`number`	`0..~1` (mode-dep.)	Increasing frequency bins, log-spaced under MODE_LOGARITHMIC.
`band7`	`number`	`0..~1` (mode-dep.)	Highest frequency bin (treble/air).

Inputs (configure once at create time):

Field	Type	Default	Notes
`mode`	`PBAudioAnalysisMode`	`MODE_LOGARITHMIC`	`MODE_RAW = 0` (raw FFT magnitudes) \| `MODE_LOGARITHMIC = 1` (perceptual log mapping, recommended for visualizers).
`amplitudeGain`	`number?` (optional)	`5.0` when omitted	Multiplier applied to `amplitude` . Only used in MODE_LOGARITHMIC.
`bandsGain`	`number?` (optional)	`0.05` when omitted	Multiplier applied to all 8 bands. Only used in MODE_LOGARITHMIC.

Values are unbounded floats — gains can push them above
1
. Clamp or scale in your system if the visual you drive needs
0..1
. For typical music at default gains, expect peaks roughly in
0..1
with normal content sitting
0..0.5
.

输出（由 renderer 填充；在你的 system 中读取）：

Field	Type	Range	Notes
`amplitude`	`number`	`0..~1` （取决于模式）	当前音频帧的汇总信号强度。
`band0`	`number`	`0..~1` （取决于模式）	最低频率区间（超低频）。
`band1..6`	`number`	`0..~1` （取决于模式）	频率依次升高的区间，在 MODE_LOGARITHMIC 下按对数间隔排列。
`band7`	`number`	`0..~1` （取决于模式）	最高频率区间（高音/空气声）。

输入（创建时配置一次）：

Field	Type	Default	Notes
`mode`	`PBAudioAnalysisMode`	`MODE_LOGARITHMIC`	`MODE_RAW = 0` （原始 FFT 幅值） \| `MODE_LOGARITHMIC = 1` （感知型对数映射，推荐用于可视化器）。
`amplitudeGain`	`number?` （可选）	`5.0` （省略时）	应用于 `amplitude` 的乘数。仅在 MODE_LOGARITHMIC 中使用。
`bandsGain`	`number?` （可选）	`0.05` （省略时）	应用于全部 8 个频段的乘数。仅在 MODE_LOGARITHMIC 中使用。

数值不受上限约束——增益可能将其推高至
1
以上。如果所驱动的视觉效果需要
0..1
范围，请在 system 中对这些值进行 clamp 或缩放。对于采用默认增益的典型音乐，峰值大致预计在
0..1
范围内，而普通内容通常位于
0..0.5
。

AudioAnalysisView

Read-only convenience shape used by

readIntoView

tryReadIntoView

typescript

type AudioAnalysisView = {
  amplitude: number;
  bands: number[]; // length 8 — bands[0] = lowest freq, bands[7] = highest
};

供

readIntoView

tryReadIntoView

使用的只读便捷结构：

typescript

type AudioAnalysisView = {
  amplitude: number;
  bands: number[]; // length 8 — bands[0] = lowest freq, bands[7] = highest
};

Reading the data

读取数据

typescript

const view: AudioAnalysisView = { amplitude: 0, bands: new Array<number>(8) };

engine.addSystem(() => {
  // Throws if the entity has no AudioAnalysis
  AudioAnalysis.readIntoView(audioEntity, view);

  // Or, defensive variant — returns false if missing, no throw
  // if (!AudioAnalysis.tryReadIntoView(audioEntity, view)) return

  // view.amplitude and view.bands[0..7] are now populated
});

readIntoView(entity, out)

— populates

out

from the latest analysis. Throws if the entity has no

AudioAnalysis

tryReadIntoView(entity, out): boolean

— same, but returns

false

instead of throwing when the component is missing.

typescript

const view: AudioAnalysisView = { amplitude: 0, bands: new Array<number>(8) };

engine.addSystem(() => {
  // Throws if the entity has no AudioAnalysis
  AudioAnalysis.readIntoView(audioEntity, view);

  // Or, defensive variant — returns false if missing, no throw
  // if (!AudioAnalysis.tryReadIntoView(audioEntity, view)) return

  // view.amplitude and view.bands[0..7] are now populated
});

readIntoView(entity, out)

——从最新分析结果填充

out

。如果实体没有

AudioAnalysis

，则抛出异常。

tryReadIntoView(entity, out): boolean

——行为相同，但当组件缺失时返回

false

，而不是抛出异常。

Common patterns

常见模式

typescript

// 1. Pulse an entity's scale to overall amplitude
const view: AudioAnalysisView = { amplitude: 0, bands: new Array<number>(8) };
engine.addSystem(() => {
  AudioAnalysis.readIntoView(audioEntity, view);
  const t = Transform.getMutable(pulseEntity);
  const s = 1 + view.amplitude * 10; // base 1, gain to taste
  t.scale = Vector3.create(s, s, s);
});

// 2. 8-bar equalizer (one entity per band, scale Y by bands[i])
for (const [entity, _] of engine.getEntitiesWith(VisualBar, Transform)) {
  const i = VisualBar.get(entity).index; // 0..7
  Transform.getMutable(entity).scale = Vector3.create(
    1,
    view.bands[i] * BAR_HEIGHT,
    1
  );
}

// 3. Bass-only kick — react to the lowest band
const bass = view.bands[0] + view.bands[1]; // sub + low
if (bass > 0.7) {
  /* trigger flash */
}

// 4. Custom gains (less sensitive amplitude, punchier bands)
AudioAnalysis.createAudioAnalysis(
  audioEntity,
  PBAudioAnalysisMode.MODE_LOGARITHMIC,
  2.0,
  0.1
);

For a complete music visualizer (audio source + amplitude sphere + 8-band equalizer bars), see

{baseDir}/references/audio-analysis-example.md

typescript

// 1. Pulse an entity's scale to overall amplitude
const view: AudioAnalysisView = { amplitude: 0, bands: new Array<number>(8) };
engine.addSystem(() => {
  AudioAnalysis.readIntoView(audioEntity, view);
  const t = Transform.getMutable(pulseEntity);
  const s = 1 + view.amplitude * 10; // base 1, gain to taste
  t.scale = Vector3.create(s, s, s);
});

// 2. 8-bar equalizer (one entity per band, scale Y by bands[i])
for (const [entity, _] of engine.getEntitiesWith(VisualBar, Transform)) {
  const i = VisualBar.get(entity).index; // 0..7
  Transform.getMutable(entity).scale = Vector3.create(
    1,
    view.bands[i] * BAR_HEIGHT,
    1
  );
}

// 3. Bass-only kick — react to the lowest band
const bass = view.bands[0] + view.bands[1]; // sub + low
if (bass > 0.7) {
  /* trigger flash */
}

// 4. Custom gains (less sensitive amplitude, punchier bands)
AudioAnalysis.createAudioAnalysis(
  audioEntity,
  PBAudioAnalysisMode.MODE_LOGARITHMIC,
  2.0,
  0.1
);

有关完整的音乐可视化器（音频源 + 振幅球 + 8 段均衡器柱），请参阅

{baseDir}/references/audio-analysis-example.md

。

Mode selection

模式选择

```
MODE_LOGARITHMIC
```
(default) — bands are log-spaced and gain-scaled to roughly fit
```
0..1
```
for typical music. Use this for visualizers, scaling, color reactivity.
```
amplitudeGain
```
and
```
bandsGain
```
apply.
```
MODE_RAW
```
— raw FFT-derived magnitudes, linearly spaced. Lower bands dominate visually because most musical energy is there.
```
amplitudeGain
```
/
```
bandsGain
```
are ignored. Use only if you intend to do your own normalization.

```
MODE_LOGARITHMIC
```
（默认）——频段按对数间隔排列，并通过增益缩放，使典型音乐的值大致落在
```
0..1
```
范围内。适用于可视化器、缩放和颜色响应。
```
amplitudeGain
```
和
```
bandsGain
```
会生效。
```
MODE_RAW
```
——采用线性间隔排列的原始 FFT 幅值。由于大部分音乐能量集中在低频区域，因此低频段在视觉上会占据主导。
```
amplitudeGain
```
/
```
bandsGain
```
会被忽略。仅在你打算自行进行归一化时使用。

Gotchas

常见陷阱

Component ID is
1212
.
amplitudeGain
and
bandsGain
are no-ops in MODE_RAW. Setting them won't error, but the renderer ignores them outside MODE_LOGARITHMIC.
Output values can exceed
1.0
with high gains or loud source material. Clamp downstream if you feed UI bars or alpha channels expecting
```
0..1
```
.
Throttled updates. The renderer runs analysis under a frame-time budget — values update each frame in normal conditions but can skip frames under heavy load. Drive smooth animations with
```
dt
```
interpolation rather than assuming a fixed update cadence.
Multi-source scenes. Each audio-emitting entity needs its own
```
AudioAnalysis
```
if you want to react to that specific source. There is no global mix-down; you pick the entity to analyze.
Works on
VideoPlayer
audio too — call
```
AudioAnalysis.createAudioAnalysis(videoEntity)
```
on the same entity that has
```
VideoPlayer
```
. The renderer taps the video's audio frame buffer the same way it does for
```
AudioSource
```
. IMPORTANT: the video must be a progressive file (MP4/WebM with direct URL), not an HLS stream (.m3u8). HLS streams play with audible sound but the renderer writes zeros to the AudioAnalysis component (its audio decodes on a path the analyzer does not tap). Verified in
```
88,-10-audio-visualization
```
, which demonstrates AudioSource and VideoPlayer side by side.
No
pitch
interaction.
```
AudioSource.pitch
```
changes playback speed; the analysis runs on the actual played audio frames, so a higher pitch shifts perceived band energy upward. This is expected, not a bug.
Don't call
readIntoView
before
createAudioAnalysis
. Reading without the component throws. Use
```
tryReadIntoView
```
if the component may not be attached yet (e.g. analysis added at runtime).

组件 ID 为
1212
。
amplitudeGain
和
bandsGain
在 MODE_RAW 下不起作用。设置它们不会报错，但 renderer 在 MODE_LOGARITHMIC 之外会忽略它们。
输出值可能超过
1.0
。较高的增益或响度较高的源素材都可能导致这种情况。如果将这些值用于要求
```
0..1
```
范围的 UI 柱或 alpha 通道，请在下游进行 clamp。
更新经过节流。 renderer 会在帧时间预算内运行分析——正常情况下值每帧更新，但在负载较重时可能跳过部分帧。驱动平滑动画时应使用
```
dt
```
插值，不要假定固定的更新频率。
多源场景。 如果希望对特定源作出响应，每个音频输出实体都需要自己的
```
AudioAnalysis
```
。不存在全局混音；你需要选择要分析的实体。
同样支持
VideoPlayer
音频——请在同一具有
```
VideoPlayer
```
的实体上调用
```
AudioAnalysis.createAudioAnalysis(videoEntity)
```
。renderer 对视频音频帧缓冲区的处理方式与对
```
AudioSource
```
相同。重要：视频必须采用渐进式文件（具有直接 URL 的 MP4/WebM），而不能使用 HLS 流（.m3u8）。 HLS 流虽然可以听到声音，但 renderer 会向
```
AudioAnalysis
```
组件写入零值（其音频解码路径不是分析器所接入的路径）。已在
```
88,-10-audio-visualization
```
中验证，该场景并排展示了
```
AudioSource
```
和
```
VideoPlayer
```
。
不存在与
pitch
的交互。
```
AudioSource.pitch
```
会改变播放速度；分析基于实际播放的音频帧运行，因此更高的 pitch 会使感知到的频段能量向上移动。这是预期行为，并非 bug。
不要在调用
createAudioAnalysis
前执行
readIntoView
。如果实体尚未附加组件就读取，调用将抛出异常。如果组件可能在运行时才附加（例如动态添加分析），请使用
```
tryReadIntoView
```
。

Permissions

权限

No special scene permission is needed beyond what the underlying

AudioSource

AudioStream

VideoPlayer

already requires. Streamed audio still needs

ALLOW_MEDIA_HOSTNAMES

scene.json

for its hostname (see

audio-video

skill).

除了底层

AudioSource

AudioStream

VideoPlayer

本身要求的权限外，无需特殊的场景权限。流式音频仍需在

scene.json

中通过

ALLOW_MEDIA_HOSTNAMES

授权其主机名（参见 audio-video skill）。

Example scenes

示例场景

audio-visualization — engine-team test scene: two independent visualizer groups side by side. Group 0:
```
AudioSource
```
+
```
AudioAnalysis.createAudioAnalysis()
```
driving 8 bars + amplitude sphere. Group 1:
```
VideoPlayer
```
(progressive MP4) +
```
AudioAnalysis.createAudioAnalysis()
```
driving its own bars + sphere from the video's soundtrack. Each group has its own pre-allocated
```
AudioAnalysisView
```
and its own
```
readIntoView
```
call per frame. Demonstrates that AudioAnalysis works identically on VideoPlayer entities, and documents the progressive-video requirement (HLS streams write zeros).

audio-visualization——引擎团队的测试场景：两个独立的可视化器组并排展示。第 0 组：
```
AudioSource
```
+
```
AudioAnalysis.createAudioAnalysis()
```
，驱动 8 根柱和振幅球。第 1 组：
```
VideoPlayer
```
（渐进式 MP4）+
```
AudioAnalysis.createAudioAnalysis()
```
，通过视频的音轨驱动自己的柱和球。每个组都有自己预先分配的
```
AudioAnalysisView
```
，并在每帧分别调用
```
readIntoView
```
。该场景证明
```
AudioAnalysis
```
在
```
VideoPlayer
```
实体上的工作方式与在
```
AudioSource
```
实体上完全相同，并记录了渐进式视频要求（HLS 流会写入零值）。

Resources

资源

```
{baseDir}/references/audio-analysis-example.md
```
— full music visualizer scene from the SDK examples (audio source + amplitude pulse + 8-bar equalizer).
Source PRs: protocol #328, js-sdk-toolchain #1256, unity-explorer #6361.
Reference scene:
```
sdk7-goerli-plaza/audio-visualization
```
.

```
{baseDir}/references/audio-analysis-example.md
```
——SDK 示例中的完整音乐可视化器场景（音频源 + 振幅脉冲 + 8 段均衡器）。
源代码 PR：protocol #328、js-sdk-toolchain #1256、unity-explorer #6361。
参考场景：
```
sdk7-goerli-plaza/audio-visualization
```
。

audio-analysis

Original

Translation

AudioAnalysis

AudioAnalysis

RULE: Requires an audio-emitting component on the same entity

规则：同一实体上必须存在音频输出组件

RULE: Audio must be playing for non-zero output

规则：音频必须正在播放才能产生非零输出

RULE: Only the Unity explorer implements this

规则：仅 Unity explorer 实现此功能

RULE: Read via
`readIntoView`
into a pre-allocated view

规则：通过
`readIntoView`
读取到预分配的 view

RULE: Use
`createAudioAnalysis`
, not
`create`

规则：使用
`createAudioAnalysis`
，而不是
`create`

Import

Import

Component fields

组件字段

AudioAnalysisView

AudioAnalysisView

Reading the data

读取数据

Common patterns

常见模式

Mode selection

模式选择

Gotchas

常见陷阱

Permissions

权限

Example scenes

示例场景

Resources

资源

audio-analysis

Original

Translation

AudioAnalysis

AudioAnalysis

RULE: Requires an audio-emitting component on the same entity

规则：同一实体上必须存在音频输出组件

RULE: Audio must be playing for non-zero output

规则：音频必须正在播放才能产生非零输出

RULE: Only the Unity explorer implements this

规则：仅 Unity explorer 实现此功能

RULE: Read via readIntoView into a pre-allocated view

规则：通过 readIntoView 读取到预分配的 view

RULE: Use createAudioAnalysis, not create

规则：使用 createAudioAnalysis，而不是 create

Import

Import

Component fields

组件字段

AudioAnalysisView

AudioAnalysisView

Reading the data

读取数据

Common patterns

常见模式

Mode selection

模式选择

Gotchas

常见陷阱

Permissions

权限

Example scenes

示例场景

Resources

资源

RULE: Read via
`readIntoView`
into a pre-allocated view

规则：通过
`readIntoView`
读取到预分配的 view

RULE: Use
`createAudioAnalysis`
, not
`create`

规则：使用
`createAudioAnalysis`
，而不是
`create`