hyperframes-audio
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseHyperFrames Audio
HyperFrames 音频
A mix is a set of relationships, not a stack of processors. Two tracks that each
sound right alone can be unlistenable together, and the fix is almost never "turn
one down" — it is finding what they are fighting over and giving it to whichever
one needs it. Every tool here exists to express one of those relationships.
Effects live on the element as , and preview and render run the
same Web Audio graph — the studio in a live context, the engine in an offline one
inside the browser it already drives. There is one implementation of each effect,
so what you hear while scrubbing is what gets written. You never tune twice.
data-fx-chainThree attributes carry everything, all on the audio/video element itself:
| Attribute | Holds |
|---|---|
| the effects, in signal order |
| envelopes on this track's volume or its effect parameters |
| the carve's own settings, so it can be re-derived |
Exact JSON for each, and the rules a lane must satisfy: .
Every effect with its parameters, ranges and units: .
references/attributes.mdreferences/fx-registry.md混音是一组关系的集合,而非处理器的堆叠。单独听来效果不错的两个轨道,放在一起可能难以入耳,而解决办法几乎从来不是“把其中一个音量调低”——而是找到它们冲突的频段,将该频段分配给更需要它的轨道。这里的每一个工具都是为了表达这类关系而存在。
效果以的形式存储在元素上,预览和渲染使用相同的Web Audio图——在实时场景中是工作室模式,在离线场景中是浏览器内置的引擎。每种效果只有一种实现方式,因此你在scrubbing时听到的内容就是最终导出的内容。无需重复调整。
data-fx-chain所有信息都通过三个属性存储在音视频元素本身:
| 属性名称 | 存储内容 |
|---|---|
| 按信号顺序排列的效果链 |
| 轨道音量或其效果参数的包络线 |
| 雕刻功能的自身设置,以便可以重新生成相关参数 |
每个属性的精确JSON格式,以及轨道必须满足的规则:。每个效果的参数、范围和单位:。
references/attributes.mdreferences/fx-registry.mdHow it fits together
整体工作流程
Two authoring surfaces write those attributes; two runtimes read them through the
same builders. That shared middle is why preview predicts the render.
mermaid
flowchart TB
voice["voice track<br/>media file"]
bed["music bed<br/>media file"]
subgraph AUTHOR["Authoring — the only things that write attributes"]
panel["Studio<br/>Voiceover carve control"]
script["scripts/carve.mjs<br/>detects the pair, dynamic by default"]
analysis["core/audioCarve.ts<br/>carveProfile · analyseCarveBands<br/>analyseCarveDuck · analyseCarveDynamics"]
panel --> analysis
script --> analysis
end
voice --> analysis
bed --> analysis
subgraph ATTRS["Written onto the bed element"]
carveAttr["data-fx-carve<br/>source · strength · dynamic"]
chainAttr["data-fx-chain<br/>peaking xN + gain, tagged fromCarve"]
autoAttr["data-automation<br/>a lane per carved parameter"]
end
analysis --> carveAttr
analysis --> chainAttr
analysis --> autoAttr
subgraph SHARED["One implementation, read by both"]
build["audioFxGraph.ts · buildFxChain"]
sched["audioFxAutomation.ts · scheduleChainAutomation"]
end
chainAttr --> build
autoAttr --> sched
build --> preview["Preview<br/>live AudioContext<br/>attachElementFxChain"]
sched --> preview
build --> render["Render<br/>OfflineAudioContext in the headless browser<br/>applyAudioFxChain"]
sched --> render
preview --> heard["what you hear while scrubbing"]
render --> wav["processed WAV<br/>+ chainTailSeconds so the mix lets the tail through"]
wav --> mix["engine · audioMixer<br/>volume lane baked into the PCM here, not in the graph"]
mix --> out["the rendered mix"]
edit["editing the attribute mid-playback"] -.->|MutationObserver| previewThe carve's own settings are never read at playback — the chain and lanes it
produced are what play. exists so strength can be changed on an
existing carve instead of guessed back out of the filters.
data-fx-carveInside a carved bed the signal runs through the dips first, then the level match,
then anything you built yourself — which is why a limiter you add still acts as
the last ceiling:
mermaid
flowchart LR
src["decoded bed"] --> p1["peaking<br/>400 Hz"]
p1 --> p2["peaking<br/>1 kHz"]
p2 --> p3["peaking<br/>1.6 kHz"]
p3 --> g["gain<br/>level match"]
g --> hand["your own effects<br/>e.g. limiter"]
hand --> dest["track gain, then out"]
l1["lane fx.n1.gain"] -.->|"envelope of the voice's<br/>level in that band"| p1
l4["lane fx.n4.gain"] -.->|"how far the bed<br/>ducks overall"| gA static carve is the same graph with fixed values and no lanes at all.
两个创作界面负责写入这些属性;两个运行时通过相同的构建器读取这些属性。正是这种共享的中间层确保了预览效果与最终渲染结果一致。
mermaid
flowchart TB
voice["voice track<br/>media file"]
bed["music bed<br/>media file"]
subgraph AUTHOR["Authoring — the only things that write attributes"]
panel["Studio<br/>Voiceover carve control"]
script["scripts/carve.mjs<br/>detects the pair, dynamic by default"]
analysis["core/audioCarve.ts<br/>carveProfile · analyseCarveBands<br/>analyseCarveDuck · analyseCarveDynamics"]
panel --> analysis
script --> analysis
end
voice --> analysis
bed --> analysis
subgraph ATTRS["Written onto the bed element"]
carveAttr["data-fx-carve<br/>source · strength · dynamic"]
chainAttr["data-fx-chain<br/>peaking xN + gain, tagged fromCarve"]
autoAttr["data-automation<br/>a lane per carved parameter"]
end
analysis --> carveAttr
analysis --> chainAttr
analysis --> autoAttr
subgraph SHARED["One implementation, read by both"]
build["audioFxGraph.ts · buildFxChain"]
sched["audioFxAutomation.ts · scheduleChainAutomation"]
end
chainAttr --> build
autoAttr --> sched
build --> preview["Preview<br/>live AudioContext<br/>attachElementFxChain"]
sched --> preview
build --> render["Render<br/>OfflineAudioContext in the headless browser<br/>applyAudioFxChain"]
sched --> render
preview --> heard["what you hear while scrubbing"]
render --> wav["processed WAV<br/>+ chainTailSeconds so the mix lets the tail through"]
wav --> mix["engine · audioMixer<br/>volume lane baked into the PCM here, not in the graph"]
mix --> out["the rendered mix"]
edit["editing the attribute mid-playback"] -.->|MutationObserver| preview雕刻功能的自身设置在播放时不会被读取——实际播放的是它生成的效果链和轨道。的存在是为了让用户可以修改已有雕刻效果的强度,而无需从滤波器中反向推导参数。
data-fx-carve在经过雕刻处理的背景音乐轨道中,信号先经过频段衰减,然后进行电平匹配,再经过你自行添加的效果——这就是为什么你添加的限制器仍能作为最后一道音量上限:
mermaid
flowchart LR
src["decoded bed"] --> p1["peaking<br/>400 Hz"]
p1 --> p2["peaking<br/>1 kHz"]
p2 --> p3["peaking<br/>1.6 kHz"]
p3 --> g["gain<br/>level match"]
g --> hand["your own effects<br/>e.g. limiter"]
hand --> dest["track gain, then out"]
l1["lane fx.n1.gain"] -.->|"envelope of the voice's<br/>level in that band"| p1
l4["lane fx.n4.gain"] -.->|"how far the bed<br/>ducks overall"| g静态雕刻效果则是使用固定值且无轨道的相同图。
Reach for a family by the problem, not the name
根据问题选择工具类别,而非工具名称
Filters (, , , , ) decide
which frequencies a track is allowed to occupy. This is the first tool for two
sources colliding, because collisions happen in bands: a bed and a voice both
want 1–3 kHz, and taking that from the bed costs the bed far less than turning
the whole thing down costs the mix. A high-pass on a voice is the standard fix
for rumble; a low-pass darkens or muffles deliberately.
highpasslowpasspeakinglowshelfhighshelfDynamics (, , , ) decide how a track's level
behaves over time. Compression narrows the distance between loud and quiet so the
quiet parts can come up. A limiter is a ceiling — it does not shape anything, it
guarantees nothing gets past. A gate removes what is below a threshold, which is
how you silence room tone between phrases. is a plain level stage, and it
is what an automation lane rides when a track has to move out of the way.
gaincompressorlimitergategainNonlinear (, ) changes the waveform's shape, which adds
harmonics that were not there. Reach for it when a track needs character or
grit rather than correction — and remember it is generative: it makes a thin
source denser, not cleaner.
saturatebitcrushTime (, , , ) puts a track in a space or gives
it width. These are the ones that most easily wreck a mix, because a tail or a
detuned copy occupies the same room a voice needs. Use them on the thing that
should sit behind something else, and keep the wet amount lower than sounds
right in isolation.
delayreverbchorusphaserThe chain is serial: each effect processes what the one before it produced. So
corrective filtering goes early, character in the middle, and a limiter last
where it can actually act as a ceiling.
滤波器(、、、、)决定轨道允许占用的频段。这是解决两个音源冲突的首选工具,因为冲突通常发生在特定频段:比如背景音乐和画外音都需要1-3kHz频段,让背景音乐放弃该频段对其整体效果的影响,远小于直接调低整个背景音乐音量对混音效果的影响。为画外音添加高通滤波器是解决低频杂音的标准方案;低通滤波器则用于刻意让声音变暗或模糊。
highpasslowpasspeakinglowshelfhighshelf动态效果(、、、)决定轨道音量随时间的变化方式。压缩器可以缩小音量的动态范围,让安静的部分更清晰。限制器是一个音量上限——它不会改变声音的形态,只会确保音量不会超过设定值。门限器会移除低于阈值的声音,常用于消除语句之间的环境噪音。是一个简单的电平控制阶段,当轨道需要为其他元素让路时,自动化轨道会控制它的参数。
gaincompressorlimitergategain非线性效果(、)会改变波形的形状,从而添加原本不存在的谐波。当轨道需要增加特色或粗糙感而非修正时,可以使用这类效果——请记住它是生成性的:它会让单薄的音源变得饱满,而非更干净。
saturatebitcrush时间类效果(、、、)为轨道营造空间感或拓宽声场。这类效果最容易破坏混音,因为混响尾音或失谐的副本会占用画外音所需的空间。请将它们应用于需要处于其他元素后方的轨道,并且湿信号量要比单独听时更低。
delayreverbchorusphaser效果链是串行的:每个效果处理前一个效果输出的信号。因此,校正性滤波应放在前面,特色效果放在中间,限制器放在最后,这样它才能真正起到音量上限的作用。
Voiceover carve
画外音雕刻
The problem it solves. A music bed under a voice makes the voice hard to
follow. The reflex is to duck the whole bed, which works and costs the bed all of
its presence — the music goes limp for the entire voiceover. But the voice does
not need the whole spectrum. It needs the few bands it actually occupies. Carve
takes only those, and the bed keeps its low end and its top, so it is still music
while the voice is still intelligible.
It is a relationship, not an effect. The settings live on the bed — the
track that gets processed — and they name the voices to listen to, exactly as a
sidechain compressor does: you select the track that gets quieter and pick what
makes it quieter. Never put a carve on a voice track. A voice carved against
itself is a bug, not a subtle mix choice.
Every voice, not one of them. is a list, because a bed usually runs
under a whole sequence — a narrator, an interview answer, a second presenter. They
are summed onto the bed's own clock before anything is measured (),
so one analysis covers all of them: the bands come from all the speech there is, and
the envelopes rise wherever any of it is happening. Voices that never play while the
bed does are left out; they cannot mask it.
sourcesmixCarveSourcesOne knob. is 0..1 and derives everything: how deep to cut, how
many bands, how wide, how far to favour intelligibility over raw voice energy,
how far the level may drop, how far under the voice to aim. Those six move
together in any real mix — a gentle carve is a shallow cut in few bands with
little ducking, a hard one is deeper in more bands with more — so they are one
relationship written once, in . Default is — a 6 dB dip in
three bands with 6 dB of level room, audible without sounding like a hole. At
the dip reaches 10 dB, which is where a carve starts being heard as an
effect rather than as room for the voice; above that is deliberate territory for
a loud bed under a quiet voice. is spectral only — one band, no level match
at all.
strengthcarveProfile0.250.50Carve by default. A bed playing under narration wants a carve; it is not a
polish step to get to if there is time. Place both tracks, run the command below,
listen. Skip it only when there is no narration for the music to sit under — a
music video, a title card, a montage cut to the track.
It always follows the voice. There is no static mode: a fixed depth thins the
bed through every pause, and once you have heard both there is no reason to want it.
Every value becomes an envelope of the speech's own level — silence leaves the bed
alone, a loud passage pushes the carve to full depth — written as ordinary automation,
which is why the lanes show up in the timeline and can be edited afterwards.
Level matching is part of it. Spectral carving cannot fix a bed that is
simply louder than the voice. So the carve also measures how far over the voice
the bed sits and writes a stage: held at one value for a static carve,
driven by an envelope for a dynamic one. That envelope releases slowly on
purpose — music that snaps back to full the instant a word ends sounds like a
machine doing it.
gainRunning it. In Studio the carve is one module at the top of a track's effect
rack — voice, strength, dynamic, and the analysis it produced, in one card. It is
there whenever another track could be the voice, and a bed with exactly one
candidate above it is carved by default, dynamically, at the default strength:
that is what a bed under narration wants, and the module is where you change or
switch it off. Several candidates leaves the picker waiting rather than guessing.
Headless —
which is the path when you are authoring a composition rather than editing one:
bash
node <SKILL_DIR>/scripts/carve.mjs --comp index.htmlThat is the whole command. It finds the voice and the bed itself, carves
dynamically at the default strength, and prints what it decided:
bed music-bed (name looks like music)
voice narration (only track left)
carve strength 0.25 dynamic
bands 400Hz -6dB q1.4, 1000Hz -3dB q1.4, 1600Hz -3.17dB q1.4
level 216-point envelope, floor -6 dBName the tracks with / (repeatable) when the automatic choice is
wrong, to push it, to see that report and write nothing.
--bed--voice--strength--dry-runHow it picks the tracks. Names first, because that is what you already told it
and the answer is explainable — in core, the same classifier
Studio's own picker uses, so the two cannot disagree. A track whose id or filename
looks like music (, , , …) is the bed; everything else that
plays over it and is not SFX-shaped is a voice. Audio elements are preferred: video
counts only when no audio track is left to be the voice, or every B-roll clip in the
composition would read as somebody talking. It refuses when it cannot tell which
track is the bed rather than carving the wrong one — typing one id is cheap.
classifyAudioNamemusicbgmbedscoreSame analysis functions as the panel, so the result is identical. Needs
on PATH and installed in the project () — the CLI inlines core rather than shipping it, so it cannot
be borrowed from there.
ffmpeg@hyperframes/corenpm i -D @hyperframes/coreWhat it writes is an ordinary chain of peaking filters plus a gain stage,
tagged . That tagging is the whole trick: a re-run replaces the
previous carve and leaves every effect you built by hand — and every lane you
drew by hand — exactly where it was. So re-carving at a new strength is safe and
repeatable, and exists so the settings can be read back rather
than guessed from the filters.
fromCarvedata-fx-carve解决的问题。背景音乐下的画外音难以听清。本能反应是调低整个背景音乐的音量,这虽然有效,但会让背景音乐完全失去存在感——在整个画外音期间,音乐变得无力。但画外音并不需要整个频谱,它只需要自己实际占用的几个频段。雕刻功能只会让背景音乐放弃这些频段,保留其低频和高频部分,这样在保证画外音清晰可辨的同时,音乐依然保持原有质感。
它是一种关系,而非单一效果。设置存储在背景音乐轨道上——即被处理的轨道,并且指定需要监听的画外音轨道,就像侧链压缩器的工作方式:你选择需要降低音量的轨道,并选择触发该操作的源。切勿在画外音轨道上添加雕刻效果。对画外音自身进行雕刻是错误操作,而非精细的混音选择。
覆盖所有画外音,而非单个。是一个列表,因为背景音乐通常会伴随一系列内容——旁白、访谈回答、第二位主持人的声音。在进行任何测量之前,它们会被混音到背景音乐的时钟上(),因此一次分析即可覆盖所有画外音:频段来自所有语音内容,包络线会在任何语音出现时上升。那些从未与背景音乐同时播放的画外音会被排除在外;它们不会影响混音。
sourcesmixCarveSources一个旋钮即可控制。的取值范围是0到1,它决定了所有参数:衰减深度、频段数量、频段宽度、清晰度与原始语音能量的权衡程度、音量允许下降的幅度、目标音量低于画外音的程度。在实际混音中,这六个参数是联动的——温和的雕刻效果意味着在少数频段进行浅度衰减,且音量降低幅度小;强烈的雕刻效果则意味着在更多频段进行深度衰减,且音量降低幅度大——因此它们被整合为一个关系,在中一次性设置。默认值为——在三个频段进行6dB的衰减,预留6dB的电平空间,效果可闻但不会让音乐听起来有缺失。当为时,衰减深度达到10dB,此时雕刻效果开始被感知为一种特效,而非为画外音腾出空间;高于该值则适用于背景音乐音量大、画外音音量小的场景。仅进行频谱处理——一个频段,完全不进行电平匹配。
strengthcarveProfile0.25strength0.50默认启用雕刻。旁白下的背景音乐需要雕刻效果;这不是有空才做的润色步骤。添加两个轨道,运行下方的命令,然后试听。只有当音乐不需要为旁白让路时才跳过该步骤——比如音乐视频、标题卡、配合音乐剪辑的蒙太奇片段。
始终跟随画外音。没有静态模式:固定深度的衰减会在画外音的每个停顿期间削弱背景音乐,一旦你听过两种效果,就不会再想使用静态模式。所有参数都会变为语音自身音量的包络线——静音时背景音乐不受影响,大声段落时雕刻效果达到最大深度——这些参数会被写入普通的自动化轨道,因此轨道会显示在时间轴上,之后可以进行编辑。
电平匹配是其中的一部分。频谱雕刻无法解决背景音乐音量明显高于画外音的问题。因此雕刻功能还会测量背景音乐比画外音高出多少,并写入一个阶段:静态雕刻时保持固定值,动态雕刻时由包络线驱动。该包络线的释放速度故意设置得较慢——如果画外音一结束音乐就立即恢复到最大音量,会显得很机械。
gain运行方式。在Studio中,雕刻功能是轨道效果架顶部的一个模块——包含画外音选择、强度、动态模式以及生成的分析结果,全部在一个卡片中。只要存在可能作为画外音的其他轨道,该模块就会显示;如果背景音乐上方恰好有一个候选轨道,默认会以动态模式、默认强度进行雕刻:这正是旁白下背景音乐所需的效果,你可以在该模块中修改设置或关闭雕刻。如果有多个候选轨道,选择器会等待用户选择,而非自动猜测。在无头模式下——即创作合成项目而非编辑项目时:
bash
node <SKILL_DIR>/scripts/carve.mjs --comp index.html这就是完整的命令。它会自动识别画外音和背景音乐,以默认强度进行动态雕刻,并输出它的决策:
bed music-bed (name looks like music)
voice narration (only track left)
carve strength 0.25 dynamic
bands 400Hz -6dB q1.4, 1000Hz -3dB q1.4, 1600Hz -3.17dB q1.4
level 216-point envelope, floor -6 dB当自动选择错误时,可以使用/(可重复)指定轨道,使用调整强度,使用查看报告但不写入任何内容。
--bed--voice--strength--dry-run轨道选择逻辑。首先根据名称判断,因为这是你已经明确告知的信息,且结果可解释——核心模块中的函数,与Studio自身选择器使用的分类器相同,因此两者的判断不会冲突。ID或文件名看起来像音乐的轨道(包含、、、等关键词)会被识别为背景音乐;所有其他与背景音乐同时播放且不属于音效类的轨道会被识别为画外音。优先选择音频元素:只有当没有音频轨道可以作为画外音,或者合成项目中的所有B-roll片段都被识别为语音时,才会考虑视频轨道。当无法区分哪个是背景音乐时,它会拒绝执行——手动输入一个ID很简单。
classifyAudioNamemusicbgmbedscore使用与面板相同的分析函数,因此结果完全一致。需要在PATH中配置,并且项目中已安装()——CLI会内联核心模块,而非从外部调用,因此无法直接借用已安装的核心模块。
ffmpeg@hyperframes/corenpm i -D @hyperframes/core它写入的内容是一组普通的峰值滤波器链加上一个阶段,并标记为。这个标记是关键:重新运行命令会替换之前的雕刻效果,而你手动添加的所有效果以及手动绘制的所有轨道都会保持原样。因此,以新强度重新雕刻是安全且可重复的,的存在是为了让设置可以被读取,而非从滤波器中反向推导。
gainfromCarvedata-fx-carveAutomation
自动化
A lane is a set of breakpoints on one parameter: in clip-local seconds
and the parameter's own units. Targets are for the track's level, or
for an effect's knob.
{t, v}volumefx.<nodeId>.<param>Only some parameters can be automated, and a lane on the others is silently
inert. A knob is automatable when a Web Audio backs it. The four
worklet-based effects — , , , — expose
none at all, so no lane on any of their parameters will ever move: to make a
compressor's behaviour change over time, automate a stage before it
instead. marks every parameter.
AudioParamcompressorlimitergatebitcrushgainreferences/fx-registry.md轨道是单个参数上的一组断点:以剪辑本地秒数为单位的,以及参数自身的单位。目标可以是轨道音量的,或是效果旋钮的。
{t, v}volumefx.<nodeId>.<param>只有部分参数可以被自动化,其他参数上的轨道会被静默忽略。当参数由Web Audio的支持时,该旋钮才可被自动化。四个基于worklet的效果——、、、——不暴露任何可自动化参数,因此它们的任何参数轨道都不会生效:如果要让压缩器的行为随时间变化,请在它之前自动化一个阶段。标记了每个参数是否可自动化。
AudioParamcompressorlimitergatebitcrushgainreferences/fx-registry.mdVerify
验证
Almost no static gate covers the mix. The linter reads for
exactly one conflict — , a volume lane on a track
that also has a GSAP tween on , where the lane wins and the tween is
ignored — and nothing validates the chain or the effect lanes at all. What
enforces those is the render: a chain it cannot parse fails the whole mix rather
than quietly writing the dry signal, because a mix that sounds plausible and is
wrong is worse than a refusal. Preview is the opposite by design: an unreadable
chain plays dry so the composition stays workable.
data-automationaudio_volume_double_automationvolumeA lane pointing at a node the chain does not have is pruned on read, not an
error — so a typo'd costs you the envelope silently. Read the ids back
out of the chain rather than assuming what was minted.
nodeIdEffects with a tail (, ) make the rendered track longer than
its source, and the mix is told how much by the chain. So a bed with reverb no
longer ends exactly at its ; that is expected, not a bug.
reverbdelaydata-durationBeyond that, a mix is verified by rendering and listening. For a carve: the voice
should be legible without the bed sounding hollowed, and with the bed
should come back up between phrases rather than staying flat. If the bed sounds
notched rather than simply quieter under the voice, the strength is too high —
that is the one failure mode with an obvious sound.
dynamic几乎没有静态门限器能适配整个混音。检查器只会读取以查找一种冲突——,即轨道上同时存在音量轨道和GSAP的补间动画,此时轨道会生效,补间动画会被忽略——而不会验证效果链或效果轨道的有效性。执行验证的是渲染过程:无法解析的效果链会导致整个混音失败,而非静默导出干信号,因为听起来合理但实际错误的混音比拒绝执行更糟糕。预览则设计相反:无法读取的效果链会播放干信号,以便合成项目可以继续编辑。
data-automationaudio_volume_double_automationvolume指向效果链中不存在的节点的轨道会在读取时被修剪,而非报错——因此输入错误的会导致包络线静默失效。请从效果链中读取ID,而非假设生成的ID。
nodeId带有尾音的效果(、)会让渲染后的轨道长于源轨道,混音会根据效果链得知延长的时长。因此带有混响的背景音乐不会恰好在其结束时停止;这是预期行为,而非错误。
reverbdelaydata-duration除此之外,混音的验证需要通过渲染和试听来完成。对于雕刻效果:画外音应清晰可辨,同时背景音乐不会听起来空洞;开启模式时,背景音乐应在画外音的停顿期间恢复音量,而非保持低音量。如果背景音乐在画外音下听起来像是被切掉了频段而非只是音量降低,说明强度设置过高——这是一种有明显特征的失败模式。
dynamic