Back to Details

video-to-music

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Sonilo Video-to-Music

Sonilo 视频转音乐

Hand Sonilo a finished video and it composes an original score to the cut — pacing, motion, and emotion matched, with transitions and beat drops aligned to the edit, at exactly the video's length. This is Sonilo's flagship capability. Every track is licensed (music licensed via Shutterstock) and cleared for commercial use on social, brand content, and advertising.
Setup: See the setup-api-key skill to connect the Sonilo MCP server and authenticate — 
sonilo login
(no key) or 
SONILO_API_KEY
.
⚠️ Cost: every tool below makes an API call that may incur charges. Only call it when the user has actually asked for a generation. Check 
get_account_services
(see the account skill) if you're unsure whether free-trial runs remain.
将成品视频交给Sonilo，它会根据视频剪辑创作原创配乐——匹配视频的节奏、画面运动和情感，转场和节拍卡点与剪辑对齐，时长与原视频完全一致。这是Sonilo的核心功能。所有曲目均已获得授权（音乐通过Shutterstock授权），可用于社交平台、品牌内容和广告等商业场景。
设置： 请查看setup-api-key技能以连接Sonilo MCP服务器并完成认证——可使用
sonilo login
（无需密钥）或
SONILO_API_KEY
。
⚠️ 费用说明： 以下所有工具调用API均可能产生费用。仅当用户明确要求生成内容时才调用该工具。若不确定是否还有免费试用额度，请调用
get_account_services
（查看account技能）。

Transport: MCP or CLI

传输方式：MCP 或 CLI

Pick one at the start of the session and stay on it. Do not mix the two inside a single job, and do not announce the choice.
  1. Sonilo MCP tools visible in this session (
    video_to_music
    and friends) — use them. This is the preferred path: it needs no shell, and it is the only one that survives a very long generation. If a call fails to authenticate — rather than failing on its inputs — this transport is not usable in this session: go to 2 instead of retrying it.
  2. No usable Sonilo MCP tools, but 
    sonilo account
    exits 0     — use the CLI commands below. Same API, same account, same credential file. Probe with 
    sonilo account
    , not 
    sonilo whoami
    : whoami exits 0 even when signed out, so it cannot tell the two states apart.
  3. Neither — stop and run the setup-api-key skill. Do not call 
    api.sonilo.com
    with curl to work around it; both transports handle uploads, polling and retries that a bare request does not.
在会话开始时选择一种方式并保持使用，不要在单个任务中混用两种方式，也无需告知用户选择的方式。
  1. 当前会话中可见的Sonilo MCP工具（
    video_to_music
    及相关工具）——优先使用该方式。此方式无需Shell环境，且是唯一支持超长生成任务的方式。若调用时认证失败（非输入参数错误），则当前会话无法使用该传输方式，请切换至方式2，不要重试。
  2. 无可用Sonilo MCP工具，但
    sonilo account
    执行返回0    ——使用下方的CLI命令。调用的是同一API、同一账户、同一凭证文件。请用
    sonilo account
    检测状态，不要用
    sonilo whoami
    ：即使未登录，
    whoami
    也会返回0，无法区分登录状态。
  3. 上述两种方式均不可用——停止操作并运行setup-api-key技能。不要使用curl直接调用
    api.sonilo.com
    来绕过限制；两种传输方式均处理了上传、轮询和重试逻辑，而裸请求不具备这些功能。

Quick Start

快速开始

MCP tool call (recommended)

MCP工具调用（推荐）

video_to_music(
    video_path="~/Desktop/trailer.mp4",
    prompt="Build suspense, then resolve with a warm cinematic finish"
)
Saves the generated file(s) to 
SONILO_MCP_BASE_PATH
(
~/Desktop
by default) and returns the saved path(s) as text.
video_to_music(
    video_path="~/Desktop/trailer.mp4",
    prompt="Build suspense, then resolve with a warm cinematic finish"
)
生成的文件会保存到
SONILO_MCP_BASE_PATH
（默认路径为
~/Desktop
），并返回保存路径的文本信息。

Python (
pip install sonilo
)

Python（需执行
pip install sonilo
）

python
from sonilo import Sonilo

client = Sonilo()  # reads SONILO_API_KEY

score = client.video_to_music.generate(video="trailer.mp4", prompt="Build suspense, then resolve with a warm cinematic finish")
score.save("score.m4a")
python
from sonilo import Sonilo

client = Sonilo()  # 读取SONILO_API_KEY

score = client.video_to_music.generate(video="trailer.mp4", prompt="Build suspense, then resolve with a warm cinematic finish")
score.save("score.m4a")

video_to_video_music: get the video back with the music muxed in

video_to_video_music：返回已合成配乐的视频

video = client.video_to_video_music.generate(video="trailer.mp4", prompt="cinematic, uplifting") video.save("scored.mp4")

`preserve_speech=True` on `video_to_music` requires the async path — use `generate_async()` instead of `generate()` to get the extra `vocals`/`mux`/`ducked` outputs; see [sonilo-python's README](https://github.com/sonilo-ai/sonilo-python#preserve-speech-async) for the full pattern.
video = client.video_to_video_music.generate(video="trailer.mp4", prompt="cinematic, uplifting") video.save("scored.mp4")

`video_to_music`中的`preserve_speech=True`参数需要使用异步调用——请使用`generate_async()`替代`generate()`，以获取额外的`vocals`/`mux`/`ducked`输出；完整调用模式请查看[sonilo-python的README](https://github.com/sonilo-ai/sonilo-python#preserve-speech-async)。

JavaScript / TypeScript (
npm install sonilo
)

JavaScript / TypeScript（需执行
npm install sonilo
）

ts
import { SoniloClient } from "sonilo";

const client = new SoniloClient(); // reads SONILO_API_KEY

const score = await client.videoToMusic.generate({
  video: "./trailer.mp4",
  prompt: "Build suspense, then resolve with a warm cinematic finish",
});

// video_to_video_music: get the video back with the music muxed in
const video = await client.videoToVideoMusic.generate({
  video: "./trailer.mp4",
  prompt: "cinematic, uplifting",
});
preserveSpeech: true
on 
videoToMusic
requires the async path — use 
.submit()
+ 
client.tasks.wait()
instead of 
.generate()
; see sonilo-js's README for the full pattern.
ts
import { SoniloClient } from "sonilo";

const client = new SoniloClient(); // 读取SONILO_API_KEY

const score = await client.videoToMusic.generate({
  video: "./trailer.mp4",
  prompt: "Build suspense, then resolve with a warm cinematic finish",
});

// video_to_video_music：返回已合成配乐的视频
const video = await client.videoToVideoMusic.generate({
  video: "./trailer.mp4",
  prompt: "cinematic, uplifting",
});
videoToMusic
中的
preserveSpeech: true
参数需要使用异步调用——请使用
.submit()
+ 
client.tasks.wait()
替代
.generate()
；完整调用模式请查看sonilo-js的README。

CLI (
npm install -g sonilo-cli
or 
pip install sonilo-cli
)

CLI（需执行
npm install -g sonilo-cli
或
pip install sonilo-cli
）

bash
sonilo video-to-music --video trailer.mp4 --prompt "Build suspense, then resolve with a warm cinematic finish" --output score.m4a
--format wav
, 
--preserve-speech
, and 
--isolate-vocals
each switch 
video-to-music
to the async submit-and-poll path.
bash
undefined
bash
sonilo video-to-music --video trailer.mp4 --prompt "Build suspense, then resolve with a warm cinematic finish" --output score.m4a
--format wav
、
--preserve-speech
和
--isolate-vocals
参数会将
video-to-music
切换为异步提交并轮询的模式。
bash
undefined

the muxed video, from the CLI

CLI生成已合成配乐的视频

sonilo video-to-video-music --video trailer.mp4 --prompt "cinematic, uplifting" --output scored.mp4
undefined
sonilo video-to-video-music --video trailer.mp4 --prompt "cinematic, uplifting" --output scored.mp4
undefined

cURL (raw REST API, no MCP host)

cURL（原生REST API，无MCP主机）

bash
curl -X POST "https://api.sonilo.com/v1/video-to-music" \
  -H "Authorization: Bearer $SONILO_API_KEY" \
  -F "video=@trailer.mp4" \
  -F "prompt=Build suspense, then resolve with a warm cinematic finish" \
  --output score.m4a
video_to_music
also accepts a 
video_url
form field instead of an uploaded file — pass one or the other, never both.
bash
curl -X POST "https://api.sonilo.com/v1/video-to-music" \
  -H "Authorization: Bearer $SONILO_API_KEY" \
  -F "video=@trailer.mp4" \
  -F "prompt=Build suspense, then resolve with a warm cinematic finish" \
  --output score.m4a
video_to_music
也接受
video_url
表单字段替代上传文件——二选一，不可同时使用。

Tools

工具列表

ToolDescription
video_to_music(video_path? | video_url?, prompt?, preserve_speech?, output_format?, ducking?, variants_num?, prompt_influence?, output_directory?)
Score a video: matches pacing/motion/emotion, matches the video's duration exactly. Returns audio only (the video itself is not muxed).
video_to_video_music(video_path? | video_url?, prompt?, segments?, keep_original_sound?, ducking?, preserve_speech?, variants_num?, prompt_influence?, output_directory?)
Same scoring, but returns a new 
.mp4
 with the music already muxed in. By default the source's own audio is dropped — see 
keep_original_sound
.
工具描述
video_to_music(video_path? | video_url?, prompt?, preserve_speech?, output_format?, ducking?, variants_num?, prompt_influence?, output_directory?)
为视频创作配乐：匹配节奏、画面运动和情感，时长与原视频完全一致。仅返回音频文件（不合成原视频）。
video_to_video_music(video_path? | video_url?, prompt?, segments?, keep_original_sound?, ducking?, preserve_speech?, variants_num?, prompt_influence?, output_directory?)
配乐功能与上述工具相同，但返回新的
.mp4
文件，已将配乐与原视频合成。默认会移除原视频的音频——详情请查看
keep_original_sound
参数说明。

Parameters

参数说明

ParameterTypeDefaultNotes
prompt
string—Optional style hint. Omit to let the footage lead entirely.
video_path
string—Absolute path, or relative to 
SONILO_MCP_BASE_PATH
. 
.mp4/.mov/.avi/.wmv/.webm/.mkv
. Max 360s (6 min), subject to the account's upload-size cap (typically 300 MB).
video_url
string—HTTPS URL to a video file. Exactly one of 
video_path
/
video_url
.
preserve_speech
bool
false
Keeps the source speech audible. On 
video_to_music
, also returns a 
vocals
speech stem and a ready-to-use 
mux
(speech+music mixed) — this makes the call run asynchronously (submit + poll) instead of streaming, so it takes a bit longer but the tool still waits for completion.
ducking
bool | nullserver default ON for 
video_to_music
, 
false
for 
video_to_video_music
Dips the generated music under the source voice. Free, best-effort. On 
video_to_video_music
it only does anything alongside 
keep_original_sound
or 
preserve_speech
— with neither, there is no source voice left in the output to duck under.
keep_original_sound
bool
false
video_to_video_music
only. This is the parameter to reach for when the result sounds wrong. By default the returned 
.mp4
carries the generated music alone — the source's dialogue, room tone, and effects are gone. Set 
true
to keep the whole source track with the music mixed under it, and add 
ducking=true
to dip the music under the voice rather than mixing it flat. 
keep_original_sound
supersedes 
preserve_speech
.
variants_num
int
1
1–10. Generates that many distinct creative directions in one request — different takes, not re-renders of one. Cost scales linearly with the count, and any value above 1 is never covered by the free trial, so confirm the number with the user first. Above 1 writes one file per variant and forces the backend's async mode.
prompt_influence
float | nullAPI default 
0.5
0–1: how strictly the music follows your prompt versus what the video itself suggests. Lower lets the footage lead, higher enforces the brief. Free, and it does not change the mode or the number of files — omit it unless the user asks for stricter or looser adherence.
output_format
string
m4a
video_to_music
only — 
video_to_video_music
has no such param and always outputs a muxed 
.mp4
. 
m4a
or 
wav
. 
wav
(and 
preserve_speech
/
ducking
) triggers the backend's async mode internally.
output_directory
string
SONILO_MCP_BASE_PATH
Absolute, or relative to the base path.
参数类型默认值说明
prompt
string—可选的风格提示词。若省略，则完全由视频内容主导配乐风格。
video_path
string—绝对路径，或相对于
SONILO_MCP_BASE_PATH
的路径。支持格式：
.mp4/.mov/.avi/.wmv/.webm/.mkv
。最大时长360秒（6分钟），同时受账户上传大小限制（通常为300 MB）。
video_url
string—视频文件的HTTPS URL。
video_path
和
video_url
二选一，不可同时使用。
preserve_speech
bool
false
保留原视频中的语音内容。在
video_to_music
中，还会返回
vocals
语音分离轨和可直接使用的
mux
（语音+音乐混合）文件——此参数会将调用切换为异步模式（提交+轮询），而非流式传输，因此耗时稍长，但工具仍会等待任务完成。
ducking
bool | null
video_to_music
默认开启，
video_to_video_music
默认关闭		自动降低配乐音量以突出原视频的语音。免费功能，效果为最佳尝试。在
video_to_video_music
中，仅当同时设置
keep_original_sound
或
preserve_speech
时该参数才生效——若两者都未设置，输出视频中已无原语音，无需降低配乐音量。
keep_original_sound
bool
false
仅适用于
video_to_video_music
。若生成结果音效异常，请优先检查此参数。默认情况下，返回的
.mp4
文件仅包含生成的配乐——原视频的对话、环境音和音效会被移除。设置为
true
可保留原视频的完整音轨，并将配乐混合在下方；若添加
ducking=true
，则会自动降低配乐音量以突出语音，而非平层混合。
keep_original_sound
优先级高于
preserve_speech
。
variants_num
int
1
取值范围1–10。一次请求生成多个不同风格的配乐——为不同创作方向，而非同一风格的重新渲染。费用随数量线性增加，且数量大于1时无法使用免费试用额度，请先与用户确认数量。数量大于1时，每个风格会生成一个文件，并强制后端使用异步模式。
prompt_influence
float | nullAPI默认值
0.5
取值范围0–1：控制配乐遵循提示词的严格程度，相对视频内容的主导程度。值越低，视频内容对配乐风格的影响越大；值越高，提示词对配乐风格的约束越强。该功能免费，且不会改变调用模式或生成文件数量——除非用户要求调整约束程度，否则无需设置。
output_format
string
m4a
仅适用于
video_to_music
——
video_to_video_music
无此参数，始终输出合成后的
.mp4
文件。支持格式：
m4a
或
wav
。设置为
wav
（或启用
preserve_speech
/
ducking
）会触发后端的异步模式。
output_directory
string
SONILO_MCP_BASE_PATH
绝对路径，或相对于基础路径的相对路径。

Prompting

提示词使用

No prompt is required — the model reads the cut. A short structured brief adds what the video can't carry: your intent — genre, the moment that must hit, the sounds that must not appear.
Before a paid call: probe the exact duration and existing audio, respect the 360 s cap (over = 422 reject, never truncated), and get sign-off — failed runs auto-refund, but your own retry is a new charge. Write the brief first, generate once, iterate on the prompt, not on rerolls.
  • Full pre-flight (inspect the video, caps, credits, verification): references/preflight.md
  • Style-prompt craft (audio brief, genre/energy wording, 
    preserve_speech
    , segmented music): references/music-prompting.md
无需提供提示词——模型会自动分析视频剪辑。简短结构化的提示词可补充视频无法传递的信息：你的创作意图，如流派、必须突出的节点、禁止出现的音效。
在付费调用前：确认视频的准确时长和已有音轨，遵守360秒时长限制（超过该限制会返回422错误，不会自动截断），并获得用户确认——失败的任务会自动退款，但重复重试会产生新的费用。请先编写提示词，生成一次后，通过调整提示词优化结果，而非重复生成。
  • 完整预检流程（检查视频、限制条件、版权、验证）：references/preflight.md
  • 风格提示词编写技巧（音频需求说明、流派/能量词汇、
    preserve_speech
    、分段配乐）：references/music-prompting.md

Workflow Tips

工作流技巧

  • Prompting: a style hint (e.g. "cinematic, uplifting") steers the result, but the prompt is optional — Sonilo already reads the footage.
  • preserve_speech
    for talking-head or narrated video:     if the source has dialogue/voiceover the user wants kept, set 
    preserve_speech=true
    . Behavior differs by tool: on 
    video_to_music
    you get the music, the isolated speech stem, and a ready-mixed combined file (the mux) — use the mux directly rather than re-mixing yourself. On 
    video_to_video_music
    there's no separate stem or mux file; it just keeps the source speech audible in the single muxed output video.
  • Want the video back with the score baked in? Use 
    video_to_video_music
    instead of 
    video_to_music
    — same inputs, but the output is a new 
    .mp4
    , not just audio. Warn the user that the source audio is dropped by default: if their video has dialogue, narration, or effects they expect to hear, pass 
    keep_original_sound=true
    (add 
    ducking=true
    to keep the voice on top), or 
    preserve_speech=true
    for the isolated speech only. A "the music is there but my voiceover vanished" report is always this.
  • Several takes in one go: 
    variants_num=3
    returns three distinct directions for one request instead of three re-rolls. It costs 3×, and it is never free-trial covered — say the price before calling.
  • Duration: always matched to the source video automatically — don't ask for it.
  • Need SFX too? To generate music and sound effects for the same video in one balanced, single-charge call, use video-to-sound rather than calling this and video-to-sfx separately.
  • Content restriction: prompts cannot reference specific artists, bands, or copyrighted lyrics.
  • 提示词使用：风格提示词（如"cinematic, uplifting"）可引导生成结果，但提示词为可选——Sonilo已能自动分析视频内容。
  • 访谈或旁白视频使用
    preserve_speech
    ：若原视频包含需要保留的对话/旁白，请设置
    preserve_speech=true
    。不同工具的行为有所区别：在
    video_to_music
    中，你会获得配乐、分离的语音轨，以及已混合好的文件（mux）——建议直接使用mux文件，无需自行混合。在
    video_to_video_music
    中，不会生成单独的音轨或混合文件，仅会在合成后的视频中保留原语音内容。
  • 需要返回已合成配乐的视频？ 使用
    video_to_video_music
    替代
    video_to_music
    ——输入参数相同，但输出为新的
    .mp4
    文件，而非仅音频。请提醒用户默认会移除原视频音频：若用户希望保留视频中的对话、旁白或音效，请设置
    keep_original_sound=true
    （添加
    ducking=true
    可突出语音），或设置
    preserve_speech=true
    仅保留分离的语音。用户反馈“配乐存在但旁白消失”通常是因为未设置该参数。
  • 一次生成多个版本：
    variants_num=3
    可在一次请求中返回3种不同风格的配乐，而非三次重复生成。费用为单次的3倍，且无法使用免费试用额度——请在调用前告知用户费用。
  • 时长：配乐时长会自动匹配原视频，无需额外设置。
  • 同时需要音效？ 若要为同一视频同时生成配乐和音效，且保证两者平衡、仅收取一次费用，请使用video-to-sound，而非分别调用本工具和video-to-sfx。
  • 内容限制：提示词不得提及特定艺术家、乐队或受版权保护的歌词。

Recovering a Timed-Out Call

超时任务恢复

video_to_music
(without any of 
preserve_speech
/
ducking
/
output_format="wav"
) streams its result in one call. Any variant that triggers the backend's async mode — and 
video_to_video_music
, which is always async — can time out on a very long 
TIME_OUT_SECONDS
. If it does, the error message includes a 
task_id
; the generation keeps running (and is already charged) on the backend. Call 
get_sfx_task(task_id)
— 
get_generation_task(task_id)
on the hosted server — to retrieve the result once ready; see the task-recovery skill.
video_to_music
（未启用
preserve_speech
/
ducking
/
output_format="wav"
时）会通过流式传输返回结果。任何触发后端异步模式的调用——以及始终为异步模式的
video_to_video_music
——在超长时间任务中可能会超时。若发生超时，错误信息中会包含
task_id
；生成任务仍会在后端继续运行（且已产生费用）。调用
get_sfx_task(task_id)
——在托管服务器上为
get_generation_task(task_id)
——即可在任务完成后获取结果；详情请查看task-recovery技能。

Output Files

输出文件

  • video_to_music
    : 
    .m4a
    by default (
    .wav
    if requested), named from the prompt (slugified) or 
    sonilo-<timestamp>.m4a
    . Multiple parallel streams get a 
    -<index>
    suffix.
  • video_to_music(preserve_speech=true)
    : also saves 
    <base>-vocals.<ext>
    (isolated speech) and 
    <base>-mux.<ext>
    (speech+music combined — the one to actually use).
  • video_to_video_music
    : a single 
    .mp4
    with the score muxed in.
  • video_to_music
    ：默认输出
    .m4a
    格式（若指定则输出
    .wav
    ），文件名由提示词转换而来（小写短横线格式），或为
    sonilo-<timestamp>.m4a
    。多个并行生成的文件会添加
    -<index>
    后缀。
  • video_to_music(preserve_speech=true)
    ：还会保存
    <base>-vocals.<ext>
    （分离的语音轨）和
    <base>-mux.<ext>
    （语音+音乐混合文件——建议直接使用此文件）。
  • video_to_video_music
    ：单个已合成配乐的
    .mp4
    文件。

Error Handling

错误处理

Common errors: 
401
invalid key, 
402
insufficient balance / trial exhausted, 
413
file too large, 
422
invalid parameters (e.g. video over the 360 s cap), 
429
rate limit. See the account skill to check trial/usage before a call, and the sonilo-mcp README's error table for exact recovery steps.
常见错误：
401
密钥无效，
402
余额不足/试用额度耗尽，
413
文件过大，
422
参数无效（如视频超过360秒时长限制），
429
请求频率超限。调用前请查看account技能检查试用/使用情况，具体恢复步骤请查看sonilo-mcp README中的错误表。