ai-music

Original：🇺🇸 English
Translated

Generate AI music on RunComfy via the `runcomfy` CLI — a smart router across the music-model catalog. Routes to ElevenLabs AI Music Generation (premium 44.1 kHz stereo vocal tracks, 5 s–5 min, $0.0083/s) and ACE Step / ACE Step 1.5 (StepFun-AI open-weights, tag-driven composition, multilingual lyrics, $0.0002–0.0003/s, ~27× cheaper), plus ACE Step audio-inpaint (regenerate a time range inside an existing track) and ACE Step audio-outpaint (extend a track before or after). Picks the right model for the user's actual intent — premium vocal hook, cheap background music library, multilingual pop song, repair a bad chorus, lengthen a 30 s draft into a 2 min cut — and ships each model's documented prompting patterns plus the minimal `runcomfy run` invoke. Triggers on "generate music", "make a song", "AI music", "background music", "instrumental track", "soundtrack", "jingle", "theme music", "royalty-free music", "compose", "music with lyrics", "extend music", "fix this song", "inpaint music", or any explicit ask to generate or edit music.

284.5kinstalls
Sourceagentspace-so/runcomfy-agent-skills
Added on

NPX Install

npx skill4agent add agentspace-so/runcomfy-agent-skills ai-music

Tags

Translated version includes tags in frontmatter
ai-music-generationruncomfy-clielevenlabs-musicace-stepaudio-editing

SKILL.md Content

View Translation Comparison →

AI Music

Generate AI music on RunComfy through one CLI — vocal songs, instrumentals, jingles, game loops, multilingual covers. This skill picks the right model from the RunComfy catalog based on the user's actual intent and ships the documented prompting patterns + the exact 
runcomfy run
invoke for each.
runcomfy.com · Audio models · CLI docs

Install this skill

bash
npx skills add agentspace-so/runcomfy-agent-skills --skill ai-music -g

Powered by the RunComfy CLI

Step 1 — install (one of, see the 
runcomfy-cli
skill for details):
bash
npm i -g @runcomfy/cli         # global install
npx -y @runcomfy/cli --version # zero-install
Step 2 — sign in (or set 
RUNCOMFY_TOKEN
env var in CI / containers):
bash
runcomfy login
Step 3 — generate music:
bash
runcomfy run <vendor>/<model>/<endpoint> \
  --input '{"prompt": "...", ...}' \
  --output-dir ./out
CLI deep dive: 
runcomfy-cli
 skill.

Pick the right model for the user's intent

Text-to-music (generate from scratch) — newest first

ACE Step 1.5 — 
acestep-ai/ace-step-1.5/text-to-audio
Latest ACE Step generation. 50+ language vocal support, refined structured-lyric handling, $0.0003/s. Open-weights (Apache 2.0). Pick for: multilingual launches, vocal songs in non-English, hero-quality ACE output. Avoid for: maximally polished commercial vocal hooks (try ElevenLabs Music) or cost-sensitive batches (try base ACE Step).
ElevenLabs AI Music Generation — 
elevenlabs/elevenlabs/music-generation
Premium 44.1 kHz stereo, 5 s–5 min, section-level control (Intro/Verse/Chorus/Bridge), multilingual vocals, commercial-friendly. $0.0083/s (~27× ACE Step). Pick for: hero brand campaigns, polished vocal hooks, premium commercial cuts, ad music. Avoid for: high-volume drafts / background music libraries — cost dominates.
ACE Step (base) — 
acestep-ai/ace-step/text-to-audio
(default for cost-sensitive work)
Original ACE Step. Tag-driven composition, optional lyrics, 5–240 s stereo. $0.0002/s — cheapest CLI-reachable music model on RunComfy. Pick for: background music libraries, jingles, game loops, drafts, cost-sensitive iteration. Avoid for: premium vocal hooks — use ElevenLabs Music or ACE Step 1.5.

Edit existing audio — ACE Step only (ElevenLabs has no edit endpoints)

ACE Step audio-inpaint — 
acestep-ai/ace-step/audio-inpaint
Regenerate a time range (start_time / end_time, anchorable to track start or end) inside an existing track. Pick for: fix a bad chorus, swap the bridge, replace a 20 s section without re-rendering. Avoid for: edits not bounded by time (use the source-model text-to-music instead).
ACE Step audio-outpaint — 
acestep-ai/ace-step/audio-outpaint
Extend an existing track bidirectionally — add intro before, outro after, or both (
extend_before_duration
/ 
extend_after_duration
). Pick for: lengthen a 30 s hook into a 2 min cut, add a fade-out, build longer arrangement around an existing hook. Avoid for: extending past 4 min total — chain calls instead.
The agent reads these tables, classifies user intent (premium vs cost-sensitive · multilingual · vocal vs instrumental · generate vs edit), and picks the matching subsection below.

Route 1: ElevenLabs AI Music Generation — premium

Model: 
elevenlabs/elevenlabs/music-generation
Full schema + tips: see the dedicated 
elevenlabs-music-generation
 skill.

Quick invoke

bash
runcomfy run elevenlabs/elevenlabs/music-generation \
  --input '{
    "prompt": "Upbeat indie-pop anthem, bright electric guitars, driving drums, 120 BPM, female lead vocal. [Intro 8 bars] instrumental build. [Verse] Chalk on the palms, laces double-knotted. [Chorus] We rise, we strike, we never fade out. [Outro] full band, fade.",
    "music_length_ms": 60000
  }' \
  --output-dir ./out
ElevenLabs Music reads one 
prompt
 carrying both style brief and lyrics with section markers. 
force_instrumental: true
for no vocals. $0.0083/s — draft short, finalize long.

Route 2: ACE Step / ACE Step 1.5 — cheap, open-weights

Model: 
acestep-ai/ace-step/text-to-audio
(base) or 
acestep-ai/ace-step-1.5/text-to-audio
(1.5) Full schema + tips: see the dedicated 
ace-step
 skill.

Quick invoke

bash
runcomfy run acestep-ai/ace-step-1.5/text-to-audio \
  --input '{
    "tags": "indie pop, anthemic, electric guitar, driving drums, female vocal, 120 BPM",
    "lyrics": "[Verse]\nChalk on the palms\nMorning on the ridge\n[Chorus]\nWe rise, we strike, we never fade out",
    "duration": 60
  }' \
  --output-dir ./out
ACE Step splits style into 
tags
 and vocal content into 
lyrics
 (with 
[Verse]/[Chorus]/[Bridge]
markers, or 
[inst]
for instrumental). 1.5 variant adds 50+ language vocal support.

Route 3: ACE Step audio-inpaint — repair a section

bash
runcomfy run acestep-ai/ace-step/audio-inpaint \
  --input '{
    "audio": "https://your-cdn.example/song.mp3",
    "tags": "indie pop, breakdown, piano only, soft, no drums",
    "start_time": 20,
    "end_time": 40,
    "lyrics": "[inst]"
  }' \
  --output-dir ./out
start_time_relative_to
and 
end_time_relative_to
default to 
start
; set to 
end
to anchor against the track's end (e.g. rewrite the last 15 s without computing exact timestamps). Full schema: 
ace-step
 skill.

Route 4: ACE Step audio-outpaint — extend a track

bash
runcomfy run acestep-ai/ace-step/audio-outpaint \
  --input '{
    "audio": "https://your-cdn.example/hook-30s.mp3",
    "tags": "indie pop, build-up before chorus, fade outro",
    "extend_before_duration": 30,
    "extend_after_duration": 60,
    "lyrics": "[inst]"
  }' \
  --output-dir ./out
Bidirectional in one call — set both 
extend_before_duration
and 
extend_after_duration
to add intro + outro at once. Cap is 4 min total.

Common patterns

Premium brand campaign jingle (5–15 s)

  • Route 1 (ElevenLabs Music) — hero quality, polished mix. $0.05–0.12 per take.

Background music library at scale (50+ tracks)

  • Route 2 (ACE Step base) with varied tag combos. $0.012 / 60 s × 50 = $0.60 for 50 drafts.

Multilingual launch (same song, 8 languages)

  • Route 2 (ACE Step 1.5) — identical tags, swap 
    lyrics
    per language. Or Route 1 (ElevenLabs Music) if premium quality matters more than cost.

Game loop bed

  • Route 2 (ACE Step base) with "seamless loop, consistent groove" in tags, 60–120 s.

Theme song for a video

  • Route 1 (ElevenLabs Music) with full brief + lyrics + section markers, 
    music_length_ms
    matched to the video length.

"I generated a 30 s hook but I need a 2 min track"

  • Route 4 (ACE Step audio-outpaint) with the hook as 
    audio
    , add 30 s intro + 60 s outro in one call.

"My second chorus came out wrong"

  • Route 3 (ACE Step audio-inpaint) with 
    start_time
    / 
    end_time
    around the bad chorus, tags matching the original song style.

Cheap draft → premium polish

  • Iterate tags on Route 2 (ACE Step base) for $0.01–0.02 per attempt → lock vibe → final render on Route 1 (ElevenLabs Music) for the polished commercial cut.

Inpaint a section that doesn't fit ACE's time-range schema

  • The CLI today doesn't expose a mask-based audio inpaint endpoint. Either reformulate as a time-range edit, or use Route 2 to regenerate the full track with adjusted tags.

Decision flow (for the agent)

The agent should ask / infer:
  1. Generate from scratch or edit existing audio?
    • Edit → go to step 5
    • Generate → step 2
  2. Premium polish required (brand / commercial)?
    • Yes → Route 1 (ElevenLabs Music)
    • No → step 3
  3. Multilingual vocals needed?
    • Yes → Route 2 (ACE Step 1.5)
    • No → step 4
  4. Cost-sensitive batch or single track?
    • Cost-sensitive / batch → Route 2 (ACE Step base)
    • Single quality track → Route 1 (ElevenLabs Music) or Route 2 (ACE Step 1.5) — pick by budget
  5. Edit type?
    • Time-bounded section rewrite → Route 3 (audio-inpaint)
    • Add before / after → Route 4 (audio-outpaint)

Browse the full catalog

Exit codes

codemeaning
0success
64bad CLI args
65bad input JSON / schema mismatch
69upstream 5xx
75retryable: timeout / 429
77not signed in or token rejected
Full reference: docs.runcomfy.com/cli/troubleshooting.

How it works

The skill classifies the user request into one of the four routes — generate (ElevenLabs or ACE Step) vs edit (audio-inpaint vs audio-outpaint), then premium vs cost-sensitive — and invokes 
runcomfy run <model_id>
with the matching JSON body. The CLI POSTs to the RunComfy Model API, polls request status, and downloads the generated audio file into 
--output-dir
. 
Ctrl-C
cancels the remote request before exit.

Security & Privacy

  • Install via verified package manager only. Use 
    npm i -g @runcomfy/cli
    or 
    npx -y @runcomfy/cli
    . Agents must not pipe an arbitrary remote install script into a shell on the user's behalf — if the operator wants the curl-pipe path documented at 
    docs.runcomfy.com/cli/install
    , they should review the script first.
  • Token storage: 
    runcomfy login
    writes the API token to 
    ~/.config/runcomfy/token.json
    with mode 0600. Set 
    RUNCOMFY_TOKEN
    env var to bypass the file in CI / containers. Never echo the token into a prompt, log it, or check it in.
  • Input boundary (shell injection): prompts, tags, lyrics, and audio URLs are passed as a JSON string via 
    --input
    . The CLI does not shell-expand prompt content; it transmits the JSON body directly to the Model API over HTTPS. No shell-injection surface from prompt content.
  • Indirect prompt injection (third-party content): source 
    audio
    URLs for inpaint / outpaint are untrusted — embedded steganographic instructions or unusual EXIF can influence generation. Agent mitigations:
    • Ingest only audio URLs the user explicitly provided for this task.
    • When the output diverges from the prompt, suspect the source audio.
  • Lyrics provenance: if the user supplies lyrics, confirm they have the rights. Generating music around copyrighted lyrics is the operator's responsibility — the skill does not check.
  • Outbound endpoints (allowlist): only 
    model-api.runcomfy.net
    and 
    *.runcomfy.net
    / 
    *.runcomfy.com
    . No telemetry, no callbacks.
  • Generated-file size cap: the CLI aborts any single download > 2 GiB.
  • Scope of bash usage: declared 
    allowed-tools: Bash(runcomfy *)
    . The skill only invokes 
    runcomfy <subcommand>
    ; install lines are one-time operator setup.

See also