novel-storyboard
**Storyboard generation for AI short dramas — the first layer in the pipeline that directly interfaces with video models. Core premise baked in: shots are generated, and the cost of adding an extra cut is nearly zero, so there's no need to skimp on the number of shots. There's only one upper limit: the single-segment generation duration of the video model (default 15 seconds).
Core mechanism: shots are assigned to beats. Each shot declares which consecutive beats of a script scene it covers (
+
), and shots cannot span scenes — a scene change always requires a new shot. This makes the relationship between storyboard and script mechanically verifiable:
| Deliverable | What it solves |
|---|
| Beat assignment | Each beat is assigned to exactly one shot, in correct order — re-run validate when the script changes, and invalid shots are flagged immediately |
| Single shot ≤ 15 seconds | Upper limit for single-segment AI video generation; long dialogues are forcibly split into shots here ( can be adjusted per model) |
| Dialogue fits within shot duration | The duration of dialogue from assigned beats ≤ shot duration — checked per shot, no guesswork |
| Storyboard frame prompts + shot descriptions | Storyboard frame prompts (, in Chinese, using character names directly) are for image models to generate frames with reference art; shot descriptions (, in Chinese, using generic identities) describe what happens in these few seconds. Shot sizes are enumerated, and Chinese shot size terms must be included in storyboard frame prompts |
| H3 video prompts (one per segment) | MiniMax H3's I2VA structure: fixed alignment instructions + integrated_multimodal_description + overall_soundscape + non_diegetic_music. Dialogue from assigned beats is placed character by character into blocks — dialogue, soundscape, and score are all included in one segment prompt |
| Seedance video prompts (one per segment, assembled programmatically) | Parallel to H3. Assembled by scripts according to the official structure: reference image declarations, segment-level blocking, per-shot quantitative fields, dialogue , sound effects , score , numbering constraints. The model only writes the content for each cut, no durations, numbers, or image references |
| Composition quantitative fields | Per-segment (who is on left/right, spacing), per-shot focal length / camera position / composition / gaze target / focus / stability — when no storyboard frames are generated, these are the sole source of composition |
| Generation batch sheet | Shots with the same scene + same lighting are grouped into one batch, sharing the same environment reference image — the AI version of a continuity sheet, automatically compiled by scripts |
| Dubbing alignment sheet | Each line of dialogue is mapped to a shot number — which video segment TTS audio should be attached to, automatically compiled by scripts |
= the directory where this file is located. The script
{baseDir}/scripts/novel-storyboard.mjs
has zero dependencies and runs directly with
.
Boundaries (what it doesn't do): No scriptwriting or dialogue editing (that's
's job), no scene/character/prop concept art generation (that's
/
' job), no video generation or editing/compositing. Lip sync is not handled for now — that's the generation pipeline's responsibility.
Step 0 — Define Inputs and Scope
is a hard prerequisite — storyboards have no meaning without a script, and
must be provided for both validate/render. Other upstream inputs are used if available:
- / : Enables character name prohibition check in prompts + displays C01 codes as character names in reports
- : Displays S01 codes as scene names in reports + embeds scene concept art in batch sheets
- : Optional mounts the shot-recipes shot formula card library (points to
shot-recipes/references/cards
How many episodes per batch: Follows the script's batch (divides up to wherever the script is written), default batch size ≤ 3 episodes.
Step 1 — Seed Working Draft
bash
node {baseDir}/scripts/novel-storyboard.mjs seed <script.json> --eps 1-3 > <workdir>/storyboard.json
Deterministic expansion: the beat list for each scene (number, action/dialogue, seconds per beat, speaker) is loaded into
, which serves as the working draft for shot cutting.
The duration of each beat is calculated, don't let the model re-estimate it. are left empty — shot cutting is the model's job.
Step 2 — Segment and Cut Shots Per Episode
One task per episode, run concurrently if possible. Each task receives:
{baseDir}/references/storyboard-pass.md
{baseDir}/references/schema.md
- The episode's draft + scene cards (anchor points and lighting prompts from ) + character cards (appearance key points from )
Process:
First segment by plot units (each segment 9–15 seconds, no scene crossing),
then cut 2–5 second shots within segments (dialogue shot/reverse shot, close-ups inserted for key actions, three-part entry — all shot cutting grammar is in
). For each cut, write one storyboard frame prompt (in Chinese, using character names directly), one shot description
(in Chinese, generic identities), and six composition fields; for each segment, write
,
, and
if there's a score.
Shot descriptions for each segment should be written according to {baseDir}/references/shot-writing.md
(protocol-agnostic: one camera movement per cut, actions must be completable, dialogue word-for-word, layered sound, no art style description). Apply the corresponding protocol syntax based on which video service you're sending to: MiniMax H3 see
{baseDir}/references/h3-prompt.md
, Seedance see
{baseDir}/references/seedance-prompt.md
.
Write one per segment, written according to
{baseDir}/references/h3-prompt.md
(internalized version of official methodology,
no external skill dependencies). Official default is English (
can be switched to Chinese),
each shot on its own line. Key points: the first line's alignment instructions and
cut timings
are derived from storyboard durations, not a single character may deviate (validate checks character by character); assigned dialogue is placed
word for word into
; camera movement terms for each cut are written on their own line; soundscape and score are placed into the latter two fields —
soundscape is also an action instruction, so if the画面 changes, the soundscape changes too.
Seedance prompts don't need to be written manually: the prompt panel in
reports and
export --protocol seedance
will assemble them on the fly from the above fields. Visual style is attached by the caller when submitting, and global constraints are provided via
.
Delete
after cutting is complete.
Step 3 — Validation ⛔ Cannot Skip
bash
node {baseDir}/scripts/novel-storyboard.mjs validate <storyboard.json> \
--script <script.json> --outline <outline.json> --cast <cast.json> \
[--shots </path/to/cards>]
All 18 quality gates are code-enforced: full beat coverage (shot-level, exactly once, in order, consecutive), segment total duration 0 < total seconds ≤ 15,
each cut 2–5 seconds, dialogue fits within shot, total episode duration within ±15% of script target, ≤ 3 people in the same frame (exceeding requires a breakdown explanation), segment numbers in consecutive E01-01 format, Chinese shot size terms present in storyboard frame prompts, camera movements use H3 vocabulary and are within their own [Shot k] section,
H3 alignment instructions derived from storyboard structure and verified character by character + cut timings checked individually,
assigned dialogue placed word for word into blocks,
prompt language matches (bidirectional check: blocks Chinese written as English and English mixed with Chinese), storyboard frame prompts are non-empty Chinese text,
video prompts contain no character names (checked for both H3 body text regardless of language and Seedance shot descriptions; storyboard frame prompts using direct names are allowed),
composition quantitative fields are complete,
Seedance shot descriptions are compliant (non-empty Chinese, no durations, shot numbers, image references, or protocol symbols), scene/character/prop reconciliation with script,
shot recipe reconciliation (optional gate, see below).
Fix violations one by one, re-run after changes, until all pass.
18th gate (optional mount): Only checked if
is provided, otherwise explicitly skipped. An optional
can be written on a cut (recipe card ID,
cut-level, not segment-level,
multi-frame recipes are expressed by consecutive shots with the same ID, not an array). The gate checks three things — ID exists in the card library, each
from the card appears in the cut's
(case-insensitive
after lowercasing both), and when the card's
lower limit ≥ 2, the number of consecutive shots with the same ID must not be lower than that limit.
The card's suggested shot sizes and camera movements are not gated, only deviations are noted in the report's "Recipe" column and at the end of
: recipes are vocabulary, not laws — optional mounts would never be used if they become too strict.
Step 4 — Output and Reporting
bash
cd <output-directory>
node {baseDir}/scripts/novel-storyboard.mjs render <drama-name>-storyboard.json --md \
--script <script.json> --outline <outline.json> --art <art.json> > <drama-name>-storyboard.md
node {baseDir}/scripts/novel-storyboard.mjs render <drama-name>-storyboard.json --html \
--script <script.json> --outline <outline.json> --art <art.json> > storyboard-report.html
Report interface language is specified with
(priority:
> top-level
field in JSON > default Chinese) — this only affects interface labels and is independent of
(H3 prompt language).
uses
to locate storyboard frames (
<directory>/<segment-number>/f<cut-order>.png
, default current directory), and
to locate scene concept art for batch sheets (
<directory>/<scene-slug>-sheet.png
, default
). Both can point to any path —
this skill does not generate these files; re-run render after downstream image generation to embed them in the report.
export --frames <directory>
will copy found storyboard frames into the production package, no manual placement needed. The report includes: KPI bar, storyboard rhythm bar (thick dividers = segment boundaries, bar width = shot duration proportion, color depth = shot size distance, click to jump to segment card), episode-by-episode storyboard table (main storyboard frame + sub-shot bars + per-cut shot rows + copy buttons for storyboard frame/H3 prompts), generation batch sheet, dubbing alignment sheet, quality gates, exportable JSON. The Markdown version includes full H3 prompts per segment, ready to copy directly.
Report in one sentence: number of episodes and shots, total duration vs target, number of generation batches, report path; explicitly mention any failed gates.
Don't say "images have been generated" — this step doesn't exist; deliverables are prompts and reference image lists.
Final output structure:
<output-directory>/
├── <drama-name>-storyboard.json
├── <drama-name>-storyboard.md
├── storyboard-report.html ← Double-click to open
├── manifest.json ← Generated by export
└── E01-01/ ← One folder per segment = all materials for one H3 generation
├── f1.png ← Main storyboard frame (place here after downstream generation)
├── f2.png … ← Sub storyboard frames
└── prompt.md ← H3 prompt (generated by export)
Five-Skill Relay (Pipeline Closed Loop Here)
novel-outline → outline.json (What: structure and episode division)
novel-characters → cast.json (Who: character concept art)
novel-art → art.json (Where: scene/prop concept art)
novel-script → script.json (Drama: scenes, beats, dialogue)
novel-storyboard → storyboard.json (How to shoot: shots, first frames, batches)
Storyboarding is the consumer end: seed consumes
, storyboard frame generation (downstream) consumes art and characters concept art as references, H3 prompts are sent directly to the video model, and the dubbing alignment sheet connects to TTS output from the script's dialogue book. Each of the five JSON files has export buttons in its respective report, and after modifications, they can be fed back into their respective render/validate processes.
Boundaries
- Report interface has built-in Chinese/English support (, default Chinese); prompt language is controlled separately by (default English)
- Durations are generation durations sent to the video model, not estimates — adjust segment upper limit with per your model, adjust cut rhythm range with
- Lip sync is not handled for now — that's the generation pipeline's responsibility
- Storyboard frames don't need to be perfect on the first try — they are composition anchors for the video model; as long as composition and assets are correct, fine-tuning is left to regeneration
Gate Failures Accumulate
and
append gate results to
in the
current directory every time; run
to summarize:
bash
node {baseDir}/scripts/novel-storyboard.mjs stats
It answers three questions: Which gate triggers most often (the rule the model most often ignores, wording should be adjusted), Which gate never triggers (may be a dead gate, or the rule has been internalized), What do failure details look like (problems that recur but have no gate can only be reviewed manually).
Add
to disable logging; silent skip if writing fails, no impact on validation.
Self-Test
bash
node {baseDir}/scripts/selftest.mjs
323 assertions, no model calls, no quota usage. Each of the 18 quality gates has failure test cases. Run this first after modifying the script.
Built-in Example
{baseDir}/examples/渡口-storyboard.json
: Complete storyboard for Episode 1 of
Ferry Crossing — 10 segments, 34 cuts, covering all 35 beats of the script, average 3.5 seconds per cut, total 119 seconds / 120 second target, 2 generation batches, each segment with full H3 video prompts (multi-image alignment + all cut timings verified and passed). Serves as a quality benchmark and self-test fixture.