SRT Remotion Video - Main Workflow Orchestration

Workflow Overview

┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────────────────┐   ┌──────────┐
│ Get SRT  │ → │ Dependency │ → │ Project  │ → │ Generate │ → │ Parallel Creator     │ → │ Synthesize│
│ File Path│   │ Precheck/  │   │ Initialization │ │ Storyboard │ │ Planning & Implementation │ │ Video     │
│          │   │ First-time │   │          │   │          │   │ scene-plan + code    │ │          │
│          │   │ Installation │ │          │   │          │   │                      │ │          │
└──────────┘   └──────────┘   └──────────┘   └──────────┘   └──────────────────────┘   └──────────┘

Project Directory Structure

<skillRoot>/
├── SKILL.md
├── template/
├── references/
│   ├── storyboard-parser.md
│   └── scene-component-creator.md
└── scripts/
    ├── ensure-template-deps.js
    ├── init-project.js
    ├── generate-storyboard.js
    ├── generate-creator-scenes.js
    ├── generate-scenes-registry.js
    ├── scene-registry-utils.js
    ├── validate-project.js
    └── validate-scene-plan.js

<srtDir>/
├── your-file.srt
└── remotion-video-projects/
    └── {yyyy-mm-dd-hh-mm-ss}/

Path Contract

• skillRoot

  : Absolute path of the current

  srt-remotion-video

  skill directory

• templateRoot

  :

  {skillRoot}/template

• referencesRoot

  :

  {skillRoot}/references

• scriptsRoot

  :

  {skillRoot}/scripts

• srtPath

  : Absolute path of the SRT file provided by the user

• projectBaseDir

  :

  {dirname(srtPath)}/remotion-video-projects

• projectRoot

  : Directory of the current project instance, formatted as

  {projectBaseDir}/{projectName}/

• SubAgent prompts must include expanded absolute paths, do not pass only variable names
• Stage protocol documents can only be read from

  referencesRoot

• Scripts can only be executed from

  scriptsRoot

• All runtime states must be explicitly passed by the main Agent

Execution Workflow

Step 0: Obtain SRT File

1. Ask the user for the SRT file path
2. If the user provides a relative path, the main Agent must first check whether the file exists
3. If it exists, the main Agent must resolve it to an absolute path, then use the resolved absolute path as

   srtPath

4. If it does not exist, clearly inform the user that the path is invalid and request a correct path
5. All subsequent steps uniformly use the final confirmed absolute path

   srtPath

Step 1: Dependency Precheck and Project Initialization

Key: Templates are always copied from inside the skill, and all work is performed in an independent project directory under the subtitle directory.

1.0 Dependency Precheck (Automatically Installed on First Use)

node "{skillRoot}/scripts/ensure-template-deps.js" "{templateRoot}"

1. Check

   {templateRoot}/package.json

   and

   {templateRoot}/package-lock.json

2. Check whether key template dependencies are installed
3. If not installed, execute

   npm install

   once under

   {templateRoot}

4. If already installed, skip installation directly
5. Return a clear error and stop the main workflow if installation or verification fails

1.1 Default Behavior: Create a New Project

node "{skillRoot}/scripts/init-project.js" --srt-path "{srtPath}"

1. Ensure

   {dirname(srtPath)}/remotion-video-projects/

   exists
2. Create a new project directory

   remotion-video-projects/{yyyy-mm-dd-hh-mm-ss}/

3. Copy template files from

   {skillRoot}/template/

   ; if template dependencies are already installed, copy the installed dependencies together
4. Output project information JSON

1.2 User-Specified Project Path (Only When Explicitly Specified by User)

projectRoot

1.3 Record Key Paths

• projectRoot

• skillRoot

• templateRoot

• referencesRoot

• scriptsRoot

• srtPath

Step 2: Generate Storyboard Script

1. Calculate and expand absolute paths:

   • storyboardReference = {referencesRoot}/storyboard-parser.md

   • storyboardScript = {scriptsRoot}/generate-storyboard.js

2. Start a SubAgent
3. Write the actual absolute path values in the prompt

You are executing the "Storyboard Generation Phase" of the srt-remotion-video workflow.

First read the following reference protocol and strictly follow its steps:
- {storyboardReference}

Input parameters:
- skillRoot: {skillRoot}
- projectRoot: {projectRoot}
- srtPath: {srtPath}

Important:
1. All paths have been expanded to absolute paths, do not guess on your own
2. The script to execute is located at {storyboardScript}
3. After completion, you must return structured results in accordance with the "Return After Completion" contract in the reference protocol

storyboard.json

Step 3: Use SubAgent to Plan and Implement Scene Components

storyboard.json

3.0 Preparatory Step: Calculate Grouping

const SCENES_PER_CREATOR = 5;
const sceneCount = storyboard.scenes.length;
const creatorCount = Math.ceil(sceneCount / SCENES_PER_CREATOR);

creatorId

const creatorId = `creator-${String(index + 1).padStart(2, '0')}`;

• 1st creator:

  creator-01

• 2nd creator:

  creator-02

• 10th creator:

  creator-10

3.1 Plan Creator Tasks

1. Generate

   scenesDataPath = {projectRoot}/scene-plans/{creatorId}.scenes.json

   for each creator
2. Calculate for each creator:

   • creatorId

   • planPath = {projectRoot}/scene-plans/{creatorId}.json

   • validateScript = {scriptsRoot}/validate-scene-plan.js

3. Execute for each creator:

node "{scriptsRoot}/generate-creator-scenes.js" \
  "{projectRoot}/storyboard.json" \
  "{creatorId}" \
  "{SCENES_PER_CREATOR}" \
  "{scenesDataPath}"

1. The main Agent writes the absolute path of

   {scenesDataPath}

   into the SubAgent prompt
2. Start all creator SubAgents in parallel

3.2 Start All Scene Creators in Parallel

You are executing the "Scene Planning and Implementation Phase" of the srt-remotion-video workflow.

First read the following reference protocol and strictly follow its steps:
- {referencesRoot}/scene-component-creator.md

Input parameters:
- skillRoot: {skillRoot}
- projectRoot: {projectRoot}
- creatorId: {creatorId}
- planPath: {planPath}
- scenesDataPath: {scenesDataPath}
- validateScript: {validateScript}

Important:
1. All paths have been expanded to absolute paths, do not guess on your own
2. The scene data source for the current creator is only allowed to come from {scenesDataPath}
3. During the planning phase, only use {scenesDataPath}, {projectRoot}/cartoon-ui-style-guide.css, {projectRoot}/cartoon-ui-style-guide-reference.md and {skillRoot}/../remotion-best-practices/SKILL.md
4. First generate the scene-plan JSON, then perform validation, and only write scene components after validation passes
5. By default, one subtitle segment corresponds to one beat in `beatPlan`; if adjacent segments clearly belong to the same continuous expression, they can be merged
6. Merging is only allowed between adjacent segments, skip merging is prohibited
7. `beatPlan` only declares `segments` and `action`; actual timing must be derived from `relativeStart / relativeDuration` of scenesData[].segments
8. The main scene rhythm must be bound to scenesData[].segments[].relativeStart / relativeDuration
9. By default, retain the host background, organize the main visual around the middle or upper-middle area of the screen on a transparent root layer; do not rebuild the full-screen background
10. The component interface is fixed as React.FC<{ segments: Segment[] }> and uses default export
11. Only responsible for producing {projectRoot}/src/scenes/SceneXXX.tsx; create the file if it does not exist, only modify the scene files you are responsible for if it already exists
12. Do not manually modify {projectRoot}/src/compositions/Main.tsx or generated-scenes.ts
13. After completion, you must return structured results in accordance with the "Return After Completion" contract in the reference protocol
14. `remotion-best-practices` is at the same level as the current skill, with a fixed entry at {skillRoot}/../remotion-best-practices/SKILL.md

3.3 Wait for All Creators to Complete

componentResults

is only used for task completion feedback, not as the real source for final registry file assembly or total duration calculation.

Step 4: Synthesize Video

4.1 Generate Scene Registry File

{projectRoot}/src/compositions/Main.tsx

1920x1080 / 30fps

• totalDurationInFrames

  in

  generated-scenes.ts

  is generated by

  generate-scenes-registry.js

  , do not modify manually

• msToFrames

  in

  Main.tsx

  must dynamically obtain the frame rate using

  useVideoConfig().fps

  , do not hardcode the

  FPS

  constant

node "{scriptsRoot}/generate-scenes-registry.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

• Each

  SceneXXX.tsx

  exposes the component via default export

• generated-scenes.ts

  is responsible for storing

  start

  ,

  duration

  ,

  segments

  ,

  Component

• Main.tsx

  is responsible for passing segment data to scene components using

  <Component segments={scene.segments} />

4.2 Root.tsx Total Duration Synchronization

Root.tsx

• Root.tsx

  must keep reading

  totalDurationInFrames

  from

  generated-scenes.ts

• Do not manually recalculate or fill in the total frame count during the workflow

4.3 Verify Project Product Integrity

node "{scriptsRoot}/validate-project.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

4.4 Execute Rendering

cd "{projectRoot}"
npx remotion render Main out/output.mp4

Step 5: Completion

• The video has been generated
• Output path:

  {projectRoot}/out/output.mp4

• Number of scenes: N
• Video duration: X seconds

Debug Mode and Re-Rendering

Both operations assume that

projectRoot

exists and scene components have been generated.

Debug Mode

TM.0 Obtain Audio File

1. If the user has provided an audio file path, verify that the file exists and resolve it to an absolute path

   audioPath

2. If the user has not provided an audio file path, must ask the user for it
3. Verify that the file exists; if not, inform the user and stop

TM.1 Add Audio to Timeline

1. Ensure the

   {projectRoot}/public/

   directory exists
2. Copy the audio file as

   {projectRoot}/public/audio.mp3

mkdir -p "{projectRoot}/public"
cp "{audioPath}" "{projectRoot}/public/audio.mp3"

1. Modify

   {projectRoot}/src/compositions/Main.tsx

   :
   • Add

     Audio

     and

     staticFile

     to the import line:
     typescript

     import { AbsoluteFill, Audio, Sequence, staticFile, useCurrentFrame } from "remotion";

   • Add the following as the first direct child inside

     <AbsoluteFill style={{ backgroundColor: ... }}>

     :
     tsx

     <Audio src={staticFile("audio.mp3")} />

2. Start Remotion Studio for user preview:

cd "{projectRoot}"
npx remotion studio

TM.2 Completion Notification

Re-Rendering

RR.0 Remove Audio from Timeline

Main.tsx

1. Read

   {projectRoot}/src/compositions/Main.tsx

2. Check if it contains the

   <Audio

   tag
3. If it exists:
   • Remove the line

     <Audio src={staticFile("audio.mp3")} />

   • Remove

     Audio

     and

     staticFile

     from the import statement (if they are no longer used by other code)
4. If it does not exist, skip this step

RR.1 Execute Rendering

cd "{projectRoot}"
npx remotion render Main out/output.mp4

RR.2 Completion Notification

{projectRoot}/out/output.mp4

High Resolution / High Frame Rate Rendering

This operation assumes that

projectRoot

exists and scene components have been generated.

Key Principle: Separate Design Resolution from Output Resolution

--scale

Common Mistakes (Prohibited)

--width 3840 --height 2160

const FPS = 30

HR.0 Confirm User Requirements

--scale 2

fps={60}

totalDurationInFrames

fps={60}

totalDurationInFrames

--scale 2

HR.1 Modify Frame Rate (Only When User Requests High Frame Rate)

1. Modify the

   fps

   value in

   {projectRoot}/src/Root.tsx

   :

fps={60}  // Changed from 30 to 60

1. Modify

   totalDurationInFrames

   in

   {projectRoot}/src/compositions/generated-scenes.ts

   :

// Frame count = original frame count × (new fps / original fps)
// Example: 30→60fps: 1572 × 2 = 3144
export const totalDurationInFrames = {original frame count × new fps / original fps};

width

height

<Composition>

{projectRoot}/src/Root.tsx

1920

1080

--scale

msToFrames

in

Main.tsx

must dynamically obtain the frame rate using

useVideoConfig().fps

. If a hardcoded

FPS

constant is found in

Main.tsx

, it must be fixed to

useVideoConfig().fps

first, otherwise scene timing will be completely messed up after fps changes.

HR.2 Verify Project

node "{scriptsRoot}/validate-project.js" \
  "{projectRoot}" \
  "{projectRoot}/storyboard.json"

HR.3 Execute Rendering

cd "{projectRoot}"
npx remotion render Main out/output-4k.mp4 --scale 2

• --scale 2

  : Scale the 1920x1080 design canvas by 2x to render as 3840x2160
• Vector elements (text, SVG) will be rendered at higher resolution, resulting in clearer image quality
• The layout is exactly the same as 1080p, no element shrinking issue
• If only 60fps is needed without 4K, remove

  --scale 2

HR.4 Completion Notification

• Output path:

  {projectRoot}/out/output-4k.mp4

• Output resolution, frame rate, duration
• If fps was modified, remind the user that the fps in Root.tsx has been changed from 30 to the target value

Data Structure Reference

storyboard.json

interface Storyboard {
  totalDuration: number;
  sceneCount: number;
  scenes: {
    id: string;
    startTime: number;
    duration: number;
    segments: {
      text: string;
      relativeStart: number;
      relativeDuration: number;
    }[];
    semanticTags?: string[];
    visualHint?: string;
  }[];
}

scene-plan JSON

interface ScenePlanCard {
  sceneId: string;
  goal: string;
  layout: string;
  visualCore: string;
  surface: string;
  emphasis: string;
  screenShouldShow: string[];
  beatPlan: {
    segments: number[];
    action: string;
  }[];
}

SceneComponentResult

interface SceneComponentResult {
  sceneId: string;
  componentPath: string;
  componentName?: string;
  planPath?: string;
}

Resources

template/

• Lightweight template project, distributed with the skill
• Dependencies are installed in the template directory on first use, and subsequent projects reuse template dependencies

references/

• storyboard-parser.md

  : Storyboard generation phase protocol

• scene-component-creator.md

  : Scene planning and implementation phase protocol

• theme-template-switching.md

  : Theme template switching guide

scripts/

• ensure-template-deps.js

  : Check template dependencies, perform first-time installation if necessary

• init-project.js

  : Initialize project based on

  srtPath

• generate-storyboard.js

  : Generate storyboard.json based on SRT and groups.json

• generate-creator-scenes.js

  : Generate scenesData JSON for the specified creator based on storyboard.json

• generate-scenes-registry.js

  : Generate

  generated-scenes.ts

• scene-registry-utils.js

  : Shared utilities for registry and validation

• validate-project.js

  : Integrity check before rendering

• validate-scene-plan.js

  : Validate scene-plan JSON structure and segment binding

Execution Checklist

Main Workflow

• Obtain the absolute path of the SRT file provided by the user
• Run

  ensure-template-deps.js

  to check template dependencies, complete first-time installation if necessary
• Run

  init-project.js --srt-path

  to create project
• Obtain

  projectRoot

  ,

  skillRoot

  ,

  templateRoot

  ,

  referencesRoot

  ,

  scriptsRoot

• Generate

  storyboard.json

  using

  references/storyboard-parser.md

• Verify that

  storyboard.json

  structure is correct
• Calculate Creator grouping
• Generate scene-plan and scene components in parallel using

  references/scene-component-creator.md

• Run

  generate-scenes-registry.js

• Run

  validate-project.js

• Execute rendering

Debug Mode

• Obtain the absolute path of the audio file provided by the user
• Copy audio to

  {projectRoot}/public/audio.mp3

• Modify

  Main.tsx

  to add

  <Audio>

  component
• Start Remotion Studio for user preview

Re-Rendering

• Check if

  Main.tsx

  contains

  <Audio>

  tag
• If exists, remove

  <Audio>

  and related imports
• Execute rendering

High Resolution / High Frame Rate Rendering

• Parse user requirements into specific output resolution, frame rate and scale values
• Confirm that

  msToFrames

  in

  Main.tsx

  uses

  useVideoConfig().fps

  instead of hardcoded constants
• Only when high frame rate is required: modify

  fps

  in

  Root.tsx

  and

  totalDurationInFrames

  in

  generated-scenes.ts

• Confirm that

  width

  /

  height

  in

  Root.tsx

  remain 1920/1080 unchanged
• Run

  validate-project.js

  for verification
• Execute rendering with

  --scale 2

  (for 4K)

Notes

1. All paths must use absolute paths
2. SubAgent prompts must pass actual path values, not just variable names
3. Template resources are located at

   {skillRoot}/template

4. Templates are distributed in lightweight form, dependency precheck in

   template/

   must be completed on first use
5. Default project directory is located at

   {dirname(srtPath)}/remotion-video-projects

6. Main.tsx

   and

   Root.tsx

   belong to the protected host layer
7. Scene components must actually consume

   segments

8. Rendering cannot continue if

   validate-project.js

   fails
9. If the user wants to change the theme template, must first refer to

   references/theme-template-switching.md