Natural Language Workflow for Converting Literary Works to Film & Video

Role Setting

Core Tasks

Must-Learn Skills

Skills are most likely located in the same directory

• novel-reader: Used to read novel content and extract elements (characters, props, scenes)
• jimeng-skill: Uses Jimeng Dreamina to generate element reference images, and generates videos through the all-around reference mode
• character-design: Used to generate character design drawings
• text-storyboard: The director creates text storyboards based on literary works, which is the core of video creation. It converts literary works into concrete audio-visual descriptions and outputs text storyboard files
• generate-film-video-prompt: Used to guide the creation of video prompts, used in conjunction with text-storyboard

Requirements

• Before starting work, you must first confirm the video art style and aspect ratio, the default is 16:9.

Workflow

Strictly follow the order, no skipping steps allowed

1. First read all must-learn skills in the same directory, and understand the video production process: novel-reader, jimeng-skill, character-design, text-storyboard, generate-film-video-prompt
2. User request: Make a video from Harry Potter and the Sorcerer's Stone.txt. (Art style and aspect ratio need to be confirmed)
3. Create a new project folder to store all files of the project, create subdirectories by episode and place them uniformly under the episode production folder (e.g. 单集制作/EP001, 单集制作/EP002, etc.). If the folder already exists, scan the folder content and resume work from the breakpoint.
4. Step 1 (Must Complete): If it is a novel file, use the novel-reader skill to read the novel content and extract characters, scenes, and props. If it is a one-sentence requirement, directly extract characters, scenes, and props.
5. Step 2 (Must Complete): View and use the text-storyboard and generate-film-video-prompt skills to create text storyboards, output one text storyboard file per episode (e.g. 单集制作/EP001/文字分镜.txt), which contains the text storyboard description of all clips in the episode. Important: Each 15-second text storyboard corresponds to a 15-second video clip, and one episode may have multiple clips
6. Step 3 (Must Complete):
   • Character Images: Use the character-design skill to generate character design prompts, write the prompts into

     角色/[Character Name].prompt.txt

     , then call jimeng-skill to read the prompts from the file and generate character images
   • Scene Images: Prepare detailed image generation prompts for each scene, write the prompts into

     场景/[Scene Name].prompt.txt

     , then call jimeng-skill to read the prompts from the file and generate scene images, which must meet:
     • The scene image must be consistent with the video aspect ratio, clearly describe the scene features, the scene image must be pure, and characters are strictly prohibited
     • All prompts must include art style requirements
7. Step 4 (Must Complete, Skipping Strictly Prohibited):
   • ⚠️ Verification Step: After completion, you must check whether the corresponding image files exist in the

     角色/

     and

     场景/

     directories
   • ⚠️ Importance: These element images are the key to maintaining the visual consistency of characters and scenes in subsequent video generation. You must not proceed to the next step without them
8. Step 5 (Only proceed after Step 4 is completed): According to the description of each clip in the text storyboard file, generate video prompts for each clip, write the prompts into

   单集制作/EPXXX/视频_ClipXXX.prompt.txt

   , you need to add the fixed requirement "No BGM, no subtitles" at the end of the prompt, then call the

   multimodal2video

   (all-around reference) mode of jimeng-skill to read the prompts from the file and generate videos, which must meet:
   • Reference the reference images of relevant characters, scenes, and props
   • Clarify the video aspect ratio, either 16:9 or 9:16
   • Use the seedance2.0 model to ensure element consistency and picture coherence

Breakpoint Resume Guide

Progress Check List

1. Check if Step 5 is completed (Highest priority):
   • Scan the episode directory: Check whether the corresponding

     .mp4

     files and corresponding

     *.prompt.txt

     files exist
   • If all videos have been generated → You can proceed to video stitching
   • If there are missing videos → You must first generate the missing videos, Skipping is strictly prohibited (if the prompt file already exists, read directly from the file; if not, generate the prompt file first)
2. Check if Step 4 is completed:
   • Scan the

     角色/

     directory: Check whether the corresponding image files and

     *.prompt.txt

     files exist
   • Scan the

     场景/

     directory: Check whether the corresponding image files and

     *.prompt.txt

     files exist
   • If all element images have been generated → You can proceed to subsequent steps
   • If there are missing element images → You must first generate the missing images, Skipping is strictly prohibited (if the prompt file already exists, read directly from the file; if not, generate the prompt file first)
3. Check if Step 3 is completed:
   • Check whether the

     角色/

     ,

     场景/

     ,

     道具/

     directories exist
   • Check whether the corresponding

     *.prompt.txt

     files exist
   • If the directory or prompt file is missing → Go back to Step 3
4. Check if Step 2 is completed:
   • Check whether the

     单集制作/

     directory exists
   • Check whether the text storyboard file exists (e.g.

     单集制作/EP001/文字分镜.txt

     ), which contains the description of all clips in the episode
   • If missing → Go back to Step 2
5. Check if Step 1 is completed:
   • Check whether there are files such as

     大纲.txt

     ,

     读取进度.txt

     , etc.
   • Check whether there are

     .txt

     description files under the

     角色/

     ,

     场景/

     ,

     道具/

     directories
   • If missing → Go back to Step 1

Common Error Handling

• Error 1: Start directly from Step 5, skip Step 4 of generating element images
  • Consequence: The characters and scenes in the generated video have no visual consistency, and the character image in each clip may be different
  • Solution: Stop immediately, go back to Step 4 to generate all element reference images first

Project Directory Structure Example

哈利波特与魔法石/
├── 哈利波特与魔法石.txt
├── 大纲.txt
├── 读取进度.txt
├── 角色/
│   ├── 哈利波特.txt
│   ├── 哈利波特.prompt.txt
│   └── 哈利波特.png
├── 场景/
│   ├── 霍格沃茨城堡.txt
│   ├── 霍格沃茨城堡.prompt.txt
│   └── 霍格沃茨城堡.png
├── 道具/
│   ├── 魔杖.txt
│   └── 魔杖.png
└── 单集制作/
    ├── EP001/
    │   ├── 文字分镜.txt
    │   ├── 视频_Clip001.prompt.txt
    │   ├── 视频_Clip001.mp4
    │   ├── 视频_Clip002.prompt.txt
    │   ├── 视频_Clip002.mp4
    │   ├── 视频_Clip003.prompt.txt
    │   └── 视频_Clip003.mp4
    ├── EP002/
    │   ├── 文字分镜.txt
    │   ├── 视频_Clip001.prompt.txt
    │   ├── 视频_Clip001.mp4
    │   ├── 视频_Clip002.prompt.txt
    │   └── 视频_Clip002.mp4
    └── EP003/
        ├── 文字分镜.txt
        ├── 视频_Clip001.prompt.txt
        └── 视频_Clip001.mp4

• All episode content is uniformly placed under the

  单集制作/

  directory
• Create subdirectories by episode (EP001, EP002, etc.)
• Each episode has only one

  文字分镜.txt

  file, which contains the text storyboard description of all clips in the episode
• Each Clip corresponds to a 15-second text storyboard description, a prompt file, and a 15-second video clip
• All image generation and video generation prompts are saved as

  *.prompt.txt

  files for easy traceability and reuse
• Shared assets such as characters, scenes, and props are placed in the project root directory