Dream-to-Video: Automated Video Generation from Dream Materials

0. Initial Environment Configuration

When a user uses this Skill for the first time, automatically configure the environment according to the following process. Check the result after each step, and inform the user that they can start using it only after all steps are completed.

Path Conventions

{W}

dream_to_video/

• Project directory:

  {W}/dream_to_video/

• Skill resource directory:

  {W}/skills/dream-to-video/

{W}

Automatic Execution (No User Action Required)

cd "<directory specified by user>" && git clone https://github.com/mediastormDev/dream-to-video-skill.git && cd dream-to-video-skill

dream-to-video-skill

{W}

python --version

Please install Python 3.10 or higher first: https://www.python.org/downloads/ Check "Add Python to PATH" during installation.

cd "{W}/dream_to_video" && pip install -r requirements.txt

playwright install chromium

cd "{W}/dream_to_video" && mkdir -p output data auth/browser_profile reference_images/indoor reference_images/outdoor

skills/dream-to-video/reference_images/

cp -n "{W}/skills/dream-to-video/reference_images/indoor/"*.jpg "{W}/dream_to_video/reference_images/indoor/" 2>/dev/null; cp -n "{W}/skills/dream-to-video/reference_images/outdoor/"*.jpg "{W}/dream_to_video/reference_images/outdoor/" 2>/dev/null; echo "reference images deployed"

-n

does not overwrite existing files. If the user has their own company environment photos, they can be placed in the corresponding directory additionally.

User Action Required

cd "{W}/dream_to_video" && python main.py login

Please scan the QR code in the browser with your TikTok/Jiemeng App to complete login. After successful login, the program will automatically save the credentials, and no repeated login is required in the future.

Self-Test Checklist

# Check 1: Python Version
python --version
# Expected: Python 3.10+

# Check 2: Core Dependencies
python -c "import playwright; import cv2; import numpy; print('deps OK')"
# Expected: Output deps OK

# Check 3: Playwright Browser
python -c "from playwright.sync_api import sync_playwright; b=sync_playwright().start(); br=b.chromium.launch(headless=True); br.close(); b.stop(); print('browser OK')"
# Expected: Output browser OK

# Check 4: Directory Structure
python -c "from pathlib import Path; dirs=['output','data','auth/browser_profile']; ok=all((Path('{W}/dream_to_video')/d).is_dir() for d in dirs); print('dirs OK' if ok else 'dirs MISSING')"
# Expected: Output dirs OK

# Check 5: Login Status
cd "{W}/dream_to_video" && python main.py verify
# Expected: Display login is valid

Environment configuration completed! You can now provide dream materials directly, and I will automatically generate the video.

1. Complete Workflow

User provides dream materials → You convert to Prompt according to rules → Submit to queue → Worker automatically generates + downloads → Notify user

After receiving materials each time, you must follow these steps:

• Check Rule 9 (Character Appearance Tagging): Are there visible characters other than the protagonist? If yes, tag them.
• Check Rule 10 (Reference Image Prefix): Is there a description of the physical environment of the company/workplace? If yes, add the prefix

  Reference image environment;

  .

cd "{W}/dream_to_video" && python -u main.py add "the complete Prompt you generated"

cd "{W}/dream_to_video" && python -u main.py worker

run_in_background: true

1. Detect if the Prompt has the

   Reference image environment;

   prefix
2. If yes → Automatically select reference images, switch to "All-round Reference" mode, upload images, and reference the images in the ProseMirror editor via

   @Image 1

   when inputting the prompt
3. If no → Directly input the prompt in the textarea
4. Configure generation settings (Seedance 2.0 / 16:9 / 15s) → Click generate → Monitor progress → Download → Post-processing effects

{W}/dream_to_video/output/

Check Status

cd "{W}/dream_to_video" && python -u main.py status

{W}/dream_to_video/output/batch_state.json

2. Video Prompt Generation Rules

Top-Level Iron Rules

1. Hardcore Realism

2. Uncanny Dream-Logic

3. Strict Adherence to Original

4. Cinematic Camera

5. Silent Visuals

6. No Names for Protagonist

7. Pure Frame

8. Space & Time (Logic & Timing)

9. Character Appearance Tagging

• Default Value: When there are no region/country words in the materials, add "East Asian appearance" to other visible characters by default
• Writing Method: Naturally integrate into the first appearance description of other characters, such as "a companion with East Asian appearance standing at the end of the corridor", "dozens of colleagues with East Asian appearance scattered in groups of three or five around the venue"
• When There Are No Other Characters: If the whole dream only has the protagonist (such as working overtime alone, staying at home alone), do not add any appearance tags

10. Company Environment Reference Image Prefix

{W}/dream_to_video/reference_images/

Reference image environment; This is a [realistic + emotional word] dream...

Output Format

Part 1: Style Opening Sentence

This is a [realistic + emotional word] dream, shot with [lens method].

Part 2: Visual Narrative (Shot 1-6)

Part 3: Environmental Sound Effects

Part 4: Technical Style Base (Mandatory)

Shot on Arri Alexa, Fisheye Lens (Fisheye 12mm), obvious barrel distortion in the frame. Letterbox (2.39:1), mandatory wide-screen movie aspect ratio. Heavy Vignette, the four corners of the frame are darkened and converge towards the center. [Fill in light and shadow description according to the scene], low-saturation cool tone. Photo-level realism. Faint digital noise and VHS-like distortion flicker at the frame edges, the image feels fragile as if it will collapse at any moment. Dream core. Liminal space.

Output Example

3. Login Instructions

• First use / Login expired: The browser will automatically open the Jiemeng website and display a login QR code, and play a prompt tone (three long beeps + two short beeps) to remind the user. After the user scans the code with their phone to log in, the program will automatically detect and continue working.
• Already logged in: Start working directly without any operation.
• Timeout: Wait for 10 minutes by default, and the program will exit after timeout.

cd "{W}/dream_to_video" && python main.py login

4. Post-Processing Effects (Elliptic Shatter)

task_XXX_YYYYMMDD_HHMMSS.mp4

task_XXX_YYYYMMDD_HHMMSS_elliptic-shatter.mp4

• The center of the video remains clear original frame
• The edge presents broken glass texture (debris scattering, slight rotation, chromatic aberration refraction)
• Outer dark border + faint debris texture
• Overall effect: Black texture border + elliptic viewing window + rotating shattered particles

effects/elliptic_shatter.py

5. Key Notes

1. The Prompt is a single piece of plain text, no line breaks, Markdown format or分段. Pass it directly as a string to

   python main.py add

   .
2. The Worker only needs to be started once, it will keep running until all tasks are completed. Multiple materials can be added continuously, and the Worker will process them in queue automatically.
3. Each task outputs two videos to

   {W}/dream_to_video/output/

   : original version + elliptic-shatter effect version.
4. There is a prompt tone (3 short beeps) after download + effect processing is completed, and the user can go to get the video when they hear the sound.
5. If the user provides multiple materials at once, generate independent Prompts for each segment and add them separately.
6. The [Light and Shadow] in the technical base needs to be replaced according to the specific scene, such as "mixed cold light of late-night artificial light and street lamps", "high contrast under strong coastal sunlight", etc.