Shadow Puppetry Animation (Code-Generated)

Applicable: When the user wants a shadow puppetry animation, such as a historical explainer or story short, with the requirement that “the shadow figures must be exquisite,” along with subtitles and music, and a final result that can be played and downloaded.

Overall workflow (in order, with a user confirmation checkpoint at every step):

Write a shot-by-shot script (Docs document)
Establish the visual style: have the program render 1–2 sample images for user approval
Arrange the full film: act-by-act visuals + subtitles
Review every shot, with particular attention to occlusion
Add music: mix sampled instruments offline
Deliver: web player (Artifact) + MP4 video

1. Script: It Must Be Shot by Shot, Not Just a Summary of Each Act

If the user provides a reference video, first use ffmpeg to extract frames (

fps=1,tile=4x2

) and crop out the subtitle bar to understand its structure before writing.

Write the script as a Docs document containing:

A production overview table: title, duration, shot count, visual references, narrative approach, and central imagery.
Recurring characters: one curious little character who asks questions and exclaims, plus one guide who tells the story. Subtitles should always use “Character Name + Dialogue,” with no narration detached from the action on screen.
Each act: a title line with its time range, followed first by “Music for This Act” (lead instrument, mode, BPM/meter, mood, reference track), and then a shot table: Shot No. | Time | Shot Size | Visual and Shadow Puppet Action | Subtitles (Character · Dialogue) | SFX.
Each subtitle should be no longer than approximately 22 Chinese characters and remain on screen for at least 2 seconds.
For historical figures, use authentic quotations whenever possible. Clearly identify any invented dialogue as dramatized in the response.
Use imagery that bookends the piece, such as a lamp lighting and going out or the waxing and waning of the moon.

A 3-minute film should have approximately 14 acts and 50 shots.

2. Visual Style: Produce Sample Images First and Wait for Approval Before Creating the Full Film

Shadow puppetry ≠ flat, colored vector characters. Users may correct the result by comparing it with real shadow puppet photographs. Key points:

Screen: a solid, translucent orange-yellow screen with an oil-lamp hotspot in the center and darkened corners (radial gradient + paper texture overlay + vignette). Do not paint realistic landscape backgrounds.
Leather: use dark cowhide as the base. Red, green, yellow, and blue should appear only as translucent dyed areas and should remain dark, such as red
```
#7a120a
```
, green
```
#114428
```
, and yellow
```
#946414
```
.
True cutouts: the patterns are holes cut all the way through, allowing the light from the screen to show through. For each color × pattern combination, generate a CanvasPattern texture: first fill in the dye, then trace a dark knife edge around each opening, and finally cut through the holes with
```
destination-out
```
. Patterns include flower, lattice, stripe (pleats), tri (zigzag), cloud, wave, scale, coin, and dense.
Yang-carved outlines: after clipping each piece of leather to its own path, draw three strokes in sequence: a wide dark line (
```
bw×2
```
) → a thin translucent slit (
```
bw×1.25
```
) → a dark line (
```
bw×0.95
```
). This creates a dark–light–dark double outline. Add one more ink line around the outer edge.
Proportions: use slender figures with a head-to-body ratio of approximately 1:9 (about 470 units tall). The torso should be narrow and straight. The lower garment should have a pleated skirt hem, a front lapel covering the knees, and a belt with a jade pendant and tassels. Female characters should have cloud shoulders, a flowing shawl, and water sleeves.
Open face: carve out the face and retain only the side-profile contour, from the forehead to the bridge of the nose as an almost continuous line smoothed with bezier. The interior of the face should remain mostly transparent. Emperors and military figures may use a solid face, with the brows and eyes traced in translucent color.
The eyes must be exquisite; do not draw them with lines of uniform thickness.
- Render willow-leaf eyebrows and sword eyebrows as filled shapes that taper at both ends and are thicker in the middle.
- For phoenix eyes, make the upper eyelid a filled shape: thin at the inner corner, thickest in the middle, and sweeping upward into the temple at the outer corner. Add a fine double-eyelid line above it and a thin lower-eyelid line.
- Place the black pupil against the upper eyelid and cut out a translucent highlight inside it.
- Add the nostril and lip contours. Female characters should have red lips and a forehead huadian, with fine hair-strand cuts at the temples.
- Render a separate enlarged close-up of the brows and eyes for the user to inspect every time.
Headwear: phoenix crowns, mianliu, soft-foot futou, winged futou or wusha, square headscarves, double buns, helmets with plumes, and opera headdresses should all be rendered as cut-out leather.
Joints and control rods: draw rivets at the joints (a dark circle with a small translucent hole). Give each puppet three control rods: one at the neck and one in each hand, extending diagonally down beyond the bottom edge of the screen.
Scenery pieces: pavilions, trees, auspicious clouds, boats, and city gates should use the same cut-out leather treatment. Render distant scenery to a half-resolution buffer, then blur and fade it to create the effect of objects receding into the distance.
Soft shadows behind the screen: first draw all shadow puppets onto a layer, scale that layer to 1/4, darken it with
```
source-in
```
, then enlarge and offset it slightly beneath the main layer to create a soft shadow. Do not apply
```
ctx.filter blur
```
to the entire image because it is too slow.
Subtitle bar: use a dark-brown gradient bar with gold scrollwork and flame motifs at both ends. Set character names in gold and dialogue in off-white, using Noto Serif SC.

How to create the samples: select a representative shot, such as Li Bai Inviting the Moon, and render one full shot and one close-up. Send them with SendUserFile, list the adjustable options, such as lighting, pattern density, and body slenderness, and wait for the user's approval before continuing.

3. Full-Film Arrangement (Canvas 2D, 1920×1080)

File structure, eventually assembled into a single HTML file:

```
core.js
```
: math utilities, paper texture, vignette, dust motes, flames, and subtitle bar.
```
puppet.js
```
: cut-out leather, puppet assembly, head pieces, hats and crowns, and handheld props.
```
scenery.js
```
: screen, blurred distant layer, puppet layer, rooftops, pavilions, trees, clouds, moon, and projections.
```
props.js
```
: scenery pieces and props for each dynasty.
```
acts.js
```
: character preset table
```
CH
```
, each act's
```
draw(ctx, act-local time, global time)
```
, and subtitle table
```
SUBS
```
.
```
engine.js
```
: select the act by time, apply a 0.5-second crossfade between acts, and handle post-processing and subtitles.

Arrangement guidelines:

Puppet pose parameters include arm angle (measured from hanging downward, with forward being positive), elbow bend, head tilt, forward body lean, and walking phase. Use
```
mk(base, override)
```
to merge character presets.
Rendering must be a pure function of time. Use a hash for particles so the animation can be seeked to any point and exported frame by frame.
Use headless Chromium (Playwright) to call
```
renderAt(t)
```
and take screenshots. Combine every three frames into a single overview image for review.

4. Review: Occlusion Rules (the User Has Explicitly Raised This Issue)

Render one frame at the midpoint of every shot and inspect each one:

Behind a character, there should only be an empty screen or blurred, faded distant scenery. A dark building or scenery piece placed behind a character can make the character appear blocked or visually merged with it.
Place buildings on either side or in the distant layer. Leave space between characters and between characters and props. Offset their walking paths.
Do not use clip to hide a character's lower body inside a cart, camel, or horse, because users interpret this as occlusion. Instead, have the character stand on the cart platform or walk in front of the animal.
When more than one group of characters shares the frame, check whether they will walk into each other and overlap at any point during the shot.

5. Music: Use Sampled Instruments Instead of Harsh Oscillator Synthesis

WebAudio oscillator-based synthesis of folk music was judged “bad-sounding.” The correct approach is:

Download a MIDI.js soundfont from GitHub; MusyngKite provides better quality. Use one mp3 per note:
```
https://raw.githubusercontent.com/gleitz/midi-js-soundfonts/gh-pages/MusyngKite/<instrument>-mp3/<note>.mp3
```
Write note names as C4, Db4, Eb4, and so on. Download only the required registers and use
```
xargs -P 16
```
for parallel downloads.
Instrument mapping:
- Banhu and jinghu → fiddle; erhu → violin; matouqin → cello
- Xun → ocarina; xiao → shakuhachi; dizi → flute; bone flute and qiangdi → pan_flute
- Suona → shanai; bili → oboe; hujia → english_horn; sheng → reed_organ
- Guzheng and qinzheng → koto; pipa and sanxian → shamisen; use rapid repeated notes for pipa tremolo; guqin → acoustic_guitar_nylon, one octave lower with slides and vibrato ornaments
- Bianzhong → tubular_bells; shiqing → kalimba
- War drums → taiko_drum; hand drums and jiegu → melodic_tom; clappers → woodblock; camel bells → agogo / tinkle_bell
- String background → string_ensemble_1; horns → french_horn
- Sound effects: seashore for water, applause for cheering, gunshot for cannon fire, and bird_tweet
- Synthesize gongs, cymbals, match strikes, cracking sounds, and similar effects with numpy, using inharmonic partials and filtered noise.
Mix offline with Python (numpy + scipy):
- Use ffmpeg to decode every mp3 into 44.1k float32 audio.
- All samples are exactly 3.1 seconds long. When a note needs to last longer, loop it using 0.45–2.2-second segments with crossfades.
- Use time-varying resampling to create slides into notes and vibrato.
- Set the volume, pan, and reverb amount for each instrument. Generate reverb using fftconvolve and a synthesized exponentially decaying impulse response.
- Finally, apply a high-pass filter and tanh soft limiting, then run
```
ffmpeg loudnorm=I=-17:TP=-1.5
```
  .
Write the score using a numbered-notation string DSL (
```
3 5 6 1'- | 6 5 3 2--
```
), where
```
'
```
indicates the higher octave,
```
,
```
indicates the lower octave,
```
-
```
adds one beat, and
```
_
```
subtracts half a beat. Use functions such as
```
mel()
```
,
```
arp()
```
,
```
perc()
```
,
```
roll()
```
, and
```
tremolo()
```
.
Arrange the music:
- Use one thematic motif throughout the film: prelude solo → full ensemble at the climax → full ensemble transposed upward → coda solo.
- Give every act three layers: lead melody + accompaniment (plucked arpeggios or a string background) + percussion.
- Align musical cues with visual events, such as bells chiming one by one, cannon fire, and rockets. Leave only one 2-second silence in the entire film.
- Use a spectrogram (
```
ffmpeg showspectrumpic
```
  ) and segmented RMS analysis to verify that the levels are balanced throughout.
Encode the result as a 160k mp3 and embed it in the page as base64. When the page loads, call decodeAudioData. During playback, use
```
BufferSource.start(when, offset)
```
to align the audio precisely with the visuals, and restart the source when the user seeks.

6. Delivery

Web player (Artifact): title + 16:9 canvas + first-frame poster (choose an attractive shot and place a large “Start Playback” button in the center) + control bar (play/pause, time, progress bar with act tick marks, current act name, mute, fullscreen) + 14 act chapter buttons. Use a consistent dark theater style, and switch the chapter buttons to 2 columns at mobile widths.
MP4:
- Export frames in parallel with multiple Playwright pages: call
```
renderAt(f/30)
```
  , then obtain
```
canvas.toDataURL('image/jpeg', .92)
```
  . On a dual-core machine, 5,400 frames take approximately 8 minutes.
- Composite the frames with ffmpeg:
```
-c:v libx264 -crf 20 -pix_fmt yuv420p -c:a aac -movflags +faststart
```
  .
- The 1080p version is approximately 115 MB, exceeding SendUserFile's 30 MB limit. Run two additional encoding passes to produce a 720p version:
```
scale=1280:720
```
  , video bitrate
```
-b:v 1150k
```
  , and audio bitrate 112k, resulting in approximately 28 MB for delivery.
- Tell the user that a 1080p HD version also exists and can be sent in segments if needed.

Common Pitfalls

Applying
```
ctx.filter
```
to a layer or puppet, such as blur or brightness, can make a single frame take several seconds. Always blur by downscaling a buffer and scaling it back up, or by using a pre-rendered sprite, such as a hand holding a control rod.
When modifying scene functions, do not replace an entire section through the end of the file, or later functions, such as
```
crescent
```
, may be accidentally deleted. After every build, run a full-film screenshot check.
If the preview environment cannot connect to Google Fonts, include a local fallback in the font stack, such as Noto Serif CJK SC.
Mirroring scenery that contains text with
```
scale(-1,1)
```
reverses the text. Instead, draw a separate version on the other side.

References (

references/

Directory)

Before establishing the visual style, use Read to review all of these images:

```
references/皮影实物参考/
```
: real shadow puppet photographs, which are the standard for the visual style.
- 01: a female puppeteer holding control rods, clearly showing the rod positions and translucent-light effect.
- 02: a female figure with a translucent cut-out crown, showing the pleated skirt hem and open face.
- 03: a scene with two characters, where the pavilion and trees are cut-out scenery pieces.
- 04: a group scene made primarily from black cut-out leather, with dense carving.
```
references/参考视频/
```
: the user's reference short film, featuring warm gold visuals and a subtitle bar with character names, together with extracted frames and subtitle-bar screenshots. Match the subtitle-bar style to this reference.
```
references/samples/
```
: visuals approved by the user.
- ```
ref_01.png
```
  and
```
ref_02.png
```
  : follow these images for lighting, contrast, and the carving of the brows and eyes.
- ```
第一版_偏亮_未采用.png
```
  : a rejected example. It is too bright overall, and the brows and eyes are too heavy.

shadow-puppet-animation

NPX Install

Tags

SKILL.md Content (Chinese)

Shadow Puppetry Animation (Code-Generated)

1. Script: It Must Be Shot by Shot, Not Just a Summary of Each Act

2. Visual Style: Produce Sample Images First and Wait for Approval Before Creating the Full Film

3. Full-Film Arrangement (Canvas 2D, 1920×1080)

4. Review: Occlusion Rules (the User Has Explicitly Raised This Issue)

5. Music: Use Sampled Instruments Instead of Harsh Oscillator Synthesis

6. Delivery

Common Pitfalls

References (
`references/`
Directory)

shadow-puppet-animation

NPX Install

Tags

SKILL.md Content (Chinese)

Shadow Puppetry Animation (Code-Generated)

1. Script: It Must Be Shot by Shot, Not Just a Summary of Each Act

2. Visual Style: Produce Sample Images First and Wait for Approval Before Creating the Full Film

3. Full-Film Arrangement (Canvas 2D, 1920×1080)

4. Review: Occlusion Rules (the User Has Explicitly Raised This Issue)

5. Music: Use Sampled Instruments Instead of Harsh Oscillator Synthesis

6. Delivery

Common Pitfalls

References (references/ Directory)

References (
`references/`
Directory)