• YouTube: full URL,

  youtu.be/<id>

  ,

  youtube.com/shorts/<id>

  , raw 11-char ID
• Loom:

  loom.com/share/<id>

  or

  loom.com/embed/<id>

• Vimeo:

  vimeo.com/<id>

• Riverside: download URL or local file
• Zoom: local

  .mp4

  from a downloaded recording
• X / IG / TikTok video: URL — defers to

  social-fetch

  for metadata, uses yt-dlp for the file
• Local file: any path to an

  .mp4

  /

  .mov

  /

  .webm

  /

  .mkv

~/Documents/videos/<source>-<slug>-<date>/

• source

  :

  youtube

  /

  loom

  /

  vimeo

  /

  riverside

  /

  zoom

  /

  local

• slug

  : kebab-case of title (first 4–6 words, max 50 chars)

• date

  :

  YYYY-MM-DD

1. Platform-provided transcript if it exists and looks complete:
   • YouTube:

     yt-dlp --write-sub --write-auto-sub --skip-download --sub-lang en --sub-format vtt

   • Loom: fetch via

     https://www.loom.com/share/<id>

     page metadata or Loom API if

     $LOOM_API_KEY

     set
   • Riverside: built-in transcripts available on the recording's share page
   • If platform transcript exists and has timestamps, use it. Skip Whisper.
2. MLX-Whisper local (default fallback — fast on Mac M-series):
   bash

   # Install once: pip install mlx-whisper
   python3 -c "import mlx_whisper; mlx_whisper.transcribe('<file>', path_or_hf_repo='mlx-community/whisper-large-v3-turbo')" \
     > "<workdir>/transcript-raw.json"

   Or via the CLI:

   mlx_whisper <file> --model mlx-community/whisper-large-v3-turbo --output-dir <workdir>

3. whisper.cpp (further fallback if MLX unavailable)

yt-dlp -f "bv*[height<=720]+ba/b[height<=720]" -o "<workdir>/video.%(ext)s" "<url>"

undefined

• transcript.txt

• metadata.json

• One-line summary in chat: title, source, duration, word count
• Path to workdir
• (Optional) Step 9 — offer to capture to second-brain

Here are N frames from a video at timestamps T1..TN. For each frame, describe what's on screen in 1–2 sentences. Flag: (a) UI changes from previous frame, (b) text visible on screen, (c) any moment that looks like a decision, action, or notable event. Also note the transcript text spoken during this window.

<workdir>/moments.md

undefined

• 00:00:15 — <one-line>
• 00:00:45 — <one-line>

• <item> [timestamp]

• <decision> [timestamp] — consider routing to /decide

• "..." [timestamp]

• <question raised but not answered>

undefined

1. Gemini native if

   $GEMINI_API_KEY

   is set (much cheaper + faster than per-frame for long videos):
   Default model:

   gemini-3.5-flash

   (released May 2026, ~$1.50 input / $9 output per 1M tokens; ~$0.15/sec of video; beats 3.1 Pro on coding/agentic benchmarks at 4× the speed). Override to

   gemini-3.1-pro

   for brand audits / high-stakes analysis where details matter;

   gemini-2.5-flash-lite

   for bulk cheap processing.
   bash

   # Step 1: Upload video via Files API
   FILE_URI=$(curl -s -X POST "https://generativelanguage.googleapis.com/upload/v1beta/files?key=$GEMINI_API_KEY" \
     -H "X-Goog-Upload-Command: start, upload, finalize" \
     -H "Content-Type: video/mp4" \
     --data-binary "@<workdir>/video.mp4" | jq -r '.file.uri')
   
   # Wait until file is ACTIVE (Gemini processes the video first)
   while true; do
     STATE=$(curl -s "$FILE_URI?key=$GEMINI_API_KEY" | jq -r '.state')
     [ "$STATE" = "ACTIVE" ] && break
     sleep 3
   done
   
   # Step 2: Generate content with the file + multimodal-analysis prompt
   curl -s -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.5-flash:generateContent?key=$GEMINI_API_KEY" \
     -H "Content-Type: application/json" \
     -d "{
       \"contents\":[{
         \"parts\":[
           {\"file_data\":{\"mime_type\":\"video/mp4\",\"file_uri\":\"$FILE_URI\"}},
           {\"text\":\"<multimodal analysis prompt — see Step 7's summary template + use-case extensions>\"}
         ]
       }]
     }"

   Files persist in Gemini Files API for ~48 hours — useful for re-querying the same video with different prompts.
2. Dense Claude vision fallback if no Gemini key:
   • Frame cadence: 1 frame per 3s (much denser than visual mode)
   • Batch through Claude vision with the multimodal-analysis prompt
   • Slower and more expensive than Gemini for long videos — warn the user before running on >10min content

• Body language / delivery: <observations on talking-head video>
• Pacing: <fast/slow/uneven>
• Visual style: <brand audit, ad review, design observations>
• Audio quality / atmosphere: <music, silence, background>

Exact extra sections depend on the use case (brand audit, ad review, talk delivery review, client-call read). Use case is inferred from the source + the user's verbal framing when invoking.

"Want to capture this to second-brain? I'll write a

call-<slug>.md

(or

meeting-

/

note-

/

resource-

) to

${SECOND_BRAIN_VAULT:-$HOME/Documents/SecondBrain}/raw/

with the summary, source URL, and transcript link."

call-

meeting-

note-

resource-

• One-line headline:

  <source> · <title> · <duration> · <mode> · <word count> words

• Workdir path
• For

  visual

  /

  multimodal

  : brief list of top 3 key moments
• For all modes: any action items / decisions flagged for triage
• If captured to second-brain: that path too

• social-fetch

  — for X/IG/TikTok URL metadata (engagement, author, replies) before video processing

• second-brain

  — capture summary as

  raw/call-<slug>.md

  ,

  meeting-

  ,

  note-

  , or

  resource-

  per source type

• decide

  — when a video contains a flagged decision, route to

  /decide

  for structured capture

• pm

  — action items flagged in summary can be triaged to project boards

• slide-deck

  — talk recordings → outline extraction → deck draft (loop)

• jab-hook

  — quotes + clip-worthy moments from podcast/talk videos feed BIP/promo posts

• skillify from-video

  — primary use case for

  visual

  mode on process recordings. the user records themselves doing a workflow (Loom/screen-share), this skill extracts transcript + key visual moments, then

  skillify

  synthesizes the workflow into a SKILL.md. "Record once, AI converts to skill."

• User picks depth, not the skill. Transcript / visual / multimodal are 3 different cost + latency profiles. Long videos (>10 min) always confirm before spending on visual/multimodal.
• Platform transcript first, Whisper second. YouTube auto-subs, Loom transcripts, Riverside built-in transcripts — all free + instant when they exist. Fall back to MLX-Whisper local only when nothing platform-provided works.
• MLX-Whisper local is the fast path on Mac. M-series machines transcribe faster than real-time. Cloud Whisper is a distant second choice — costs money, network dependency, worse latency on typical durations.
• Frame cadence by source type. Screen-share / demos need 1 frame per 5s (UI changes fast); talking-head podcasts need 1 per 30s (slow change). Default 15s if unsure. Wrong cadence = missed key moments OR wasted vision-pass cost.
• 720p is plenty. Downloading 1080p / 4K for transcription + frame analysis wastes bandwidth + storage.

  yt-dlp -f "bv*[height<=720]+ba/b[height<=720]"

  is the default.
• Scene-change detection catches slide transitions. When the video is a slide presentation, add

  ffmpeg -vf "select='gt(scene,0.3)'"

  to force a frame on each detected slide change — more reliable than pure time-based sampling.
• Multimodal cost warning is non-optional. Gemini multimodal on a 60-min video is meaningfully expensive. Warn before running; offer transcript-only as fallback if the user isn't sure.
• Summary format includes routing hints.

  ## Decisions flagged

  +

  ## Action items flagged

  sections signal

  /decide

  and

  /pm

  follow-ups. Downstream composability lives in the summary structure.