SmolVLM - Local Image Analysis

Quick Usage

Describe an Image

python ~/.claude/skills/smolvlm/scripts/view_image.py /path/to/image.png

Ask a Question About an Image

python ~/.claude/skills/smolvlm/scripts/view_image.py /path/to/image.png "What text is visible?"

Specific Tasks

# Extract text (OCR)
python ~/.claude/skills/smolvlm/scripts/view_image.py screenshot.png "Extract all text"

# UI analysis
python ~/.claude/skills/smolvlm/scripts/view_image.py ui.png "Describe the UI elements"

# Detailed description
python ~/.claude/skills/smolvlm/scripts/view_image.py photo.jpg --detailed

Effective Prompts

General Description

• "Describe this image"

  - Basic description

• "Describe this image in detail, including colors, composition, and any text"

  - Comprehensive

Text Extraction (OCR)

• "Extract all visible text from this image"

• "What text appears in this screenshot?"

• "Read the text in this document"

UI/Screenshot Analysis

• "Describe the user interface elements"

• "What buttons and controls are visible?"

• "Identify the application and its current state"

Visual Question Answering

• "How many [objects] are in this image?"

• "What color is the [object]?"

• "Is there a [object] in this image?"

Code/Technical

• "What programming language is shown?"

• "Describe what this code does"

• "Identify any errors in this code screenshot"

Model Details

Requirements

• macOS with Apple Silicon (M1/M2/M3)
• Python 3.10+
• mlx-vlm package:

  uv pip install mlx-vlm --system

Troubleshooting