Vidlens

VidLens

Extract structured artifacts from video files for LLM analysis using ffmpeg/ffprobe + Bun.

LLMs cannot process video directly. VidLens extracts small, structured artifacts — frame images, contact sheets, metadata, and clips — so agents can "see" video content through images.

Prerequisites

• ffmpeg and ffprobe (included with ffmpeg)
• Bun runtime

# macOS
brew install ffmpeg
# Bun
curl -fsSL https://bun.sh/install | bash

Installation (Claude Code)

/plugin marketplace add mplibunao/mp-claude-marketplace
/plugin install vidlens@mp-claude-marketplace

Usage

As a Claude Code skill

Invoke the skill with /vidlens:vidlens. The environment variable $CLAUDE_PLUGIN_ROOT points to the plugin's installed directory.

CLI examples

# Get video metadata
bun ${CLAUDE_PLUGIN_ROOT}/skills/vidlens/scripts/vidlens.ts info --input video.mp4 --out metadata.json

# Extract frames at 1 fps, scaled to 1280px width
bun ${CLAUDE_PLUGIN_ROOT}/skills/vidlens/scripts/vidlens.ts frames --input video.mp4 --fps 1 --outdir frames/ --scale 1280

# Detect scene changes
bun ${CLAUDE_PLUGIN_ROOT}/skills/vidlens/scripts/vidlens.ts scenes --input video.mp4 --threshold 0.3 --outdir scenes/

# Create 4x4 contact sheet
bun ${CLAUDE_PLUGIN_ROOT}/skills/vidlens/scripts/vidlens.ts contact-sheet --input video.mp4 --grid 4x4 --out overview.jpg

# Extract clip (fast, stream copy)
bun ${CLAUDE_PLUGIN_ROOT}/skills/vidlens/scripts/vidlens.ts clip --input video.mp4 --start 00:01:00 --duration 00:00:30 --out clip.mp4

# Extract clip (precise, re-encode)
bun ${CLAUDE_PLUGIN_ROOT}/skills/vidlens/scripts/vidlens.ts clip --input video.mp4 --start 00:01:00 --duration 00:00:30 --out clip.mp4 --reencode

Guardrails

These rules must be followed when using VidLens:

1. Always run info first, then choose a sampling strategy based on the video's duration and resolution.
2. Confirm with the user before extracting thousands of frames. After running info, estimate duration * fps. If the result is >= 1000, ask the user to confirm and suggest alternatives:
   • Lower fps (e.g., 0.2-1 fps)
   • Contact sheet (e.g., 4x4)
   • Scene detection (threshold 0.3-0.5)

Exit codes

Code	Meaning
0	Success
2	Missing ffmpeg/ffprobe or input file
3	ffmpeg/ffprobe error (non-zero exit or unusable output)
4	Unexpected error / usage error

Development

cd vidlens
bun install
bun test

License

MIT