
Shorts
- 50 installs
- 172 repo stars
- Updated July 11, 2026
- agricidaniel/claude-shorts
shorts is a Claude Code skill that extracts viral-ready vertical short clips from long videos, rendering animated captions with Remotion and exporting per-platform files.
About
shorts is an interactive longform-to-shortform video creator that extracts viral-ready short clips from long videos. It transcribes with faster-whisper, has Claude score and present candidate segments interactively, snaps boundaries to natural cut points, renders premium animated captions with Remotion, and exports platform-optimized files with FFmpeg. A developer uses it to turn one long video into vertical clips for YouTube Shorts, TikTok, and Reels. It runs a 10-step interactive pipeline.
- Extracts viral-ready short clips from long videos with Claude as orchestrator
- Transcribes with faster-whisper, scores segments, renders animated captions with Remotion
- Exports platform-optimized files for YouTube Shorts, TikTok, and Instagram Reels
Shorts by the numbers
- 50 all-time installs (skills.sh)
- Ranked #897 of 1,335 Generative Media skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
shorts capabilities & compatibility
Free and local; benefits from an NVIDIA GPU for faster transcription and encoding.
- Capabilities
- video generation · transcription · caption rendering
- Use cases
- video generation · transcription · marketing
- Runs
- Runs locally
- Pricing
- Free
What shorts says it does
Interactive longform-to-shortform video creator. Extracts viral-ready short clips
Remotion renders premium animated captions (Bold/Bounce/Clean styles)
FFmpeg exports platform-optimized files (YouTube Shorts, TikTok, Instagram Reels).
npx skills add https://github.com/agricidaniel/claude-shorts --skill shortsAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 50 |
|---|---|
| repo stars | ★ 172 |
| Last updated | July 11, 2026 |
| Repository | agricidaniel/claude-shorts ↗ |
What it does
Turn a long video into captioned vertical short clips for YouTube Shorts, TikTok, and Instagram Reels.
Who is it for?
Extracting captioned vertical clips from a long video for Shorts, TikTok, and Reels.
Skip if: Writing social copy or generating video from a text prompt.
When should I use this skill?
The user says shorts, short clips, extract clips, reels from video, or create shorts.
What you get
Captioned 9:16 short clips exported per platform from a chosen long video.
- Captioned vertical short clips
- Cleaned transcript
- Platform-optimized export files
By the numbers
- 10-step interactive pipeline
- scores 8-12 candidate segments
- 3 caption styles: Bold, Bounce, Clean
Files
shorts — Interactive Shortform Video Creator
You are an interactive shortform video producer. You guide the user through a 10-step pipeline where YOU (Claude) analyze the transcript, identify the best segments, present them for approval, snap boundaries to natural audio cut points, and render premium vertical videos with animated captions.
Pre-Flight
Before starting, locate the project root:
# Try common locations in priority order
SHORTS_ROOT=""
for dir in "$HOME/.claude/skills/shorts" "$HOME/.claude/skills/claude-shorts" "$HOME/claude-shorts" "$(pwd)"; do
if [ -f "$dir/SKILL.md" ]; then
SHORTS_ROOT="$dir"
break
fi
done
if [ -z "$SHORTS_ROOT" ]; then
echo "ERROR: shorts skill project root not found. Please run from the project directory or install with install.sh"
fiSet up the temp directory (configurable via SHORTS_TMP environment variable):
SHORTS_TMP="${SHORTS_TMP:-/tmp/claude-shorts}"
mkdir -p "$SHORTS_TMP/clips"10-Step Interactive Pipeline
Step 1: PREFLIGHT
Run safety checks on the input video:
bash "$SHORTS_ROOT/scripts/preflight.sh" INPUT_FILE [OUTPUT_DIR]If preflight fails, report errors and stop. If warnings exist, report them and ask the user whether to proceed.
Also detect GPU capabilities:
bash "$SHORTS_ROOT/scripts/detect_gpu.sh"Report to user: input duration, resolution, GPU status, estimated processing time.
Step 2: TRANSCRIBE
Transcribe with faster-whisper (GPU-accelerated, word-level timestamps). Audio extraction is handled internally by transcribe.py:
VENV="$HOME/.video-skill"
[ -d "$VENV" ] || VENV="$HOME/.shorts-skill"
source "$VENV/bin/activate"
python3 "$SHORTS_ROOT/scripts/transcribe.py" INPUT_FILE \
--output $SHORTS_TMP/transcript.jsonOutput is dual-format JSON:
segments[]— WhisperX-style with word timestamps (for Claude to read)captions[]— Remotion-native{text, startMs, endMs}array (for rendering)
Report to user: transcription time, word count, language detected.
Step 3: DETECT CONTENT TYPE
Auto-detect whether the video is talking-head, screen recording, or podcast:
python3 "$SHORTS_ROOT/scripts/detect_content.py" INPUT_FILE \
--output $SHORTS_TMP/content_type.jsonReport detected type to user. Ask if they want to override.
- talking-head: Face-tracked center crop to 9:16
- screen: Letterboxed framed layout (content centered, dark padding)
- podcast: Side-by-side speaker tracking or center crop
Step 4: ANALYZE — Claude Reads Transcript
Read the full transcript directly:
Read $SHORTS_TMP/transcript.jsonAlso load the scoring rubric:
Read $SHORTS_ROOT/references/scoring-rubric.mdScore 8-12 candidate segments (15-55 seconds each) on 5 dimensions:
| Dimension | Weight | What to look for |
|---|---|---|
| Hook strength | 0.30 | Bold claims, curiosity gaps, value promises, pattern interrupts |
| Standalone coherence | 0.25 | Makes complete sense without any context from the rest of the video |
| Emotional intensity | 0.20 | Strong opinions, surprise reveals, humor, passion |
| Value density | 0.15 | Actionable insights, data points, frameworks per second |
| Payoff quality | 0.10 | Satisfying conclusion — punchline, reveal, call-to-action |
Weighted score = sum of (dimension_score * weight), scale 0-100.
For each candidate, identify:
- Start/end timestamps (to the nearest second)
- A suggested hook line (first 3 seconds of text overlay)
- Brief rationale (1 sentence explaining why this segment works)
Transcript cleanup: While analyzing, also produce cleaned captions for rendering. Read the captions[] array from transcript.json, then: 1. Remove filler words (um, uh, you know, like, sort of, I mean, right, basically, actually) 2. Fix obvious transcription errors based on surrounding context 3. Consolidate incomplete sentence fragments where appropriate 4. Keep all timestamps unchanged — only modify the text field
Write the cleaned transcript to $SHORTS_TMP/transcript_cleaned.json using the same JSON structure as transcript.json (both segments and captions arrays). The captions array should contain the cleaned text; copy segments as-is.
Step 5: PRESENT — Show Candidates Interactively
Present candidates in a formatted table:
| # | Time | Dur | Score | Hook | Why |
|---|---------------|------|-------|-----------------------------------|----------------------------------------|
| 1 | 04:22 → 05:01 | 39s | 87 | "Nobody talks about this..." | Contrarian take with data backing |
| 2 | 12:45 → 13:28 | 43s | 82 | "Here's the exact framework..." | Complete actionable method, clean arc |
| 3 | 08:11 → 08:52 | 41s | 79 | "I tested this for 6 months..." | Personal story + surprising result |Then ask the user using AskUserQuestion: 1. Which segments? — "all", specific numbers, or "none, re-analyze" 2. Caption style? — bold (ALL CAPS pop-in), bounce (bouncy colorful), clean (minimal fade) 3. Platform? — youtube, tiktok, instagram, or all
Step 6: APPROVE — Interactive Adjustment Loop
After user selects segments:
- Show selected segments with exact timestamps
- Allow timecode adjustments ("move segment 2 start back 3 seconds")
- Confirm final selections
- Estimate render time (~15-30s per segment with Remotion)
Write approved segments to:
cat > $SHORTS_TMP/approved_segments.json << 'EOF'
{
"segments": [
{
"id": 1,
"start": 262.0,
"end": 301.0,
"hook_line1": "Nobody talks about this...",
"hook_line2": "The hidden cost of scaling",
"score": 87
}
],
"style": "bold",
"platform": "all",
"content_type": "talking-head"
}
EOFStep 7: SNAP BOUNDARIES — Audio-Aware Cut Points
Snap segment boundaries to natural audio cut points so clips never cut mid-word or mid-sentence:
python3 "$SHORTS_ROOT/scripts/snap_boundaries.py" \
--segments $SHORTS_TMP/approved_segments.json \
--transcript $SHORTS_TMP/transcript.json \
--input-video INPUT_FILE \
--output $SHORTS_TMP/snapped_segments.jsonThe script: 1. Loads word-level timestamps from the transcript 2. Snaps start times to the nearest word boundary (prefers sentence starts) 3. Extends end times to the next sentence boundary (. ? !) if within 3 seconds 4. Adds 300ms padding after the last word 5. Uses FFmpeg silencedetect to find natural pauses near cut points 6. Enforces min 5s / max 60s duration, clamps to video bounds
Use --no-silence to skip silence detection (faster, word-boundary snapping only).
Report to user: adjustment deltas per segment (e.g., "start +150ms, end +362ms").
From this point forward, use snapped_segments.json instead of approved_segments.json.
Step 8: PREPARE — Extract Clips + Compute Reframe
Extract each snapped segment via FFmpeg stream copy (near-instant, lossless). Use the snapped start/end times from $SHORTS_TMP/snapped_segments.json:
ffmpeg -y -ss START -to END -i INPUT_FILE -c copy \
$SHORTS_TMP/clips/clip_01.mp4Compute reframe coordinates for each clip:
python3 "$SHORTS_ROOT/scripts/compute_reframe.py" \
--clips-dir $SHORTS_TMP/clips/ \
--content-type CONTENT_TYPE \
--output $SHORTS_TMP/reframe.jsonReport to user: clips extracted, content type per clip, reframe strategy.
Step 9: RENDER via Remotion
Render all snapped segments with the selected caption style:
node "$SHORTS_ROOT/remotion/render.mjs" \
--segments $SHORTS_TMP/snapped_segments.json \
--reframe $SHORTS_TMP/reframe.json \
--captions $SHORTS_TMP/transcript_cleaned.json \
--style STYLE \
--clips-dir $SHORTS_TMP/clips/ \
--output-dir $SHORTS_TMP/render/The render script: 1. Bundles the Remotion project once (~5-10s) 2. Opens a shared Chrome instance 3. Renders each segment sequentially (~15-30s each) 4. Outputs 1080x1920 MP4 files
Report progress to user as each segment renders.
Step 10: EXPORT — Platform-Optimized Encoding
Export rendered shorts with platform-specific encoding:
bash "$SHORTS_ROOT/scripts/export.sh" \
--input-dir $SHORTS_TMP/render/ \
--platform PLATFORM \
--output-dir ./shorts/Platform encoding specs:
- YouTube Shorts: H.264 High 4.2, 12 Mbps, AAC 192k
- TikTok: H.264, CRF 18, -preset slow, AAC 128k
- Instagram Reels: H.264 High 4.2, 4.5 Mbps maxrate 5000k, AAC 128k
- All: Exports all three variants per clip
With NVENC GPU: h264_nvenc -preset p5 -tune hq for 5-10x faster encoding.
Present final summary table:
| # | File | Platform | Duration | Size |
|---|---------------------------|-----------|----------|--------|
| 1 | shorts/short_01_yt.mp4 | YouTube | 39s | 12.3MB |
| 1 | shorts/short_01_tt.mp4 | TikTok | 39s | 8.7MB |
| 1 | shorts/short_01_ig.mp4 | Instagram | 39s | 7.1MB |Post-export validation: Run validation on all exported files:
bash "$SHORTS_ROOT/scripts/validate.sh" --output-dir ./shorts/Checks: file is playable, resolution is 1080x1920, audio track exists and isn't silent, file size is within platform limits, video codec is H.264, duration is 3-90 seconds. If any file fails, report the issues to the user. Failed files should be re-rendered or re-exported before delivery.
Important Rules
1. Always run preflight before any processing 2. Always present segments for approval — never auto-render without user confirmation 3. Always report costs — Remotion rendering is free (local), only potential cost is GPU power 4. Handle errors gracefully — if any step fails, report the error and suggest fixes 5. Clean up on success — offer to delete $SHORTS_TMP/ after export 6. Respect the user's choices — if they say "re-analyze", go back to Step 4 7. Stream copy for extraction — never re-encode when cutting segments (use -c copy) 8. One segment at a time for progress reporting during render 9. Load references when needed — scoring-rubric.md for Step 4, caption-styles.md for style questions
Caption Style Reference
| Style | Font | Look | Best for |
|---|---|---|---|
| bold | Montserrat Bold | ALL CAPS, pop-in, yellow active word | Business, education, motivation |
| bounce | Bangers | Bouncy scale, rotating bright colors | Entertainment, reactions, energy |
| clean | Inter Bold | Minimal fade-in, white + shadow | Professional, calm, interviews |
Load references/caption-styles.md for detailed visual specs and spring configs.
Configurable Parameters
These defaults work well for most content. Offer alternatives when the user has specific needs.
| Parameter | Default | Flag/Var | When to change |
|---|---|---|---|
| Whisper model | large-v3 | --model small | Low VRAM (< 6 GB) |
| Screen zoom | 0.55 | --zoom 0.4 | More context visible in screen recordings |
| Cursor tracking | enabled | --no-cursor-track | Static screen content (slides, documents) |
| Silence detection | enabled | --no-silence | Faster processing, word-boundary-only snapping |
| Score threshold | 60 | (SKILL.md instruction) | Lower for longer videos with fewer highlights |
| Segment duration | 15-55s | (SKILL.md instruction) | Adjust per platform (TikTok prefers 21-34s) |
| Temp directory | /tmp/claude-shorts/ | SHORTS_TMP env var | Systems with limited /tmp space |
| Export platform | all | --platform youtube | Single-platform targeting |
Error Recovery
- Transcription fails: Check venv activation, try
--model smallfor less VRAM - Remotion render fails: Check
cd remotion && npm install, verify node_modules exists - Export fails: Check FFmpeg version (
ffmpeg -version), try CPU encoding if NVENC fails - Out of disk space: Clean $SHORTS_TMP/, check with
df -h /tmp
{
"name": "shorts",
"version": "1.1.0",
"description": "Interactive longform-to-shortform video creator. Extracts viral-ready short clips from long videos with Remotion-rendered animated captions, AI segment scoring, and platform-optimized exports for YouTube Shorts, TikTok, and Instagram Reels.",
"author": "AgriciDaniel",
"license": "MIT",
"homepage": "https://github.com/AgriciDaniel/claude-shorts"
}
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
labels:
- "dependencies"
open-pull-requests-limit: 5
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
labels:
- "dependencies"name: Bug Report
description: Report a bug or unexpected behavior
labels: ["bug"]
body:
- type: markdown
attributes:
value: Thanks for reporting a bug! Please fill out the details below.
- type: textarea
id: description
attributes:
label: What happened?
description: Describe the bug clearly
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
description: What did you expect to happen?
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to reproduce
description: How can we reproduce this?
placeholder: |
1. ...
2. ...
3. See error
validations:
required: true
- type: input
id: environment
attributes:
label: Environment
description: OS, runtime version, or any relevant environment details
placeholder: "e.g., macOS 14, Python 3.12, Node 20"
- type: textarea
id: logs
attributes:
label: Error output
description: Paste any error messages or logs
render: shell
blank_issues_enabled: false
contact_links:
- name: Discussions
url: https://github.com/AgriciDaniel/claude-shorts/issues
about: Ask questions and share ideasname: Feature Request
description: Suggest a new feature or improvement
labels: ["enhancement"]
body:
- type: markdown
attributes:
value: We'd love to hear your ideas!
- type: textarea
id: problem
attributes:
label: Problem or use case
description: What problem does this solve or what would it enable?
validations:
required: true
- type: textarea
id: solution
attributes:
label: Proposed solution
description: How would you like this to work?
validations:
required: true
- type: dropdown
id: area
attributes:
label: Area
options:
- Bug fix
- New feature
- Documentation
- Performance
- Other
Summary
<!-- What does this PR do? Keep it brief. -->
Changes
-
Checklist
- [ ] Changes are tested and working
- [ ] Documentation updated (if applicable)
- [ ] No secrets or API keys committed
changelog:
categories:
- title: New Features
labels:
- enhancement
- title: Bug Fixes
labels:
- bug
- title: Documentation
labels:
- documentation
- title: Dependencies
labels:
- dependencies
- title: Other Changes
labels:
- "*"
# Generated outputs
shorts/
/tmp/
# Remotion cache
.remotion/
# Dependencies
node_modules/
# Python
__pycache__/
*.pyc
*.pyo
*.egg-info/
dist/
build/
# Virtual environments
.venv/
venv/
# Environment variables
.env
.env.local
.env.*.local
# OS files
.DS_Store
Thumbs.db
*.swp
*.swo
*~
# IDE
.vscode/
.idea/
*.code-workspace
# Temporary / intermediate files
*.wav
*.tmp
Changelog
v1.1.0
- Renamed caption styles from creator names to generic: Bold, Bounce, Clean
- Renamed skill command from
/claude-shortsto/shorts - Audio loudnorm normalization in export pipeline
- Skip existing exports instead of overwriting
- Use segment ID for clip/output filenames instead of array index
- Render retry logic (1 automatic retry before skipping)
- Snap boundaries tolerance fix for edge-case word alignment
- Batch frame extraction for content detection
- Configurable temp directory via
$SHORTS_TMP
v1.0.0 — Initial Release
- 10-step interactive pipeline for longform-to-shortform video creation
- 3 caption styles: Bold, Bounce, Clean
- Remotion v4 rendering with spring animations and word-level karaoke highlighting
- GPU-accelerated transcription via faster-whisper
- Audio-aware boundary snapping (never cuts mid-word or mid-sentence)
- Cursor tracking for screen recordings
- Face tracking for talking-head content
- Platform-optimized export for YouTube Shorts, TikTok, and Instagram Reels
- NVENC GPU encoding with automatic CPU fallback
claude-shorts
Standalone interactive shortform video creator. Extracts viral-ready short clips from longform videos using an interactive Claude Code workflow with Remotion-rendered premium captions.
Architecture
- Claude Code is the orchestrator — reads transcripts, scores segments, presents to user interactively
- faster-whisper handles GPU-accelerated transcription with word-level timestamps
- Remotion renders 1080x1920 vertical video with animated captions (single-pass)
- FFmpeg extracts audio, cuts segments (stream copy), and does platform-specific export encoding
Key Commands
bash setup.sh # Install Python + Node dependencies (one-time)
bash install.sh # Copy skill to ~/.claude/skills/shorts/Python Virtual Environment
Scripts use ~/.video-skill/ venv if it exists (shared with claude-video), otherwise ~/.shorts-skill/. Activate before running Python scripts:
source ~/.video-skill/bin/activate # or ~/.shorts-skill/bin/activateRemotion
The Remotion project lives in remotion/. After bash setup.sh:
cd remotion && npx remotion preview # Preview compositions in browser
node remotion/render.mjs # Headless render (used by pipeline)Temp Files
Pipeline stores intermediate files in $SHORTS_TMP (defaults to /tmp/claude-shorts/). Override with export SHORTS_TMP="/path/to/dir" for systems with limited /tmp space. The directory is created automatically and cleaned between runs.
Dependencies
- FFmpeg (system) — audio extraction, segment cutting, export encoding
- Python 3.10+ with faster-whisper, mediapipe, numpy, opencv-python
- Node.js 18+ with remotion, @remotion/captions, zod
- NVIDIA GPU recommended (NVENC encoding, CUDA transcription)
Contributor Covenant Code of Conduct
Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
Our Standards
Examples of behavior that contributes to a positive environment:
- Using welcoming and inclusive language
- Being respectful of differing viewpoints and experiences
- Gracefully accepting constructive criticism
- Focusing on what is best for the community
- Showing empathy towards other community members
Examples of unacceptable behavior:
- The use of sexualized language or imagery, and sexual attention or advances
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information without explicit permission
- Other conduct which could reasonably be considered inappropriate
Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue on this repository.
All complaints will be reviewed and investigated promptly and fairly.
Attribution
This Code of Conduct is adapted from the Contributor Covenant, version 2.1, available at https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
Contributing to claude-shorts
Thanks for your interest in contributing! Here's how to get involved.
Reporting Bugs
Open a GitHub Issue with:
- Your OS, Python version, Node.js version, and GPU info
- The full error output (copy from terminal)
- The command or step that failed
- Input video details (duration, resolution, codec) if relevant
Suggesting Features
Use GitHub Discussions for feature ideas and questions.
Pull Requests
1. Fork the repository 2. Create a feature branch (git checkout -b feature/my-feature) 3. Make your changes 4. Test with a short video clip (under 5 minutes) before submitting 5. Submit a PR with a clear description of what changed and why
Development Setup
git clone https://github.com/YOUR_USERNAME/claude-shorts.git
cd claude-shorts
bash setup.sh
bash install.shGuidelines
- All Python scripts should output JSON for Claude Code to parse
- Shell scripts should use
set -euo pipefailfor safety - Test both GPU and CPU code paths when possible
- Keep dependencies minimal — don't add packages for single-use operations
#!/usr/bin/env bash
# Install claude-shorts skill to ~/.claude/skills/
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_DIR="$HOME/.claude/skills/shorts"
echo "=== Installing shorts skill ==="
# Create skill directory
mkdir -p "$SKILL_DIR"
mkdir -p "$SKILL_DIR/scripts"
mkdir -p "$SKILL_DIR/references"
mkdir -p "$SKILL_DIR/remotion"
# Copy SKILL.md (the skill definition)
cp "$SCRIPT_DIR/SKILL.md" "$SKILL_DIR/SKILL.md"
# Copy scripts
for f in "$SCRIPT_DIR"/scripts/*.sh "$SCRIPT_DIR"/scripts/*.py; do
[ -f "$f" ] && cp "$f" "$SKILL_DIR/scripts/"
done
# Make scripts executable
chmod +x "$SKILL_DIR"/scripts/*.sh 2>/dev/null || true
chmod +x "$SKILL_DIR"/scripts/*.py 2>/dev/null || true
# Copy references
for f in "$SCRIPT_DIR"/references/*.md; do
[ -f "$f" ] && cp "$f" "$SKILL_DIR/references/"
done
# Copy Remotion project (excluding node_modules)
rsync -a --exclude='node_modules' --exclude='.remotion' \
"$SCRIPT_DIR/remotion/" "$SKILL_DIR/remotion/"
echo ""
echo "Installed to: $SKILL_DIR"
echo ""
echo "Files copied:"
find "$SKILL_DIR" -type f | sort | while read -r f; do
echo " ${f#$SKILL_DIR/}"
done
echo ""
echo "Next steps:"
echo " 1. Run 'bash setup.sh' to install dependencies (if not done)"
echo " 2. Use /shorts in Claude Code to start the interactive pipeline"
MIT License
Copyright (c) 2026 Daniel Agrici
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
claude-shorts
!claude-shorts header
Interactive longform-to-shortform video creator powered by Claude Code. Extracts viral-ready vertical clips from long videos using Claude as the intelligent orchestrator with Remotion-rendered premium animated captions.
How It Works
Claude Code guides you through a 10-step interactive pipeline:
1. Preflight - Validates input video, checks disk space, detects GPU 2. Transcribe - GPU-accelerated transcription with word-level timestamps (faster-whisper) 3. Detect Content - Auto-classifies: talking-head, screen recording, or podcast 4. Analyze - Claude reads the full transcript and scores 8-12 candidate segments 5. Present - Shows candidates in a formatted table with scores, hooks, and rationale 6. Approve - You pick segments, adjust timecodes, choose caption style and platform 7. Snap Boundaries - Aligns cut points to word boundaries, sentence endings, and audio silences 8. Prepare - Extracts clips (FFmpeg stream copy) and computes reframe coordinates 9. Render - Remotion renders 1080x1920 vertical video with animated captions 10. Export - Platform-optimized encoding (YouTube Shorts, TikTok, Instagram Reels)
Demo
Demo video/GIF coming soon — showing the full pipeline from input to rendered short with Bold-style captions.
Features
- Claude-powered segment scoring - 5-dimension rubric (hook strength, coherence, emotion, value density, payoff) with weighted scoring. No heuristic keyword matching - Claude understands narrative arcs.
- 3 caption styles - Bold (ALL CAPS, yellow highlights), Bounce (bouncy spring, rotating colors), Clean (minimal fade-in)
- Cursor tracking - For screen recordings, detects mouse cursor via frame differencing and smoothly pans the crop to follow it
- Audio-aware boundary snapping - Never cuts mid-word or mid-sentence. Extends to natural sentence endings and silence points.
- Remotion rendering - React-based single-pass rendering with spring animations, word-level karaoke highlighting, hook text overlays, and progress bars
- GPU acceleration - CUDA for transcription, NVENC for export encoding (falls back to CPU gracefully)
Prerequisites
- FFmpeg (system package)
- Python 3.10+
- Node.js 18+
- Claude Code (CLI)
- NVIDIA GPU recommended (for CUDA transcription + NVENC encoding)
Installation
# Clone the repository
git clone https://github.com/AgriciDaniel/claude-shorts.git
cd claude-shorts
# Install Python + Node.js dependencies
bash setup.sh
# Install as a Claude Code skill
bash install.shWindows
claude-shorts requires Unix tools (FFmpeg, bash). On Windows, use WSL 2:
# Inside WSL
git clone https://github.com/AgriciDaniel/claude-shorts.git
cd claude-shorts
bash setup.sh
bash install.shWhat setup.sh does
- Creates a Python virtual environment at
~/.shorts-skill/(or reuses~/.video-skill/if it exists) - Installs
faster-whisper,mediapipe,numpy,opencv-python, and PyTorch (CUDA or CPU variant) - Runs
npm installin theremotion/directory - Checks for system dependencies (FFmpeg, jq)
Usage
In Claude Code, invoke the skill:
/shortsThen provide your video file when prompted. Claude will:
1. Transcribe the video 2. Present scored segment candidates 3. Ask which segments to render, caption style, and target platform 4. Render and export the final shorts
Example Interaction
You: /shorts ~/Videos/my-talk.mp4
Claude: [Transcribes, detects content type, scores segments]
| # | Time | Dur | Score | Hook |
|---|---------------|------|-------|--------------------------------|
| 1 | 04:22 - 05:01 | 39s | 87 | "Nobody talks about this..." |
| 2 | 12:45 - 13:28 | 43s | 82 | "Here's the exact framework." |
| 3 | 08:11 - 08:52 | 41s | 79 | "I tested this for 6 months." |
Claude: Which segments? Caption style? Platform?
You: 1 and 3, bounce style, youtube
Claude: [Snaps boundaries, extracts clips, renders, exports]
Output: shorts/short_01_yt.mp4, shorts/short_03_yt.mp4Project Structure
claude-shorts/
├── SKILL.md # 10-step interactive pipeline (Claude Code skill)
├── CLAUDE.md # Project-level instructions
├── install.sh # Install to ~/.claude/skills/
├── setup.sh # Python + Node dependency installer
│
├── scripts/
│ ├── transcribe.py # faster-whisper GPU transcription
│ ├── detect_content.py # MediaPipe content type classifier
│ ├── compute_reframe.py # Face tracking + cursor tracking + crop
│ ├── snap_boundaries.py # Audio-aware boundary snapping
│ ├── preflight.sh # Input validation + disk space check
│ ├── detect_gpu.sh # NVIDIA NVENC detection
│ └── export.sh # Platform-specific FFmpeg encoding
│
├── remotion/
│ ├── package.json # Remotion v4 + React 19 + Zod
│ ├── render.mjs # Bundle-once-render-many orchestrator
│ ├── remotion.config.ts
│ └── src/
│ ├── Root.tsx # Composition registry
│ ├── ShortVideo.tsx # Main composition
│ ├── types.ts # Zod schemas for props
│ ├── components/
│ │ ├── VideoFrame.tsx # Reframed video with animated crop pan
│ │ ├── Captions.tsx # Style dispatcher
│ │ ├── BoldCaptions.tsx # Bold ALL CAPS, pop-in spring
│ │ ├── BounceCaptions.tsx # Bouncy scale, bright colors
│ │ ├── CleanCaptions.tsx # Minimal fade-in
│ │ ├── HookOverlay.tsx # First 3.5s hook text
│ │ └── ProgressBar.tsx # Bottom progress indicator
│ ├── hooks/
│ │ └── useCaptionPages.ts # @remotion/captions TikTok-style pages
│ └── styles/
│ ├── fonts.ts # @font-face declarations
│ └── theme.ts # Color palettes per style
│
└── references/
├── scoring-rubric.md # 5-dimension scoring criteria
├── caption-styles.md # Visual specs + spring configs
├── platform-specs.md # YouTube/TikTok/Instagram encoding
└── remotion-patterns.md # Remotion best practicesCaption Styles
| Style | Font | Animation | Best For |
|---|---|---|---|
| Bold | Montserrat Bold | Pop-in spring, yellow active word | Business, education |
| Bounce | Bangers | Bouncy scale 70-120-100%, rotating colors | Entertainment, energy |
| Clean | Inter Bold | Fade-in opacity, white + shadow | Professional, interviews |
Platform Export Specs
| Platform | Codec | Bitrate | Audio |
|---|---|---|---|
| YouTube Shorts | H.264 High 4.2 | 12 Mbps | AAC 192k |
| TikTok | H.264 | CRF 18, -preset slow | AAC 128k |
| Instagram Reels | H.264 High 4.2 | 4.5 Mbps (max 5000k) | AAC 128k |
Content Type Strategies
| Content Type | Reframe Strategy | Zoom |
|---|---|---|
| Talking-head | Face-tracked center crop (MediaPipe) | 9:16 exact |
| Screen recording | Cursor-tracked pan with moderate zoom | 55% of source width |
| Podcast | Dominant speaker tracking | 9:16 exact |
Dependencies
Python (installed via setup.sh)
- faster-whisper - GPU-accelerated Whisper
- mediapipe - Face detection for content classification + reframing
- numpy - Array operations for cursor tracking smoothing
- opencv-python - Frame differencing for cursor detection
Node.js (installed via setup.sh)
- Remotion v4 - React-based video rendering
- @remotion/captions - TikTok-style word-level captions
- React 19 - Component framework
- Zod - Runtime type validation for props
Fonts
Caption fonts are bundled from Google Fonts under the SIL Open Font License:
- Montserrat Bold (Bold style)
- Bangers Regular (Bounce style)
- Inter Bold (Clean style)
System
How Segment Scoring Works
Claude scores each candidate on 5 weighted dimensions:
| Dimension | Weight | What Claude Looks For |
|---|---|---|
| Hook Strength | 0.30 | Bold claims, curiosity gaps, value promises, pattern interrupts |
| Standalone Coherence | 0.25 | Makes complete sense without any context from the rest of the video |
| Emotional Intensity | 0.20 | Strong opinions, surprise reveals, humor, passion |
| Value Density | 0.15 | Actionable insights, data points, frameworks per second |
| Payoff Quality | 0.10 | Satisfying conclusion - punchline, reveal, call-to-action |
Final score = weighted sum, scale 0-100. Minimum threshold: 60.
Support
- Issues: GitHub Issues
- Discussions: GitHub Discussions
License
MIT
---
Author
Built by Agrici Daniel - AI Workflow Architect.
- Blog - Deep dives on AI marketing automation
- AI Marketing Hub - Free community, 2,800+ members
- YouTube - Tutorials and demos
- All open-source tools
Caption Styles — Visual Specifications
Three premium caption presets for different content tones. Each uses @remotion/captions createTikTokStyleCaptions() for word-level timing.
Bold Style
Best for: Business, education, motivation, "guru" content.
| Property | Value |
|---|---|
| Font | Montserrat Bold (800 weight) |
| Font Size | 72px |
| Text Transform | UPPERCASE |
| Words Per Page | 2-3 |
| Combine Window | 800ms |
| Text Color | White (#FFFFFF) |
| Active Word | Yellow (#FFD700) |
| Text Shadow | 3px solid black outline (all 4 directions) |
| Position | Bottom: 350px from bottom edge |
| Animation | Pop-in spring: {mass:1, damping:12, stiffness:200} |
| Timing | Page appears at first word, disappears after last word |
Visual reference: All caps, word-by-word yellow highlight, bold pop-in entrance.
Spring config explained:
damping: 12— moderate bounce, settles in ~0.4sstiffness: 200— snappy entrance- Scale goes from 0 → slight overshoot (~1.05) → 1.0
Bounce Style
Best for: Entertainment, reactions, challenges, high-energy content.
| Property | Value |
|---|---|
| Font | Bangers (Google Fonts, OFL) |
| Font Size | 84px |
| Text Transform | UPPERCASE |
| Words Per Page | 1-2 |
| Combine Window | 600ms |
| Text Color | Rotating (see below) |
| Text Shadow | 4px solid black outline (all 4 directions) |
| Position | Bottom: 350px from bottom edge |
| Animation | Bouncy scale: 70% → ~120% → 100% |
| Spring Config | {mass:1, damping:8, stiffness:180} |
Color rotation (per page, cycling): 1. Cyan (#00FFFF) 2. Magenta (#FF00FF) 3. Lime (#00FF00) 4. Yellow (#FFFF00) 5. Orange (#FF6600) 6. Hot Pink (#FF0066)
Spring config explained:
damping: 8— very bouncy, overshoots significantly- Scale: starts at 0.7, springs up to ~1.2, settles at 1.0
- Creates the "explosive word pop" effect
Clean Style
Best for: Professional, interviews, calm content, podcasts.
| Property | Value |
|---|---|
| Font | Inter Bold (700 weight) |
| Font Size | 56px |
| Text Transform | None (preserves original case) |
| Words Per Page | 3-5 |
| Combine Window | 1500ms |
| Text Color | White (#FFFFFF) |
| Active Word | Light gray (#E0E0E0) |
| Text Shadow | 2px 2px 8px rgba(0,0,0,0.6) — soft shadow |
| Position | Bottom: 350px from bottom edge |
| Animation | Fade-in opacity 0→1 over 6 frames (200ms at 30fps) |
Visual reference: Minimal, readable, doesn't distract from the speaker. Subtle active-word highlighting through slight color change rather than bold contrast.
Font Files
All fonts must be placed in remotion/public/fonts/:
Montserrat-Bold.ttf— from Google Fonts (OFL license)Bangers-Regular.ttf— from Google Fonts (OFL license)Inter-Bold.ttf— from Google Fonts (OFL license)
Download from: https://fonts.google.com/
Caption Positioning
All styles share the same vertical positioning:
- Bottom offset: 350px from bottom edge of 1920px frame
- This places captions above the platform's built-in UI elements (like/comment/share buttons)
- Safe zone for TikTok: 150px from bottom, 64px from sides
- Safe zone for YouTube Shorts: 120px from bottom
- Our 350px offset clears all platform UI with margin
Integration with Hook Overlay
The hook overlay occupies the top 200px zone (first 3.5s only). Captions appear in the bottom zone. No overlap.
When both are visible (0.3s - 3.5s), the visual hierarchy is: 1. Hook text (top, large, attention-grabbing) 2. Video content (center) 3. Captions (bottom, word-level) 4. Progress bar (very bottom, subtle)
Platform Export Specifications
Encoding specs for each target platform, optimized for quality and upload compatibility.
YouTube Shorts
| Property | Value |
|---|---|
| Resolution | 1080x1920 |
| Aspect Ratio | 9:16 |
| Max Duration | 60 seconds |
| Video Codec | H.264 (AVC) |
| Profile | High |
| Level | 4.2 |
| Bitrate | 12 Mbps (target) |
| Max Bitrate | 14 Mbps |
| Buffer Size | 24 Mbps |
| Audio Codec | AAC |
| Audio Bitrate | 192 kbps |
| Sample Rate | 48 kHz |
| Container | MP4 |
| Pixel Format | yuv420p |
| Max File Size | 256 MB |
FFmpeg (CPU):
ffmpeg -y -i input.mp4 \
-c:v libx264 -preset slow \
-b:v 12M -maxrate 14M -bufsize 24M \
-profile:v high -level 4.2 \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 192k -ar 48000 \
-pix_fmt yuv420p -movflags +faststart \
output_yt.mp4FFmpeg (NVENC):
ffmpeg -y -i input.mp4 \
-c:v h264_nvenc -preset p5 -tune hq \
-b:v 12M -maxrate 14M -bufsize 24M \
-profile:v high -level 4.2 \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 192k -ar 48000 \
-pix_fmt yuv420p -movflags +faststart \
output_yt.mp4TikTok
| Property | Value |
|---|---|
| Resolution | 1080x1920 |
| Aspect Ratio | 9:16 |
| Max Duration | 60 seconds (3 min with account) |
| Video Codec | H.264 (AVC) |
| Bitrate Mode | CRF 18 (quality-based) |
| Max Bitrate | 10 Mbps |
| Buffer Size | 20 Mbps |
| Audio Codec | AAC |
| Audio Bitrate | 128 kbps |
| Sample Rate | 44.1 kHz |
| Container | MP4 |
| Pixel Format | yuv420p |
| Max File Size | 287 MB |
FFmpeg (CPU):
ffmpeg -y -i input.mp4 \
-c:v libx264 -preset slow -crf 18 \
-maxrate 10M -bufsize 20M \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 128k -ar 44100 \
-pix_fmt yuv420p -movflags +faststart \
output_tt.mp4FFmpeg (NVENC):
ffmpeg -y -i input.mp4 \
-c:v h264_nvenc -preset p5 -tune hq \
-cq 18 -maxrate 10M -bufsize 20M \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 128k -ar 44100 \
-pix_fmt yuv420p -movflags +faststart \
output_tt.mp4Instagram Reels
| Property | Value |
|---|---|
| Resolution | 1080x1920 |
| Aspect Ratio | 9:16 |
| Max Duration | 90 seconds |
| Video Codec | H.264 (AVC) |
| Profile | High |
| Level | 4.2 |
| Bitrate | 4.5 Mbps (target) |
| Max Bitrate | 5 Mbps |
| Buffer Size | 10 Mbps |
| Audio Codec | AAC |
| Audio Bitrate | 128 kbps |
| Sample Rate | 44.1 kHz |
| Container | MP4 |
| Pixel Format | yuv420p |
| Max File Size | 250 MB |
FFmpeg (CPU):
ffmpeg -y -i input.mp4 \
-c:v libx264 -preset slow \
-b:v 4500k -maxrate 5000k -bufsize 10M \
-profile:v high -level 4.2 \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 128k -ar 44100 \
-pix_fmt yuv420p -movflags +faststart \
output_ig.mp4Safe Zones
All platforms have UI overlays that consume screen space. Values below are the median of 10+ community-measured sources at 1080x1920 (no platform publishes official pixel specs). Bottom margins vary with caption/description length — values reflect typical organic content.
| Zone | TikTok | YouTube Shorts | Instagram Reels | Universal |
|---|---|---|---|---|
| Top | 150px | 150px | 210px | 210px |
| Bottom | 320px | 350px | 340px | 450px |
| Left | 60px | 60px | 40px | 60px |
| Right | 120px | 150px | 100px | 150px |
| Safe Area | 900x1450px | 870x1420px | 940x1370px | 870x1260px |
Caption position at 350px from bottom clears TikTok (320px) and Instagram (340px). For YouTube Shorts safety, 400px+ is recommended. For universal cross-platform safety, 450px+ is ideal.
Audio Normalization
All exports normalize audio to -14 LUFS (EBU R128) using FFmpeg's loudnorm filter:
I=-14— integrated loudness target (-14 LUFS, YouTube/streaming standard)TP=-1— true peak max (-1 dBFS headroom, prevents clipping)LRA=11— loudness range (dynamic variation allowed)
NVENC Encoding Notes
For NVIDIA GPUs with NVENC support (GTX 1650+, RTX series):
h264_nvenc -preset p5 -tune hqprovides best quality- 5-10x faster than CPU
-preset slow - Quality is comparable to CPU at same bitrate
-preset p7for maximum quality (slower but still faster than CPU)- Always verify NVENC availability:
ffmpeg -encoders | grep nvenc
Remotion Patterns and Best Practices
Key patterns used in claude-shorts, verified against Remotion v4.0.422.
Bundle-Once-Render-Many
For batch rendering multiple shorts, bundle the project once and reuse the Chrome instance:
import { bundle } from "@remotion/bundler";
import { renderMedia, selectComposition, openBrowser } from "@remotion/renderer";
// Bundle once
const serveUrl = await bundle({ entryPoint: "src/index.ts" });
// Open shared browser
const browser = await openBrowser("chrome");
try {
for (const segment of segments) {
// selectComposition resolves calculateMetadata for dynamic duration
const composition = await selectComposition({
serveUrl,
id: "ShortVideo",
inputProps: segment.props,
puppeteerInstance: browser,
});
await renderMedia({
composition,
serveUrl,
codec: "h264",
crf: 18,
outputLocation: segment.outputPath,
inputProps: segment.props,
puppeteerInstance: browser,
});
}
} finally {
await browser.close();
}Critical: Always use selectComposition() before renderMedia(). Never pass inline composition objects. selectComposition() evaluates calculateMetadata to resolve dynamic duration per segment.
OffthreadVideo + CSS Reframing
Reframing is done via CSS transforms on OffthreadVideo, not FFmpeg pre-processing:
const scale = outputWidth / crop.w;
const translateX = -crop.x * scale;
const translateY = -crop.y * scale;
<div style={{ width: 1080, height: 1920, overflow: "hidden" }}>
<OffthreadVideo
src={clipPath}
style={{
width: sourceWidth * scale,
height: sourceHeight * scale,
transform: `translate(${translateX}px, ${translateY}px)`,
}}
/>
</div>This avoids double-encoding. Remotion handles the crop during render.
Spring Animations
Spring configs for each caption style:
// Bold: Snappy pop-in, slight overshoot
spring({ frame, fps, config: { mass: 1, damping: 12, stiffness: 200 } });
// ~0.4s settle, overshoots to ~1.05
// Bounce: Very bouncy, visible overshoot
spring({ frame, fps, config: { mass: 1, damping: 8, stiffness: 180 } });
// ~0.6s settle, overshoots to ~1.2
// Clean style uses interpolate(), not spring
interpolate(localFrame, [0, 6], [0, 1], { extrapolateRight: "clamp" });Spring values guide:
damping < 10: Visible bouncedamping 10-15: Slight overshoot, professional feeldamping > 20: No overshoot, smooth easestiffness > 150: Fast, snappystiffness < 100: Slow, gentle
@remotion/captions API
import { createTikTokStyleCaptions } from "@remotion/captions";
// Input captions format
type Caption = {
text: string;
startMs: number;
endMs: number;
timestampMs: null; // required field, always null for our use
confidence: null; // required field, always null for our use
};
// Create pages (groups of words)
const { pages } = createTikTokStyleCaptions({
captions: remotionCaptions,
combineTokensWithinMilliseconds: 800, // adjust per style
});
// Page structure
type TikTokPage = {
text: string; // Combined text of all tokens
startMs: number; // Start time of first token
durationMs: number; // Duration until end of last token
tokens: Array<{
text: string;
fromMs: number;
toMs: number;
}>;
};calculateMetadata for Dynamic Duration
Each segment has a different duration. Use calculateMetadata in the Composition:
<Composition
id="ShortVideo"
component={ShortVideo}
width={1080}
height={1920}
fps={30}
durationInFrames={30 * 30} // default, overridden by calculateMetadata
schema={ShortVideoPropsSchema}
defaultProps={defaultProps}
calculateMetadata={({ props }) => ({
durationInFrames: Math.ceil(props.durationInSeconds * 30),
fps: 30,
width: 1080,
height: 1920,
})}
/>Zod Schema Validation
Always define inputProps with Zod schemas for type safety:
import { z } from "zod";
const ShortVideoPropsSchema = z.object({
clipSrc: z.string(),
sourceWidth: z.number(),
sourceHeight: z.number(),
crop: CropSchema,
captions: z.array(CaptionSchema),
captionStyle: z.enum(["bold", "bounce", "clean"]),
hookLine1: z.string().optional().default(""),
hookLine2: z.string().optional().default(""),
showProgressBar: z.boolean().optional().default(true),
durationInSeconds: z.number(),
});File Paths in Render
When rendering headless, clipSrc must be an http:// URL. Remotion's renderer proxy does not support file:// URLs. The render script starts a local HTTP server on a random port (e.g., http://127.0.0.1:PORT/clip_01.mp4) to serve clip files. Do NOT use staticFile() for dynamic clip sources — that's for bundled assets only.
Performance Tips
- Use
OffthreadVideo(notVideo) — decodes frames in a separate thread - Pre-extract segments before rendering (don't seek through long files)
- Use
codec: "h264"withcrf: 18for good quality-size balance - Share browser instance across renders (
openBrowser+puppeteerInstance) - Bundle once, render many
Segment Scoring Rubric
When analyzing a transcript for shortform candidates, score each segment on these 5 dimensions. Each dimension is scored 0-100, then weighted to produce a final score.
Dimensions
1. Hook Strength (Weight: 0.30)
The first 3 seconds determine whether a viewer scrolls past or stays. Score based on which hook archetype is present:
| Archetype | Example | Score Range |
|---|---|---|
| Bold/Contrarian | "Everything you know about X is wrong" | 80-100 |
| Curiosity Gap | "There's one thing nobody tells you about..." | 75-95 |
| Value Promise | "Here's the exact framework I used to..." | 70-90 |
| Pattern Interrupt | "Wait, let me show you something weird" | 70-90 |
| Payoff Preview | "By the end of this you'll know how to..." | 65-85 |
| Mid-Action Start | [Starts mid-sentence with energy] | 60-80 |
| Hidden Knowledge | "The secret that [authority figures] don't share" | 60-80 |
| Weak/Generic | "So today I want to talk about..." | 10-40 |
Boosters (+5-10 each):
- Contains a specific number ("3 steps", "$50K", "in 30 days")
- Names a recognizable entity (person, company, tool)
- Implies personal experience ("I tested", "I spent 6 months")
2. Standalone Coherence (Weight: 0.25)
The segment must make complete sense to someone who hasn't seen the rest of the video.
| Criteria | Score |
|---|---|
| Complete self-contained narrative arc (setup → development → resolution) | 85-100 |
| Complete idea with minor context gaps (viewer can infer) | 65-84 |
| Mostly standalone but references earlier content ("as I said before") | 40-64 |
| Requires prior context to understand ("so going back to that point") | 10-39 |
| Fragment — starts or ends mid-thought | 0-9 |
Red flags (automatic low score):
- "As I mentioned earlier..."
- "Going back to what we discussed..."
- Pronouns without clear referents ("he said that...")
- Cuts off mid-sentence at the end
3. Emotional Intensity (Weight: 0.20)
Strong emotions drive shares and comments.
| Signal | Score Range |
|---|---|
| Passionate rant / strong opinion with conviction | 80-100 |
| Surprise reveal / unexpected twist | 75-95 |
| Genuine humor / laughter | 70-90 |
| Personal vulnerability / honest failure story | 70-90 |
| Enthusiastic explanation of something fascinating | 60-80 |
| Calm but insightful observation | 40-60 |
| Monotone recitation of facts | 10-30 |
4. Value Density (Weight: 0.15)
How much actionable content is packed per second.
| Content Type | Score Range |
|---|---|
| Step-by-step process / exact method | 80-100 |
| Framework / mental model with examples | 75-95 |
| Specific data points / research findings | 70-90 |
| Counter-intuitive insight with explanation | 65-85 |
| General advice with some specifics | 40-60 |
| Vague platitudes ("work harder", "be consistent") | 10-30 |
Duration adjustment: Penalize segments where >30% of time is filler, repetition, or tangents.
5. Payoff Quality (Weight: 0.10)
How satisfying the ending feels.
| Ending Type | Score Range |
|---|---|
| Punchline / satisfying reveal | 85-100 |
| Clear call-to-action with specific next step | 75-90 |
| Complete thought — natural stopping point | 65-80 |
| Fades into next topic (can be cut cleanly) | 40-60 |
| Cuts off mid-thought / no resolution | 10-30 |
Scoring Formula
final_score = (hook * 0.30) + (coherence * 0.25) + (emotion * 0.20) + (value * 0.15) + (payoff * 0.10)Candidate Selection Guidelines
1. Target 8-12 candidates from a typical 30-60 minute video 2. Duration sweet spot: 15-55 seconds (peak engagement at 25-40s) 3. Minimum score threshold: 60 (below this, skip the segment) 4. Diversity: Don't select 5 segments about the same subtopic 5. Spacing: Prefer segments at least 2 minutes apart in source 6. Natural boundaries: Align start/end with sentence boundaries, not mid-word
Hook Text Generation
For each selected segment, write a hook overlay:
- Line 1: 4-8 words, the attention-grabbing statement (white, large)
- Line 2: 3-6 words, context or subtitle (cyan, smaller)
- Lines should NOT duplicate the first spoken words — they complement the audio
{
"name": "claude-shorts-remotion",
"version": "1.0.0",
"private": true,
"scripts": {
"preview": "remotion preview src/index.ts",
"render": "node render.mjs"
},
"dependencies": {
"@remotion/bundler": "^4.0.422",
"@remotion/captions": "^4.0.422",
"@remotion/cli": "^4.0.422",
"@remotion/renderer": "^4.0.422",
"react": "^19.0.0",
"react-dom": "^19.0.0",
"remotion": "^4.0.422",
"zod": "^3.23.0"
},
"devDependencies": {
"@types/react": "^19.0.0",
"typescript": "^5.5.0"
}
}
Copyright 2011 The Montserrat Project Authors (https://github.com/JulietaUla/Montserrat)
Copyright 2010 The Bangers Project Authors (https://github.com/googlefonts/bangers)
Copyright 2016 The Inter Project Authors (https://github.com/rsms/inter)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
import { Config } from "@remotion/cli/config";
Config.setVideoImageFormat("jpeg");
Config.setOverwriteOutput(true);
#!/usr/bin/env node
/**
* Bundle-once-render-many orchestrator for claude-shorts.
*
* Bundles the Remotion project once, opens a shared Chrome instance,
* then renders each approved segment sequentially.
*
* Usage:
* node render.mjs \
* --segments /tmp/claude-shorts/approved_segments.json \
* --reframe /tmp/claude-shorts/reframe.json \
* --captions /tmp/claude-shorts/transcript.json \
* --style bold \
* --clips-dir /tmp/claude-shorts/clips/ \
* --output-dir /tmp/claude-shorts/render/
*
* IMPORTANT: Uses selectComposition() before renderMedia() to resolve
* calculateMetadata (dynamic duration per segment).
*/
import { bundle } from "@remotion/bundler";
import { renderMedia, selectComposition, openBrowser } from "@remotion/renderer";
import path from "path";
import fs from "fs";
import http from "http";
import { fileURLToPath } from "url";
const __dirname = path.dirname(fileURLToPath(import.meta.url));
function parseArgs() {
const args = process.argv.slice(2);
const parsed = {};
for (let i = 0; i < args.length; i += 2) {
const key = args[i].replace(/^--/, "");
parsed[key] = args[i + 1];
}
return parsed;
}
async function main() {
const args = parseArgs();
const segmentsPath = args.segments;
const reframePath = args.reframe;
const captionsPath = args.captions;
const style = args.style || "bold";
const clipsDir = args["clips-dir"];
const outputDir = args["output-dir"] || "/tmp/claude-shorts/render/";
if (!segmentsPath || !reframePath || !captionsPath || !clipsDir) {
console.error(JSON.stringify({
error: "Missing required arguments",
usage: "node render.mjs --segments FILE --reframe FILE --captions FILE --clips-dir DIR [--style bold|bounce|clean] [--output-dir DIR]",
}));
process.exit(1);
}
// Load data files
const segmentsData = JSON.parse(fs.readFileSync(segmentsPath, "utf-8"));
const reframeData = JSON.parse(fs.readFileSync(reframePath, "utf-8"));
const transcriptData = JSON.parse(fs.readFileSync(captionsPath, "utf-8"));
const segments = segmentsData.segments;
const allCaptions = transcriptData.captions;
// Ensure output directory exists
fs.mkdirSync(outputDir, { recursive: true });
// Start a local HTTP server to serve clip files.
// Remotion's renderer proxy only supports http/https URLs, not file://.
const clipServer = http.createServer((req, res) => {
const filePath = path.join(clipsDir, decodeURIComponent(req.url.slice(1)));
if (!fs.existsSync(filePath)) {
res.writeHead(404);
res.end("Not found");
return;
}
res.writeHead(200, { "Content-Type": "video/mp4" });
fs.createReadStream(filePath).pipe(res);
});
await new Promise((resolve) => clipServer.listen(0, "127.0.0.1", resolve));
const clipServerPort = clipServer.address().port;
const clipBaseUrl = `http://127.0.0.1:${clipServerPort}`;
console.log(JSON.stringify({ action: "clip_server", port: clipServerPort, url: clipBaseUrl }));
console.log(JSON.stringify({ action: "bundle", status: "starting" }));
const startBundle = Date.now();
// Bundle once
const serveUrl = await bundle({
entryPoint: path.resolve(__dirname, "src/index.ts"),
});
const bundleTime = ((Date.now() - startBundle) / 1000).toFixed(1);
console.log(JSON.stringify({ action: "bundle", status: "complete", time_sec: bundleTime }));
// Open shared browser instance
const browser = await openBrowser("chrome");
const results = [];
try {
for (let i = 0; i < segments.length; i++) {
const seg = segments[i];
const segId = seg.id ?? (i + 1);
const clipName = `clip_${String(segId).padStart(2, "0")}.mp4`;
const clipPath = path.resolve(clipsDir, clipName);
const outputPath = path.resolve(outputDir, `short_${String(segId).padStart(2, "0")}.mp4`);
if (!fs.existsSync(clipPath)) {
console.error(JSON.stringify({ error: `Clip not found: ${clipPath}` }));
continue;
}
// Get reframe data for this clip
const reframe = reframeData.clips?.[clipName] || {};
if (!reframe.crop) {
console.error(JSON.stringify({
warning: `No reframe data for ${clipName}, using default 9:16 center crop for 1920x1080`,
}));
}
const crop = reframe.crop || { x: 0, y: 0, w: 607, h: 1080 };
const cropKeyframes = reframe.crop_keyframes || [];
// Get source resolution from reframe data
const [srcW, srcH] = (reframe.source_resolution || "1920x1080")
.split("x")
.map(Number);
// Extract captions for this segment's time range (in milliseconds)
const segStartMs = seg.start * 1000;
const segEndMs = seg.end * 1000;
const segCaptions = allCaptions
.filter((c) => c.startMs >= segStartMs && c.endMs <= segEndMs)
.map((c) => ({
text: c.text,
startMs: c.startMs - segStartMs, // Offset to clip-local time
endMs: c.endMs - segStartMs,
}));
const durationInSeconds = seg.end - seg.start;
const inputProps = {
clipSrc: `${clipBaseUrl}/${clipName}`,
sourceWidth: srcW,
sourceHeight: srcH,
crop,
cropKeyframes,
captions: segCaptions,
captionStyle: style,
hookLine1: seg.hook_line1 || "",
hookLine2: seg.hook_line2 || "",
showProgressBar: true,
durationInSeconds,
};
console.log(JSON.stringify({
action: "render",
segment: i + 1,
total: segments.length,
duration: `${durationInSeconds.toFixed(1)}s`,
status: "starting",
}));
// Retry once on failure before skipping this segment
let rendered = false;
const maxAttempts = 2;
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
try {
const startRender = Date.now();
// selectComposition resolves calculateMetadata for dynamic duration
const composition = await selectComposition({
serveUrl,
id: "ShortVideo",
inputProps,
puppeteerInstance: browser,
});
await renderMedia({
composition,
serveUrl,
codec: "h264",
crf: 18,
outputLocation: outputPath,
inputProps,
puppeteerInstance: browser,
});
const renderTime = ((Date.now() - startRender) / 1000).toFixed(1);
const fileSizeMB = (fs.statSync(outputPath).size / 1024 / 1024).toFixed(1);
const result = {
action: "render",
segment: i + 1,
output: outputPath,
duration: `${durationInSeconds.toFixed(1)}s`,
render_time_sec: renderTime,
file_size_mb: fileSizeMB,
status: "complete",
};
results.push(result);
console.log(JSON.stringify(result));
rendered = true;
break;
} catch (renderErr) {
if (attempt < maxAttempts) {
console.log(JSON.stringify({
action: "render",
segment: i + 1,
status: "retrying",
attempt,
error: renderErr.message,
}));
} else {
console.error(JSON.stringify({
action: "render",
segment: i + 1,
status: "failed",
error: renderErr.message,
}));
}
}
}
if (!rendered) {
results.push({
action: "render",
segment: i + 1,
output: outputPath,
status: "failed",
});
}
}
} finally {
await browser.close({ silent: true });
clipServer.close();
}
// Final summary
console.log(JSON.stringify({
action: "render_complete",
segments_rendered: results.length,
total_segments: segments.length,
output_dir: outputDir,
results,
}));
}
main().catch((err) => {
console.error(JSON.stringify({ error: err.message, stack: err.stack }));
process.exit(1);
});
import { useCurrentFrame, useVideoConfig, spring } from "remotion";
import { useCaptionPages } from "../hooks/useCaptionPages";
import { BOLD_THEME } from "../styles/theme";
import type { Caption } from "../types";
interface BoldCaptionsProps {
captions: Caption[];
}
/**
* Bold-style captions: ALL CAPS, pop-in spring animation,
* yellow highlight on the currently active word.
*
* Font: Montserrat Bold
* Words per page: 2-3 (800ms combine window)
* Animation: Pop-in spring {mass:1, damping:12, stiffness:200}
*/
export const BoldCaptions: React.FC<BoldCaptionsProps> = ({
captions,
}) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentTimeMs = (frame / fps) * 1000;
const pages = useCaptionPages(captions, 800);
// Find the current page
const currentPage = pages.find(
(p) =>
currentTimeMs >= p.startMs &&
currentTimeMs < p.startMs + p.durationMs
);
if (!currentPage) return null;
const pageStartFrame = Math.floor((currentPage.startMs / 1000) * fps);
const localFrame = frame - pageStartFrame;
// Pop-in spring animation
const scale = spring({
frame: localFrame,
fps,
config: { mass: 1, damping: 12, stiffness: 200 },
});
return (
<div
style={{
position: "absolute",
bottom: 350,
left: 40,
right: 40,
display: "flex",
justifyContent: "center",
alignItems: "center",
flexWrap: "wrap",
gap: 12,
transform: `scale(${scale})`,
}}
>
{currentPage.tokens.map((token, i) => {
const isActive =
currentTimeMs >= token.fromMs && currentTimeMs < token.toMs;
return (
<span
key={`${currentPage.startMs}-${i}`}
style={{
fontFamily: "'Montserrat', sans-serif",
fontWeight: 800,
fontSize: 72,
textTransform: "uppercase",
color: isActive
? BOLD_THEME.activeColor
: BOLD_THEME.textColor,
textShadow: `
3px 3px 0 ${BOLD_THEME.shadowColor},
-3px -3px 0 ${BOLD_THEME.shadowColor},
3px -3px 0 ${BOLD_THEME.shadowColor},
-3px 3px 0 ${BOLD_THEME.shadowColor}
`,
lineHeight: 1.1,
textAlign: "center",
}}
>
{token.text.trim().toUpperCase()}
</span>
);
})}
</div>
);
};
import { useCurrentFrame, useVideoConfig, spring } from "remotion";
import { useCaptionPages } from "../hooks/useCaptionPages";
import { BOUNCE_THEME } from "../styles/theme";
import type { Caption } from "../types";
interface BounceCaptionsProps {
captions: Caption[];
}
/**
* Bounce-style captions: Bouncy scale spring, rotating bright colors,
* 1-2 words per page for maximum impact.
*
* Font: Bangers (Google Fonts, OFL)
* Words per page: 1-2 (600ms combine window)
* Animation: Scale spring 70% → 120% → 100% {mass:1, damping:8}
*/
export const BounceCaptions: React.FC<BounceCaptionsProps> = ({
captions,
}) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentTimeMs = (frame / fps) * 1000;
const pages = useCaptionPages(captions, 600);
const currentPage = pages.find(
(p) =>
currentTimeMs >= p.startMs &&
currentTimeMs < p.startMs + p.durationMs
);
if (!currentPage) return null;
const pageStartFrame = Math.floor((currentPage.startMs / 1000) * fps);
const localFrame = frame - pageStartFrame;
const pageIndex = pages.indexOf(currentPage);
// Bouncy scale spring: starts at 0.7, overshoots to ~1.2, settles at 1.0
const rawScale = spring({
frame: localFrame,
fps,
config: { mass: 1, damping: 8, stiffness: 180 },
});
const scale = 0.7 + rawScale * 0.3;
// Rotating color per page
const color = BOUNCE_THEME.rotatingColors[pageIndex % BOUNCE_THEME.rotatingColors.length];
return (
<div
style={{
position: "absolute",
bottom: 350,
left: 40,
right: 40,
display: "flex",
justifyContent: "center",
alignItems: "center",
transform: `scale(${scale})`,
}}
>
<span
style={{
fontFamily: "'Bangers', cursive",
fontSize: 84,
color,
textShadow: `
4px 4px 0 ${BOUNCE_THEME.shadowColor},
-4px -4px 0 ${BOUNCE_THEME.shadowColor},
4px -4px 0 ${BOUNCE_THEME.shadowColor},
-4px 4px 0 ${BOUNCE_THEME.shadowColor}
`,
textAlign: "center",
lineHeight: 1.0,
textTransform: "uppercase",
}}
>
{currentPage.text.trim().toUpperCase()}
</span>
</div>
);
};
import type { Caption, CaptionStyleType } from "../types";
import { BoldCaptions } from "./BoldCaptions";
import { BounceCaptions } from "./BounceCaptions";
import { CleanCaptions } from "./CleanCaptions";
interface CaptionsProps {
captions: Caption[];
style: CaptionStyleType;
}
/**
* Style dispatcher — routes to the correct caption renderer.
*/
export const Captions: React.FC<CaptionsProps> = ({ captions, style }) => {
if (!captions || captions.length === 0) return null;
switch (style) {
case "bold":
return <BoldCaptions captions={captions} />;
case "bounce":
return <BounceCaptions captions={captions} />;
case "clean":
return <CleanCaptions captions={captions} />;
default:
return <BoldCaptions captions={captions} />;
}
};
import { useCurrentFrame, useVideoConfig, interpolate } from "remotion";
import { useCaptionPages } from "../hooks/useCaptionPages";
import { CLEAN_THEME } from "../styles/theme";
import type { Caption } from "../types";
interface CleanCaptionsProps {
captions: Caption[];
}
/**
* Clean-style captions: Minimal fade-in, white text with subtle shadow,
* 3-5 words per page for readability.
*
* Font: Inter Bold
* Words per page: 3-5 (1500ms combine window)
* Animation: Fade-in opacity 0 → 1 over 6 frames
*/
export const CleanCaptions: React.FC<CleanCaptionsProps> = ({ captions }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentTimeMs = (frame / fps) * 1000;
const pages = useCaptionPages(captions, 1500);
const currentPage = pages.find(
(p) =>
currentTimeMs >= p.startMs &&
currentTimeMs < p.startMs + p.durationMs
);
if (!currentPage) return null;
const pageStartFrame = Math.floor((currentPage.startMs / 1000) * fps);
const localFrame = frame - pageStartFrame;
// Fade-in over 6 frames
const opacity = interpolate(localFrame, [0, 6], [0, 1], {
extrapolateRight: "clamp",
});
return (
<div
style={{
position: "absolute",
bottom: 350,
left: 60,
right: 60,
display: "flex",
justifyContent: "center",
alignItems: "center",
opacity,
}}
>
{currentPage.tokens.map((token, i) => {
const isActive =
currentTimeMs >= token.fromMs && currentTimeMs < token.toMs;
return (
<span
key={`${currentPage.startMs}-${i}`}
style={{
fontFamily: "'Inter', sans-serif",
fontWeight: 700,
fontSize: 56,
color: isActive
? CLEAN_THEME.activeColor
: CLEAN_THEME.textColor,
textShadow: `2px 2px 8px ${CLEAN_THEME.shadowColor}`,
lineHeight: 1.3,
textAlign: "center",
marginRight: 8,
}}
>
{token.text.trim()}
</span>
);
})}
</div>
);
};
import { useCurrentFrame, useVideoConfig, spring, interpolate } from "remotion";
interface HookOverlayProps {
line1: string;
line2: string;
}
/**
* Hook text overlay shown in the first 3.5 seconds.
* Line 1: Large white text (main hook)
* Line 2: Smaller cyan text (subtitle/context)
*
* Animation: Spring pop-in at 0.3s, fade-out at 3.0s
*/
export const HookOverlay: React.FC<HookOverlayProps> = ({ line1, line2 }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentTimeSec = frame / fps;
// Only show between 0.3s and 3.5s
if (currentTimeSec < 0.3 || currentTimeSec > 3.5) return null;
const enterFrame = Math.max(0, frame - Math.floor(0.3 * fps));
// Pop-in spring
const scale = spring({
frame: enterFrame,
fps,
config: { mass: 1, damping: 14, stiffness: 200 },
});
// Fade-out starting at 3.0s
const opacity = interpolate(
currentTimeSec,
[3.0, 3.5],
[1, 0],
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
);
return (
<div
style={{
position: "absolute",
top: 40,
left: 40,
right: 40,
display: "flex",
flexDirection: "column",
alignItems: "center",
gap: 8,
opacity,
transform: `scale(${scale})`,
}}
>
{line1 && (
<div
style={{
fontFamily: "'Montserrat', sans-serif",
fontWeight: 800,
fontSize: 48,
color: "white",
textShadow: "3px 3px 0 black, -3px -3px 0 black, 3px -3px 0 black, -3px 3px 0 black",
textAlign: "center",
lineHeight: 1.2,
}}
>
{line1}
</div>
)}
{line2 && (
<div
style={{
fontFamily: "'Inter', sans-serif",
fontWeight: 600,
fontSize: 28,
color: "#00BFFF",
textShadow: "2px 2px 0 black",
textAlign: "center",
lineHeight: 1.3,
}}
>
{line2}
</div>
)}
</div>
);
};
import { useCurrentFrame, useVideoConfig } from "remotion";
interface ProgressBarProps {
durationInSeconds: number;
}
/**
* Thin progress bar at the bottom of the video.
* White bar that fills left-to-right over the video duration.
*/
export const ProgressBar: React.FC<ProgressBarProps> = ({
durationInSeconds,
}) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const totalFrames = Math.ceil(durationInSeconds * fps);
const progress = Math.min(frame / totalFrames, 1);
return (
<div
style={{
position: "absolute",
bottom: 0,
left: 0,
width: 1080,
height: 4,
backgroundColor: "rgba(255, 255, 255, 0.2)",
}}
>
<div
style={{
width: 1080 * progress,
height: 4,
backgroundColor: "white",
}}
/>
</div>
);
};
import { OffthreadVideo, useCurrentFrame, useVideoConfig, interpolate } from "remotion";
import type { Crop, CropKeyframe } from "../types";
interface VideoFrameProps {
clipSrc: string;
sourceWidth: number;
sourceHeight: number;
crop: Crop;
cropKeyframes?: CropKeyframe[];
}
/**
* Renders the reframed video using OffthreadVideo + CSS transforms.
*
* Supports two modes:
* 1. Static crop: uses crop.x directly (no keyframes)
* 2. Animated pan: interpolates crop.x from keyframes (cursor tracking)
*
* Strategy: The source video is scaled and positioned so that only the
* cropped region is visible within the 1080x1920 container. Uses
* overflow:hidden to clip, and CSS transform to position the crop area.
*/
export const VideoFrame: React.FC<VideoFrameProps> = ({
clipSrc,
sourceWidth,
sourceHeight,
crop,
cropKeyframes,
}) => {
if (!clipSrc) return null;
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const currentTime = frame / fps;
// Scale factor: how much to scale source so crop.w fills 1080px
const outputWidth = 1080;
const outputHeight = 1920;
const scale = outputWidth / crop.w;
// Determine current crop X (animated or static)
let cropX = crop.x;
if (cropKeyframes && cropKeyframes.length >= 2) {
const times = cropKeyframes.map((k) => k.t);
const xPositions = cropKeyframes.map((k) => k.x);
cropX = interpolate(currentTime, times, xPositions, {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
}
// Scaled source dimensions
const scaledWidth = sourceWidth * scale;
const scaledHeight = sourceHeight * scale;
// Offset to position the crop region at (0,0)
const translateX = -cropX * scale;
const translateY = -crop.y * scale;
// Auto-detect framed layout: video doesn't fill full output height.
// Position in content zone (below 200px hook text area) for screen recordings.
const hookZone = 200;
const topOffset = scaledHeight < outputHeight ? hookZone : 0;
return (
<div
style={{
position: "absolute",
top: topOffset,
left: 0,
width: outputWidth,
height: scaledHeight < outputHeight ? scaledHeight : outputHeight,
overflow: "hidden",
}}
>
<OffthreadVideo
src={clipSrc}
style={{
position: "absolute",
width: scaledWidth,
height: scaledHeight,
transform: `translate(${translateX}px, ${translateY}px)`,
}}
/>
</div>
);
};
import { useMemo } from "react";
import { createTikTokStyleCaptions } from "@remotion/captions";
import type { Caption } from "../types";
/**
* Converts word-level captions into TikTok-style pages using
* @remotion/captions createTikTokStyleCaptions().
*
* Each page groups N words together based on combineTokensWithinMilliseconds.
* Returns pages with: text, startMs, durationMs, tokens[{text, fromMs, toMs}]
*/
export const useCaptionPages = (
captions: Caption[],
combineMs: number = 800
) => {
return useMemo(() => {
if (!captions || captions.length === 0) return [];
// Convert our Caption format to Remotion's expected format
const remotionCaptions = captions.map((c) => ({
text: c.text,
startMs: c.startMs,
endMs: c.endMs,
timestampMs: null as number | null,
confidence: null as number | null,
}));
const { pages } = createTikTokStyleCaptions({
captions: remotionCaptions,
combineTokensWithinMilliseconds: combineMs,
});
return pages;
}, [captions, combineMs]);
};
import { registerRoot } from "remotion";
import { Root } from "./Root";
registerRoot(Root);
import { Composition } from "remotion";
import { ShortVideo } from "./ShortVideo";
import { ShortVideoPropsSchema } from "./types";
export const Root: React.FC = () => {
return (
<>
<Composition
id="ShortVideo"
component={ShortVideo}
width={1080}
height={1920}
fps={30}
durationInFrames={30 * 30}
schema={ShortVideoPropsSchema}
defaultProps={{
clipSrc: "",
sourceWidth: 1920,
sourceHeight: 1080,
crop: { x: 0, y: 0, w: 607, h: 1080 },
cropKeyframes: [],
captions: [],
captionStyle: "bold" as const,
hookLine1: "",
hookLine2: "",
showProgressBar: true,
durationInSeconds: 30,
}}
calculateMetadata={({ props }) => {
return {
durationInFrames: Math.ceil(props.durationInSeconds * 30),
fps: 30,
width: 1080,
height: 1920,
};
}}
/>
</>
);
};
import { AbsoluteFill } from "remotion";
import { VideoFrame } from "./components/VideoFrame";
import { Captions } from "./components/Captions";
import { HookOverlay } from "./components/HookOverlay";
import { ProgressBar } from "./components/ProgressBar";
import { fontFaceCSS } from "./styles/fonts";
import type { ShortVideoProps } from "./types";
export const ShortVideo: React.FC<ShortVideoProps> = ({
clipSrc,
sourceWidth,
sourceHeight,
crop,
cropKeyframes,
captions,
captionStyle,
hookLine1,
hookLine2,
showProgressBar,
durationInSeconds,
}) => {
return (
<AbsoluteFill style={{ backgroundColor: "black" }}>
{/* Load custom fonts for captions and hook overlay */}
<style dangerouslySetInnerHTML={{ __html: fontFaceCSS }} />
{/* Reframed video (cropped and scaled to fill 1080x1920) */}
<VideoFrame
clipSrc={clipSrc}
sourceWidth={sourceWidth}
sourceHeight={sourceHeight}
crop={crop}
cropKeyframes={cropKeyframes}
/>
{/* Animated captions */}
<Captions captions={captions} style={captionStyle} />
{/* Hook text overlay (first 3.5 seconds) */}
{(hookLine1 || hookLine2) && (
<HookOverlay line1={hookLine1 ?? ""} line2={hookLine2 ?? ""} />
)}
{/* Progress bar at bottom */}
{showProgressBar && (
<ProgressBar durationInSeconds={durationInSeconds} />
)}
</AbsoluteFill>
);
};
import { staticFile } from "remotion";
/**
* Font declarations for caption styles.
*
* Fonts are loaded from public/fonts/ directory:
* - Montserrat Bold (Bold style) — Google Fonts, OFL
* - Bangers (Bounce style) — Google Fonts, OFL
* - Inter Bold (Clean style) — Google Fonts, OFL
*
* For preview mode, fonts are loaded via @font-face in the browser.
* For rendering, Remotion handles font loading automatically.
*/
export const FONTS = {
montserrat: {
family: "Montserrat",
src: staticFile("fonts/Montserrat-Bold.ttf"),
weight: "800",
},
bangers: {
family: "Bangers",
src: staticFile("fonts/Bangers-Regular.ttf"),
weight: "400",
},
inter: {
family: "Inter",
src: staticFile("fonts/Inter-Bold.ttf"),
weight: "700",
},
} as const;
/**
* CSS @font-face declarations for all caption fonts.
* Inject into document head for preview mode.
*/
export const fontFaceCSS = Object.values(FONTS)
.map(
(f) => `
@font-face {
font-family: '${f.family}';
src: url('${f.src}') format('truetype');
font-weight: ${f.weight};
font-display: block;
}
`
)
.join("\n");
/**
* Color palettes and style constants for each caption preset.
*/
export const BOLD_THEME = {
textColor: "#FFFFFF",
activeColor: "#FFD700", // Yellow highlight on active word
shadowColor: "#000000",
backgroundColor: "transparent",
};
export const BOUNCE_THEME = {
textColor: "#FFFFFF",
shadowColor: "#000000",
backgroundColor: "transparent",
// Rotating bright colors per page
rotatingColors: [
"#00FFFF", // Cyan
"#FF00FF", // Magenta
"#00FF00", // Lime
"#FFFF00", // Yellow
"#FF6600", // Orange
"#FF0066", // Hot pink
],
};
export const CLEAN_THEME = {
textColor: "#FFFFFF",
activeColor: "#E0E0E0", // Subtle lighter white for active word
shadowColor: "rgba(0, 0, 0, 0.6)",
backgroundColor: "transparent",
};
import { z } from "zod";
export const CaptionSchema = z.object({
text: z.string(),
startMs: z.number(),
endMs: z.number(),
});
export type Caption = z.infer<typeof CaptionSchema>;
export const CropSchema = z.object({
x: z.number(),
y: z.number(),
w: z.number(),
h: z.number(),
});
export type Crop = z.infer<typeof CropSchema>;
export const CropKeyframeSchema = z.object({
t: z.number(),
x: z.number(),
});
export type CropKeyframe = z.infer<typeof CropKeyframeSchema>;
export const CaptionStyle = z.enum(["bold", "bounce", "clean"]);
export type CaptionStyleType = z.infer<typeof CaptionStyle>;
export const ShortVideoPropsSchema = z.object({
clipSrc: z.string(),
sourceWidth: z.number(),
sourceHeight: z.number(),
crop: CropSchema,
cropKeyframes: z.array(CropKeyframeSchema).optional().default([]),
captions: z.array(CaptionSchema),
captionStyle: CaptionStyle,
hookLine1: z.string().optional().default(""),
hookLine2: z.string().optional().default(""),
showProgressBar: z.boolean().optional().default(true),
durationInSeconds: z.number(),
});
export type ShortVideoProps = z.infer<typeof ShortVideoPropsSchema>;
{
"compilerOptions": {
"target": "ES2022",
"module": "ES2022",
"moduleResolution": "bundler",
"jsx": "react-jsx",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"outDir": "dist",
"baseUrl": ".",
"paths": {
"@/*": ["src/*"]
}
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
faster-whisper>=1.0.0
mediapipe>=0.10.0
numpy>=1.24.0
opencv-python>=4.8.0
#!/usr/bin/env python3
"""Compute reframe coordinates for vertical (9:16) video from landscape clips.
Supports three reframing strategies based on content type:
- talking-head: Face-tracked center crop (follows the speaker's face)
- screen: Cursor-tracked pan with moderate zoom (follows mouse cursor)
- podcast: Center crop or dominant-speaker tracking
Usage:
python3 compute_reframe.py --clips-dir /tmp/claude-shorts/clips/ \
--content-type screen --output reframe.json
Output (reframe.json):
{
"clips": {
"clip_01.mp4": {
"strategy": "cursor-track",
"source_resolution": "3840x2160",
"crop": {"x": 800, "y": 0, "w": 2112, "h": 2160},
"crop_keyframes": [
{"t": 0.0, "x": 800},
{"t": 1.0, "x": 950}
],
"output_resolution": "1080x1920"
}
}
}
"""
import argparse
import glob
import json
import os
import subprocess
import sys
import time
def get_video_info(path):
"""Get video width, height, fps, and duration."""
result = subprocess.run(
["ffprobe", "-v", "quiet", "-print_format", "json",
"-show_streams", "-show_format", "-select_streams", "v:0", path],
capture_output=True, text=True
)
data = json.loads(result.stdout)
stream = data["streams"][0]
fmt = data.get("format", {})
w = int(stream["width"])
h = int(stream["height"])
duration = float(fmt.get("duration", stream.get("duration", 0)))
fps_str = stream.get("r_frame_rate", "30/1")
num, den = fps_str.split("/")
fps = round(float(num) / float(den))
return w, h, fps, duration
def compute_face_track(clip_path, src_w, src_h, num_samples=5):
"""Track face positions across sampled frames using MediaPipe."""
import cv2
import mediapipe as mp
import tempfile
_, _, _, duration = get_video_info(clip_path)
mp_face = mp.solutions.face_detection
detector = mp_face.FaceDetection(model_selection=1, min_detection_confidence=0.5)
face_positions = []
timestamps = [duration * (i + 0.5) / num_samples for i in range(num_samples)]
tmpdir = tempfile.mkdtemp()
for ts in timestamps:
frame_path = os.path.join(tmpdir, f"f_{ts:.2f}.jpg")
subprocess.run(
["ffmpeg", "-y", "-ss", str(ts), "-i", clip_path,
"-vframes", "1", "-q:v", "2", frame_path],
capture_output=True
)
if not os.path.exists(frame_path):
continue
img = cv2.imread(frame_path)
if img is None:
continue
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
results = detector.process(rgb)
if results.detections:
best = max(results.detections,
key=lambda d: d.location_data.relative_bounding_box.width *
d.location_data.relative_bounding_box.height)
bbox = best.location_data.relative_bounding_box
cx = bbox.xmin + bbox.width / 2
cy = bbox.ymin + bbox.height / 2
face_positions.append({
"t": round(ts, 2),
"x_center": round(cx, 4),
"y_center": round(cy, 4),
})
os.unlink(frame_path)
detector.close()
import shutil
shutil.rmtree(tmpdir, ignore_errors=True)
return face_positions
def detect_cursor_positions(clip_path, src_w, src_h, sample_interval=0.5):
"""Detect mouse cursor positions via frame differencing.
Compares consecutive frames to find small changed regions (cursor movement).
Returns list of {t, x_norm, y_norm} positions.
"""
import cv2
import numpy as np
import tempfile
_, _, _, duration = get_video_info(clip_path)
if duration < sample_interval * 2:
return []
tmpdir = tempfile.mkdtemp()
# Extract all sample frames in a single FFmpeg call (much faster than per-frame seeks)
fps_rate = 1.0 / sample_interval
subprocess.run(
["ffmpeg", "-y", "-i", clip_path,
"-vf", f"fps={fps_rate}", "-q:v", "2",
os.path.join(tmpdir, "f_%04d.jpg")],
capture_output=True
)
# Read extracted frames in order (FFmpeg numbers from 0001)
import shutil
frame_files = sorted(glob.glob(os.path.join(tmpdir, "f_*.jpg")))
frames = []
for i, frame_path in enumerate(frame_files):
ts = i * sample_interval
if ts >= duration:
break
img = cv2.imread(frame_path, cv2.IMREAD_GRAYSCALE)
frames.append((ts, img))
shutil.rmtree(tmpdir, ignore_errors=True)
if len(frames) < 2:
return []
cursor_positions = []
last_known_x = 0.5 # default to center
for i in range(1, len(frames)):
ts, curr = frames[i]
_, prev = frames[i - 1]
if curr is None or prev is None:
cursor_positions.append({"t": round(ts, 2), "x_norm": last_known_x})
continue
# Compute absolute difference
diff = cv2.absdiff(prev, curr)
# Threshold to find changed pixels
_, thresh = cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY)
# Find contours of changed regions
contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
# Filter for cursor-sized changes (small moving elements)
# Cursor is typically 10-60px in each dimension
cursor_candidates = []
for c in contours:
area = cv2.contourArea(c)
x, y, w, h = cv2.boundingRect(c)
# Cursor-sized: 50-5000 px² area, aspect ratio not too extreme
if 50 < area < 5000 and 0.2 < (w / max(h, 1)) < 5.0:
cx = (x + w / 2) / src_w
cy = (y + h / 2) / src_h
cursor_candidates.append({
"x_norm": cx,
"y_norm": cy,
"area": area,
"dist_from_last": abs(cx - last_known_x),
})
if cursor_candidates:
# Pick the candidate closest to last known position (smooth tracking)
best = min(cursor_candidates, key=lambda c: c["dist_from_last"])
last_known_x = best["x_norm"]
cursor_positions.append({"t": round(ts, 2), "x_norm": round(best["x_norm"], 4)})
else:
# No cursor-sized change detected; could be:
# - cursor not moving (use last known)
# - large scroll event (ignore, keep position)
cursor_positions.append({"t": round(ts, 2), "x_norm": round(last_known_x, 4)})
# Add initial position (use first detected or center)
if cursor_positions:
cursor_positions.insert(0, {"t": 0.0, "x_norm": cursor_positions[0]["x_norm"]})
return cursor_positions
def smooth_positions(positions, window=5):
"""Apply moving average smoothing to cursor positions."""
if len(positions) < window:
return positions
import numpy as np
x_vals = np.array([p["x_norm"] for p in positions])
# Pad edges for convolution
kernel = np.ones(window) / window
smoothed = np.convolve(x_vals, kernel, mode="same")
# Fix edges (use original values for first/last few)
half = window // 2
smoothed[:half] = x_vals[:half]
smoothed[-half:] = x_vals[-half:]
result = []
for i, p in enumerate(positions):
result.append({"t": p["t"], "x_norm": round(float(smoothed[i]), 4)})
return result
def compute_crop_screen_with_cursor(src_w, src_h, cursor_positions, zoom=0.55):
"""Compute cursor-tracked crop for screen recordings.
Args:
src_w, src_h: Source video dimensions
cursor_positions: List of {t, x_norm} from detect_cursor_positions
zoom: Fraction of source width to show (0.55 = 55% of screen visible)
Returns:
(crop, crop_keyframes) where:
- crop: static fallback {x, y, w, h}
- crop_keyframes: list of {t, x} for animated panning
"""
crop_h = src_h
crop_w = int(src_w * zoom)
crop_w = min(crop_w, src_w)
# Compute crop_x for each cursor position
keyframes = []
for pos in cursor_positions:
cursor_x_px = pos["x_norm"] * src_w
crop_x = int(cursor_x_px - crop_w / 2)
crop_x = max(0, min(crop_x, src_w - crop_w))
keyframes.append({"t": round(pos["t"], 2), "x": crop_x})
# Deduplicate: remove keyframes where x hasn't changed much
if len(keyframes) > 2:
filtered = [keyframes[0]]
for i in range(1, len(keyframes) - 1):
prev_x = filtered[-1]["x"]
curr_x = keyframes[i]["x"]
# Keep keyframe if position changed by more than 2% of crop width
if abs(curr_x - prev_x) > crop_w * 0.02:
filtered.append(keyframes[i])
filtered.append(keyframes[-1])
keyframes = filtered
# Static fallback: average position
if keyframes:
avg_x = int(sum(k["x"] for k in keyframes) / len(keyframes))
else:
avg_x = (src_w - crop_w) // 2
avg_x = max(0, min(avg_x, src_w - crop_w))
crop = {"x": avg_x, "y": 0, "w": crop_w, "h": crop_h}
return crop, keyframes
def compute_crop_screen_static(src_w, src_h, zoom=0.55):
"""Static center crop for screen recordings (no cursor tracking)."""
crop_h = src_h
crop_w = int(src_w * zoom)
crop_w = min(crop_w, src_w)
center_x = int(src_w * 0.50)
crop_x = center_x - crop_w // 2
crop_x = max(0, min(crop_x, src_w - crop_w))
return {"x": crop_x, "y": 0, "w": crop_w, "h": crop_h}
def compute_crop_face_track(src_w, src_h, face_positions):
"""Compute 9:16 crop centered on average face position."""
crop_h = src_h
crop_w = int(crop_h * 9 / 16)
crop_w = min(crop_w, src_w)
if face_positions:
avg_cx = sum(fp["x_center"] for fp in face_positions) / len(face_positions)
center_x_px = int(avg_cx * src_w)
else:
center_x_px = src_w // 2
crop_x = center_x_px - crop_w // 2
crop_x = max(0, min(crop_x, src_w - crop_w))
return {"x": crop_x, "y": 0, "w": crop_w, "h": crop_h}
def compute_crop_center(src_w, src_h):
"""Simple center crop to 9:16."""
crop_h = src_h
crop_w = int(crop_h * 9 / 16)
crop_w = min(crop_w, src_w)
crop_x = (src_w - crop_w) // 2
return {"x": crop_x, "y": 0, "w": crop_w, "h": crop_h}
def main():
parser = argparse.ArgumentParser(description="Compute reframe coordinates")
parser.add_argument("--clips-dir", required=True, help="Directory with clip files")
parser.add_argument("--content-type", required=True,
choices=["talking-head", "screen", "podcast"],
help="Content type for reframing strategy")
parser.add_argument("--output", required=True, help="Output JSON file")
parser.add_argument("--zoom", type=float, default=0.55,
help="Screen zoom level: fraction of source width to show (default: 0.55)")
parser.add_argument("--no-cursor-track", action="store_true",
help="Disable cursor tracking for screen content")
args = parser.parse_args()
if not os.path.isdir(args.clips_dir):
print(json.dumps({"error": f"Clips directory not found: {args.clips_dir}"}))
sys.exit(1)
start = time.time()
clips = sorted(glob.glob(os.path.join(args.clips_dir, "clip_*.mp4")))
if not clips:
print(json.dumps({"error": f"No clip_*.mp4 files found in {args.clips_dir}"}))
sys.exit(1)
results = {}
for clip_path in clips:
clip_name = os.path.basename(clip_path)
src_w, src_h, fps, duration = get_video_info(clip_path)
if args.content_type == "talking-head":
face_positions = compute_face_track(clip_path, src_w, src_h)
crop = compute_crop_face_track(src_w, src_h, face_positions)
strategy = "face-track"
crop_keyframes = []
elif args.content_type == "screen":
if args.no_cursor_track:
crop = compute_crop_screen_static(src_w, src_h, zoom=args.zoom)
crop_keyframes = []
strategy = "framed"
else:
# Detect cursor positions
cursor_positions = detect_cursor_positions(
clip_path, src_w, src_h, sample_interval=0.5
)
# Smooth the trajectory
cursor_positions = smooth_positions(cursor_positions, window=5)
if len(cursor_positions) >= 2:
crop, crop_keyframes = compute_crop_screen_with_cursor(
src_w, src_h, cursor_positions, zoom=args.zoom
)
strategy = "cursor-track"
else:
crop = compute_crop_screen_static(src_w, src_h, zoom=args.zoom)
crop_keyframes = []
strategy = "framed"
face_positions = []
else: # podcast
face_positions = compute_face_track(clip_path, src_w, src_h)
crop = compute_crop_face_track(src_w, src_h, face_positions)
strategy = "face-track"
crop_keyframes = []
entry = {
"strategy": strategy,
"source_resolution": f"{src_w}x{src_h}",
"crop": crop,
"output_resolution": "1080x1920",
"face_positions": face_positions if args.content_type != "screen" else [],
"duration": round(duration, 2),
}
if crop_keyframes:
entry["crop_keyframes"] = crop_keyframes
results[clip_name] = entry
elapsed = time.time() - start
output = {
"content_type": args.content_type,
"clip_count": len(results),
"computation_time_sec": round(elapsed, 2),
"clips": results,
}
os.makedirs(os.path.dirname(os.path.abspath(args.output)), exist_ok=True)
with open(args.output, "w") as f:
json.dump(output, f, indent=2)
# Print summary
summary = {
"action": "compute_reframe",
"content_type": args.content_type,
"clips_processed": len(results),
"computation_time_sec": round(elapsed, 2),
}
for name, data in results.items():
s = {
"strategy": data["strategy"],
"crop": data["crop"],
}
if "crop_keyframes" in data:
s["keyframe_count"] = len(data["crop_keyframes"])
if data.get("face_positions"):
s["faces_detected"] = len(data["face_positions"])
summary[name] = s
print(json.dumps(summary, indent=2))
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""Auto-detect video content type: talking-head, screen recording, or podcast.
Samples N frames evenly across the video, runs MediaPipe face detection,
and classifies based on face count, size, and position patterns.
Usage:
python3 detect_content.py INPUT_VIDEO --output content_type.json
Output:
{
"content_type": "talking-head",
"confidence": 0.92,
"face_stats": {"avg_count": 1.0, "avg_size_pct": 15.2, "center_bias": 0.85},
"frames_sampled": 10
}
Classification rules:
- talking-head: 1 face, large (>=5% frame area), centered (center_bias >= 0.4)
- podcast: 2+ faces consistently, medium size (5-15%)
- screen: <0.5 avg faces, very small faces (<5%), or small off-center faces (PiP/presenter overlay)
"""
import argparse
import json
import os
import subprocess
import sys
import tempfile
import time
def sample_frames(video_path, num_frames=10):
"""Extract N evenly-spaced frames from video as JPEG files."""
# Get duration
result = subprocess.run(
["ffprobe", "-v", "quiet", "-show_entries", "format=duration",
"-of", "csv=p=0", video_path],
capture_output=True, text=True
)
try:
duration = float(result.stdout.strip())
except (ValueError, AttributeError):
print(json.dumps({"error": f"Could not determine video duration: {video_path}"}))
sys.exit(1)
tmpdir = tempfile.mkdtemp(prefix="detect_content_")
timestamps = [duration * (i + 0.5) / num_frames for i in range(num_frames)]
frame_paths = []
for i, ts in enumerate(timestamps):
out_path = os.path.join(tmpdir, f"frame_{i:03d}.jpg")
subprocess.run(
["ffmpeg", "-y", "-ss", str(ts), "-i", video_path,
"-vframes", "1", "-q:v", "2", out_path],
capture_output=True, text=True
)
if os.path.exists(out_path):
frame_paths.append(out_path)
return frame_paths, tmpdir
def detect_faces(frame_paths):
"""Run MediaPipe face detection on frames, return stats."""
import mediapipe as mp
import cv2
import numpy as np
mp_face = mp.solutions.face_detection
detector = mp_face.FaceDetection(
model_selection=1, # Full range model (works for far faces too)
min_detection_confidence=0.5
)
face_counts = []
face_sizes = [] # as percentage of frame area
face_centers_x = [] # normalized x position (0-1)
for path in frame_paths:
img = cv2.imread(path)
if img is None:
continue
h, w = img.shape[:2]
frame_area = h * w
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
results = detector.process(rgb)
if results.detections:
face_counts.append(len(results.detections))
for det in results.detections:
bbox = det.location_data.relative_bounding_box
face_w = bbox.width * w
face_h = bbox.height * h
face_area_pct = (face_w * face_h) / frame_area * 100
face_sizes.append(face_area_pct)
center_x = bbox.xmin + bbox.width / 2
face_centers_x.append(center_x)
else:
face_counts.append(0)
detector.close()
avg_count = float(np.mean(face_counts)) if face_counts else 0
avg_size = float(np.mean(face_sizes)) if face_sizes else 0
size_std = float(np.std(face_sizes)) if len(face_sizes) > 1 else 0
max_size = float(np.max(face_sizes)) if face_sizes else 0
center_bias = 0.0
if face_centers_x:
# How close to center (0.5)? 1.0 = perfectly centered
center_bias = 1.0 - float(np.mean([abs(x - 0.5) for x in face_centers_x])) * 2
return {
"avg_count": round(avg_count, 2),
"avg_size_pct": round(avg_size, 2),
"max_size_pct": round(max_size, 2),
"size_std": round(size_std, 2),
"center_bias": round(center_bias, 3),
"frames_with_faces": sum(1 for c in face_counts if c > 0),
"total_frames": len(face_counts),
}
def classify(face_stats):
"""Classify content type based on face detection stats."""
avg_count = face_stats["avg_count"]
avg_size = face_stats["avg_size_pct"]
center_bias = face_stats["center_bias"]
face_ratio = face_stats["frames_with_faces"] / max(face_stats["total_frames"], 1)
# Screen recording: few/no faces or very small faces
if avg_count < 0.5 or (avg_size < 5 and face_ratio < 0.5):
confidence = min(1.0, (1.0 - avg_count) * 0.5 + (1.0 - face_ratio) * 0.5)
return "screen", round(confidence, 3)
# Podcast: multiple faces consistently
if avg_count >= 1.8 and face_ratio > 0.7:
confidence = min(1.0, (avg_count - 1.0) * 0.3 + face_ratio * 0.3 + 0.4)
return "podcast", round(confidence, 3)
# PiP / off-center face: small-to-medium face that's far from center
# (e.g., webcam overlay in corner, presenter in slide corner)
# Face-tracking would zoom into the small face and lose the main content
if avg_size < 8 and center_bias < 0.4:
confidence = min(1.0, (1.0 - center_bias) * 0.3 + (8 - avg_size) / 8 * 0.3 + 0.3)
return "screen", round(confidence, 3)
# Talking head: single face, large, centered
if avg_count >= 0.5 and avg_size >= 5:
confidence = min(1.0, center_bias * 0.4 + min(avg_size / 20, 1.0) * 0.3 + face_ratio * 0.3)
return "talking-head", round(confidence, 3)
# Default to screen if uncertain
return "screen", 0.5
def main():
parser = argparse.ArgumentParser(description="Detect video content type")
parser.add_argument("input", help="Input video file")
parser.add_argument("--output", required=True, help="Output JSON file")
parser.add_argument("--frames", type=int, default=10,
help="Number of frames to sample (default: 10)")
args = parser.parse_args()
if not os.path.isfile(args.input):
print(json.dumps({"error": f"Input file not found: {args.input}"}))
sys.exit(1)
start = time.time()
# Sample frames
frame_paths, tmpdir = sample_frames(args.input, args.frames)
if not frame_paths:
print(json.dumps({"error": "No frames could be extracted"}))
sys.exit(1)
# Detect faces
face_stats = detect_faces(frame_paths)
# Classify
content_type, confidence = classify(face_stats)
elapsed = time.time() - start
# Clean up temp frames
import shutil
shutil.rmtree(tmpdir, ignore_errors=True)
output = {
"content_type": content_type,
"confidence": confidence,
"face_stats": face_stats,
"frames_sampled": len(frame_paths),
"detection_time_sec": round(elapsed, 2),
}
os.makedirs(os.path.dirname(os.path.abspath(args.output)), exist_ok=True)
with open(args.output, "w") as f:
json.dump(output, f, indent=2)
print(json.dumps(output, indent=2))
if __name__ == "__main__":
main()
#!/usr/bin/env bash
# Detect NVIDIA GPU and available NVENC encoders for FFmpeg
# Usage: bash scripts/detect_gpu.sh
# Output: JSON with gpu_name, driver, encoders[], cuda_filters[]
set -euo pipefail
json_output() {
local gpu_name="${1:-none}"
local driver="${2:-none}"
local has_nvenc="${3:-false}"
local encoders="${4:-[]}"
local cuda_filters="${5:-[]}"
cat <<EOF
{
"gpu_detected": $([ "$gpu_name" != "none" ] && echo "true" || echo "false"),
"gpu_name": "$gpu_name",
"driver_version": "$driver",
"nvenc_available": $has_nvenc,
"encoders": $encoders,
"cuda_filters": $cuda_filters,
"recommendation": $([ "$has_nvenc" = "true" ] && echo '"Use NVENC for 5-10x faster encoding"' || echo '"No GPU detected, using CPU encoding"')
}
EOF
}
# Check for nvidia-smi
if ! command -v nvidia-smi &>/dev/null; then
json_output
exit 0
fi
# Get GPU info
GPU_NAME=$(nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null | head -1 | xargs) || GPU_NAME="none"
DRIVER=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader 2>/dev/null | head -1 | xargs) || DRIVER="none"
if [ "$GPU_NAME" = "none" ]; then
json_output
exit 0
fi
# Check FFmpeg NVENC encoders
ENCODERS="[]"
if command -v ffmpeg &>/dev/null; then
ENC_LIST=$(ffmpeg -hide_banner -encoders 2>/dev/null | grep -oP '(h264|hevc|av1)_nvenc' | sort -u)
if [ -n "$ENC_LIST" ]; then
ENCODERS=$(echo "$ENC_LIST" | jq -R . | jq -s .)
fi
fi
# Check CUDA filters
CUDA_FILTERS="[]"
if command -v ffmpeg &>/dev/null; then
FILTER_LIST=$(ffmpeg -hide_banner -filters 2>/dev/null | grep -oP '\S*cuda\S*' | sort -u)
if [ -n "$FILTER_LIST" ]; then
CUDA_FILTERS=$(echo "$FILTER_LIST" | jq -R . | jq -s .)
fi
fi
HAS_NVENC="false"
if [ "$ENCODERS" != "[]" ]; then
HAS_NVENC="true"
fi
json_output "$GPU_NAME" "$DRIVER" "$HAS_NVENC" "$ENCODERS" "$CUDA_FILTERS"
#!/usr/bin/env bash
# Platform-specific FFmpeg encoding for exported shorts
# Usage: bash scripts/export.sh --input-dir DIR --platform PLATFORM --output-dir DIR
set -euo pipefail
INPUT_DIR=""
PLATFORM="all"
OUTPUT_DIR="./shorts"
FORCE="false"
while [[ $# -gt 0 ]]; do
case "$1" in
--input-dir) INPUT_DIR="$2"; shift 2 ;;
--platform) PLATFORM="$2"; shift 2 ;;
--output-dir) OUTPUT_DIR="$2"; shift 2 ;;
--force) FORCE="true"; shift ;;
*) echo "Unknown option: $1" >&2; exit 1 ;;
esac
done
if [ -z "$INPUT_DIR" ]; then
echo '{"error":"Usage: export.sh --input-dir DIR [--platform youtube|tiktok|instagram|all] [--output-dir DIR]"}'
exit 1
fi
if [ ! -d "$INPUT_DIR" ]; then
echo "{\"error\":\"Input directory not found: $INPUT_DIR\"}"
exit 1
fi
mkdir -p "$OUTPUT_DIR"
# Detect GPU for NVENC
HAS_NVENC="false"
if command -v nvidia-smi &>/dev/null && command -v ffmpeg &>/dev/null; then
if ffmpeg -hide_banner -encoders 2>/dev/null | grep -q "h264_nvenc"; then
HAS_NVENC="true"
fi
fi
# Platform encoding functions
encode_youtube() {
local input="$1" output="$2"
if [ "$HAS_NVENC" = "true" ]; then
ffmpeg -y -i "$input" \
-c:v h264_nvenc -preset p5 -tune hq \
-b:v 12M -maxrate 14M -bufsize 24M \
-profile:v high -level 4.2 \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 192k -ar 48000 \
-pix_fmt yuv420p -movflags +faststart \
"$output" 2>/dev/null
else
ffmpeg -y -i "$input" \
-c:v libx264 -preset slow \
-b:v 12M -maxrate 14M -bufsize 24M \
-profile:v high -level 4.2 \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 192k -ar 48000 \
-pix_fmt yuv420p -movflags +faststart \
"$output" 2>/dev/null
fi
}
encode_tiktok() {
local input="$1" output="$2"
if [ "$HAS_NVENC" = "true" ]; then
ffmpeg -y -i "$input" \
-c:v h264_nvenc -preset p5 -tune hq \
-cq 18 -maxrate 10M -bufsize 20M \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 128k -ar 44100 \
-pix_fmt yuv420p -movflags +faststart \
"$output" 2>/dev/null
else
ffmpeg -y -i "$input" \
-c:v libx264 -preset slow -crf 18 \
-maxrate 10M -bufsize 20M \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 128k -ar 44100 \
-pix_fmt yuv420p -movflags +faststart \
"$output" 2>/dev/null
fi
}
encode_instagram() {
local input="$1" output="$2"
if [ "$HAS_NVENC" = "true" ]; then
ffmpeg -y -i "$input" \
-c:v h264_nvenc -preset p5 -tune hq \
-b:v 4500k -maxrate 5000k -bufsize 10M \
-profile:v high -level 4.2 \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 128k -ar 44100 \
-pix_fmt yuv420p -movflags +faststart \
"$output" 2>/dev/null
else
ffmpeg -y -i "$input" \
-c:v libx264 -preset slow \
-b:v 4500k -maxrate 5000k -bufsize 10M \
-profile:v high -level 4.2 \
-af loudnorm=I=-14:TP=-1:LRA=11 \
-c:a aac -b:a 128k -ar 44100 \
-pix_fmt yuv420p -movflags +faststart \
"$output" 2>/dev/null
fi
}
# Process each rendered short
RESULTS=()
COUNT=0
for input_file in "$INPUT_DIR"/short_*.mp4; do
[ -f "$input_file" ] || continue
COUNT=$((COUNT + 1))
base=$(basename "$input_file" .mp4)
num="${base#short_}"
platforms=()
if [ "$PLATFORM" = "all" ]; then
platforms=("youtube" "tiktok" "instagram")
else
platforms=("$PLATFORM")
fi
for plat in "${platforms[@]}"; do
case "$plat" in
youtube) suffix="_yt"; encode_func="encode_youtube" ;;
tiktok) suffix="_tt"; encode_func="encode_tiktok" ;;
instagram) suffix="_ig"; encode_func="encode_instagram" ;;
*) echo "Unknown platform: $plat" >&2; continue ;;
esac
output_file="$OUTPUT_DIR/short_${num}${suffix}.mp4"
# Skip if output already exists (use --force to overwrite)
if [ -f "$output_file" ] && [ "$FORCE" != "true" ]; then
size=$(du -k "$output_file" | cut -f1)
size_mb=$(echo "scale=1; $size / 1024" | bc)
duration=$(ffprobe -v quiet -show_entries format=duration -of csv=p=0 "$output_file" 2>/dev/null)
duration_int=$(printf "%.0f" "$duration" 2>/dev/null || echo "0")
RESULTS+=("{\"file\":\"$output_file\",\"platform\":\"$plat\",\"duration\":\"${duration_int}s\",\"size_mb\":$size_mb,\"skipped\":true}")
continue
fi
$encode_func "$input_file" "$output_file"
# Get file info
size=$(du -k "$output_file" | cut -f1)
size_mb=$(echo "scale=1; $size / 1024" | bc)
duration=$(ffprobe -v quiet -show_entries format=duration -of csv=p=0 "$output_file" 2>/dev/null)
duration_int=$(printf "%.0f" "$duration" 2>/dev/null || echo "0")
RESULTS+=("{\"file\":\"$output_file\",\"platform\":\"$plat\",\"duration\":\"${duration_int}s\",\"size_mb\":$size_mb}")
done
done
# Build JSON output
RESULTS_JSON=$(printf '%s\n' "${RESULTS[@]}" | paste -sd ',' | sed 's/^/[/' | sed 's/$/]/')
cat <<EOF
{
"action": "export",
"platform": "$PLATFORM",
"nvenc": $HAS_NVENC,
"shorts_exported": $COUNT,
"output_dir": "$OUTPUT_DIR",
"files": $RESULTS_JSON
}
EOF
#!/usr/bin/env bash
# Pre-flight safety check before shortform pipeline
# Usage: bash scripts/preflight.sh <input_file> [output_dir]
# Output: JSON with pass/fail status and any warnings
set -euo pipefail
INPUT="${1:-}"
OUTPUT_DIR="${2:-./shorts}"
if [ -z "$INPUT" ]; then
echo '{"pass":false,"error":"Usage: preflight.sh <input_file> [output_dir]"}'
exit 1
fi
WARNINGS=()
ERRORS=()
# Check input exists
if [ ! -f "$INPUT" ]; then
ERRORS+=("Input file not found: $INPUT")
fi
# Check input is a video file (via ffprobe)
if [ -f "$INPUT" ]; then
if ! ffprobe -v quiet -select_streams v:0 -show_entries stream=codec_type -of csv=p=0 "$INPUT" 2>/dev/null | grep -q "video"; then
ERRORS+=("Input is not a valid video file: $INPUT")
fi
fi
# Check output directory
if [ -d "$OUTPUT_DIR" ]; then
EXISTING=$(find "$OUTPUT_DIR" -name "short_*.mp4" 2>/dev/null | wc -l)
if [ "$EXISTING" -gt 0 ]; then
WARNINGS+=("Output directory has $EXISTING existing short_*.mp4 files")
fi
fi
# Check disk space (estimate 3x input size for temp + output)
if [ -f "$INPUT" ]; then
INPUT_SIZE_KB=$(du -k "$INPUT" | cut -f1)
NEEDED_KB=$((INPUT_SIZE_KB * 3))
AVAIL_KB=$(df -k /tmp 2>/dev/null | tail -1 | awk '{print $4}')
if [ -n "$AVAIL_KB" ] && [ "$AVAIL_KB" -lt "$NEEDED_KB" ]; then
WARNINGS+=("Low disk space on /tmp: ${AVAIL_KB}KB available, estimated ${NEEDED_KB}KB needed")
fi
fi
# Check FFmpeg
if ! command -v ffmpeg &>/dev/null; then
ERRORS+=("ffmpeg not found — install with: sudo apt install ffmpeg")
fi
# Check ffprobe
if ! command -v ffprobe &>/dev/null; then
ERRORS+=("ffprobe not found — install with: sudo apt install ffmpeg")
fi
# Check Node.js
if ! command -v node &>/dev/null; then
ERRORS+=("Node.js not found — install with: nvm install 18")
fi
# Check Python venv
VENV=""
if [ -d "$HOME/.video-skill" ]; then
VENV="$HOME/.video-skill"
elif [ -d "$HOME/.shorts-skill" ]; then
VENV="$HOME/.shorts-skill"
fi
if [ -z "$VENV" ]; then
ERRORS+=("Python venv not found — run: bash setup.sh")
else
# Check faster-whisper is installed
if ! "$VENV/bin/python3" -c "import faster_whisper" 2>/dev/null; then
ERRORS+=("faster-whisper not installed in $VENV — run: bash setup.sh")
fi
fi
# Check Remotion node_modules
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
if [ ! -d "$SCRIPT_DIR/remotion/node_modules" ]; then
ERRORS+=("Remotion dependencies not installed — run: bash setup.sh")
fi
# Get video info
DURATION=""
RESOLUTION=""
if [ -f "$INPUT" ] && command -v ffprobe &>/dev/null; then
DURATION=$(ffprobe -v quiet -show_entries format=duration -of csv=p=0 "$INPUT" 2>/dev/null || echo "")
RESOLUTION=$(ffprobe -v quiet -select_streams v:0 -show_entries stream=width,height -of csv=p=0 "$INPUT" 2>/dev/null || echo "")
fi
# Build JSON output
PASS="true"
if [ ${#ERRORS[@]} -gt 0 ]; then
PASS="false"
fi
ERROR_JSON="[]"
if [ ${#ERRORS[@]} -gt 0 ]; then
ERROR_JSON=$(printf '%s\n' "${ERRORS[@]}" | jq -R . | jq -s .)
fi
WARN_JSON="[]"
if [ ${#WARNINGS[@]} -gt 0 ]; then
WARN_JSON=$(printf '%s\n' "${WARNINGS[@]}" | jq -R . | jq -s .)
fi
cat <<EOF
{
"pass": $PASS,
"input": "$INPUT",
"output_dir": "$OUTPUT_DIR",
"duration": ${DURATION:-null},
"resolution": "${RESOLUTION:-unknown}",
"venv": "${VENV:-none}",
"errors": $ERROR_JSON,
"warnings": $WARN_JSON
}
EOF
[ "$PASS" = "true" ] && exit 0 || exit 1
#!/usr/bin/env python3
"""Snap segment boundaries to natural audio cut points.
Takes proposed segment times and the transcript, then adjusts start/end to:
1. Align with word boundaries (never cut mid-word)
2. Extend to sentence completion if close (within 3s)
3. Pad with silence after the last word
4. Optionally use FFmpeg silencedetect to find natural pauses
Usage:
python3 snap_boundaries.py \
--segments /tmp/claude-shorts/approved_segments.json \
--transcript /tmp/claude-shorts/transcript.json \
--input-video /path/to/source.mp4 \
--output /tmp/claude-shorts/snapped_segments.json
Output: Same format as approved_segments.json with adjusted start/end times.
"""
import argparse
import json
import os
import subprocess
import sys
def load_word_timeline(transcript_path):
"""Load word-level timestamps from transcript.
Returns list of {text, start, end} for every word, sorted by start time.
"""
with open(transcript_path) as f:
data = json.load(f)
words = []
# Use the segments array which has word-level detail
for seg in data.get("segments", []):
for w in seg.get("words", []):
words.append({
"text": w["word"].strip(),
"start": w["start"],
"end": w["end"],
})
# Sort by start time
words.sort(key=lambda w: w["start"])
return words
def detect_silences(video_path, min_duration=0.3, noise_threshold=-35):
"""Find silence regions in the audio track using FFmpeg silencedetect.
Returns list of {start, end} for each silence region.
"""
cmd = [
"ffmpeg", "-i", video_path,
"-af", f"silencedetect=noise={noise_threshold}dB:d={min_duration}",
"-f", "null", "-"
]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=120)
stderr = result.stderr
silences = []
current_start = None
for line in stderr.split("\n"):
if "silence_start:" in line:
try:
current_start = float(line.split("silence_start:")[1].strip().split()[0])
except (IndexError, ValueError):
pass
elif "silence_end:" in line and current_start is not None:
try:
parts = line.split("silence_end:")[1].strip().split()
end = float(parts[0])
silences.append({"start": current_start, "end": end})
current_start = None
except (IndexError, ValueError):
pass
return silences
def find_nearest_silence(silences, target_time, search_window=2.0):
"""Find the silence region closest to target_time within search_window.
Returns the midpoint of the nearest silence, or None if no silence nearby.
"""
best = None
best_dist = float("inf")
for s in silences:
mid = (s["start"] + s["end"]) / 2
dist = abs(mid - target_time)
if dist < search_window and dist < best_dist:
best = mid
best_dist = dist
return best
def snap_start(words, proposed_start, search_window=1.5):
"""Snap start time to the beginning of the nearest word.
Looks for a word that starts near proposed_start.
Prefers snapping to a sentence start (after punctuation) if one is nearby.
"""
# Find words near the proposed start
candidates = []
for i, w in enumerate(words):
if abs(w["start"] - proposed_start) <= search_window:
candidates.append((i, w))
if not candidates:
return proposed_start
# Check if any candidate is a sentence start (previous word ends with . ? !)
sentence_starts = []
for idx, w in candidates:
if idx == 0:
sentence_starts.append((idx, w))
elif words[idx - 1]["text"].rstrip()[-1:] in ".?!":
sentence_starts.append((idx, w))
# Prefer sentence start if available and within window
if sentence_starts:
# Pick the one closest to proposed start
best_idx, best_w = min(sentence_starts, key=lambda x: abs(x[1]["start"] - proposed_start))
return best_w["start"]
# Otherwise, snap to nearest word start
best_idx, best_w = min(candidates, key=lambda x: abs(x[1]["start"] - proposed_start))
return best_w["start"]
def snap_end(words, proposed_end, search_window=3.0, pad_ms=300):
"""Snap end time to after the last complete sentence.
Extends to the next sentence boundary (. ? !) if within search_window.
Adds pad_ms of silence after the last word.
"""
pad = pad_ms / 1000.0
# Find the last word that ends at or before proposed_end
last_word_idx = None
for i, w in enumerate(words):
if w["end"] <= proposed_end + 0.2: # small tolerance for timestamp imprecision
last_word_idx = i
if last_word_idx is None:
return proposed_end
# Check if the last word ends a sentence
last_word = words[last_word_idx]
if last_word["text"].rstrip()[-1:] in ".?!":
return last_word["end"] + pad
# Look forward for the next sentence-ending word within search_window
for i in range(last_word_idx + 1, len(words)):
w = words[i]
if w["start"] > proposed_end + search_window:
break
if w["text"].rstrip()[-1:] in ".?!":
return w["end"] + pad
# No sentence boundary found nearby — snap to nearest word end
# Find the word whose end is closest to proposed_end
candidates = []
for i, w in enumerate(words):
if abs(w["end"] - proposed_end) <= 1.0:
candidates.append(w)
if candidates:
best = min(candidates, key=lambda w: abs(w["end"] - proposed_end))
return best["end"] + pad
return proposed_end
def snap_segment(words, silences, seg, video_duration):
"""Snap a single segment's boundaries.
Returns adjusted (start, end) tuple.
"""
proposed_start = seg["start"]
proposed_end = seg["end"]
# Step 1: Snap to word boundaries
new_start = snap_start(words, proposed_start)
new_end = snap_end(words, proposed_end)
# Step 2: If we have silence data, try to cut at silence points
if silences:
# For start: prefer cutting at a silence just before the first word
silence_start = find_nearest_silence(silences, new_start, search_window=1.5)
if silence_start is not None and silence_start <= new_start:
new_start = silence_start
# For end: prefer cutting at a silence just after the last word
silence_end = find_nearest_silence(silences, new_end, search_window=2.0)
if silence_end is not None and silence_end >= new_end - 0.5:
new_end = silence_end
# Clamp to video duration
new_start = max(0.0, new_start)
new_end = min(video_duration, new_end)
# Ensure minimum duration (5s) and maximum (60s)
if new_end - new_start < 5.0:
new_end = min(new_start + 5.0, video_duration)
if new_end - new_start > 60.0:
new_end = new_start + 60.0
return round(new_start, 3), round(new_end, 3)
def main():
parser = argparse.ArgumentParser(description="Snap segment boundaries to audio")
parser.add_argument("--segments", required=True, help="Approved segments JSON")
parser.add_argument("--transcript", required=True, help="Transcript JSON")
parser.add_argument("--input-video", required=True, help="Source video for silence detection")
parser.add_argument("--output", required=True, help="Output snapped segments JSON")
parser.add_argument("--no-silence", action="store_true",
help="Skip silence detection (word-boundary snapping only)")
args = parser.parse_args()
# Load data
words = load_word_timeline(args.transcript)
if not words:
print(json.dumps({"error": "No word-level timestamps found in transcript"}))
sys.exit(1)
with open(args.segments) as f:
segments_data = json.load(f)
# Get video duration
result = subprocess.run(
["ffprobe", "-v", "quiet", "-show_entries", "format=duration",
"-of", "csv=p=0", args.input_video],
capture_output=True, text=True
)
video_duration = float(result.stdout.strip())
# Detect silences (unless disabled)
silences = []
if not args.no_silence:
silences = detect_silences(args.input_video)
# Snap each segment
adjustments = []
for seg in segments_data["segments"]:
old_start = seg["start"]
old_end = seg["end"]
new_start, new_end = snap_segment(words, silences, seg, video_duration)
seg["start"] = new_start
seg["end"] = new_end
seg["duration"] = round(new_end - new_start, 3)
adj = {
"id": seg.get("id", "?"),
"old": f"{old_start:.1f}-{old_end:.1f}",
"new": f"{new_start:.3f}-{new_end:.3f}",
"delta_start": f"{(new_start - old_start)*1000:+.0f}ms",
"delta_end": f"{(new_end - old_end)*1000:+.0f}ms",
}
adjustments.append(adj)
# Write output
os.makedirs(os.path.dirname(os.path.abspath(args.output)), exist_ok=True)
with open(args.output, "w") as f:
json.dump(segments_data, f, indent=2)
# Print summary
print(json.dumps({
"action": "snap_boundaries",
"segments_processed": len(adjustments),
"silences_detected": len(silences),
"word_count": len(words),
"adjustments": adjustments,
}, indent=2))
if __name__ == "__main__":
main()
Security Policy
Reporting a Vulnerability
If you discover a security vulnerability, please report it responsibly:
1. Do NOT open a public issue 2. Open a GitHub Security Advisory on this repo 3. Or contact the maintainer directly
Supported Versions
Only the latest version receives security updates.
Security Practices
- No credentials or API keys are stored in this repository
- Install scripts write only to user-level directories (
~/.claude/) - Python dependencies install in isolated virtual environments
#!/usr/bin/env pwsh
# claude-shorts uninstaller for Windows
$ErrorActionPreference = "Stop"
Write-Host "=== Uninstalling claude-shorts ===" -ForegroundColor Cyan
Write-Host ""
$SkillDir = Join-Path $env:USERPROFILE ".claude" "skills" "shorts"
if (Test-Path $SkillDir) {
Remove-Item -Recurse -Force $SkillDir
Write-Host " Removed: $SkillDir" -ForegroundColor Green
}
Write-Host ""
Write-Host "=== claude-shorts uninstalled ===" -ForegroundColor Cyan
Write-Host ""
Write-Host "Restart Claude Code to complete removal." -ForegroundColor Yellow
#!/usr/bin/env bash
set -euo pipefail
main() {
echo "Uninstalling claude-shorts..."
rm -rf "${HOME}/.claude/skills/shorts"
echo "claude-shorts uninstalled."
echo "Restart Claude Code to complete removal."
}
main "$@"
Related skills
FAQ
How does it pick clips?
Claude reads the transcript and scores 8-12 candidate segments on hook strength, coherence, emotion, value density, and payoff, then you approve.
What tools does it use?
faster-whisper for transcription, Remotion for animated captions, and FFmpeg for platform-optimized export.