
Universal Video Clipper
- 4 installs
- 10 repo stars
- Updated March 9, 2026
- dkyazzentwatwa/vid-clipper
Turns a YouTube URL or local video file into 3-7 short vertical clips optimized for Instagram Reels and TikTok using AI highlight analysis.
About
A video-clipping skill that analyzes long-form video and extracts short viral-worthy segments in both original and 9:16 vertical formats. A creator uses it to produce Reels and TikToks from tutorials, podcasts, or uploaded footage.
- Accepts both YouTube URLs and local/uploaded video files
- Outputs 15-60s clips in standard plus 9:16 vertical with optional animated captions
Universal Video Clipper by the numbers
- 4 all-time installs (skills.sh)
- Ranked #1,141 of 1,337 Generative Media skills by installs in the Skillselion catalog
- Data as of Jul 29, 2026 (Skillselion catalog sync)
npx skills add https://github.com/dkyazzentwatwa/vid-clipper --skill universal-video-clipperAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 4 |
|---|---|
| repo stars | ★ 10 |
| Last updated | March 9, 2026 |
| Repository | dkyazzentwatwa/vid-clipper ↗ |
What it does
Turns a YouTube URL or local video file into 3-7 short vertical clips optimized for Instagram Reels and TikTok using AI highlight analysis.
Files
Universal Video Clipper
Transform long-form videos into viral-worthy short clips optimized for Instagram Reels and TikTok using AI-powered analysis. Supports both YouTube URLs and local/uploaded video files.
---
How to Talk to This Tool
Use natural language to create clips:
For YouTube Videos
"Create viral clips from this YouTube video: https://www.youtube.com/watch?v=VIDEO_ID"
"Turn this YouTube video into Instagram Reels"
"Extract the best highlights from this tutorial for TikTok"
For Local/Uploaded Videos
"Create short clips from this video file: /path/to/video.mp4"
"Make Reels from my uploaded video"
With Additional Options
"Create 5 viral clips from this video with animated captions"
"Extract highlights from this video, skip downloading since I already have it"
---
What You Get
For each video processed:
- 3-7 viral-worthy clips (15-60 seconds each)
- Dual formats - Standard (original aspect ratio) + Instagram 9:16 vertical
- Optional animated captions - CapCut-style word-by-word highlighting
- Summary report with:
- Virality scores (1-10)
- Suggested captions with emojis
- Hashtag recommendations
- Target audience insights
Sample Output Structure
downloads/{video_id}/
├── original.mp4 # Downloaded/copied video
├── original.json # Whisper transcript with timestamps
├── metadata.json # Video info
├── analysis_request.md # AI analysis prompt
├── clip_recommendations.json # AI-generated clip suggestions
├── SUMMARY.md # Final report
└── clips/
├── clip_001_hook.mp4 # Standard clip
├── clip_001_hook_instagram.mp4 # 9:16 vertical
├── clip_001_hook_captioned.mp4 # With animated captions
└── ...---
How It Works
Step 1: Video Ingest
- YouTube URLs: Downloads via yt-dlp
- Local files: Copies to working directory
- Automatically detects source type
Step 2: Transcription
- Uses Whisper for speech-to-text
- Generates timestamps for each segment
- Falls back from "base" to "tiny" model if needed
Step 3: AI Analysis (Interactive)
The script pauses for AI analysis: 1. Review analysis_request.md (contains transcript + instructions) 2. Run AI on this prompt to generate clip recommendations 3. Save results as clip_recommendations.json 4. Press Enter to continue
Step 4: Generate Clips
- Validates clip timestamps
- Creates standard and Instagram 9:16 versions
- Optionally adds animated captions
---
Prerequisites
Verify these are installed:
# Check installations
ffmpeg -version # Video processing
yt-dlp --version # YouTube downloader (only for YouTube URLs)
whisper --help # Audio transcriptionIf missing, install:
# Install Python dependencies
pip install -r assets/requirements.txt
# Install ffmpeg (macOS)
brew install ffmpeg
# Install ffmpeg (Linux)
sudo apt install ffmpeg
# Install Node.js 18+ (for animated captions)
brew install node---
Technical Reference
Skill Package Layout
universal-video-clipper/
├── SKILL.md
├── assets/requirements.txt
└── scripts/
├── ai_clip_generator.py
├── prompt_templates/clip_analysis_prompt.md
└── remotion-captions/ # Remotion caption renderer projectCLI Command
# YouTube input
python3 scripts/ai_clip_generator.py "https://www.youtube.com/watch?v=VIDEO_ID"
# Local/uploaded file input
python3 scripts/ai_clip_generator.py "/absolute/path/to/uploaded_video.mp4"
# Skip download if video exists
python3 scripts/ai_clip_generator.py <source> --skip-download
# Skip transcription if done
python3 scripts/ai_clip_generator.py <source> --skip-transcription
# With animated captions
python3 scripts/ai_clip_generator.py <source> --add-captions
python3 scripts/ai_clip_generator.py <source> --add-captions --caption-style scaling
python3 scripts/ai_clip_generator.py <source> --add-captions --caption-color "#FF6600"Caption Styles
| Style | Description | Best For |
|---|---|---|
background | Animated highlight box behind words (default) | CapCut-style, modern IG look |
scaling | Words scale up with spring animation | Energetic, punchy content |
colored | Active word highlighted in accent color | Clean, professional look |
---
Clip Selection Criteria
The AI analysis identifies viral-worthy moments based on:
1. Strong Hooks (0-3 seconds) - Bold claims, surprising statements, visual demonstrations 2. Value Bombs - Actionable tips, "aha!" moments, problem-solution demos 3. Emotional Peaks - Excitement, surprise, humor, impressive demonstrations 4. Story Arcs - Complete narratives with setup → demonstration → payoff
Constraints:
- Duration: 15-60 seconds (optimal: 20-45 seconds)
- Self-contained: Each clip makes sense independently
- Natural boundaries: No mid-sentence cuts
- Platform: Optimized for Instagram Reels and TikTok (9:16 mobile vertical)
---
JSON Output Format
The AI analysis must return JSON in this format:
{
"clips": [
{
"clip_number": 1,
"start_time": 5.2,
"end_time": 32.8,
"duration": 27.6,
"title": "Hook: AI Creates Apple Shortcuts",
"description": "Opens with bold claim and immediate demonstration",
"virality_score": 9,
"virality_factors": ["strong_hook", "visual_demo", "trending_topic"],
"suggested_caption": "🤯 I made AI create Apple Shortcuts!",
"content_type": "hook",
"target_audience": "iOS users, automation enthusiasts",
"key_moments": ["0:05 - Bold claim", "0:15 - First demo"]
}
],
"video_summary": "Brief summary of video content",
"overall_theme": "AI Automation",
"target_audience": "Tech enthusiasts, developers",
"hashtag_suggestions": ["#AI", "#automation", "#tech"]
}See references/clip_analysis_prompt.md for the complete prompt template.
---
Troubleshooting
Quick Fixes
| Issue | Solution |
|---|---|
| Download fails | brew upgrade yt-dlp |
| Transcription fails | pip install openai-whisper |
| FFmpeg not found | brew install ffmpeg (macOS) / sudo apt install ffmpeg (Linux) |
| Invalid JSON from AI | Remove markdown code blocks (json), ensure all required fields present |
---
Key Functions
In scripts/ai_clip_generator.py:
is_youtube_url(input)- Detect source typeparse_youtube_url(url)- Extract video IDprepare_local_video(path, video_id, output_dir)- Ingest local file inputdownload_video(url, video_id, output_dir)- Download with cookie fallbackstranscribe_video(video_path, output_dir)- Whisper transcriptiongenerate_analysis_prompt(transcript, metadata, output_dir)- Create Claude promptvalidate_clip_recommendations(json_path, duration)- Validate AI outputgenerate_clips(video_path, recommendations, output_dir)- FFmpeg clip generationgenerate_captions_for_clip(...)- Remotion caption generationconvert_whisper_to_captions(...)- Whisper JSON to Remotion formatgenerate_summary_report(...)- Create final report
---
Design Decisions
1. File-based AI integration - Simple, no API costs, easy debugging 2. Interactive analysis step - User reviews prompt and AI output before generation 3. Dual input support - Handles both YouTube and local uploads 4. Whisper base model - Balance of speed and accuracy 5. Dual clip formats - Standard landscape + Instagram 9:16 vertical 6. Remotion for captions - React-based rendering for animated word-by-word captions 7. Caption positioning - Captions positioned within video content (not black bars) for 9:16 letterboxed videos
---
Next Steps After Clip Generation
1. Review clips - Check quality and content 2. Add captions - Use --add-captions flag or video editing software 3. Add music - Use trending audio in Instagram/TikTok editor 4. Post with optimized captions - Use suggestions from SUMMARY.md 5. Track performance - Monitor which clips perform best
# Auto detect text files and perform LF normalization
* text=auto
# ============================================
# AI Video Clipper - Git Ignore Rules
# ============================================
# Downloaded videos and generated clips
downloads/
*.mp4
*.m4a
*.webm
*.mkv
*.avi
*.mov
# Transcription and analysis files
*.json
!package.json
!tsconfig.json
!requirements.txt
*.srt
*.vtt
*.txt
!requirements.txt
# Markdown outputs (generated files)
downloads/**/SUMMARY.md
downloads/**/analysis_request.md
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
# Virtual environments
venv/
env/
ENV/
.venv/
virtualenv/
# Distribution / packaging
dist/
build/
*.egg-info/
.eggs/
# Testing
.pytest_cache/
.coverage
htmlcov/
.tox/
# Logs
logs/
*.log
# OS generated files
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
# IDE and editor files
.vscode/
.idea/
*.swp
*.swo
*~
.project
.settings/
*.sublime-project
*.sublime-workspace
# Claude Code local config (optional - uncomment if you want to ignore)
# .claude/
# Temporary files
tmp/
temp/
*.tmp
*.bak
# FFmpeg temp files
ffmpeg2pass-*.log
# Keep structure with .gitkeep files
!downloads/.gitkeep
!logs/.gitkeep
!prompt_templates/
# Skill package (if you want to version it separately)
# skill-package/
*.skill
# Node.js / Remotion
node_modules/
package-lock.json
.remotion/
AGENTS.md
Project Purpose
vid-clipper converts long-form videos into short, viral-ready social clips (Reels/TikTok) using: 1. video ingest/download, 2. Whisper transcription, 3. AI-based clip recommendation, 4. FFmpeg clip rendering, 5. optional caption workflows.
---
How to Talk to This Project (Natural Language)
AI agents and users should interact using natural language prompts. Here are the recommended phrasings:
For YouTube Videos
"Create viral clips from this YouTube video: https://www.youtube.com/watch?v=VIDEO_ID"
"Turn this YouTube video into Instagram Reels"
"Extract the best highlights from this tutorial for TikTok"
"Make short clips from this YouTube video with animated captions"
For Local/Uploaded Videos
"Create short clips from this video file: /path/to/video.mp4"
"Make Reels from my uploaded video"
With Additional Options
"Create 5 viral clips from this video, skip downloading since I already have it"
"Extract highlights from this YouTube video with scaling caption style"
---
Primary Skill To Use
Use the universal-video-clipper skill for all video clipping tasks. It handles both YouTube URLs and local video files.
Skill file:
/Users/cypher/.codex/skills/universal-video-clipper/SKILL.md
---
Standard Operating Workflow
A) Any Video Source (YouTube or Local)
Run:
python3 /Users/cypher/Public/code/vid-clipper/ai_clip_generator.py "<youtube-url-or-local-path>"Examples:
# YouTube
python3 /Users/cypher/Public/code/vid-clipper/ai_clip_generator.py "https://www.youtube.com/watch?v=VIDEO_ID"
# Local file
python3 /Users/cypher/Public/code/vid-clipper/ai_clip_generator.py "/path/to/video.mp4"C) Interactive Analysis Step (Required)
The script pauses after generating analysis_request.md and waits for:
clip_recommendations.jsonin the video output folder.
Agent must: 1. Read transcript and metadata. 2. Produce valid JSON recommendations (3-7 clips, timestamps within bounds). 3. Save JSON to the expected path. 4. Continue the paused run (press Enter programmatically).
---
Dependencies
Verify before major runs:
yt-dlp --version
whisper --help
ffmpeg -versionInstall if needed:
pip install -r /Users/cypher/Public/code/vid-clipper/requirements.txt
brew install ffmpeg---
Output Contract
For each processed video, expected artifacts live under:
/Users/cypher/Public/code/vid-clipper/downloads/<video_id>/
Required files:
original.*(downloaded/copied source)original.json(transcript)metadata.jsonanalysis_request.mdclip_recommendations.jsonSUMMARY.mdclips/*.mp4(standard +_instagram.mp4variants)
---
JSON Recommendation Requirements
clip_recommendations.json must contain:
- top-level
clipsarray, - each clip with:
start_time,end_time,title,description, - non-overlapping, valid ranges,
- timestamp boundaries inside source duration,
- recommended duration target: 15-60 seconds.
---
Project-Specific Guardrails
1. Do not claim success without verifying rendered files exist. 2. If transcription succeeds but no JSON transcript appears, inspect Whisper output naming/path behavior. 3. For silent videos (no audio stream), generate a placeholder transcript and continue with visual-only recommendations. 4. Keep clip titles filesystem-safe and concise. 5. Preserve existing user changes in unrelated files.
---
Fast Validation Checklist
After each run:
find /Users/cypher/Public/code/vid-clipper/downloads/<video_id> -maxdepth 2 -type f | sortConfirm:
- summary exists,
- recommendations JSON exists,
- expected number of clip files were generated,
- instagram variants exist.
---
Troubleshooting Shortcuts
- Download issues:
brew upgrade yt-dlp - Whisper missing:
pip install openai-whisper - FFmpeg missing:
brew install ffmpeg - Invalid recommendation JSON: remove markdown fences and revalidate required fields.
---
Definition of Done
A task is complete when: 1. Pipeline reaches final success banner, 2. clips are rendered and discoverable on disk, 3. SUMMARY.md is generated, 4. output paths are reported clearly to the user.
#!/usr/bin/env python3
"""
AI-Powered Video Clipper
Automatically identifies and creates viral-worthy clips from YouTube videos or local files using Claude AI.
Usage:
python ai_clip_generator.py <youtube_url_or_local_file>
Examples:
python ai_clip_generator.py "https://www.youtube.com/watch?v=QE_Nt5dMLHI"
python ai_clip_generator.py "/path/to/video.mp4"
"""
import subprocess
import os
import sys
import json
import re
import shutil
from pathlib import Path
from datetime import datetime
import argparse
def parse_youtube_url(url):
"""Extract video ID from YouTube URL"""
patterns = [
r'(?:youtube\.com\/watch\?v=|youtu\.be\/)([^&\n?#]+)',
r'youtube\.com\/embed\/([^&\n?#]+)',
r'youtube\.com\/v\/([^&\n?#]+)'
]
for pattern in patterns:
match = re.search(pattern, url)
if match:
return match.group(1)
raise ValueError(f"Could not extract video ID from URL: {url}")
def is_youtube_url(value):
"""Check if input looks like a YouTube URL"""
return bool(re.search(r"(youtube\.com|youtu\.be)", str(value)))
def slugify(value):
"""Create filesystem-safe slug"""
safe = re.sub(r"[^\w\s-]", "", value).strip().lower()
safe = re.sub(r"[-\s]+", "_", safe)
return safe or "video"
def get_local_video_metadata(video_path, source_id):
"""Extract metadata from local video using ffprobe"""
cmd = [
"ffprobe",
"-v", "error",
"-show_format",
"-show_streams",
"-of", "json",
str(video_path)
]
try:
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
info = json.loads(result.stdout)
except Exception:
info = {}
duration = 0
fmt = info.get("format", {})
if fmt.get("duration"):
try:
duration = int(float(fmt["duration"]))
except Exception:
duration = 0
return {
"video_id": source_id,
"title": video_path.stem,
"duration": duration,
"uploader": "Local Upload",
"upload_date": datetime.now().strftime("%Y%m%d"),
"view_count": 0,
"description": f"Local video file: {video_path.name}",
"source_type": "local",
"source_path": str(video_path.resolve())
}
def has_audio_stream(video_path):
"""Return True if ffprobe detects at least one audio stream."""
cmd = [
"ffprobe",
"-v", "error",
"-select_streams", "a",
"-show_entries", "stream=index",
"-of", "json",
str(video_path),
]
try:
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
data = json.loads(result.stdout or "{}")
return len(data.get("streams", [])) > 0
except Exception:
return False
def prepare_local_video(source_path, source_id, output_dir):
"""Copy local video into output directory and generate metadata"""
print("\n📥 Preparing local video...")
local_path = Path(source_path).expanduser().resolve()
if not local_path.exists() or not local_path.is_file():
print(f"\n❌ Local video not found: {local_path}")
sys.exit(1)
suffix = local_path.suffix if local_path.suffix else ".mp4"
target_path = output_dir / f"original{suffix}"
shutil.copy2(local_path, target_path)
metadata = get_local_video_metadata(target_path, source_id)
metadata_path = output_dir / "metadata.json"
with open(metadata_path, "w") as f:
json.dump(metadata, f, indent=2)
print(f" ✓ Copied: {local_path.name}")
print(f" ✓ Duration: {metadata.get('duration', 0)}s")
return target_path, metadata
def download_video(url, video_id, output_dir):
"""Download video using yt-dlp with multiple fallback methods"""
print("\n📥 Downloading video...")
print(f" Video ID: {video_id}")
video_path = output_dir / "original.mp4"
metadata_path = output_dir / "metadata.json"
# Try multiple download methods (most reliable first)
download_methods = [
{
"name": "Android client (most reliable)",
"cmd": [
"yt-dlp",
"--extractor-args", "youtube:player_client=android",
"-f", "best[ext=mp4]/best",
"--write-info-json",
"-o", str(video_path),
url
]
},
{
"name": "Chrome cookies",
"cmd": [
"yt-dlp",
"--cookies-from-browser", "chrome",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
"--write-info-json",
"-o", str(video_path),
url
]
},
{
"name": "Firefox cookies",
"cmd": [
"yt-dlp",
"--cookies-from-browser", "firefox",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
"--write-info-json",
"-o", str(video_path),
url
]
},
{
"name": "No cookies",
"cmd": [
"yt-dlp",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
"--write-info-json",
"-o", str(video_path),
url
]
}
]
for method in download_methods:
print(f"\n Trying: {method['name']}...")
try:
result = subprocess.run(
method['cmd'],
capture_output=True,
text=True,
check=True
)
print(f" ✓ Download successful!")
# Read metadata
info_json_path = output_dir / "original.info.json"
if info_json_path.exists():
with open(info_json_path, 'r') as f:
metadata = json.load(f)
# Extract relevant metadata
clean_metadata = {
"video_id": video_id,
"title": metadata.get("title", "Unknown"),
"duration": metadata.get("duration", 0),
"uploader": metadata.get("uploader", "Unknown"),
"upload_date": metadata.get("upload_date", "Unknown"),
"view_count": metadata.get("view_count", 0),
"description": metadata.get("description", "")[:500] # First 500 chars
}
with open(metadata_path, 'w') as f:
json.dump(clean_metadata, f, indent=2)
print(f" Title: {clean_metadata['title']}")
print(f" Duration: {clean_metadata['duration']}s")
return video_path, clean_metadata
return video_path, {"video_id": video_id}
except subprocess.CalledProcessError as e:
print(f" ✗ Failed: {e.stderr[:200]}")
continue
except FileNotFoundError:
print(f" ✗ yt-dlp not found. Install with: pip install yt-dlp")
sys.exit(1)
print("\n❌ All download methods failed!")
print("\nManual download instructions:")
print(f"1. Visit: {url}")
print(f"2. Download the video manually")
print(f"3. Save as: {video_path}")
print(f"4. Run this script again")
sys.exit(1)
def transcribe_video(video_path, output_dir):
"""Transcribe video using Whisper"""
print("\n🎙️ Transcribing video...")
video_path = Path(video_path).resolve()
output_dir = Path(output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
transcript_json = output_dir / "original.json"
# Check if transcription already exists
if transcript_json.exists():
print(" ✓ Found existing transcription")
with open(transcript_json, 'r') as f:
return json.load(f)
# Check if whisper is available
try:
subprocess.run(["whisper", "--help"], capture_output=True, check=True)
except FileNotFoundError:
print(" ✗ Whisper not found. Install with: pip install openai-whisper")
sys.exit(1)
# Handle silent videos gracefully (e.g., screen recordings with no audio track)
if not has_audio_stream(video_path):
print(" ⚠️ No audio stream detected; creating placeholder transcript")
fallback = {
"text": "[No audio track detected]",
"segments": [
{
"start": 0.0,
"end": 0.0,
"text": "[No audio track detected in this video. Use visual moments for clip selection.]",
}
],
}
with open(transcript_json, "w") as f:
json.dump(fallback, f, indent=2)
return fallback
# Try base model first, fallback to tiny if it fails
models = ["base", "tiny"]
for model in models:
print(f" Trying Whisper model: {model}...")
try:
cmd = [
"whisper",
str(video_path),
"--model", model,
"--output_format", "json",
"--output_dir", str(output_dir)
]
subprocess.run(
cmd,
capture_output=True,
text=True,
check=True
)
print(f" ✓ Transcription complete with {model} model")
# Whisper can emit either <stem>.json or "original.json" depending on input path handling.
stem_transcript_json = output_dir / f"{video_path.stem}.json"
if not transcript_json.exists() and stem_transcript_json.exists():
transcript_json = stem_transcript_json
# Load and return the transcript
if transcript_json.exists():
with open(transcript_json, 'r') as f:
return json.load(f)
else:
print(f" ✗ Warning: Transcript file not created")
if model == models[-1]:
print(f"\n❌ Transcription failed: Output file not found")
sys.exit(1)
continue
except subprocess.CalledProcessError as e:
print(f" ✗ Failed with {model} model")
if model == models[-1]: # Last model
print(f"\n❌ Transcription failed: {e.stderr[:200]}")
sys.exit(1)
continue
def format_transcript_for_claude(transcript_data):
"""Format Whisper transcript with timestamps for Claude analysis"""
formatted_lines = []
for segment in transcript_data.get("segments", []):
timestamp = segment.get("start", 0)
text = segment.get("text", "").strip()
# Format as [MM:SS] text
mins = int(timestamp // 60)
secs = int(timestamp % 60)
formatted_lines.append(f"[{mins:02d}:{secs:02d}] {text}")
return "\n".join(formatted_lines)
def generate_analysis_prompt(transcript_data, metadata, output_dir):
"""Generate Claude analysis prompt from template"""
print("\n📝 Generating Claude analysis prompt...")
template_path = Path(__file__).parent / "prompt_templates" / "clip_analysis_prompt.md"
if not template_path.exists():
print(f" ✗ Template not found: {template_path}")
sys.exit(1)
with open(template_path, 'r') as f:
template = f.read()
# Format metadata
metadata_str = f"""
- **Title**: {metadata.get('title', 'Unknown')}
- **Duration**: {metadata.get('duration', 0)} seconds
- **Uploader**: {metadata.get('uploader', 'Unknown')}
- **Video ID**: {metadata.get('video_id', 'Unknown')}
"""
# Format transcript
transcript_str = format_transcript_for_claude(transcript_data)
# Replace placeholders
prompt = template.replace("{metadata}", metadata_str.strip())
prompt = prompt.replace("{transcript}", transcript_str)
# Save prompt
prompt_path = output_dir / "analysis_request.md"
with open(prompt_path, 'w') as f:
f.write(prompt)
print(f" ✓ Prompt saved to: {prompt_path}")
return prompt_path
def invoke_claude_analysis(prompt_path, output_path):
"""Invoke Claude Code for analysis (interactive mode)"""
# Check if recommendations already exist
if output_path.exists():
print("\n✓ Found existing clip recommendations")
return
print("\n🤖 Claude AI Analysis Required")
print("=" * 60)
print("\nNext steps:")
print(f"1. Review the analysis prompt at:")
print(f" {prompt_path}")
print(f"\n2. Run Claude Code on this prompt to generate clip recommendations")
print(f"\n3. Save Claude's JSON output to:")
print(f" {output_path}")
print(f"\n4. Press Enter when the JSON file is ready...")
print("=" * 60)
input("\nPress Enter to continue...")
# Validate output exists
if not output_path.exists():
print(f"\n❌ Output file not found: {output_path}")
print("Please create the file and run again.")
sys.exit(1)
print(" ✓ Found clip recommendations file")
def validate_clip_recommendations(json_path, video_duration):
"""Validate Claude's clip recommendations"""
print("\n✅ Validating clip recommendations...")
try:
with open(json_path, 'r') as f:
data = json.load(f)
except json.JSONDecodeError as e:
print(f" ✗ Invalid JSON: {e}")
sys.exit(1)
if "clips" not in data:
print(" ✗ Missing 'clips' field in JSON")
sys.exit(1)
clips = data["clips"]
if not clips or len(clips) == 0:
print(" ✗ No clips found in recommendations")
sys.exit(1)
print(f" Found {len(clips)} clips")
# Validate each clip
for i, clip in enumerate(clips, 1):
required_fields = ["start_time", "end_time", "title", "description"]
for field in required_fields:
if field not in clip:
print(f" ✗ Clip {i} missing required field: {field}")
sys.exit(1)
start = clip["start_time"]
end = clip["end_time"]
# Validate timestamps
if start >= end:
print(f" ✗ Clip {i}: start_time ({start}) >= end_time ({end})")
sys.exit(1)
duration = end - start
if duration < 15 or duration > 60:
print(f" ⚠️ Clip {i}: duration {duration:.1f}s outside 15-60s range")
if end > video_duration:
print(f" ✗ Clip {i}: end_time ({end}) exceeds video duration ({video_duration})")
sys.exit(1)
print(" ✓ All clips validated successfully")
return data
def format_time(seconds):
"""Convert seconds to MM:SS format"""
mins, secs = divmod(int(seconds), 60)
return f"{mins}:{secs:02d}"
def run_ffmpeg(cmd, description="Running ffmpeg"):
"""Run an ffmpeg command with error handling"""
try:
result = subprocess.run(
cmd,
capture_output=True,
text=True
)
# Check return code explicitly instead of check=True
# ffmpeg may output warnings to stderr but still succeed
if result.returncode != 0:
# Only print stderr on actual failure
stderr_clean = result.stderr.strip()
if stderr_clean:
# Extract just the error line if possible
error_lines = [l for l in stderr_clean.split('\n') if 'error' in l.lower() or 'failed' in l.lower()]
if error_lines:
print(f" ✗ Error: {error_lines[0][:200]}")
else:
print(f" ✗ Error: {stderr_clean[:200]}")
else:
print(f" ✗ FFmpeg failed with exit code {result.returncode}")
return False
return True
except Exception as e:
print(f" ✗ Error running ffmpeg: {e}")
return False
def generate_clips(video_path, recommendations, output_dir, add_captions=False, caption_style="background", caption_color="#FFFF00", whisper_json_path=None):
"""Generate video clips based on recommendations"""
print("\n✂️ Generating clips...")
clips_dir = output_dir / "clips"
clips_dir.mkdir(exist_ok=True)
clips = recommendations["clips"]
success_count = 0
generated_files = []
# Check Remotion availability if captions requested
if add_captions:
available, error = check_remotion_available()
if not available:
print(f"\n ⚠️ Captions disabled: {error}")
add_captions = False
for clip in clips:
clip_num = clip.get("clip_number", 0)
title = clip.get("title", f"clip_{clip_num}")
start_time = clip.get("start_time", 0)
end_time = clip.get("end_time", 0)
# Create safe filename
safe_title = re.sub(r'[^\w\s-]', '', title)
safe_title = re.sub(r'[-\s]+', '_', safe_title)
filename = f"clip_{clip_num:03d}_{safe_title[:30]}"
print(f"\n 📹 Creating clip {clip_num}: {title}")
print(f" Time: {format_time(start_time)} - {format_time(end_time)}")
output_path = clips_dir / f"{filename}.mp4"
instagram_path = clips_dir / f"{filename}_instagram.mp4"
# Create standard clip
cmd = [
"ffmpeg", "-i", str(video_path),
"-ss", str(start_time),
"-to", str(end_time),
"-c:v", "libx264", "-c:a", "aac",
"-preset", "fast", "-crf", "23",
"-y", str(output_path)
]
if not run_ffmpeg(cmd, f"Creating clip {clip_num}"):
continue
print(f" ✓ Created {filename}.mp4")
generated_files.append(output_path)
# Create Instagram vertical version (9:16 aspect ratio)
cmd_instagram = [
"ffmpeg", "-i", str(output_path),
"-vf", "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:-1:-1:color=black",
"-c:v", "libx264", "-c:a", "aac",
"-preset", "fast", "-crf", "23",
"-y", str(instagram_path)
]
if run_ffmpeg(cmd_instagram, f"Creating Instagram version {clip_num}"):
print(f" ✓ Created {filename}_instagram.mp4 (9:16)")
generated_files.append(instagram_path)
success_count += 1
# Generate captioned versions if requested
if add_captions and whisper_json_path:
captioned_path = clips_dir / f"{filename}_captioned.mp4"
captioned_instagram_path = clips_dir / f"{filename}_instagram_captioned.mp4"
# Generate captioned standard version
if generate_captions_for_clip(
output_path, whisper_json_path,
start_time, end_time,
captioned_path, caption_style, caption_color
):
generated_files.append(captioned_path)
# Generate captioned Instagram version
cmd_captioned_ig = [
"ffmpeg", "-i", str(captioned_path),
"-vf", "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:-1:-1:color=black",
"-c:v", "libx264", "-c:a", "aac",
"-preset", "fast", "-crf", "23",
"-y", str(captioned_instagram_path)
]
if run_ffmpeg(cmd_captioned_ig, f"Creating captioned Instagram version {clip_num}"):
print(f" ✓ Created {filename}_instagram_captioned.mp4")
generated_files.append(captioned_instagram_path)
print(f"\n ✅ Successfully created {success_count}/{len(clips)} clips")
return generated_files
def generate_summary_report(output_dir, recommendations, generated_files, metadata):
"""Generate a summary report"""
print("\n📊 Generating summary report...")
report_path = output_dir / "SUMMARY.md"
report = f"""# Video Clipping Summary Report
## Video Information
- **Title**: {metadata.get('title', 'Unknown')}
- **Video ID**: {metadata.get('video_id', 'Unknown')}
- **Duration**: {metadata.get('duration', 0)} seconds
- **Generated**: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
## Analysis Results
- **Total Clips Recommended**: {len(recommendations.get('clips', []))}
- **Overall Theme**: {recommendations.get('overall_theme', 'N/A')}
- **Target Audience**: {recommendations.get('target_audience', 'N/A')}
- **Hashtags**: {', '.join(recommendations.get('hashtag_suggestions', []))}
## Generated Clips
"""
for i, clip in enumerate(recommendations.get('clips', []), 1):
report += f"""### Clip {i}: {clip.get('title', 'Untitled')}
- **Time**: {format_time(clip.get('start_time', 0))} - {format_time(clip.get('end_time', 0))}
- **Duration**: {clip.get('duration', 0):.1f}s
- **Virality Score**: {clip.get('virality_score', 0)}/10
- **Description**: {clip.get('description', 'N/A')}
- **Suggested Caption**: {clip.get('suggested_caption', 'N/A')}
- **Content Type**: {clip.get('content_type', 'N/A')}
"""
report += f"""## Output Files
Generated {len(generated_files)} files in `clips/` directory:
"""
for filepath in sorted(generated_files):
size_mb = filepath.stat().st_size / (1024 * 1024)
report += f"- `{filepath.name}` ({size_mb:.1f} MB)\n"
report += """
## Next Steps
1. **Review each clip** for quality and content
2. **Add captions** using Remotion or editing software (Phase 2)
3. **Add trending audio/music** in Instagram/TikTok editor
4. **Post with optimized captions** and hashtags from recommendations
5. **Track performance** and iterate on successful patterns
---
Generated by AI Video Clipper
"""
with open(report_path, 'w') as f:
f.write(report)
print(f" ✓ Report saved to: {report_path}")
return report_path
def check_remotion_available():
"""Check if Remotion is installed and available"""
remotion_dir = Path(__file__).parent / "remotion-captions"
if not remotion_dir.exists():
return False, "Remotion project not found"
node_modules = remotion_dir / "node_modules"
if not node_modules.exists():
return False, "Remotion dependencies not installed. Run: cd remotion-captions && npm install"
return True, None
def convert_whisper_to_captions(whisper_json_path, start_time, end_time):
"""Convert Whisper JSON to Remotion caption format"""
with open(whisper_json_path, 'r') as f:
whisper_data = json.load(f)
captions = []
offset_ms = start_time * 1000
for segment in whisper_data.get("segments", []):
# Skip segments outside clip range
if segment.get("end", 0) < start_time or segment.get("start", 0) > end_time:
continue
# Use word-level timestamps if available
words = segment.get("words", [])
if words:
for word in words:
word_start = word.get("start", 0)
word_end = word.get("end", 0)
# Skip words outside clip range
if word_end < start_time or word_start > end_time:
continue
start_ms = max(0, word_start * 1000 - offset_ms)
end_ms = word_end * 1000 - offset_ms
captions.append({
"text": word.get("word", "").strip(),
"startMs": start_ms,
"endMs": end_ms,
"timestampMs": start_ms,
"confidence": word.get("probability")
})
else:
# Fall back to segment-level - distribute timing across words
text = segment.get("text", "").strip()
words_list = text.split()
seg_start = segment.get("start", 0)
seg_end = segment.get("end", 0)
seg_duration = (seg_end - seg_start) * 1000
if words_list:
word_duration = seg_duration / len(words_list)
for i, word_text in enumerate(words_list):
word_start_ms = seg_start * 1000 + i * word_duration - offset_ms
word_end_ms = word_start_ms + word_duration
if word_start_ms < 0:
continue
captions.append({
"text": word_text,
"startMs": max(0, word_start_ms),
"endMs": word_end_ms,
"timestampMs": max(0, word_start_ms),
"confidence": None
})
return captions
def generate_captions_for_clip(clip_path, whisper_json_path, start_time, end_time, output_path, style="background", accent_color="#FFFF00"):
"""Generate captioned video using Remotion"""
print(f" 🎬 Generating captions with style: {style}...")
remotion_dir = Path(__file__).parent / "remotion-captions"
# Check if Remotion is available
available, error = check_remotion_available()
if not available:
print(f" ⚠️ {error}")
return False
# Convert Whisper JSON to captions
captions = convert_whisper_to_captions(whisper_json_path, start_time, end_time)
if not captions:
print(f" ⚠️ No captions found for this time range")
return False
# Calculate duration in frames
fps = 30
duration_frames = int((end_time - start_time) * fps)
# Copy clip to Remotion public folder for serving
import shutil
public_dir = remotion_dir / "public"
public_dir.mkdir(exist_ok=True)
temp_video_name = f"temp_clip_{Path(clip_path).stem}.mp4"
temp_video_path = public_dir / temp_video_name
shutil.copy2(clip_path, temp_video_path)
# Create props JSON for Remotion - use staticFile path
props = {
"videoSrc": temp_video_name, # Remotion will serve from public/
"captions": captions,
"style": style,
"accentColor": accent_color,
"fontFamily": "Inter, system-ui, sans-serif",
"durationInFrames": duration_frames
}
# Write props to temp file
props_file = Path(output_path).parent / f".caption_props_{Path(output_path).stem}.json"
with open(props_file, 'w') as f:
json.dump(props, f)
# Build the render command using Remotion CLI
cmd = [
"npx", "remotion", "render",
"src/index.ts",
"CaptionedClip",
str(Path(output_path).resolve()),
"--props", str(props_file.resolve()),
"--overwrite"
]
try:
result = subprocess.run(
cmd,
cwd=str(remotion_dir),
capture_output=True,
text=True,
timeout=600 # 10 minute timeout
)
# Clean up temp files
if props_file.exists():
props_file.unlink()
if temp_video_path.exists():
temp_video_path.unlink()
if result.returncode != 0:
print(f" ✗ Caption render failed: {result.stderr[:300]}")
return False
print(f" ✓ Captions generated!")
return True
except subprocess.TimeoutExpired:
print(f" ✗ Caption render timed out")
if props_file.exists():
props_file.unlink()
if temp_video_path.exists():
temp_video_path.unlink()
return False
except FileNotFoundError:
print(f" ✗ Node.js/npx not found. Install Node.js 18+")
if props_file.exists():
props_file.unlink()
if temp_video_path.exists():
temp_video_path.unlink()
return False
def main():
parser = argparse.ArgumentParser(
description="AI-Powered Video Clipper - Create viral clips from YouTube URLs or local video files"
)
parser.add_argument("source", help="YouTube URL or local video file path")
parser.add_argument("--skip-download", action="store_true", help="Skip download if video already exists")
parser.add_argument("--skip-transcription", action="store_true", help="Skip transcription if it already exists")
parser.add_argument("--add-captions", action="store_true", help="Generate animated captions using Remotion (Phase 2)")
parser.add_argument("--caption-style", choices=["background", "scaling", "colored"], default="background",
help="Caption animation style (default: background)")
parser.add_argument("--caption-color", default="#FFFF00", help="Accent color for captions (default: #FFFF00)")
args = parser.parse_args()
print("=" * 60)
print(" AI-Powered Video Clipper")
print(" Automated Viral Clip Generation")
print("=" * 60)
# Check for ffmpeg
try:
subprocess.run(["ffmpeg", "-version"], capture_output=True, check=True)
except (subprocess.CalledProcessError, FileNotFoundError):
print("\n❌ ffmpeg not found. Install with: brew install ffmpeg")
sys.exit(1)
try:
source = args.source
source_type = "youtube" if is_youtube_url(source) else "local"
if source_type == "youtube":
video_id = parse_youtube_url(source)
else:
local_path = Path(source).expanduser().resolve()
if not local_path.exists():
print(f"\n❌ Local file not found: {local_path}")
sys.exit(1)
video_id = f"local_{slugify(local_path.stem)}_{int(local_path.stat().st_mtime)}"
print(f"\n✓ Source type: {source_type}")
print(f"✓ Video ID: {video_id}")
# Setup directories
downloads_dir = Path("downloads")
downloads_dir.mkdir(exist_ok=True)
output_dir = downloads_dir / video_id
output_dir.mkdir(exist_ok=True)
print(f"✓ Output directory: {output_dir}")
# Download or copy video (look for video files only)
video_extensions = {'.mp4', '.mov', '.mkv', '.avi', '.webm'}
existing_video = next((p for p in output_dir.glob("original.*") if p.is_file() and p.suffix.lower() in video_extensions), None)
if source_type == "youtube":
video_path = output_dir / "original.mp4"
if args.skip_download and existing_video:
video_path = existing_video
print(f"\n✓ Using existing video: {video_path}")
metadata_path = output_dir / "metadata.json"
if metadata_path.exists():
with open(metadata_path, 'r') as f:
metadata = json.load(f)
else:
metadata = {"video_id": video_id, "source_type": "youtube"}
else:
video_path, metadata = download_video(source, video_id, output_dir)
metadata["source_type"] = "youtube"
else:
if args.skip_download and existing_video:
video_path = existing_video
print(f"\n✓ Using existing local video: {video_path}")
metadata_path = output_dir / "metadata.json"
if metadata_path.exists():
with open(metadata_path, 'r') as f:
metadata = json.load(f)
else:
metadata = {"video_id": video_id, "source_type": "local"}
else:
video_path, metadata = prepare_local_video(source, video_id, output_dir)
# Transcribe video
transcript_json = output_dir / "original.json"
if args.skip_transcription and transcript_json.exists():
print(f"\n✓ Using existing transcription")
with open(transcript_json, 'r') as f:
transcript_data = json.load(f)
else:
transcript_data = transcribe_video(video_path, output_dir)
# Generate Claude prompt
prompt_path = generate_analysis_prompt(transcript_data, metadata, output_dir)
# Invoke Claude analysis (interactive)
recommendations_path = output_dir / "clip_recommendations.json"
invoke_claude_analysis(prompt_path, recommendations_path)
# Validate recommendations
recommendations = validate_clip_recommendations(recommendations_path, metadata.get("duration", 999999))
# Generate clips
generated_files = generate_clips(
video_path, recommendations, output_dir,
add_captions=args.add_captions,
caption_style=args.caption_style,
caption_color=args.caption_color,
whisper_json_path=transcript_json
)
# Generate summary report
report_path = generate_summary_report(output_dir, recommendations, generated_files, metadata)
print("\n" + "=" * 60)
print(" ✅ Video Clipping Complete!")
print("=" * 60)
print(f"\n📁 All files saved to: {output_dir}")
print(f"📊 Summary report: {report_path}")
print(f"🎬 Generated {len(generated_files)} clip files")
print("\n🎯 Next Steps:")
print("1. Review clips in the clips/ directory")
print("2. Use suggested captions from SUMMARY.md")
if not args.add_captions:
print("3. Generate captions with: --add-captions --caption-style [background|scaling|colored]")
print("4. Post to Instagram/TikTok with recommended hashtags")
except KeyboardInterrupt:
print("\n\n❌ Cancelled by user")
sys.exit(1)
except Exception as e:
print(f"\n❌ Error: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()
How to Turn Any Video Into Viral Shorts (Using Claude Code)
literally just drop a youtube link or video file into claude and get instagram-ready clips in minutes
---
what you need
- claude code (the desktop app)
- this skill file:
universal-video-clipper.skill
that's it. claude handles the rest.
---
setup (2 minutes)
step 1: download the skill file from the github repo
step 2: open claude code, paste this:
install this skill: universal-video-clipper.skillclaude will load it up. done.
---
how to use it
just paste a link or file path and ask for clips:
"make viral clips from this: https://youtube.com/watch?v=..."
"turn this into reels: ~/downloads/my-video.mp4"
"extract highlights from this tutorial for tiktok: [url]"
that's literally it. claude will:
- download the video
- transcribe it
- find the best moments
- cut 3-7 clips (15-60 seconds each)
- make instagram 9:16 versions
- write captions and hashtags for you
---
pro tip: animated captions
want that capcut-style word-by-word caption effect? just say:
"make clips with animated captions from: [url]"
you'll get clips with text that bounces along with the audio.
---
what you get
claude creates a folder with everything:
downloads/video-id/
├── clips/
│ ├── clip_001_hook.mp4 # original format
│ ├── clip_001_hook_instagram.mp4 # 9:16 vertical
│ └── clip_001_hook_captioned.mp4 # with animated text
└── SUMMARY.md # captions & hashtags ready to copyopen SUMMARY.md - it has suggested captions with emojis, hashtags, and virality scores for each clip. just copy and post.
---
the one thing to know
at one point claude will pause and say "press enter to continue"
this is normal. claude is analyzing the transcript to pick the best clips. just hit enter and let it cook.
---
example workflow
you: "create viral clips from this: https://youtu.be/dQw4w9WgXcQ"
claude: downloads, transcribes, analyzes, cuts clips
claude: "done! created 5 clips in downloads/dQw4w9WgXcQ/clips/"
you: copy captions from SUMMARY.md → paste into instagram → post
---
try it on any video. works with youtube links or files on your computer.
Universal Video Clipper Skill - Installation Guide
How to Use This Skill
Once installed, talk to your AI assistant using natural language:
For YouTube Videos
"Create viral clips from this YouTube video: https://www.youtube.com/watch?v=yKeWqGkCaRQ"
"Turn my video into Instagram Reels"
"Extract the best moments from this tutorial"
For Local Video Files
"Create clips from this video file: /path/to/video.mp4"
"Make Reels from my uploaded video"
The AI will automatically use this skill to: 1. Accept YouTube URLs or local video files 2. Download/copy and transcribe audio with timestamps 3. Analyze the content for viral moments 4. Generate optimized clips (standard + Instagram 9:16) 5. Provide ready-to-use captions and hashtags
---
What Is This Skill?
The Universal Video Clipper skill is a packaged, reusable AI skill that transforms any long-form video (YouTube or local file) into viral-worthy short clips optimized for Instagram Reels and TikTok.
What You Get
For each video processed:
- 3-7 viral-worthy clips (15-60 seconds each)
- Dual formats - Standard + Instagram 9:16 vertical
- Optional animated captions - CapCut-style word highlighting
- Summary report with virality scores, captions, and hashtags
Workflow Overview
YouTube URL or Local Video File
↓
Download video (yt-dlp) OR Copy local file
↓
Transcribe audio (Whisper with timestamps)
↓
Generate AI analysis prompt
↓
[AI: Run analysis on the prompt]
↓
Generate clips (FFmpeg)
↓
✨ Viral-ready clips + captions + hashtags---
Installation
Option 1: Import the Skill (Recommended)
1. Locate the skill file: universal-video-clipper.skill 2. Import into Claude Code using the skills manager 3. The skill will be available in all future AI assistant sessions
Option 2: Manual Installation
# Extract the skill package
unzip universal-video-clipper.skill -d ~/.claude/skills/
# Verify installation
ls ~/.claude/skills/universal-video-clipper/---
What's Included
The skill package contains:
universal-video-clipper/
├── SKILL.md # Main skill instructions
├── scripts/
│ ├── ai_clip_generator.py # Main orchestrator
│ └── remotion-captions/ # Remotion caption renderer project
│ ├── package.json
│ ├── remotion.config.ts
│ └── src/
├── references/
│ ├── clip_analysis_prompt.md # AI prompt template
│ └── troubleshooting.md # Common issues & solutions
└── assets/
└── requirements.txt # Python dependencies---
Prerequisites
Before using the skill, install these dependencies:
# Install Python packages
pip install yt-dlp openai-whisper
# Install ffmpeg (macOS)
brew install ffmpeg
# Install ffmpeg (Linux)
sudo apt install ffmpegVerify installation:
ffmpeg -version
yt-dlp --version---
Example Usage
Natural Language (Recommended)
Simply tell your AI assistant what you want:
User: Create viral clips from https://www.youtube.com/watch?v=yKeWqGkCaRQ
AI: [Automatically uses the universal-video-clipper skill]
✅ Downloaded video: "I Made Claude Into A Cybersecurity Agent"
✅ Transcribed with Whisper
✅ Generated AI analysis prompt
🤖 Running AI analysis on the transcript...
✅ Generated 5 viral-worthy clips
✅ Created Instagram 9:16 versions
✅ Summary report with captions ready!---
Advanced Features
Skip Flags
If you already have files processed:
# Skip download if video already exists
python3 scripts/ai_clip_generator.py <url> --skip-download
# Skip transcription if already done
python3 scripts/ai_clip_generator.py <url> --skip-transcription
# Both (for testing different clip selections)
python3 scripts/ai_clip_generator.py <url> --skip-download --skip-transcriptionCustom Clip Selection
You can manually edit the clip_recommendations.json file to:
- Adjust timestamps
- Change clip titles
- Modify suggested captions
- Add/remove clips
Then re-run with --skip-download --skip-transcription to regenerate only the clips.
---
Troubleshooting
Quick Fixes
| Issue | Solution |
|---|---|
| Download fails | brew upgrade yt-dlp |
| Whisper missing | pip install openai-whisper |
| FFmpeg missing | brew install ffmpeg (macOS) / sudo apt install ffmpeg (Linux) |
See references/troubleshooting.md for detailed solutions covering:
- Download failures (403 errors, cookies, updates)
- Transcription issues (memory, speed, model selection)
- FFmpeg problems (audio sync, encoding)
- JSON validation errors
- Performance optimization
---
Design Philosophy
This skill balances automation with human oversight:
- ✅ Automated: Download, transcription, clip generation, formatting
- 🤝 Interactive: AI analysis step (you review the prompt and output)
- ✅ Validated: Comprehensive checks ensure quality output
- 🎯 Optimized: Built on proven patterns from viral content
This approach ensures:
- Quality control - You see what AI recommends before generating
- No API costs - File-based analysis (no API charges)
- Easy debugging - All intermediate files are saved
- Flexibility - Edit recommendations before final generation
---
What Makes This Skill Powerful
AI-Powered Clip Selection
The skill uses AI to analyze videos for:
- Strong Hooks (0-3 sec attention grabbers)
- Value Bombs (actionable tips, "aha!" moments)
- Emotional Peaks (excitement, surprise, humor)
- Story Arcs (complete narratives with payoff)
Optimized for Social Media
- Duration: 15-60 seconds (optimal for Reels/TikTok)
- Format: Dual output (landscape + 9:16 vertical)
- Quality: Professional FFmpeg encoding
- Captions: Ready-to-use captions with emojis
- Hashtags: AI-generated hashtag suggestions
Automation Features
- Cookie-based downloads - Handles private/age-restricted videos
- Whisper transcription - Timestamp-accurate audio-to-text
- Fallback mechanisms - Multiple download/transcription methods
- Comprehensive validation - Ensures AI output is valid
- Batch-friendly - Skip flags for efficient re-processing
---
Real-World Example
Input: 3-minute cybersecurity demo video
Output:
- 5 viral-worthy clips (21s, 43s, 33s, 27s, 43s)
- 10 total files (standard + Instagram versions)
- Virality scores: 8/10, 9/10, 8/10, 9/10, 8/10
- Ready-to-use captions with emojis
- Hashtags: #cybersecurity #AI #infosec #pentest #opensource
Time saved: Manual editing (1-2 hours) → AI workflow (10-15 minutes)
---
Next Steps
After generating clips:
1. Review - Check quality and content 2. Edit - Add captions (automated captions optional) 3. Enhance - Add trending music in Instagram/TikTok editor 4. Post - Use suggested captions and hashtags 5. Track - Monitor performance and iterate
---
Future Enhancements (Roadmap)
- ✨ Automated caption generation with Remotion
- ✨ Direct Claude API integration (no manual step)
- ✨ Web UI for clip review and editing
- ✨ Batch processing multiple videos
- ✨ Social media posting integration
- ✨ Background music addition
- ✨ Thumbnail extraction and optimization
---
Support
The skill includes:
- SKILL.md - Complete usage documentation
- references/troubleshooting.md - Common issues and solutions
- references/clip_analysis_prompt.md - AI prompt template
- scripts/ai_clip_generator.py - Well-documented Python code
- scripts/remotion-captions/ - Remotion project used by
--add-captions
All files are included in the skill package and available in context when using the skill.
---
License
Open source - feel free to fork, modify, and share!
AI Video Clip Analysis Task
Objective
Analyze the provided YouTube video transcript and identify 3-7 viral-worthy moments suitable for Instagram Reels and TikTok (9:16 vertical format).
Video Metadata
{metadata}
Video Transcript (with timestamps)
{transcript}
Selection Criteria
Identify clips that have HIGH viral potential based on:
1. Strong Hooks (0-3 seconds)
- Bold claims or surprising statements
- Visual demonstrations that grab attention
- Questions that create curiosity
2. Value Bombs (quick insights)
- Actionable tips or techniques
- "Aha!" moments or revelations
- Problem-solution demonstrations
3. Emotional Peaks
- Excitement, surprise, or humor
- Relatable frustrations solved
- Impressive demonstrations
4. Story Arcs (complete narratives)
- Setup → Demonstration → Payoff
- Before → After transformations
- Challenge → Solution sequences
Constraints
- Duration: 15-60 seconds per clip (optimal: 20-45 seconds)
- Self-contained: Each clip must make sense on its own
- No mid-sentence cuts: Start and end at natural speech boundaries
- Audience: Tech enthusiasts, developers, and early adopters on social media
- Platform: Instagram Reels and TikTok (9:16 vertical, mobile-first)
Output Format
Return ONLY valid JSON in this exact format (no markdown code blocks, no additional text):
{
"clips": [
{
"clip_number": 1,
"start_time": 5.2,
"end_time": 32.8,
"duration": 27.6,
"title": "Hook: AI Creates Apple Shortcuts",
"description": "Opens with bold claim and immediate demonstration of AI creating shortcuts",
"virality_score": 9,
"virality_factors": ["strong_hook", "visual_demo", "trending_topic"],
"suggested_caption": "🤯 I made AI create Apple Shortcuts from scratch!",
"content_type": "hook",
"target_audience": "iOS users, automation enthusiasts",
"key_moments": ["0:05 - Bold claim", "0:15 - First demo", "0:28 - Payoff"]
}
],
"video_summary": "Brief 1-2 sentence summary of the full video content and main topic",
"overall_theme": "Main theme or category (e.g., 'AI Automation', 'Tech Tutorial', 'Product Demo')",
"target_audience": "Primary audience for these clips",
"hashtag_suggestions": ["#apple", "#AI", "#automation", "#shortcuts", "#tech"]
}Field Definitions
- clip_number: Sequential number (1, 2, 3...)
- start_time: Start timestamp in seconds (decimal allowed, e.g., 5.2)
- end_time: End timestamp in seconds
- duration: Calculated duration (end_time - start_time)
- title: Short, punchy title (max 60 characters)
- description: 1-2 sentence description of what happens in the clip
- virality_score: 1-10 rating (8+ recommended for clips worth creating)
- virality_factors: Array of factors from: ["strong_hook", "visual_demo", "trending_topic", "emotional_peak", "value_bomb", "surprise_factor", "relatable_problem", "quick_win", "story_arc"]
- suggested_caption: Ready-to-use Instagram/TikTok caption with emojis
- content_type: One of: "hook", "demo", "tutorial", "value_bomb", "story", "payoff"
- target_audience: Specific audience segment for this clip
- key_moments: Array of 2-4 specific timestamps with brief descriptions
Examples of High-Performing Clips
Example 1: Tech Demo Hook
- Duration: 28 seconds
- Structure: Bold claim (3s) → Visual proof (20s) → Call-to-action (5s)
- Why it works: Immediate demonstration, trending topic, shareable moment
Example 2: Problem-Solution Value Bomb
- Duration: 35 seconds
- Structure: Relatable problem (8s) → Solution demo (22s) → Results (5s)
- Why it works: Addresses pain point, shows clear before/after, actionable
Example 3: Story Arc
- Duration: 45 seconds
- Structure: Setup/challenge (10s) → Process (25s) → Payoff/result (10s)
- Why it works: Complete narrative, emotional journey, satisfying conclusion
Quality Guidelines
- Prioritize clips with visual demonstrations over talking heads
- Favor clips where the speaker's energy/enthusiasm is high
- Avoid clips that require prior context from earlier in the video
- Each clip should have a clear "hook" in the first 3 seconds
- Ensure clips end on a strong note (payoff, punchline, or call-to-action)
Important Notes
1. Return ONLY the JSON object - no markdown formatting, no code blocks, no explanatory text 2. Validate all timestamps are within the video duration 3. Ensure no clips overlap in time 4. Order clips chronologically by start_time 5. Only recommend clips with virality_score >= 7 6. Aim for 3-7 clips total (quality over quantity)
Now analyze the transcript above and generate your recommendations.
AI Video Clipper
Transform long-form YouTube videos or local video files into viral-ready short clips for Instagram Reels and TikTok using AI-powered analysis.
---
Quick Start (Choose One)
Option 1: Use with an AI Assistant (Easiest)
Give the skill file to your AI assistant (Claude Code, Codex, etc.):
"Install this skill: universal-video-clipper.skill"Then simply ask:
"Create viral clips from this YouTube video: https://www.youtube.com/watch?v=VIDEO_ID"
Option 2: Use with Claude Code / Codex
Tell your AI assistant to install this project:
"Install this project into your skill catalog"The AI will automatically use the skill when you ask for video clips.
Option 3: Run Directly (No AI)
python ai_clip_generator.py "https://www.youtube.com/watch?v=VIDEO_ID"
python ai_clip_generator.py "/path/to/video.mp4"---
How to Use This Tool
Once installed (or running directly), use natural language:
For YouTube Videos
"Create viral clips from this YouTube video: https://www.youtube.com/watch?v=VIDEO_ID"
"Turn this YouTube video into Instagram Reels: [URL]"
"Create viral clips from this YouTube video with animated captions: [URL]"
"Extract the best highlights from this tutorial for TikTok"
For Local/Uploaded Videos
"Create short clips from this video file: /path/to/video.mp4"
"Make Reels from my uploaded video"
"Make Reels from my uploaded video with scaling caption style"
With Additional Options
"Create 5 viral clips from this video with animated captions"
"Extract highlights from this YouTube video, skip downloading since I already have it"
---
What This Tool Does
1. Analyzes your video - Downloads (YouTube) or ingests (local files) the source video 2. Transcribes audio - Uses Whisper to create timestamped text transcripts 3. Finds viral moments - AI analyzes the content for engaging clips (hooks, value bombs, demos) 4. Generates clips - Creates 15-60 second clips optimized for social media 5. Formats for platforms - Outputs both standard and 9:16 vertical versions
---
What You Get
For each video processed, you'll receive:
- Standard clips - Original aspect ratio versions
- Instagram versions - 9:16 vertical format (1080x1920) with padding
- Captioned variants - Optional animated word-by-word captions (CapCut-style)
- Summary report - Suggested captions, hashtags, virality scores
Sample Output
downloads/{video_id}/
├── original.mp4 # Source video
├── original.json # Transcript with timestamps
├── metadata.json # Video info
├── analysis_request.md # AI analysis prompt
├── clip_recommendations.json # AI-generated clip suggestions
├── SUMMARY.md # Final report with captions & hashtags
└── clips/
├── clip_001_hook.mp4 # Standard clip
├── clip_001_hook_instagram.mp4 # 9:16 vertical
├── clip_001_hook_captioned.mp4 # With animated captions
└── ...---
Quick Example
YouTube Input
Input: https://www.youtube.com/watch?v=VIDEO_ID
Output: 3-5 ready-to-post clips with:
- Virality scores (1-10)
- Suggested captions with emojis
- Hashtag recommendations
- Standard + Instagram versions
Local File Input
Input: /Users/you/Videos/my_recording.mp4
Output: Same as above - works with any MP4, MOV, or video file on your computer
---
Prerequisites
Before first use, ensure these are installed:
| Tool | Purpose | Install Command |
|---|---|---|
| Python 3.8+ | Runtime | (usually pre-installed) |
| ffmpeg | Video processing | brew install ffmpeg (macOS) / sudo apt install ffmpeg (Linux) |
| yt-dlp | YouTube downloads | pip install yt-dlp |
| whisper | Audio transcription | pip install openai-whisper |
| remotion (optional, captions) | Animated caption rendering | cd remotion-captions && npm install |
Verify installation:
ffmpeg -version
yt-dlp --version
whisper --help
cd remotion-captions && npx remotion --help---
Technical Reference
CLI Command (Alternative to Natural Language)
If you prefer direct script execution:
# YouTube video
python ai_clip_generator.py "https://www.youtube.com/watch?v=VIDEO_ID"
# Local video file (MP4, MOV, etc.)
python ai_clip_generator.py "/path/to/video.mp4"
python ai_clip_generator.py "~/Movies/my_recording.mov"
# With options
python ai_clip_generator.py "URL" --add-captions
python ai_clip_generator.py "/path/to/video.mp4" --add-captions
python ai_clip_generator.py "URL" --skip-downloadCaption Styles
| Style | Description | Best For |
|---|---|---|
background | CapCut-style animated box behind active word | Most content (default) |
scaling | Word pops/bounces when spoken | Energetic, punchy videos |
colored | Active word highlighted in accent color | Professional, clean look |
Skip Flags
# Re-run with existing files
python ai_clip_generator.py "URL" --skip-download
python ai_clip_generator.py "URL" --skip-download --skip-transcription---
Workflow Details
Step 1: Video Ingest
- YouTube URLs: Downloads via yt-dlp (tries Chrome cookies → Firefox → no cookies)
- Local files: Copies to working directory (supports MP4, MOV, MKV, and most video formats)
Step 2: Transcription
- Uses OpenAI Whisper for speech-to-text
- Generates timestamps for each segment
- Falls back from "base" to "tiny" model if needed
Step 3: AI Analysis
The script pauses and waits for AI analysis: 1. Review analysis_request.md (contains transcript + instructions) 2. Run AI on this prompt to generate clip recommendations 3. Save results as clip_recommendations.json 4. Press Enter to continue
Step 4: Clip Generation
- Validates clip timestamps
- Uses FFmpeg to extract segments
- Creates dual formats: standard + Instagram 9:16
Step 5: Summary Report
- Generates
SUMMARY.mdwith captions, hashtags, virality scores
---
JSON Recommendation Format
When AI analyzes the transcript, it should return JSON like:
{
"clips": [
{
"clip_number": 1,
"start_time": 5.2,
"end_time": 32.8,
"duration": 27.6,
"title": "Hook: AI Creates Apple Shortcuts",
"description": "Opens with bold claim and immediate demonstration",
"virality_score": 9,
"virality_factors": ["strong_hook", "visual_demo", "trending_topic"],
"suggested_caption": "🤯 I made AI create Apple Shortcuts!",
"content_type": "hook"
}
],
"video_summary": "Brief summary of video content",
"overall_theme": "AI Automation",
"hashtag_suggestions": ["#AI", "#automation", "#tech"]
}---
Troubleshooting
| Issue | Solution |
|---|---|
| Download fails | brew upgrade yt-dlp or use browser cookies |
| Whisper not found | pip install openai-whisper |
| FFmpeg missing | brew install ffmpeg (macOS) / sudo apt install ffmpeg (Linux) |
| Invalid JSON from AI | Remove markdown code blocks (json), validate required fields |
---
Understanding the Output
Virality Score (1-10)
- 8-10: Highly recommended, strong viral potential
- 6-7: Good, worth posting
- Below 6: Consider adjusting timestamps or skipping
Content Types
- hook - Attention-grabbing opening (0-3 seconds)
- value_bomb - Actionable tips, "aha!" moments
- demo - Technical demonstrations
- story - Complete narrative with payoff
---
Project Overview
This tool is designed for two types of users:
1. AI Agents - Use natural language prompts shown in the "How to Use This Tool" section above 2. Direct users - Run CLI commands shown in "Technical Reference" section
Features
- Smart clip detection based on viral content patterns
- Automated workflow from URL to final clips
- Social media optimization (15-60 second duration, 9:16 format)
- Comprehensive reporting with captions and hashtags
Phase 2 Features (Implemented)
- Animated caption generation using Remotion
- Word-by-word sync with Whisper timestamps
- Three customizable caption styles
---
AI Assistant Integration (Optional)
If you use an AI assistant (like Claude Code), you can import the packaged skill:
`universal-video-clipper.skill` - A packaged skill file (ZIP archive) containing:
- SKILL.md with instructions for the AI
- Reference documentation
- Helper scripts
scripts/remotion-captions/for animated caption rendering
To use it, import universal-video-clipper.skill into your AI assistant's skill manager.
---
License
Open source - feel free to use, modify, and distribute.
---
Ready? Try: "Create viral clips from this YouTube video: https://www.youtube.com/watch?v=QE_Nt5dMLHI"
yt-dlp
openai-whisper
# Note: ffmpeg required (install via homebrew: brew install ffmpeg)
#!/bin/bash
#
# Install Remotion dependencies for animated captions
# Requires: Node.js 18+
#
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
REMOTION_DIR="$PROJECT_DIR/remotion-captions"
echo "========================================"
echo " Remotion Captions Setup"
echo "========================================"
echo ""
# Check Node.js version
check_node() {
if ! command -v node &> /dev/null; then
echo "❌ Node.js not found!"
echo ""
echo "Install Node.js 18+ using one of these methods:"
echo " brew install node # macOS"
echo " nvm install 18 # Using nvm"
echo " fnm install 18 # Using fnm"
exit 1
fi
NODE_VERSION=$(node -v | cut -d'v' -f2 | cut -d'.' -f1)
if [ "$NODE_VERSION" -lt 18 ]; then
echo "❌ Node.js 18+ required (found v$NODE_VERSION)"
echo "Please upgrade Node.js and try again."
exit 1
fi
echo "✓ Node.js $(node -v) detected"
}
# Check npm
check_npm() {
if ! command -v npm &> /dev/null; then
echo "❌ npm not found!"
exit 1
fi
echo "✓ npm $(npm -v) detected"
}
# Install dependencies
install_deps() {
echo ""
echo "Installing Remotion dependencies..."
echo ""
cd "$REMOTION_DIR"
if [ -d "node_modules" ]; then
echo "⚠️ node_modules already exists. Reinstalling..."
rm -rf node_modules
fi
npm install
echo ""
echo "✓ Dependencies installed successfully!"
}
# Verify installation
verify_install() {
echo ""
echo "Verifying installation..."
cd "$REMOTION_DIR"
# Check if remotion CLI works
if npx remotion --version &> /dev/null; then
echo "✓ Remotion CLI: $(npx remotion --version)"
else
echo "❌ Remotion CLI verification failed"
exit 1
fi
echo ""
echo "✅ Installation complete!"
echo ""
echo "========================================"
echo " Usage"
echo "========================================"
echo ""
echo "Generate clips with animated captions:"
echo " python3 ai_clip_generator.py <url> --add-captions"
echo ""
echo "Choose caption style:"
echo " --caption-style background # CapCut style (default)"
echo " --caption-style scaling # Pop/bounce effect"
echo " --caption-style colored # Clean highlight"
echo ""
echo "Preview captions in browser:"
echo " cd remotion-captions && npx remotion studio"
echo ""
}
# Main
main() {
check_node
check_npm
install_deps
verify_install
}
main
yt-dlp
openai-whisper
# Note: ffmpeg required (install via homebrew: brew install ffmpeg)
#!/usr/bin/env python3
"""
AI-Powered Video Clipper
Automatically identifies and creates viral-worthy clips from YouTube videos or local files using Claude AI.
Usage:
python ai_clip_generator.py <youtube_url_or_local_file>
Examples:
python ai_clip_generator.py "https://www.youtube.com/watch?v=QE_Nt5dMLHI"
python ai_clip_generator.py "/path/to/video.mp4"
"""
import subprocess
import os
import sys
import json
import re
import shutil
from pathlib import Path
from datetime import datetime
import argparse
def parse_youtube_url(url):
"""Extract video ID from YouTube URL"""
patterns = [
r'(?:youtube\.com\/watch\?v=|youtu\.be\/)([^&\n?#]+)',
r'youtube\.com\/embed\/([^&\n?#]+)',
r'youtube\.com\/v\/([^&\n?#]+)'
]
for pattern in patterns:
match = re.search(pattern, url)
if match:
return match.group(1)
raise ValueError(f"Could not extract video ID from URL: {url}")
def is_youtube_url(value):
"""Check if input looks like a YouTube URL"""
return bool(re.search(r"(youtube\.com|youtu\.be)", str(value)))
def slugify(value):
"""Create filesystem-safe slug"""
safe = re.sub(r"[^\w\s-]", "", value).strip().lower()
safe = re.sub(r"[-\s]+", "_", safe)
return safe or "video"
def get_local_video_metadata(video_path, source_id):
"""Extract metadata from local video using ffprobe"""
cmd = [
"ffprobe",
"-v", "error",
"-show_format",
"-show_streams",
"-of", "json",
str(video_path)
]
try:
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
info = json.loads(result.stdout)
except Exception:
info = {}
duration = 0
fmt = info.get("format", {})
if fmt.get("duration"):
try:
duration = int(float(fmt["duration"]))
except Exception:
duration = 0
return {
"video_id": source_id,
"title": video_path.stem,
"duration": duration,
"uploader": "Local Upload",
"upload_date": datetime.now().strftime("%Y%m%d"),
"view_count": 0,
"description": f"Local video file: {video_path.name}",
"source_type": "local",
"source_path": str(video_path.resolve())
}
def has_audio_stream(video_path):
"""Return True if ffprobe detects at least one audio stream."""
cmd = [
"ffprobe",
"-v", "error",
"-select_streams", "a",
"-show_entries", "stream=index",
"-of", "json",
str(video_path),
]
try:
result = subprocess.run(cmd, capture_output=True, text=True, check=True)
data = json.loads(result.stdout or "{}")
return len(data.get("streams", [])) > 0
except Exception:
return False
def prepare_local_video(source_path, source_id, output_dir):
"""Copy local video into output directory and generate metadata"""
print("\n📥 Preparing local video...")
local_path = Path(source_path).expanduser().resolve()
if not local_path.exists() or not local_path.is_file():
print(f"\n❌ Local video not found: {local_path}")
sys.exit(1)
suffix = local_path.suffix if local_path.suffix else ".mp4"
target_path = output_dir / f"original{suffix}"
shutil.copy2(local_path, target_path)
metadata = get_local_video_metadata(target_path, source_id)
metadata_path = output_dir / "metadata.json"
with open(metadata_path, "w") as f:
json.dump(metadata, f, indent=2)
print(f" ✓ Copied: {local_path.name}")
print(f" ✓ Duration: {metadata.get('duration', 0)}s")
return target_path, metadata
def download_video(url, video_id, output_dir):
"""Download video using yt-dlp with multiple fallback methods"""
print("\n📥 Downloading video...")
print(f" Video ID: {video_id}")
video_path = output_dir / "original.mp4"
metadata_path = output_dir / "metadata.json"
# Try multiple download methods (most reliable first)
download_methods = [
{
"name": "Android client (most reliable)",
"cmd": [
"yt-dlp",
"--extractor-args", "youtube:player_client=android",
"-f", "best[ext=mp4]/best",
"--write-info-json",
"-o", str(video_path),
url
]
},
{
"name": "Chrome cookies",
"cmd": [
"yt-dlp",
"--cookies-from-browser", "chrome",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
"--write-info-json",
"-o", str(video_path),
url
]
},
{
"name": "Firefox cookies",
"cmd": [
"yt-dlp",
"--cookies-from-browser", "firefox",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
"--write-info-json",
"-o", str(video_path),
url
]
},
{
"name": "No cookies",
"cmd": [
"yt-dlp",
"-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]",
"--write-info-json",
"-o", str(video_path),
url
]
}
]
for method in download_methods:
print(f"\n Trying: {method['name']}...")
try:
result = subprocess.run(
method['cmd'],
capture_output=True,
text=True,
check=True
)
print(f" ✓ Download successful!")
# Read metadata
info_json_path = output_dir / "original.info.json"
if info_json_path.exists():
with open(info_json_path, 'r') as f:
metadata = json.load(f)
# Extract relevant metadata
clean_metadata = {
"video_id": video_id,
"title": metadata.get("title", "Unknown"),
"duration": metadata.get("duration", 0),
"uploader": metadata.get("uploader", "Unknown"),
"upload_date": metadata.get("upload_date", "Unknown"),
"view_count": metadata.get("view_count", 0),
"description": metadata.get("description", "")[:500] # First 500 chars
}
with open(metadata_path, 'w') as f:
json.dump(clean_metadata, f, indent=2)
print(f" Title: {clean_metadata['title']}")
print(f" Duration: {clean_metadata['duration']}s")
return video_path, clean_metadata
return video_path, {"video_id": video_id}
except subprocess.CalledProcessError as e:
print(f" ✗ Failed: {e.stderr[:200]}")
continue
except FileNotFoundError:
print(f" ✗ yt-dlp not found. Install with: pip install yt-dlp")
sys.exit(1)
print("\n❌ All download methods failed!")
print("\nManual download instructions:")
print(f"1. Visit: {url}")
print(f"2. Download the video manually")
print(f"3. Save as: {video_path}")
print(f"4. Run this script again")
sys.exit(1)
def transcribe_video(video_path, output_dir):
"""Transcribe video using Whisper"""
print("\n🎙️ Transcribing video...")
video_path = Path(video_path).resolve()
output_dir = Path(output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True)
transcript_json = output_dir / "original.json"
# Check if transcription already exists
if transcript_json.exists():
print(" ✓ Found existing transcription")
with open(transcript_json, 'r') as f:
return json.load(f)
# Check if whisper is available
try:
subprocess.run(["whisper", "--help"], capture_output=True, check=True)
except FileNotFoundError:
print(" ✗ Whisper not found. Install with: pip install openai-whisper")
sys.exit(1)
# Handle silent videos gracefully (e.g., screen recordings with no audio track)
if not has_audio_stream(video_path):
print(" ⚠️ No audio stream detected; creating placeholder transcript")
fallback = {
"text": "[No audio track detected]",
"segments": [
{
"start": 0.0,
"end": 0.0,
"text": "[No audio track detected in this video. Use visual moments for clip selection.]",
}
],
}
with open(transcript_json, "w") as f:
json.dump(fallback, f, indent=2)
return fallback
# Try base model first, fallback to tiny if it fails
models = ["base", "tiny"]
for model in models:
print(f" Trying Whisper model: {model}...")
try:
cmd = [
"whisper",
str(video_path),
"--model", model,
"--output_format", "json",
"--output_dir", str(output_dir)
]
subprocess.run(
cmd,
capture_output=True,
text=True,
check=True
)
print(f" ✓ Transcription complete with {model} model")
# Whisper can emit either <stem>.json or "original.json" depending on input path handling.
stem_transcript_json = output_dir / f"{video_path.stem}.json"
if not transcript_json.exists() and stem_transcript_json.exists():
transcript_json = stem_transcript_json
# Load and return the transcript
if transcript_json.exists():
with open(transcript_json, 'r') as f:
return json.load(f)
else:
print(f" ✗ Warning: Transcript file not created")
if model == models[-1]:
print(f"\n❌ Transcription failed: Output file not found")
sys.exit(1)
continue
except subprocess.CalledProcessError as e:
print(f" ✗ Failed with {model} model")
if model == models[-1]: # Last model
print(f"\n❌ Transcription failed: {e.stderr[:200]}")
sys.exit(1)
continue
def format_transcript_for_claude(transcript_data):
"""Format Whisper transcript with timestamps for Claude analysis"""
formatted_lines = []
for segment in transcript_data.get("segments", []):
timestamp = segment.get("start", 0)
text = segment.get("text", "").strip()
# Format as [MM:SS] text
mins = int(timestamp // 60)
secs = int(timestamp % 60)
formatted_lines.append(f"[{mins:02d}:{secs:02d}] {text}")
return "\n".join(formatted_lines)
def generate_analysis_prompt(transcript_data, metadata, output_dir):
"""Generate Claude analysis prompt from template"""
print("\n📝 Generating Claude analysis prompt...")
template_path = Path(__file__).parent / "prompt_templates" / "clip_analysis_prompt.md"
if not template_path.exists():
print(f" ✗ Template not found: {template_path}")
sys.exit(1)
with open(template_path, 'r') as f:
template = f.read()
# Format metadata
metadata_str = f"""
- **Title**: {metadata.get('title', 'Unknown')}
- **Duration**: {metadata.get('duration', 0)} seconds
- **Uploader**: {metadata.get('uploader', 'Unknown')}
- **Video ID**: {metadata.get('video_id', 'Unknown')}
"""
# Format transcript
transcript_str = format_transcript_for_claude(transcript_data)
# Replace placeholders
prompt = template.replace("{metadata}", metadata_str.strip())
prompt = prompt.replace("{transcript}", transcript_str)
# Save prompt
prompt_path = output_dir / "analysis_request.md"
with open(prompt_path, 'w') as f:
f.write(prompt)
print(f" ✓ Prompt saved to: {prompt_path}")
return prompt_path
def invoke_claude_analysis(prompt_path, output_path):
"""Invoke Claude Code for analysis (interactive mode)"""
# Check if recommendations already exist
if output_path.exists():
print("\n✓ Found existing clip recommendations")
return
print("\n🤖 Claude AI Analysis Required")
print("=" * 60)
print("\nNext steps:")
print(f"1. Review the analysis prompt at:")
print(f" {prompt_path}")
print(f"\n2. Run Claude Code on this prompt to generate clip recommendations")
print(f"\n3. Save Claude's JSON output to:")
print(f" {output_path}")
print(f"\n4. Press Enter when the JSON file is ready...")
print("=" * 60)
input("\nPress Enter to continue...")
# Validate output exists
if not output_path.exists():
print(f"\n❌ Output file not found: {output_path}")
print("Please create the file and run again.")
sys.exit(1)
print(" ✓ Found clip recommendations file")
def validate_clip_recommendations(json_path, video_duration):
"""Validate Claude's clip recommendations"""
print("\n✅ Validating clip recommendations...")
try:
with open(json_path, 'r') as f:
data = json.load(f)
except json.JSONDecodeError as e:
print(f" ✗ Invalid JSON: {e}")
sys.exit(1)
if "clips" not in data:
print(" ✗ Missing 'clips' field in JSON")
sys.exit(1)
clips = data["clips"]
if not clips or len(clips) == 0:
print(" ✗ No clips found in recommendations")
sys.exit(1)
print(f" Found {len(clips)} clips")
# Validate each clip
for i, clip in enumerate(clips, 1):
required_fields = ["start_time", "end_time", "title", "description"]
for field in required_fields:
if field not in clip:
print(f" ✗ Clip {i} missing required field: {field}")
sys.exit(1)
start = clip["start_time"]
end = clip["end_time"]
# Validate timestamps
if start >= end:
print(f" ✗ Clip {i}: start_time ({start}) >= end_time ({end})")
sys.exit(1)
duration = end - start
if duration < 15 or duration > 60:
print(f" ⚠️ Clip {i}: duration {duration:.1f}s outside 15-60s range")
if end > video_duration:
print(f" ✗ Clip {i}: end_time ({end}) exceeds video duration ({video_duration})")
sys.exit(1)
print(" ✓ All clips validated successfully")
return data
def format_time(seconds):
"""Convert seconds to MM:SS format"""
mins, secs = divmod(int(seconds), 60)
return f"{mins}:{secs:02d}"
def run_ffmpeg(cmd, description="Running ffmpeg"):
"""Run an ffmpeg command with error handling"""
try:
result = subprocess.run(
cmd,
capture_output=True,
text=True
)
# Check return code explicitly instead of check=True
# ffmpeg may output warnings to stderr but still succeed
if result.returncode != 0:
# Only print stderr on actual failure
stderr_clean = result.stderr.strip()
if stderr_clean:
# Extract just the error line if possible
error_lines = [l for l in stderr_clean.split('\n') if 'error' in l.lower() or 'failed' in l.lower()]
if error_lines:
print(f" ✗ Error: {error_lines[0][:200]}")
else:
print(f" ✗ Error: {stderr_clean[:200]}")
else:
print(f" ✗ FFmpeg failed with exit code {result.returncode}")
return False
return True
except Exception as e:
print(f" ✗ Error running ffmpeg: {e}")
return False
def generate_clips(video_path, recommendations, output_dir, add_captions=False, caption_style="background", caption_color="#FFFF00", whisper_json_path=None):
"""Generate video clips based on recommendations"""
print("\n✂️ Generating clips...")
clips_dir = output_dir / "clips"
clips_dir.mkdir(exist_ok=True)
clips = recommendations["clips"]
success_count = 0
generated_files = []
# Check Remotion availability if captions requested
if add_captions:
available, error = check_remotion_available()
if not available:
print(f"\n ⚠️ Captions disabled: {error}")
add_captions = False
for clip in clips:
clip_num = clip.get("clip_number", 0)
title = clip.get("title", f"clip_{clip_num}")
start_time = clip.get("start_time", 0)
end_time = clip.get("end_time", 0)
# Create safe filename
safe_title = re.sub(r'[^\w\s-]', '', title)
safe_title = re.sub(r'[-\s]+', '_', safe_title)
filename = f"clip_{clip_num:03d}_{safe_title[:30]}"
print(f"\n 📹 Creating clip {clip_num}: {title}")
print(f" Time: {format_time(start_time)} - {format_time(end_time)}")
output_path = clips_dir / f"{filename}.mp4"
instagram_path = clips_dir / f"{filename}_instagram.mp4"
# Create standard clip
cmd = [
"ffmpeg", "-i", str(video_path),
"-ss", str(start_time),
"-to", str(end_time),
"-c:v", "libx264", "-c:a", "aac",
"-preset", "fast", "-crf", "23",
"-y", str(output_path)
]
if not run_ffmpeg(cmd, f"Creating clip {clip_num}"):
continue
print(f" ✓ Created {filename}.mp4")
generated_files.append(output_path)
# Create Instagram vertical version (9:16 aspect ratio)
cmd_instagram = [
"ffmpeg", "-i", str(output_path),
"-vf", "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:-1:-1:color=black",
"-c:v", "libx264", "-c:a", "aac",
"-preset", "fast", "-crf", "23",
"-y", str(instagram_path)
]
if run_ffmpeg(cmd_instagram, f"Creating Instagram version {clip_num}"):
print(f" ✓ Created {filename}_instagram.mp4 (9:16)")
generated_files.append(instagram_path)
success_count += 1
# Generate captioned versions if requested
if add_captions and whisper_json_path:
captioned_path = clips_dir / f"{filename}_captioned.mp4"
captioned_instagram_path = clips_dir / f"{filename}_instagram_captioned.mp4"
# Generate captioned standard version
if generate_captions_for_clip(
output_path, whisper_json_path,
start_time, end_time,
captioned_path, caption_style, caption_color
):
generated_files.append(captioned_path)
# Generate captioned Instagram version
cmd_captioned_ig = [
"ffmpeg", "-i", str(captioned_path),
"-vf", "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:-1:-1:color=black",
"-c:v", "libx264", "-c:a", "aac",
"-preset", "fast", "-crf", "23",
"-y", str(captioned_instagram_path)
]
if run_ffmpeg(cmd_captioned_ig, f"Creating captioned Instagram version {clip_num}"):
print(f" ✓ Created {filename}_instagram_captioned.mp4")
generated_files.append(captioned_instagram_path)
print(f"\n ✅ Successfully created {success_count}/{len(clips)} clips")
return generated_files
def generate_summary_report(output_dir, recommendations, generated_files, metadata):
"""Generate a summary report"""
print("\n📊 Generating summary report...")
report_path = output_dir / "SUMMARY.md"
report = f"""# Video Clipping Summary Report
## Video Information
- **Title**: {metadata.get('title', 'Unknown')}
- **Video ID**: {metadata.get('video_id', 'Unknown')}
- **Duration**: {metadata.get('duration', 0)} seconds
- **Generated**: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}
## Analysis Results
- **Total Clips Recommended**: {len(recommendations.get('clips', []))}
- **Overall Theme**: {recommendations.get('overall_theme', 'N/A')}
- **Target Audience**: {recommendations.get('target_audience', 'N/A')}
- **Hashtags**: {', '.join(recommendations.get('hashtag_suggestions', []))}
## Generated Clips
"""
for i, clip in enumerate(recommendations.get('clips', []), 1):
report += f"""### Clip {i}: {clip.get('title', 'Untitled')}
- **Time**: {format_time(clip.get('start_time', 0))} - {format_time(clip.get('end_time', 0))}
- **Duration**: {clip.get('duration', 0):.1f}s
- **Virality Score**: {clip.get('virality_score', 0)}/10
- **Description**: {clip.get('description', 'N/A')}
- **Suggested Caption**: {clip.get('suggested_caption', 'N/A')}
- **Content Type**: {clip.get('content_type', 'N/A')}
"""
report += f"""## Output Files
Generated {len(generated_files)} files in `clips/` directory:
"""
for filepath in sorted(generated_files):
size_mb = filepath.stat().st_size / (1024 * 1024)
report += f"- `{filepath.name}` ({size_mb:.1f} MB)\n"
report += """
## Next Steps
1. **Review each clip** for quality and content
2. **Add captions** using Remotion or editing software (Phase 2)
3. **Add trending audio/music** in Instagram/TikTok editor
4. **Post with optimized captions** and hashtags from recommendations
5. **Track performance** and iterate on successful patterns
---
Generated by AI Video Clipper
"""
with open(report_path, 'w') as f:
f.write(report)
print(f" ✓ Report saved to: {report_path}")
return report_path
def check_remotion_available():
"""Check if Remotion is installed and available"""
remotion_dir = Path(__file__).parent / "remotion-captions"
if not remotion_dir.exists():
return False, "Remotion project not found"
node_modules = remotion_dir / "node_modules"
if not node_modules.exists():
return False, "Remotion dependencies not installed. Run: cd remotion-captions && npm install"
return True, None
def convert_whisper_to_captions(whisper_json_path, start_time, end_time):
"""Convert Whisper JSON to Remotion caption format"""
with open(whisper_json_path, 'r') as f:
whisper_data = json.load(f)
captions = []
offset_ms = start_time * 1000
for segment in whisper_data.get("segments", []):
# Skip segments outside clip range
if segment.get("end", 0) < start_time or segment.get("start", 0) > end_time:
continue
# Use word-level timestamps if available
words = segment.get("words", [])
if words:
for word in words:
word_start = word.get("start", 0)
word_end = word.get("end", 0)
# Skip words outside clip range
if word_end < start_time or word_start > end_time:
continue
start_ms = max(0, word_start * 1000 - offset_ms)
end_ms = word_end * 1000 - offset_ms
captions.append({
"text": word.get("word", "").strip(),
"startMs": start_ms,
"endMs": end_ms,
"timestampMs": start_ms,
"confidence": word.get("probability")
})
else:
# Fall back to segment-level - distribute timing across words
text = segment.get("text", "").strip()
words_list = text.split()
seg_start = segment.get("start", 0)
seg_end = segment.get("end", 0)
seg_duration = (seg_end - seg_start) * 1000
if words_list:
word_duration = seg_duration / len(words_list)
for i, word_text in enumerate(words_list):
word_start_ms = seg_start * 1000 + i * word_duration - offset_ms
word_end_ms = word_start_ms + word_duration
if word_start_ms < 0:
continue
captions.append({
"text": word_text,
"startMs": max(0, word_start_ms),
"endMs": word_end_ms,
"timestampMs": max(0, word_start_ms),
"confidence": None
})
return captions
def generate_captions_for_clip(clip_path, whisper_json_path, start_time, end_time, output_path, style="background", accent_color="#FFFF00"):
"""Generate captioned video using Remotion"""
print(f" 🎬 Generating captions with style: {style}...")
remotion_dir = Path(__file__).parent / "remotion-captions"
# Check if Remotion is available
available, error = check_remotion_available()
if not available:
print(f" ⚠️ {error}")
return False
# Convert Whisper JSON to captions
captions = convert_whisper_to_captions(whisper_json_path, start_time, end_time)
if not captions:
print(f" ⚠️ No captions found for this time range")
return False
# Calculate duration in frames
fps = 30
duration_frames = int((end_time - start_time) * fps)
# Copy clip to Remotion public folder for serving
import shutil
public_dir = remotion_dir / "public"
public_dir.mkdir(exist_ok=True)
temp_video_name = f"temp_clip_{Path(clip_path).stem}.mp4"
temp_video_path = public_dir / temp_video_name
shutil.copy2(clip_path, temp_video_path)
# Create props JSON for Remotion - use staticFile path
props = {
"videoSrc": temp_video_name, # Remotion will serve from public/
"captions": captions,
"style": style,
"accentColor": accent_color,
"fontFamily": "Inter, system-ui, sans-serif",
"durationInFrames": duration_frames
}
# Write props to temp file
props_file = Path(output_path).parent / f".caption_props_{Path(output_path).stem}.json"
with open(props_file, 'w') as f:
json.dump(props, f)
# Build the render command using Remotion CLI
cmd = [
"npx", "remotion", "render",
"src/index.ts",
"CaptionedClip",
str(Path(output_path).resolve()),
"--props", str(props_file.resolve()),
"--overwrite"
]
try:
result = subprocess.run(
cmd,
cwd=str(remotion_dir),
capture_output=True,
text=True,
timeout=600 # 10 minute timeout
)
# Clean up temp files
if props_file.exists():
props_file.unlink()
if temp_video_path.exists():
temp_video_path.unlink()
if result.returncode != 0:
print(f" ✗ Caption render failed: {result.stderr[:300]}")
return False
print(f" ✓ Captions generated!")
return True
except subprocess.TimeoutExpired:
print(f" ✗ Caption render timed out")
if props_file.exists():
props_file.unlink()
if temp_video_path.exists():
temp_video_path.unlink()
return False
except FileNotFoundError:
print(f" ✗ Node.js/npx not found. Install Node.js 18+")
if props_file.exists():
props_file.unlink()
if temp_video_path.exists():
temp_video_path.unlink()
return False
def main():
parser = argparse.ArgumentParser(
description="AI-Powered Video Clipper - Create viral clips from YouTube URLs or local video files"
)
parser.add_argument("source", help="YouTube URL or local video file path")
parser.add_argument("--skip-download", action="store_true", help="Skip download if video already exists")
parser.add_argument("--skip-transcription", action="store_true", help="Skip transcription if it already exists")
parser.add_argument("--add-captions", action="store_true", help="Generate animated captions using Remotion (Phase 2)")
parser.add_argument("--caption-style", choices=["background", "scaling", "colored"], default="background",
help="Caption animation style (default: background)")
parser.add_argument("--caption-color", default="#FFFF00", help="Accent color for captions (default: #FFFF00)")
args = parser.parse_args()
print("=" * 60)
print(" AI-Powered Video Clipper")
print(" Automated Viral Clip Generation")
print("=" * 60)
# Check for ffmpeg
try:
subprocess.run(["ffmpeg", "-version"], capture_output=True, check=True)
except (subprocess.CalledProcessError, FileNotFoundError):
print("\n❌ ffmpeg not found. Install with: brew install ffmpeg")
sys.exit(1)
try:
source = args.source
source_type = "youtube" if is_youtube_url(source) else "local"
if source_type == "youtube":
video_id = parse_youtube_url(source)
else:
local_path = Path(source).expanduser().resolve()
if not local_path.exists():
print(f"\n❌ Local file not found: {local_path}")
sys.exit(1)
video_id = f"local_{slugify(local_path.stem)}_{int(local_path.stat().st_mtime)}"
print(f"\n✓ Source type: {source_type}")
print(f"✓ Video ID: {video_id}")
# Setup directories
downloads_dir = Path("downloads")
downloads_dir.mkdir(exist_ok=True)
output_dir = downloads_dir / video_id
output_dir.mkdir(exist_ok=True)
print(f"✓ Output directory: {output_dir}")
# Download or copy video (look for video files only)
video_extensions = {'.mp4', '.mov', '.mkv', '.avi', '.webm'}
existing_video = next((p for p in output_dir.glob("original.*") if p.is_file() and p.suffix.lower() in video_extensions), None)
if source_type == "youtube":
video_path = output_dir / "original.mp4"
if args.skip_download and existing_video:
video_path = existing_video
print(f"\n✓ Using existing video: {video_path}")
metadata_path = output_dir / "metadata.json"
if metadata_path.exists():
with open(metadata_path, 'r') as f:
metadata = json.load(f)
else:
metadata = {"video_id": video_id, "source_type": "youtube"}
else:
video_path, metadata = download_video(source, video_id, output_dir)
metadata["source_type"] = "youtube"
else:
if args.skip_download and existing_video:
video_path = existing_video
print(f"\n✓ Using existing local video: {video_path}")
metadata_path = output_dir / "metadata.json"
if metadata_path.exists():
with open(metadata_path, 'r') as f:
metadata = json.load(f)
else:
metadata = {"video_id": video_id, "source_type": "local"}
else:
video_path, metadata = prepare_local_video(source, video_id, output_dir)
# Transcribe video
transcript_json = output_dir / "original.json"
if args.skip_transcription and transcript_json.exists():
print(f"\n✓ Using existing transcription")
with open(transcript_json, 'r') as f:
transcript_data = json.load(f)
else:
transcript_data = transcribe_video(video_path, output_dir)
# Generate Claude prompt
prompt_path = generate_analysis_prompt(transcript_data, metadata, output_dir)
# Invoke Claude analysis (interactive)
recommendations_path = output_dir / "clip_recommendations.json"
invoke_claude_analysis(prompt_path, recommendations_path)
# Validate recommendations
recommendations = validate_clip_recommendations(recommendations_path, metadata.get("duration", 999999))
# Generate clips
generated_files = generate_clips(
video_path, recommendations, output_dir,
add_captions=args.add_captions,
caption_style=args.caption_style,
caption_color=args.caption_color,
whisper_json_path=transcript_json
)
# Generate summary report
report_path = generate_summary_report(output_dir, recommendations, generated_files, metadata)
print("\n" + "=" * 60)
print(" ✅ Video Clipping Complete!")
print("=" * 60)
print(f"\n📁 All files saved to: {output_dir}")
print(f"📊 Summary report: {report_path}")
print(f"🎬 Generated {len(generated_files)} clip files")
print("\n🎯 Next Steps:")
print("1. Review clips in the clips/ directory")
print("2. Use suggested captions from SUMMARY.md")
if not args.add_captions:
print("3. Generate captions with: --add-captions --caption-style [background|scaling|colored]")
print("4. Post to Instagram/TikTok with recommended hashtags")
except KeyboardInterrupt:
print("\n\n❌ Cancelled by user")
sys.exit(1)
except Exception as e:
print(f"\n❌ Error: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()
{
"name": "remotion-captions",
"version": "1.0.0",
"description": "Animated captions for video clips using Remotion",
"type": "module",
"scripts": {
"dev": "remotion studio",
"build": "remotion bundle",
"render": "remotion render",
"upgrade": "remotion upgrade"
},
"dependencies": {
"@remotion/bundler": "^4.0.0",
"@remotion/captions": "^4.0.0",
"@remotion/cli": "^4.0.0",
"@remotion/paths": "^4.0.0",
"@remotion/player": "^4.0.0",
"@remotion/renderer": "^4.0.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"remotion": "^4.0.0"
},
"devDependencies": {
"@types/react": "^18.3.3",
"@types/node": "^20.0.0",
"ts-node": "^10.9.0",
"typescript": "^5.5.0",
"prettier": "^3.3.0"
}
}
import { Config } from "@remotion/cli/config";
Config.setVideoImageFormat("jpeg");
Config.setOverwriteOutput(true);
import React from "react";
import {
AbsoluteFill,
OffthreadVideo,
staticFile,
useCurrentFrame,
useVideoConfig,
} from "remotion";
import { RemotionCaption } from "./convert-whisper";
import {
ColoredWords,
ScalingWords,
AnimatedBackground,
CaptionStyle,
} from "./CaptionStyles";
export interface CaptionedClipProps {
videoSrc: string;
captions: RemotionCaption[];
style: CaptionStyle;
accentColor: string;
fontFamily: string;
durationInFrames: number;
}
const CaptionRenderer: React.FC<{
style: CaptionStyle;
captions: RemotionCaption[];
currentFrame: number;
fps: number;
accentColor: string;
fontFamily: string;
}> = ({ style, ...props }) => {
switch (style) {
case "colored":
return <ColoredWords {...props} />;
case "scaling":
return <ScalingWords {...props} />;
case "background":
default:
return <AnimatedBackground {...props} />;
}
};
export const CaptionedClip: React.FC<CaptionedClipProps> = ({
videoSrc,
captions,
style,
accentColor,
fontFamily,
}) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
// Use staticFile for videos in the public folder
const resolvedVideoSrc = videoSrc.startsWith("/") || videoSrc.startsWith("http")
? videoSrc
: staticFile(videoSrc);
return (
<AbsoluteFill
style={{
backgroundColor: "#000000",
}}
>
{/* Video layer */}
<AbsoluteFill>
<OffthreadVideo
src={resolvedVideoSrc}
style={{
width: "100%",
height: "100%",
objectFit: "contain",
}}
/>
</AbsoluteFill>
{/* Caption overlay layer - positioned at bottom of video content (not black bars) */}
{/* For 16:9 video in 9:16 frame: video is ~608px tall, centered, so bottom of video is ~656px from frame bottom */}
<div
style={{
position: "absolute",
bottom: 700,
left: 0,
right: 0,
display: "flex",
justifyContent: "center",
alignItems: "center",
}}
>
<CaptionRenderer
style={style}
captions={captions}
currentFrame={frame}
fps={fps}
accentColor={accentColor}
fontFamily={fontFamily}
/>
</div>
</AbsoluteFill>
);
};
import React, { useMemo, useRef, useEffect, useState } from "react";
import { interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
import { CaptionStyleProps } from "./types";
import { RemotionCaption } from "../convert-whisper";
/**
* AnimatedBackground Style (CapCut Signature Style)
*
* Shows a short phrase at the bottom with the active word highlighted.
* Words stay in one fixed position - only the highlight moves.
*/
export const AnimatedBackground: React.FC<CaptionStyleProps> = ({
captions,
currentFrame,
fps,
accentColor,
fontFamily,
}) => {
const containerRef = useRef<HTMLDivElement>(null);
const wordRefs = useRef<Map<number, HTMLSpanElement>>(new Map());
const [wordPositions, setWordPositions] = useState<Map<number, DOMRect>>(new Map());
const currentTimeMs = (currentFrame / fps) * 1000;
// Find the currently active word index in the full captions array
const activeWordIndex = useMemo(() => {
return captions.findIndex(
(cap) => currentTimeMs >= cap.startMs && currentTimeMs < cap.endMs
);
}, [captions, currentTimeMs]);
// Get a window of words to display (current word + context)
const visibleWords = useMemo(() => {
if (activeWordIndex < 0) {
// Show first few words if nothing active yet
return captions.slice(0, 4).map((cap, i) => ({
word: cap.text,
index: i,
isActive: false,
}));
}
// Show 3-5 words centered around the active word
const windowSize = 4;
const halfWindow = Math.floor(windowSize / 2);
const start = Math.max(0, activeWordIndex - halfWindow);
const end = Math.min(captions.length, start + windowSize);
const actualStart = Math.max(0, end - windowSize);
return captions.slice(actualStart, end).map((cap, i) => ({
word: cap.text,
index: actualStart + i,
isActive: actualStart + i === activeWordIndex,
}));
}, [captions, activeWordIndex]);
// Update word positions for highlight box
useEffect(() => {
const positions = new Map<number, DOMRect>();
wordRefs.current.forEach((element, index) => {
if (element) {
positions.set(index, element.getBoundingClientRect());
}
});
setWordPositions(positions);
}, [visibleWords, currentFrame]);
if (visibleWords.length === 0) return null;
const containerRect = containerRef.current?.getBoundingClientRect();
const activeVisibleIndex = visibleWords.findIndex((w) => w.isActive);
const activeWordRect = wordPositions.get(activeVisibleIndex);
return (
<div
ref={containerRef}
style={{
position: "relative",
display: "flex",
flexDirection: "row",
justifyContent: "center",
alignItems: "center",
gap: "8px",
padding: "12px 24px",
backgroundColor: "rgba(0, 0, 0, 0.6)",
borderRadius: "8px",
}}
>
{/* Animated highlight box */}
{activeVisibleIndex >= 0 && activeWordRect && containerRect && (
<div
style={{
position: "absolute",
left: activeWordRect.left - containerRect.left - 6,
top: activeWordRect.top - containerRect.top - 4,
width: activeWordRect.width + 12,
height: activeWordRect.height + 8,
backgroundColor: accentColor,
borderRadius: "6px",
transition: "all 0.1s ease-out",
zIndex: 0,
}}
/>
)}
{/* Words in a single row */}
{visibleWords.map((item, index) => (
<span
key={`${item.word}-${item.index}`}
ref={(el) => {
if (el) wordRefs.current.set(index, el);
}}
style={{
fontFamily,
fontSize: "32px",
fontWeight: 700,
color: item.isActive ? "#000000" : "#FFFFFF",
position: "relative",
zIndex: item.isActive ? 2 : 1,
textTransform: "uppercase",
letterSpacing: "0.02em",
whiteSpace: "nowrap",
}}
>
{item.word}
</span>
))}
</div>
);
};
import React from "react";
import { interpolate, useCurrentFrame, useVideoConfig } from "remotion";
import { CaptionStyleProps, getActiveWords } from "./types";
/**
* ColoredWords Style
*
* Clean, readable captions where the active word is highlighted
* in an accent color. Simple and professional.
*/
export const ColoredWords: React.FC<CaptionStyleProps> = ({
captions,
currentFrame,
fps,
accentColor,
fontFamily,
}) => {
const words = getActiveWords(captions, currentFrame, fps);
if (words.length === 0) return null;
return (
<div
style={{
display: "flex",
flexWrap: "wrap",
justifyContent: "center",
alignItems: "center",
gap: "6px 10px",
padding: "16px 40px",
maxWidth: "85%",
}}
>
{words.map((word, index) => {
const isActive = word.isActive;
// Smooth opacity transition
const progress = interpolate(
currentFrame,
[word.startFrame - 3, word.startFrame, word.endFrame, word.endFrame + 3],
[0.6, 1, 1, 0.6],
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
);
return (
<span
key={`${word.word}-${index}`}
style={{
fontFamily,
fontSize: "36px",
fontWeight: 700,
color: isActive ? accentColor : "#FFFFFF",
textShadow: isActive
? `0 0 12px ${accentColor}80, 1px 1px 3px rgba(0,0,0,0.9)`
: "1px 1px 3px rgba(0,0,0,0.9)",
opacity: progress,
transition: "color 0.1s ease-out",
textTransform: "uppercase",
letterSpacing: "0.01em",
}}
>
{word.word}
</span>
);
})}
</div>
);
};
export { ColoredWords } from "./ColoredWords";
export { ScalingWords } from "./ScalingWords";
export { AnimatedBackground } from "./AnimatedBackground";
export type { CaptionStyle, CaptionStyleProps } from "./types";
import React from "react";
import { interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
import { CaptionStyleProps, getActiveWords } from "./types";
/**
* ScalingWords Style
*
* Punchy, dynamic captions where each word scales up with a spring
* animation when it becomes active. Great for energetic content.
*/
export const ScalingWords: React.FC<CaptionStyleProps> = ({
captions,
currentFrame,
fps,
accentColor,
fontFamily,
}) => {
const words = getActiveWords(captions, currentFrame, fps);
if (words.length === 0) return null;
return (
<div
style={{
display: "flex",
flexWrap: "wrap",
justifyContent: "center",
alignItems: "center",
gap: "6px 10px",
padding: "16px 40px",
maxWidth: "85%",
}}
>
{words.map((word, index) => {
const isActive = word.isActive;
// Spring animation for scale
const scaleProgress = spring({
frame: currentFrame - word.startFrame,
fps,
config: {
damping: 12,
stiffness: 200,
mass: 0.5,
},
});
// Scale up when active, return to normal after
const scale = isActive
? interpolate(scaleProgress, [0, 1], [1, 1.2])
: interpolate(
currentFrame,
[word.endFrame, word.endFrame + 5],
[1.2, 1],
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
);
// Opacity based on timing
const opacity = interpolate(
currentFrame,
[word.startFrame - 5, word.startFrame, word.endFrame + 10, word.endFrame + 15],
[0.5, 1, 1, 0.5],
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
);
return (
<span
key={`${word.word}-${index}`}
style={{
fontFamily,
fontSize: "36px",
fontWeight: 700,
color: "#FFFFFF",
textShadow: isActive
? `0 0 16px ${accentColor}, 0 0 32px ${accentColor}80, 1px 1px 3px rgba(0,0,0,0.9)`
: "1px 1px 3px rgba(0,0,0,0.9)",
transform: `scale(${scale})`,
opacity,
display: "inline-block",
textTransform: "uppercase",
letterSpacing: "0.01em",
WebkitTextStroke: isActive ? `1px ${accentColor}` : "none",
}}
>
{word.word}
</span>
);
})}
</div>
);
};
import { RemotionCaption } from "../convert-whisper";
export type CaptionStyle = "colored" | "scaling" | "background";
export interface CaptionStyleProps {
captions: RemotionCaption[];
currentFrame: number;
fps: number;
accentColor: string;
fontFamily: string;
}
export interface WordTiming {
word: string;
startFrame: number;
endFrame: number;
isActive: boolean;
}
/**
* Get words visible at the current frame with their timing info.
* Groups words into "pages" - sentences or groups that appear together.
*/
export function getActiveWords(
captions: RemotionCaption[],
currentFrame: number,
fps: number,
windowMs: number = 3000 // Show 3 seconds of context
): WordTiming[] {
const currentTimeMs = (currentFrame / fps) * 1000;
const windowStart = currentTimeMs - windowMs / 2;
const windowEnd = currentTimeMs + windowMs / 2;
return captions
.filter((cap) => cap.startMs <= windowEnd && cap.endMs >= windowStart)
.map((cap) => ({
word: cap.text,
startFrame: Math.round((cap.startMs / 1000) * fps),
endFrame: Math.round((cap.endMs / 1000) * fps),
isActive: currentTimeMs >= cap.startMs && currentTimeMs < cap.endMs,
}));
}
/**
* Find the currently active word index.
*/
export function getActiveWordIndex(
captions: RemotionCaption[],
currentFrame: number,
fps: number
): number {
const currentTimeMs = (currentFrame / fps) * 1000;
return captions.findIndex(
(cap) => currentTimeMs >= cap.startMs && currentTimeMs < cap.endMs
);
}
/**
* Converts vid-clipper Whisper JSON format to Remotion Caption objects.
* Supports both segment-level and word-level timestamps.
*/
export interface WhisperWord {
word: string;
start: number;
end: number;
probability?: number;
}
export interface WhisperSegment {
id: number;
seek: number;
start: number;
end: number;
text: string;
tokens: number[];
temperature: number;
avg_logprob: number;
compression_ratio: number;
no_speech_prob: number;
words?: WhisperWord[];
}
export interface WhisperTranscript {
text: string;
segments: WhisperSegment[];
language: string;
}
export interface RemotionCaption {
text: string;
startMs: number;
endMs: number;
timestampMs: number;
confidence: number | null;
}
/**
* Convert Whisper JSON transcript to Remotion Caption array.
* Uses word-level timestamps when available for precise caption timing.
*/
export function convertWhisperToRemotionCaptions(
whisperJson: WhisperTranscript,
clipStartTime?: number,
clipEndTime?: number
): RemotionCaption[] {
const captions: RemotionCaption[] = [];
const offsetMs = (clipStartTime || 0) * 1000;
for (const segment of whisperJson.segments) {
// Skip segments outside clip range if specified
if (clipStartTime !== undefined && segment.end < clipStartTime) continue;
if (clipEndTime !== undefined && segment.start > clipEndTime) continue;
// Use word-level timestamps if available
if (segment.words && segment.words.length > 0) {
for (const word of segment.words) {
// Skip words outside clip range
if (clipStartTime !== undefined && word.end < clipStartTime) continue;
if (clipEndTime !== undefined && word.start > clipEndTime) continue;
const startMs = Math.max(0, word.start * 1000 - offsetMs);
const endMs = word.end * 1000 - offsetMs;
// Clamp to clip duration if specified
if (clipEndTime !== undefined) {
const clipDurationMs = (clipEndTime - (clipStartTime || 0)) * 1000;
if (startMs >= clipDurationMs) continue;
}
captions.push({
text: word.word.trim(),
startMs,
endMs,
timestampMs: startMs,
confidence: word.probability ?? null,
});
}
} else {
// Fall back to segment-level timestamps
// Split segment text into words and distribute time evenly
const words = segment.text.trim().split(/\s+/);
const segmentDuration = (segment.end - segment.start) * 1000;
const wordDuration = segmentDuration / words.length;
for (let i = 0; i < words.length; i++) {
const wordStart = segment.start * 1000 + i * wordDuration - offsetMs;
const wordEnd = wordStart + wordDuration;
if (wordStart < 0) continue;
captions.push({
text: words[i],
startMs: Math.max(0, wordStart),
endMs: wordEnd,
timestampMs: Math.max(0, wordStart),
confidence: null,
});
}
}
}
return captions;
}
/**
* Extract captions for a specific clip timerange from the full transcript.
*/
export function extractClipCaptions(
whisperJson: WhisperTranscript,
startTime: number,
endTime: number
): RemotionCaption[] {
return convertWhisperToRemotionCaptions(whisperJson, startTime, endTime);
}
/**
* Read and parse Whisper JSON file.
*/
export async function loadWhisperTranscript(
jsonPath: string
): Promise<WhisperTranscript> {
const response = await fetch(jsonPath);
if (!response.ok) {
throw new Error(`Failed to load Whisper transcript: ${jsonPath}`);
}
return response.json();
}
import { registerRoot } from "remotion";
import { RemotionRoot } from "./Root";
registerRoot(RemotionRoot);
#!/usr/bin/env npx ts-node
/**
* CLI script to render a captioned clip.
*
* Usage:
* npx ts-node src/render-clip.ts \
* --video /path/to/clip.mp4 \
* --whisper /path/to/original.json \
* --output /path/to/output.mp4 \
* --start 10.5 \
* --end 40.2 \
* --style background \
* --color "#FFFF00"
*/
import { bundle } from "@remotion/bundler";
import { renderMedia, selectComposition } from "@remotion/renderer";
import { convertWhisperToRemotionCaptions, WhisperTranscript } from "./convert-whisper";
import * as fs from "fs";
import * as path from "path";
interface RenderOptions {
videoPath: string;
whisperPath: string;
outputPath: string;
startTime: number;
endTime: number;
style: "colored" | "scaling" | "background";
accentColor: string;
fontFamily: string;
}
function parseArgs(): RenderOptions {
const args = process.argv.slice(2);
const options: Partial<RenderOptions> = {
style: "background",
accentColor: "#FFFF00",
fontFamily: "Inter, system-ui, sans-serif",
};
for (let i = 0; i < args.length; i += 2) {
const key = args[i];
const value = args[i + 1];
switch (key) {
case "--video":
options.videoPath = value;
break;
case "--whisper":
options.whisperPath = value;
break;
case "--output":
options.outputPath = value;
break;
case "--start":
options.startTime = parseFloat(value);
break;
case "--end":
options.endTime = parseFloat(value);
break;
case "--style":
options.style = value as RenderOptions["style"];
break;
case "--color":
options.accentColor = value;
break;
case "--font":
options.fontFamily = value;
break;
}
}
if (!options.videoPath || !options.whisperPath || !options.outputPath) {
console.error("Required: --video, --whisper, --output");
process.exit(1);
}
if (options.startTime === undefined || options.endTime === undefined) {
console.error("Required: --start, --end");
process.exit(1);
}
return options as RenderOptions;
}
async function main() {
const options = parseArgs();
console.log("Loading Whisper transcript...");
const whisperJson: WhisperTranscript = JSON.parse(
fs.readFileSync(options.whisperPath, "utf-8")
);
console.log("Converting captions for clip range...");
const captions = convertWhisperToRemotionCaptions(
whisperJson,
options.startTime,
options.endTime
);
console.log(`Found ${captions.length} caption words for clip`);
const fps = 30;
const durationInFrames = Math.ceil((options.endTime - options.startTime) * fps);
console.log("Bundling Remotion project...");
const bundleLocation = await bundle({
entryPoint: path.join(__dirname, "index.ts"),
onProgress: (progress) => {
if (progress % 20 === 0) {
console.log(`Bundle progress: ${progress}%`);
}
},
});
console.log("Selecting composition...");
const composition = await selectComposition({
serveUrl: bundleLocation,
id: "CaptionedClip",
inputProps: {
videoSrc: options.videoPath,
captions,
style: options.style,
accentColor: options.accentColor,
fontFamily: options.fontFamily,
durationInFrames,
},
});
console.log("Rendering video with captions...");
await renderMedia({
composition: {
...composition,
durationInFrames,
},
serveUrl: bundleLocation,
codec: "h264",
outputLocation: options.outputPath,
inputProps: {
videoSrc: options.videoPath,
captions,
style: options.style,
accentColor: options.accentColor,
fontFamily: options.fontFamily,
durationInFrames,
},
onProgress: ({ progress }) => {
const percent = Math.round(progress * 100);
if (percent % 10 === 0) {
process.stdout.write(`\rRendering: ${percent}%`);
}
},
});
console.log(`\nDone! Output saved to: ${options.outputPath}`);
}
main().catch((err) => {
console.error("Render failed:", err);
process.exit(1);
});
import React from "react";
import { Composition, staticFile } from "remotion";
import { CaptionedClip } from "./CaptionedClip";
import type { RemotionCaption } from "./convert-whisper";
import type { CaptionStyle } from "./CaptionStyles";
// Schema for input props - matches CaptionedClipProps
const defaultCaptions: RemotionCaption[] = [];
export const RemotionRoot: React.FC = () => {
return (
<>
<Composition
id="CaptionedClip"
// eslint-disable-next-line @typescript-eslint/no-explicit-any
component={CaptionedClip as any}
durationInFrames={30 * 30}
fps={30}
width={1080}
height={1920}
defaultProps={{
videoSrc: staticFile("sample.mp4"),
captions: defaultCaptions,
style: "background" as CaptionStyle,
accentColor: "#FFFF00",
fontFamily: "Inter, system-ui, sans-serif",
durationInFrames: 30 * 30,
}}
calculateMetadata={async ({ props }) => {
return {
durationInFrames:
(props as { durationInFrames?: number }).durationInFrames ??
30 * 30,
};
}}
/>
</>
);
};