
Document To Narration
- 765 installs
- 133 repo stars
- Updated February 24, 2026
- jwynia/agent-skills
document-to-narration is a Claude Code skill that converts long documents, reports, or chapters into listenable narration scripts for podcasts, audiobooks, or agent voice output without manual rewriting.
About
document-to-narration is a Claude Code skill from jwynia/agent-skills that transforms long-form written material—documentation, reports, or book chapters—into narration scripts suitable for podcasts, audiobooks, or agent voice synthesis. Developers reach for document-to-narration when they have dense technical or narrative prose that must become spoken-word content without hand-editing every paragraph for audio pacing and clarity. The skill automates the structural rewrite from reading format to listenable format, preserving meaning while adapting sentence rhythm, transitions, and emphasis for audio delivery. It fits documentation pipelines, content repurposing workflows, and agent systems that need text-to-speech or voice-output scripts generated from existing markdown or report files.
- Document-to-script conversion
- Narrative pacing guidance
- Section-aware chunking
- Voice-friendly phrasing
- Multi-format source ingestion
Document To Narration by the numbers
- 765 all-time installs (skills.sh)
- +27 installs in the week ending Aug 2, 2026 (Skillselion tracking)
- Ranked #140 of 688 Office & Documents skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/jwynia/agent-skills --skill document-to-narrationAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 765 |
|---|---|
| repo stars | ★ 133 |
| Last updated | February 24, 2026 |
| Repository | jwynia/agent-skills ↗ |
How do you turn technical docs into narration scripts?
Turn long docs, reports, or chapters into listenable narration scripts for podcasts, audiobooks, or agent voice output without manual rewriting.
Who is it for?
Developers or technical writers repurposing long documentation or reports into audio-ready narration without manual paragraph rewriting.
Skip if: Developers who only need written API documentation or markdown formatting without audio adaptation should skip document-to-narration.
When should I use this skill?
The user has long docs, reports, or chapters that need conversion into podcast, audiobook, or agent voice narration scripts.
What you get
Listenable narration scripts adapted from source documents for podcast, audiobook, or agent voice delivery.
- narration script
- audio-ready prose
- podcast episode script
Files
Document to Narration
Convert written documents into narrated video scripts with precise word-level timing.
Core Principle
The agent interprets; the document guides. Rather than rigid template-based splits, this skill uses agent judgment to find where the content naturally breathes, argues, and transitions. The document's argument flow determines scene breaks, not a predetermined structure.
When to Use This Skill
Use this skill when:
- Converting a blog post or essay to video narration
- Preparing content for TTS audio generation
- Breaking long-form content into digestible scenes
- Creating word-level synchronized captions for video
Do NOT use this skill when:
- The content is already in scene/script format
- You need real-time voice synthesis (this is batch processing)
- Working with dialogue or multi-speaker content (single voice only)
Prerequisites
- Deno installed (https://deno.land/)
- Python 3.12 with venv support
- ffmpeg for audio conversion
- whisper-cpp (installed via @remotion/install-whisper-cpp)
- TTS model at
tts/model/(not in git due to size - see Model Setup below)
Complete Pipeline
There are two approaches: per-scene (legacy) and full narration (recommended).
Full Narration Pipeline (Recommended)
Generates a single audio file for consistent volume and pacing:
Document (.md)
↓ [agent interprets scene breaks]
Scene .txt files (01-scene-name.txt, 02-scene-name.txt, ...)
↓ [TTS via narrate-full.py - SINGLE PASS]
full-narration.wav (one consistent audio file)
↓ [Whisper via transcribe-full.py]
full-narration.json + full-narration.vtt (word-level timing)
↓ [extract-scene-boundaries.py]
Scene timing boundaries for video compositionPer-Scene Pipeline (Legacy)
Generates separate audio per scene - can cause volume inconsistencies:
Scene .txt files
↓ [TTS via narrate-scenes.py - MULTIPLE PASSES]
Scene .wav files (volume may vary between scenes)
↓ [concatenate]
Combined audio (may have clipping at boundaries)Warning: Per-scene TTS generates audio with different volume levels and pacing. When concatenated, this causes audible jumps and clipping. Use the full narration pipeline instead.
Quick Start
Full Narration Pipeline (Recommended)
cd .claude/skills/document-to-narration
source tts/.venv/bin/activate
# 1. Split document into scenes (manual or scripted)
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/
# 2. Generate single audio file
python scripts/narrate-full.py ./output/scenes/
# 3. Transcribe with word-level timestamps
python scripts/transcribe-full.py ./output/full-narration.wav
# 4. Extract scene boundaries for video timing
python scripts/extract-scene-boundaries.py ./output/scenes/ ./output/full-narration.json --typescriptLegacy Per-Scene Pipeline
# 1. Split document into scenes
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/
# 2. Generate audio per scene (may have volume inconsistencies)
source tts/.venv/bin/activate
python scripts/narrate-scenes.py ./output/scenes/
# 3. Transcribe (DEPRECATED: transcribe-scenes.ts requires whisper-cpp)
# Use transcribe-full.py instead after concatenating audioInstructions
Step 1: Setup (First Time Only)
Create Python Virtual Environment
cd .claude/skills/document-to-narration/tts
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtTTS Model Setup
The fine-tuned voice model (~7.8GB) is not included in git due to size. Place your Qwen3-TTS model files in tts/model/:
tts/model/
├── config.json
├── generation_config.json
├── model.safetensors # Main model weights
├── tokenizer_config.json
├── vocab.json
├── merges.txt
└── speech_tokenizer/
└── ...Install Whisper (if not already installed)
The @remotion/install-whisper-cpp package handles this:
import { installWhisperCpp, downloadWhisperModel } from '@remotion/install-whisper-cpp';
await installWhisperCpp({ to: './whisper-cpp', version: '1.5.5' });
await downloadWhisperModel({ model: 'medium', folder: './whisper-cpp' });Step 2: Prepare Your Document
The skill works best with:
- Markdown documents with clear heading structure (H1, H2)
- Well-structured arguments with distinct sections
- Content that reads naturally aloud
Step 3: Run the Pipeline
deno run -A scripts/full-pipeline.ts /path/to/essay.md --output ./output/essay-name/Step 4: Review Output
output/essay-name/
├── scenes/
│ ├── 01-opening-hook.txt # Scene script
│ ├── 01-opening-hook.wav # Generated audio
│ ├── 01-opening-hook.vtt # Word-level captions
│ ├── 02-core-argument.txt
│ ├── 02-core-argument.wav
│ ├── 02-core-argument.vtt
│ └── ...
└── manifest.json # Complete timing dataScene Boundary Heuristics
The agent identifies scene breaks using these heuristics:
Strong Boundaries (Almost Always Break)
- H2 heading changes
- "Here's the thing" / "The point is" pivot statements
- Major metaphor introduction
- Explicit enumeration ("First...", "Second...")
- Significant perspective shifts
Moderate Boundaries (Consider Breaking)
- Long paragraph after short ones (or vice versa)
- Example-to-principle transitions
- "But" / "However" / "Meanwhile" at paragraph start
- Question-then-answer patterns
Weak Boundaries (Usually Keep Together)
- Paragraph-to-paragraph within same example
- Sequential evidence for same point
- Build-up to a punchline/reveal
Scene Length Guidance
- Target: 100-300 words per scene (30-90 seconds of audio)
- Minimum: 50 words (avoid micro-scenes)
- Maximum: 500 words (avoid cognitive overload)
Anti-Patterns
The Paragraph Slicer
Pattern: Breaking at every paragraph or heading mechanically. Problem: Ignores argument flow. Scenes feel choppy and disconnected. Fix: Look for rhetorical units, not structural units. Multiple paragraphs often form one scene.
The Wall of Text
Pattern: Keeping entire sections as single scenes. Problem: Creates TTS audio that's too long. Loses natural breathing room. Fix: Target 100-300 words. Find the natural pause point within sections.
The Verbatim Transcriber
Pattern: Copying written text exactly without spoken adaptation. Problem: Written conventions don't work when spoken. Parentheticals, complex punctuation, and nested clauses confuse TTS and listeners. Fix: Apply adaptation rules. Read it aloud mentally.
The Over-Adapter
Pattern: Rewriting content so heavily it loses the author's voice. Problem: The result doesn't sound like the original author. Fix: Preserve voice, adjust mechanics. If the author uses rhetorical questions, keep them.
Available Scripts
scripts/split-to-scenes.ts
Parse a markdown document and output scene text files.
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/
deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --output ./output/ --adapt
deno run --allow-read scripts/split-to-scenes.ts input.md --dry-runOptions:
--output- Directory for scene files (created if doesn't exist)--adapt- Apply spoken adaptation rules--dry-run- Preview scene breaks without writing files
Output: Numbered .txt files and initial manifest.json
scripts/narrate-full.py (Recommended)
Generate a single TTS audio file from all scene files. Produces consistent volume and pacing.
python scripts/narrate-full.py ./output/scenes/
python scripts/narrate-full.py ./output/scenes/ --force
python scripts/narrate-full.py ./output/scenes/ --speaker jwynia
python scripts/narrate-full.py ./output/scenes/ --output ./custom/path/audio.wavOptions:
--force- Regenerate even if output exists--speaker- Speaker name (default: jwynia)--output- Custom output path (default:../full-narration.wav)
Output: Single full-narration.wav in parent directory of scenes
scripts/narrate-scenes.py (Legacy)
Generate TTS audio for each scene file separately. Not recommended - can cause volume inconsistencies when concatenated.
python scripts/narrate-scenes.py ./output/scenes/
python scripts/narrate-scenes.py ./output/scenes/ --force
python scripts/narrate-scenes.py ./output/scenes/ --speaker jwyniaOptions:
--force- Regenerate even if output exists--speaker- Speaker name (default: jwynia)
Output: .wav files alongside each .txt file
scripts/transcribe-full.py (Recommended)
Transcribe audio with word-level timestamps using Python's openai-whisper.
python scripts/transcribe-full.py ./output/full-narration.wav
python scripts/transcribe-full.py ./output/full-narration.wav --model large-v3
python scripts/transcribe-full.py ./output/full-narration.wav --output-dir ./captions/Options:
--model- Whisper model: tiny, base, small, medium, large, large-v2, large-v3 (default: medium)--output-dir- Output directory (default: same as audio file)
Output:
.vttfile with word-level timestamps.jsonfile with captions array for Remotion
Dependencies: Requires openai-whisper in Python environment:
pip install openai-whisperscripts/extract-scene-boundaries.py
Extract scene timing boundaries from transcript by matching scene opening phrases.
# Human-readable table
python scripts/extract-scene-boundaries.py ./output/scenes/ ./output/full-narration.json
# JSON output
python scripts/extract-scene-boundaries.py ./output/scenes/ ./output/full-narration.json --json
# TypeScript for Video.tsx
python scripts/extract-scene-boundaries.py ./output/scenes/ ./output/full-narration.json --typescriptOptions:
--json- Output as JSON array--typescript- Output as TypeScript code for Video.tsx scenes array
Output: Scene numbers, slugs, start times, and durations
scripts/transcribe-scenes.ts (Deprecated)
Deprecated: Requires whisper-cpp binary which may not be installed. Use transcribe-full.py instead.Transcribe per-scene audio files using whisper-cpp.
deno run --allow-read --allow-write --allow-run scripts/transcribe-scenes.ts ./output/scenes/Output: .vtt files with word-level timestamps
scripts/full-pipeline.ts
Orchestrate the complete pipeline.
deno run -A scripts/full-pipeline.ts input.md --output ./output/project-name/Options:
--output- Output directory (required)--adapt- Apply spoken adaptation--skip-tts- Skip audio generation (text only)--skip-transcribe- Skip Whisper transcription
Output Format
manifest.json
{
"source": "appliance-vs-trade-tool-draft.md",
"created_at": "2024-01-15T10:30:00Z",
"total_scenes": 9,
"total_duration_seconds": 420,
"scenes": [
{
"number": 1,
"slug": "popcorn-opening",
"word_count": 185,
"audio_duration_seconds": 55.2,
"files": {
"text": "scenes/01-popcorn-opening.txt",
"audio": "scenes/01-popcorn-opening.wav",
"captions": "scenes/01-popcorn-opening.vtt"
},
"captions": [
{ "text": "Two", "startMs": 0, "endMs": 180, "confidence": 0.98 },
{ "text": "people", "startMs": 180, "endMs": 450, "confidence": 0.97 }
]
}
]
}VTT Format
WEBVTT
00:00.000 --> 00:00.180
Two
00:00.180 --> 00:00.450
people
00:00.450 --> 00:00.720
walk
00:00.720 --> 00:01.100
intoSpoken Adaptation
When --adapt is enabled, the skill transforms written conventions to spoken equivalents:
| Written | Spoken |
|---|---|
| Parenthetical asides | Em-dash or separate sentence |
| "e.g." | "for example" |
| "i.e." | "that is" |
| Long nested clauses | Split into multiple sentences |
| Semicolons | Periods |
*emphasis* | Context-appropriate stress |
Preserve:
- Author's voice and tone
- Rhetorical questions
- Deliberate repetition
- Key phrases and memorable formulations
Integration
With remotion-designer
- Pass manifest scene list to remotion-designer
- Each scene becomes a visual design unit
- Word-level timing drives text animation
With Remotion Compositions
import { Audio, useCurrentFrame, Sequence } from 'remotion';
import manifest from './output/manifest.json';
// Use scene durations for Sequence timing
{manifest.scenes.map((scene, i) => (
<Sequence
from={accumulatedFrames}
durationInFrames={scene.audio_duration_seconds * fps}
>
<Audio src={staticFile(scene.files.audio)} />
<CaptionRenderer captions={scene.captions} />
</Sequence>
))}Technical Notes
WAV Format Conversion
Whisper requires 16kHz mono WAV. The pipeline handles conversion automatically:
ffmpeg -i input.wav -ar 16000 -ac 1 output_16khz.wavTTS Model
The fine-tuned voice model (~7.8GB) is bundled at tts/model/. Uses Qwen3-TTS with custom speaker embedding.
Performance
- TTS: ~5-30 seconds per sentence (Apple Silicon MPS or NVIDIA CUDA)
- Whisper: ~0.5-2x realtime depending on model size
- Full essay (~2000 words): ~10-20 minutes total processing
What This Skill Does NOT Do
- Generate video visuals (use remotion-designer)
- Real-time voice synthesis
- Multi-speaker dialogue
- Edit or improve the content's argument
- Make editorial changes beyond mechanical spoken adaptation
# TTS model files (too large for git)
tts/model/*
!tts/model/.gitkeep
# Python virtual environment
tts/.venv/
# Output directories
output/
# Whisper installation
whisper.cpp/
# Temporary files
*.pyc
__pycache__/
tmp/
{
"_meta": {
"description": "Rules for adapting written text to spoken delivery",
"version": "1.0"
},
"replacements": {
"abbreviations": {
"e.g.": "for example",
"i.e.": "that is",
"etc.": "and so on",
"vs.": "versus",
"vs": "versus",
"w/": "with",
"b/c": "because",
"w/o": "without",
"approx.": "approximately",
"esp.": "especially",
"incl.": "including"
},
"symbols": {
"&": "and",
"%": "percent",
"+": "plus",
"=": "equals",
"~": "approximately"
}
},
"punctuation_patterns": {
"parenthetical_to_dash": {
"description": "Convert parenthetical asides to em-dashes or separate sentences",
"example_before": "The tool (which was available to both) produced different results.",
"example_after": "The tool — which was available to both — produced different results."
},
"semicolon_to_period": {
"description": "Break semicolon-joined clauses into separate sentences",
"example_before": "The first person found the floor; the second discovered the ceiling.",
"example_after": "The first person found the floor. The second discovered the ceiling."
},
"ellipsis_handling": {
"description": "Replace ellipsis with period or natural pause",
"example_before": "And then... it clicked.",
"example_after": "And then it clicked."
}
},
"sentence_structure": {
"max_clause_depth": 2,
"split_triggers": [
"which, in turn,",
"although it should be noted that",
"despite the fact that",
"in addition to the fact that",
"notwithstanding the possibility that"
]
},
"emphasis_handling": {
"italics": {
"description": "Context determines stress; often no change needed",
"note": "TTS handles emphasis through sentence context"
},
"bold": {
"description": "Usually structural emphasis; may need spoken equivalent",
"note": "Consider adding 'importantly' or similar if emphasis is critical"
},
"ALL_CAPS": {
"description": "Strong emphasis; preserve meaning through word choice",
"note": "Convert to lowercase but ensure context conveys emphasis"
}
},
"preserve_patterns": [
"rhetorical_questions",
"deliberate_repetition",
"parallel_structure",
"author_voice_phrases",
"memorable_formulations"
],
"transition_phrases": {
"section_start": [
"Now,",
"Here's where things get interesting:",
"Let's look at",
"Consider this:"
],
"returning_to_point": [
"Back to our main question:",
"Returning to the point:",
"This brings us back to:"
],
"contrast_pivot": [
"But here's the thing:",
"However,",
"On the other hand,"
],
"building": [
"And this connects to",
"Building on this,",
"Taking this further,"
],
"conclusion": [
"So what does this mean?",
"The key takeaway is:",
"In the end,"
]
}
}
Spoken Adaptation Guide
Principle: Sound Natural When Read Aloud
The test for spoken adaptation: read the text aloud. Does it flow? Do you stumble? Would a listener understand on first hearing?
When to Adapt
Adapt text when:
- Parenthetical asides break the flow
- Sentences are too long or nested
- Abbreviations would sound awkward spoken
- Punctuation creates unnatural pauses
- Academic hedging obscures the point
Do NOT adapt when:
- The author's voice would be lost
- Rhetorical devices are intentional
- The original phrasing is memorable
- Simplification would change meaning
Adaptation Categories
1. Structural Simplification
Written: "The tool — which, it should be noted, was available to both participants — produced dramatically different results."
Spoken: "The tool was available to both participants. But it produced dramatically different results."
2. Parenthetical Handling
Written: "They found the floor (where anyone can get results) but never discovered the ceiling."
Spoken Options:
- "They found the floor — where anyone can get results — but never discovered the ceiling."
- "They found the floor, where anyone can get results. But they never discovered the ceiling."
3. Academic Hedging
Written: "It could be argued that the distinction between..."
Spoken (if author's voice permits): "The distinction between..."
Written: "Research suggests that..."
Spoken: "Research shows that..." (if the research is definitive)
4. List Handling
Written: "The elements include: clarity, brevity, and impact."
Spoken: "The elements include clarity, brevity, and impact."
For longer lists: Written: "1. First item 2. Second item 3. Third item"
Spoken: "First: [item]. Second: [item]. And third: [item]."
5. Abbreviation Expansion
| Written | Spoken |
|---|---|
| e.g. | for example |
| i.e. | that is |
| etc. | and so on |
| vs. | versus |
| w/ | with |
6. Emphasis Preservation
Italics in writing often indicate emphasis, but TTS needs context cues.
Written: "The microwave doesn't have a 'more skill' setting."
Spoken: "The microwave doesn't have a 'more skill' setting." (Natural stress from sentence structure)
If emphasis is critical and might be lost: Spoken: "The microwave... it simply doesn't have a 'more skill' setting."
What NOT to Change
- Author's distinctive phrasing
- Rhetorical questions (these work great in audio)
- Parallel structure ("Some... others...")
- Deliberate fragment sentences
- Characteristic word choices
- The argument's logic or claims
Scene Transition Language
When a section break becomes a scene break, sometimes you need to add spoken transitions:
| Context | Transition Type | Example |
|---|---|---|
| New major section | Introduction | "Now..." or "Here's where..." |
| Returning to main thread | Callback | "Back to our question:" |
| Contrast or pivot | Pivot | "But here's the thing:" |
| Building on previous | Connection | "And this connects to..." |
| Conclusion | Summary | "So what does this mean?" |
Use sparingly. Only add transitions when the audio would feel jarring without them.
Quality Checklist
Before finalizing adapted text:
- [ ] Read it aloud - does it flow naturally?
- [ ] Check for tongue-twisters or awkward consonant clusters
- [ ] Verify the author's voice is preserved
- [ ] Confirm no meaning was changed
- [ ] Ensure transitions feel organic, not mechanical
- [ ] Verify abbreviations are expanded
- [ ] Check sentence length (aim for 15-25 words average)
#!/usr/bin/env python3
"""
extract-scene-boundaries.py - Extract scene timing boundaries from transcript.
After generating audio with narrate-full.py and transcribing with transcribe-full.py,
this script finds where each scene starts in the audio by matching the opening
words of each scene text file against the transcript.
Usage:
python extract-scene-boundaries.py ./narration/scenes/ ./narration/full-narration.json
# Output as JSON for programmatic use
python extract-scene-boundaries.py ./narration/scenes/ ./narration/full-narration.json --json
# Output TypeScript code for Video.tsx
python extract-scene-boundaries.py ./narration/scenes/ ./narration/full-narration.json --typescript
"""
import argparse
import json
import re
import sys
from pathlib import Path
def load_transcript(json_path: Path) -> list[dict]:
"""Load word-level timestamps from transcript JSON."""
with open(json_path, "r", encoding="utf-8") as f:
data = json.load(f)
words = []
for seg in data.get("segments", []):
for w in seg.get("words", []):
words.append({
"word": w["word"].strip(),
"start": w["start"],
"end": w["end"],
})
return words
def get_scene_opening_phrases(scenes_dir: Path) -> list[tuple[str, str]]:
"""Get opening phrase from each scene file. Returns list of (filename, phrase)."""
scene_files = sorted(scenes_dir.glob("*.txt"))
phrases = []
for f in scene_files:
text = f.read_text(encoding="utf-8").strip()
# Get first ~5 words as opening phrase
words = text.split()[:5]
phrase = " ".join(words)
phrases.append((f.stem, phrase))
return phrases
def normalize_word(word: str) -> str:
"""Normalize word for matching (lowercase, remove punctuation)."""
return re.sub(r"[^\w]", "", word.lower())
def find_phrase_in_transcript(phrase: str, words: list[dict], start_from: int = 0) -> float | None:
"""Find where a phrase starts in the transcript. Returns start time or None."""
phrase_words = phrase.split()
phrase_normalized = [normalize_word(w) for w in phrase_words]
# Search through transcript
for i in range(start_from, len(words) - len(phrase_normalized) + 1):
match = True
for j, pw in enumerate(phrase_normalized):
transcript_word = normalize_word(words[i + j]["word"])
# Allow partial match (phrase word is prefix of transcript word)
if not transcript_word.startswith(pw[:3]): # Match at least first 3 chars
match = False
break
if match:
return words[i]["start"]
return None
def extract_boundaries(scenes_dir: Path, transcript_path: Path) -> list[dict]:
"""Extract scene boundaries from transcript."""
words = load_transcript(transcript_path)
phrases = get_scene_opening_phrases(scenes_dir)
if not words:
print("Error: No words found in transcript", file=sys.stderr)
sys.exit(1)
if not phrases:
print("Error: No scene files found", file=sys.stderr)
sys.exit(1)
boundaries = []
last_end = 0
search_from = 0
for i, (slug, phrase) in enumerate(phrases):
start_time = find_phrase_in_transcript(phrase, words, search_from)
if start_time is None:
print(f"Warning: Could not find scene '{slug}' starting with '{phrase}'", file=sys.stderr)
# Use last known position
start_time = last_end
else:
# Update search position to avoid finding same phrase twice
search_from = next(
(j for j, w in enumerate(words) if w["start"] >= start_time + 1),
search_from
)
boundaries.append({
"number": i + 1,
"slug": slug,
"start_seconds": start_time,
"opening_phrase": phrase,
})
last_end = start_time
# Calculate durations
total_duration = words[-1]["end"] if words else 0
for i, b in enumerate(boundaries):
if i + 1 < len(boundaries):
b["duration_seconds"] = round(boundaries[i + 1]["start_seconds"] - b["start_seconds"], 2)
else:
b["duration_seconds"] = round(total_duration - b["start_seconds"], 2)
return boundaries
def format_table(boundaries: list[dict], total_duration: float) -> str:
"""Format boundaries as readable table."""
lines = [
"Scene Boundaries",
"=" * 60,
f"{'#':<4} {'Slug':<30} {'Start':>8} {'Duration':>10}",
"-" * 60,
]
for b in boundaries:
lines.append(
f"{b['number']:<4} {b['slug']:<30} {b['start_seconds']:>7.2f}s {b['duration_seconds']:>9.2f}s"
)
lines.append("-" * 60)
lines.append(f"{'Total':<35} {total_duration:>18.2f}s")
return "\n".join(lines)
def format_json(boundaries: list[dict]) -> str:
"""Format boundaries as JSON."""
return json.dumps(boundaries, indent=2)
def format_typescript(boundaries: list[dict]) -> str:
"""Format boundaries as TypeScript for Video.tsx."""
lines = [
"// Scene timing data (at 30fps)",
"// Timings extracted from full-narration.wav transcript",
"const FPS = 30;",
"const scenes = [",
]
for b in boundaries:
# Convert slug to component name
parts = b["slug"].split("-")
# Handle numbered prefix (e.g., "01-kitchen-opening" -> "Scene01KitchenOpening")
if parts[0].isdigit():
num = parts[0]
name_parts = parts[1:]
else:
num = str(b["number"]).zfill(2)
name_parts = parts
component_name = f"Scene{num}{''.join(p.title() for p in name_parts)}"
lines.append(f" {{")
lines.append(f" number: {b['number']},")
lines.append(f" slug: '{'-'.join(name_parts)}',")
lines.append(f" durationSeconds: {b['duration_seconds']}, // {b['start_seconds']:.2f}s - {b['start_seconds'] + b['duration_seconds']:.2f}s")
lines.append(f" component: {component_name},")
lines.append(f" }},")
lines.append("];")
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(
description="Extract scene timing boundaries from transcript",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python extract-scene-boundaries.py ./narration/scenes/ ./narration/full-narration.json
python extract-scene-boundaries.py ./narration/scenes/ ./narration/full-narration.json --json
python extract-scene-boundaries.py ./narration/scenes/ ./narration/full-narration.json --typescript
""",
)
parser.add_argument(
"scenes_dir",
type=Path,
help="Directory containing scene .txt files",
)
parser.add_argument(
"transcript",
type=Path,
help="Path to transcript JSON file (from transcribe-full.py)",
)
parser.add_argument(
"--json",
action="store_true",
help="Output as JSON",
)
parser.add_argument(
"--typescript",
action="store_true",
help="Output as TypeScript for Video.tsx",
)
args = parser.parse_args()
scenes_dir = args.scenes_dir.resolve()
transcript_path = args.transcript.resolve()
if not scenes_dir.exists():
print(f"Error: Scenes directory not found: {scenes_dir}", file=sys.stderr)
sys.exit(1)
if not transcript_path.exists():
print(f"Error: Transcript not found: {transcript_path}", file=sys.stderr)
sys.exit(1)
# Extract boundaries
boundaries = extract_boundaries(scenes_dir, transcript_path)
# Calculate total
total_duration = sum(b["duration_seconds"] for b in boundaries)
# Output
if args.json:
print(format_json(boundaries))
elif args.typescript:
print(format_typescript(boundaries))
else:
print(format_table(boundaries, total_duration))
if __name__ == "__main__":
main()
#!/usr/bin/env -S deno run --allow-read --allow-write --allow-run --allow-env
/**
* full-pipeline.ts
*
* Orchestrate the complete document-to-narration pipeline:
* 1. Split document into scenes
* 2. Generate TTS audio for each scene
* 3. Transcribe audio for word-level timing
*
* Usage:
* deno run -A scripts/full-pipeline.ts input.md --output ./output/project-name/
* deno run -A scripts/full-pipeline.ts input.md --output ./output/ --skip-tts
* deno run -A scripts/full-pipeline.ts input.md --output ./output/ --by-headings
*/
import { dirname, fromFileUrl, join } from 'https://deno.land/std@0.208.0/path/mod.ts';
// ============================================================================
// Configuration
// ============================================================================
const SCRIPT_DIR = dirname(fromFileUrl(import.meta.url));
const SKILL_DIR = join(SCRIPT_DIR, '..');
const TTS_DIR = join(SKILL_DIR, 'tts');
// ============================================================================
// Utilities
// ============================================================================
async function runCommand(
cmd: string,
args: string[],
options: { cwd?: string; env?: Record<string, string> } = {}
): Promise<{ success: boolean; stdout: string; stderr: string }> {
const command = new Deno.Command(cmd, {
args,
cwd: options.cwd,
env: options.env,
stdout: 'piped',
stderr: 'piped',
});
const result = await command.output();
return {
success: result.success,
stdout: new TextDecoder().decode(result.stdout),
stderr: new TextDecoder().decode(result.stderr),
};
}
async function fileExists(path: string): Promise<boolean> {
try {
await Deno.stat(path);
return true;
} catch {
return false;
}
}
// ============================================================================
// Pipeline Steps
// ============================================================================
async function splitDocument(
inputFile: string,
outputDir: string,
byHeadings: boolean,
boundariesFile?: string
): Promise<boolean> {
console.log('\n' + '='.repeat(60));
console.log('STEP 1: Splitting document into scenes');
console.log('='.repeat(60));
const splitScript = join(SCRIPT_DIR, 'split-to-scenes.ts');
const args = [
'run',
'--allow-read',
'--allow-write',
splitScript,
inputFile,
'--output',
outputDir,
];
if (boundariesFile) {
args.push('--boundaries', boundariesFile);
} else if (byHeadings) {
args.push('--by-headings');
} else {
// Default to by-headings if no boundaries specified
console.log(' Using default: splitting by H2 headings');
args.push('--by-headings');
}
const result = await runCommand('deno', args);
if (result.stdout) console.log(result.stdout);
if (result.stderr) console.error(result.stderr);
return result.success;
}
async function generateAudio(outputDir: string, force: boolean): Promise<boolean> {
console.log('\n' + '='.repeat(60));
console.log('STEP 2: Generating TTS audio');
console.log('='.repeat(60));
const scenesDir = join(outputDir, 'scenes');
const narrateScript = join(SCRIPT_DIR, 'narrate-scenes.py');
// Check for Python venv
const venvPython = join(TTS_DIR, '.venv', 'bin', 'python');
const hasPythonVenv = await fileExists(venvPython);
let pythonCmd: string;
if (hasPythonVenv) {
pythonCmd = venvPython;
console.log(` Using venv Python: ${venvPython}`);
} else {
console.log(' Warning: No venv found at tts/.venv');
console.log(' Trying system Python...');
pythonCmd = 'python3';
}
const args = [narrateScript, scenesDir];
if (force) {
args.push('--force');
}
const result = await runCommand(pythonCmd, args);
if (result.stdout) console.log(result.stdout);
if (result.stderr) console.error(result.stderr);
return result.success;
}
async function transcribeAudio(
outputDir: string,
whisperPath: string,
model: string,
force: boolean
): Promise<boolean> {
console.log('\n' + '='.repeat(60));
console.log('STEP 3: Transcribing audio for word-level timing');
console.log('='.repeat(60));
const scenesDir = join(outputDir, 'scenes');
const transcribeScript = join(SCRIPT_DIR, 'transcribe-scenes.ts');
const args = [
'run',
'--allow-read',
'--allow-write',
'--allow-run',
transcribeScript,
scenesDir,
'--whisper-path',
whisperPath,
'--model',
model,
];
if (force) {
args.push('--force');
}
const result = await runCommand('deno', args);
if (result.stdout) console.log(result.stdout);
if (result.stderr) console.error(result.stderr);
return result.success;
}
// ============================================================================
// Main
// ============================================================================
function printHelp() {
console.log(`
full-pipeline.ts - Complete document-to-narration pipeline
USAGE:
deno run -A scripts/full-pipeline.ts <input.md> --output <dir> [options]
OPTIONS:
--output <dir> Output directory (required)
--by-headings Split by H2 headings (default if no boundaries)
--boundaries <file> Use agent-specified scene boundaries
--skip-tts Skip audio generation
--skip-transcribe Skip Whisper transcription
--force Force regeneration of all files
--whisper-path <dir> Path to whisper.cpp (default: ./whisper.cpp)
--whisper-model <name> Whisper model (default: medium)
--help, -h Show this help
PREREQUISITES:
1. Python venv at tts/.venv with requirements installed
2. whisper.cpp installed with model downloaded
EXAMPLE:
# Full pipeline with default settings
deno run -A scripts/full-pipeline.ts essay.md --output ./output/my-essay/
# Split only (no audio)
deno run -A scripts/full-pipeline.ts essay.md --output ./output/ --skip-tts --skip-transcribe
# With custom whisper settings
deno run -A scripts/full-pipeline.ts essay.md --output ./output/ --whisper-model large-v3-turbo
PIPELINE:
1. Split document into scene .txt files
2. Generate .wav audio for each scene (TTS)
3. Transcribe .wav to .vtt with word-level timing
4. Update manifest.json with complete data
`);
}
async function main() {
const args = Deno.args;
if (args.includes('--help') || args.includes('-h') || args.length === 0) {
printHelp();
Deno.exit(0);
}
// Parse arguments
const inputFile = args.find((a) => !a.startsWith('--'));
const byHeadings = args.includes('--by-headings');
const skipTTS = args.includes('--skip-tts');
const skipTranscribe = args.includes('--skip-transcribe');
const force = args.includes('--force');
const outputIdx = args.indexOf('--output');
const outputDir = outputIdx >= 0 ? args[outputIdx + 1] : null;
const boundariesIdx = args.indexOf('--boundaries');
const boundariesFile = boundariesIdx >= 0 ? args[boundariesIdx + 1] : undefined;
const whisperPathIdx = args.indexOf('--whisper-path');
const whisperPath = whisperPathIdx >= 0 ? args[whisperPathIdx + 1] : './whisper.cpp';
const whisperModelIdx = args.indexOf('--whisper-model');
const whisperModel = whisperModelIdx >= 0 ? args[whisperModelIdx + 1] : 'medium';
// Validate
if (!inputFile) {
console.error('Error: No input file specified');
Deno.exit(1);
}
if (!outputDir) {
console.error('Error: --output directory required');
Deno.exit(1);
}
if (!(await fileExists(inputFile))) {
console.error(`Error: Input file not found: ${inputFile}`);
Deno.exit(1);
}
// Create output directory
await Deno.mkdir(outputDir, { recursive: true });
console.log('Document to Narration Pipeline');
console.log('='.repeat(60));
console.log(`Input: ${inputFile}`);
console.log(`Output: ${outputDir}`);
console.log(`Skip TTS: ${skipTTS}`);
console.log(`Skip Transcribe: ${skipTranscribe}`);
// Step 1: Split document
const splitSuccess = await splitDocument(inputFile, outputDir, byHeadings, boundariesFile);
if (!splitSuccess) {
console.error('\nError: Document splitting failed');
Deno.exit(1);
}
// Step 2: Generate audio
if (!skipTTS) {
const audioSuccess = await generateAudio(outputDir, force);
if (!audioSuccess) {
console.error('\nWarning: Audio generation had errors');
// Continue anyway - some files may have succeeded
}
} else {
console.log('\n[Skipping TTS audio generation]');
}
// Step 3: Transcribe
if (!skipTTS && !skipTranscribe) {
const transcribeSuccess = await transcribeAudio(outputDir, whisperPath, whisperModel, force);
if (!transcribeSuccess) {
console.error('\nWarning: Transcription had errors');
}
} else if (skipTranscribe) {
console.log('\n[Skipping Whisper transcription]');
}
// Summary
console.log('\n' + '='.repeat(60));
console.log('PIPELINE COMPLETE');
console.log('='.repeat(60));
console.log(`Output directory: ${outputDir}`);
console.log(`\nGenerated files:`);
const scenesDir = join(outputDir, 'scenes');
if (await fileExists(scenesDir)) {
for await (const entry of Deno.readDir(scenesDir)) {
console.log(` ${entry.name}`);
}
}
const manifestPath = join(outputDir, 'manifest.json');
if (await fileExists(manifestPath)) {
console.log(`\nManifest: ${manifestPath}`);
}
}
main();
#!/usr/bin/env python3
"""
narrate-full.py - Generate a single TTS audio file from all scene files.
Combines all scene .txt files and generates a single consistent audio file,
avoiding volume inconsistencies that occur when concatenating separately
generated audio files.
Usage:
python narrate-full.py ./output/scenes/
python narrate-full.py ./output/scenes/ --force
python narrate-full.py ./output/scenes/ --speaker other_voice
"""
import argparse
import json
import sys
from pathlib import Path
import soundfile as sf
import torch
from qwen_tts import Qwen3TTSModel
# Directory setup (relative to script location)
SCRIPT_DIR = Path(__file__).parent.resolve()
SKILL_DIR = SCRIPT_DIR.parent
MODEL_DIR = SKILL_DIR / "tts" / "model"
# Default speaker name (from fine-tuning)
DEFAULT_SPEAKER = "jwynia"
def load_model():
"""Load the fine-tuned TTS model."""
print("Loading TTS model...")
print(f" Model path: {MODEL_DIR}")
if not MODEL_DIR.exists():
print(f"Error: Model directory not found at {MODEL_DIR}")
print("Make sure the model has been moved from inbox/portable-narrator/model/")
sys.exit(1)
# Determine device
if torch.backends.mps.is_available():
device = "mps"
elif torch.cuda.is_available():
device = "cuda"
else:
device = "cpu"
print(f" Device: {device}")
tts = Qwen3TTSModel.from_pretrained(
str(MODEL_DIR),
device_map=device,
dtype=torch.float32,
attn_implementation="sdpa",
)
print("Model loaded!\n")
return tts
def find_scene_files(scenes_dir: Path) -> list[Path]:
"""Find all .txt scene files in the directory."""
files = list(scenes_dir.glob("*.txt"))
# Sort by scene number (assumes NN-slug.txt format)
files.sort(key=lambda f: f.name)
return files
def combine_scene_texts(scene_files: list[Path]) -> str:
"""Combine all scene texts into a single string with paragraph breaks."""
texts = []
for f in scene_files:
text = f.read_text(encoding="utf-8").strip()
if text:
texts.append(text)
print(f" Added: {f.name} ({len(text.split())} words)")
# Join with double newline for natural pause between scenes
combined = "\n\n".join(texts)
total_words = len(combined.split())
print(f"\nCombined: {len(texts)} scenes, {total_words} words total")
return combined
def narrate_text(tts, text: str, output_path: Path, speaker: str) -> dict | None:
"""Generate speech for the combined text. Returns metadata or None on error."""
print(f"\nGenerating speech for {len(text.split())} words...")
print("(This may take a few minutes for longer texts)")
try:
wavs, sr = tts.generate_custom_voice(
text=text,
speaker=speaker,
)
except Exception as e:
print(f"Error generating speech: {e}")
return None
# Save output
sf.write(str(output_path), wavs[0], sr)
duration = len(wavs[0]) / sr
print(f"\nSaved: {output_path}")
print(f"Duration: {duration:.1f}s ({duration/60:.1f} minutes)")
print(f"Sample rate: {sr} Hz")
return {
"duration_seconds": duration,
"sample_rate": sr,
"samples": len(wavs[0]),
}
def main():
parser = argparse.ArgumentParser(
description="Generate single TTS audio from all scene files",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python narrate-full.py ./output/scenes/
python narrate-full.py ./output/scenes/ --force
python narrate-full.py ./output/scenes/ --speaker custom_voice
""",
)
parser.add_argument(
"scenes_dir",
type=Path,
help="Directory containing scene .txt files",
)
parser.add_argument(
"--force",
"-f",
action="store_true",
help="Force regeneration even if output exists",
)
parser.add_argument(
"--speaker",
"-s",
type=str,
default=DEFAULT_SPEAKER,
help=f"Speaker name (default: {DEFAULT_SPEAKER})",
)
parser.add_argument(
"--output",
"-o",
type=Path,
default=None,
help="Output file path (default: <scenes_dir>/../full-narration.wav)",
)
args = parser.parse_args()
scenes_dir = args.scenes_dir.resolve()
if not scenes_dir.exists():
print(f"Error: Directory not found: {scenes_dir}")
sys.exit(1)
# Determine output path
if args.output:
output_path = args.output.resolve()
else:
output_path = scenes_dir.parent / "full-narration.wav"
# Check if output exists
if output_path.exists() and not args.force:
print(f"Output already exists: {output_path}")
print("Use --force to regenerate.")
sys.exit(0)
# Find scene files
scene_files = find_scene_files(scenes_dir)
if not scene_files:
print(f"No .txt files found in: {scenes_dir}")
sys.exit(1)
print(f"Found {len(scene_files)} scene files:")
# Combine all scene texts
combined_text = combine_scene_texts(scene_files)
if not combined_text.strip():
print("Error: No text content found in scene files")
sys.exit(1)
# Load model
tts = load_model()
# Generate audio
result = narrate_text(tts, combined_text, output_path, args.speaker)
if result:
print("\n" + "=" * 40)
print("SUCCESS: Full narration generated!")
print(f"Output: {output_path}")
print(f"Duration: {result['duration_seconds']:.1f}s")
else:
print("\nFailed to generate narration")
sys.exit(1)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
narrate-scenes.py - Generate TTS audio for scene files.
Converts scene .txt files to .wav audio using the bundled fine-tuned voice model.
Usage:
# Process all txt files in scenes directory
python narrate-scenes.py ./output/scenes/
# Force regeneration of existing outputs
python narrate-scenes.py ./output/scenes/ --force
# Use a different speaker
python narrate-scenes.py ./output/scenes/ --speaker other_voice
"""
import argparse
import json
import os
import sys
from pathlib import Path
import soundfile as sf
import torch
from qwen_tts import Qwen3TTSModel
# Directory setup (relative to script location)
SCRIPT_DIR = Path(__file__).parent.resolve()
SKILL_DIR = SCRIPT_DIR.parent
MODEL_DIR = SKILL_DIR / "tts" / "model"
# Default speaker name (from fine-tuning)
DEFAULT_SPEAKER = "jwynia"
def load_model():
"""Load the fine-tuned TTS model."""
print("Loading TTS model...")
print(f" Model path: {MODEL_DIR}")
if not MODEL_DIR.exists():
print(f"Error: Model directory not found at {MODEL_DIR}")
print("Make sure the model has been moved from inbox/portable-narrator/model/")
sys.exit(1)
# Determine device
if torch.backends.mps.is_available():
device = "mps"
elif torch.cuda.is_available():
device = "cuda"
else:
device = "cpu"
print(f" Device: {device}")
tts = Qwen3TTSModel.from_pretrained(
str(MODEL_DIR),
device_map=device,
dtype=torch.float32,
attn_implementation="sdpa",
)
print("Model loaded!\n")
return tts
def find_scene_files(scenes_dir: Path) -> list[Path]:
"""Find all .txt scene files in the directory."""
files = list(scenes_dir.glob("*.txt"))
# Sort by scene number (assumes NN-slug.txt format)
files.sort(key=lambda f: f.name)
return files
def get_output_path(input_path: Path) -> Path:
"""Get the output path for an input file."""
return input_path.with_suffix(".wav")
def read_text_file(filepath: Path) -> str:
"""Read and return the contents of a text file."""
with open(filepath, "r", encoding="utf-8") as f:
return f.read().strip()
def narrate_file(tts, input_path: Path, output_path: Path, speaker: str) -> dict | None:
"""Generate speech for a single file. Returns metadata or None on error."""
print(f"Processing: {input_path.name}")
# Read text
text = read_text_file(input_path)
if not text:
print(f" Skipping: file is empty")
return None
# Show text preview
preview = text[:100].replace("\n", " ")
if len(text) > 100:
preview += "..."
print(f" Text: {preview}")
# Generate speech
print(f" Generating speech...")
try:
wavs, sr = tts.generate_custom_voice(
text=text,
speaker=speaker,
)
except Exception as e:
print(f" Error generating speech: {e}")
return None
# Save output
sf.write(str(output_path), wavs[0], sr)
duration = len(wavs[0]) / sr
print(f" Saved: {output_path.name} ({duration:.1f}s)")
return {
"duration_seconds": duration,
"sample_rate": sr,
"samples": len(wavs[0]),
}
def update_manifest(scenes_dir: Path, audio_metadata: dict[str, dict]):
"""Update manifest.json with audio information."""
manifest_path = scenes_dir.parent / "manifest.json"
if not manifest_path.exists():
print(f"Warning: No manifest.json found at {manifest_path}")
return
with open(manifest_path, "r") as f:
manifest = json.load(f)
# Update each scene with audio info
for scene in manifest.get("scenes", []):
text_file = scene.get("files", {}).get("text", "")
if text_file:
# Convert text filename to audio filename
txt_name = Path(text_file).name
wav_name = txt_name.replace(".txt", ".wav")
if wav_name in audio_metadata:
meta = audio_metadata[wav_name]
scene["audio_duration_seconds"] = meta["duration_seconds"]
scene["files"]["audio"] = f"scenes/{wav_name}"
# Calculate total duration
total_duration = sum(
s.get("audio_duration_seconds", 0) for s in manifest.get("scenes", [])
)
manifest["total_duration_seconds"] = total_duration
# Write updated manifest
with open(manifest_path, "w") as f:
json.dump(manifest, f, indent=2)
print(f"\nUpdated manifest: {manifest_path}")
print(f"Total audio duration: {total_duration:.1f}s")
def main():
parser = argparse.ArgumentParser(
description="Generate TTS audio for scene files",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python narrate-scenes.py ./output/scenes/
python narrate-scenes.py ./output/scenes/ --force
python narrate-scenes.py ./output/scenes/ --speaker custom_voice
""",
)
parser.add_argument(
"scenes_dir",
type=Path,
help="Directory containing scene .txt files",
)
parser.add_argument(
"--force",
"-f",
action="store_true",
help="Force regeneration even if output exists",
)
parser.add_argument(
"--speaker",
"-s",
type=str,
default=DEFAULT_SPEAKER,
help=f"Speaker name (default: {DEFAULT_SPEAKER})",
)
args = parser.parse_args()
scenes_dir = args.scenes_dir.resolve()
if not scenes_dir.exists():
print(f"Error: Directory not found: {scenes_dir}")
sys.exit(1)
# Find scene files
scene_files = find_scene_files(scenes_dir)
if not scene_files:
print(f"No .txt files found in: {scenes_dir}")
sys.exit(0)
# Filter out files that already have output (unless --force)
files_to_process = []
for input_path in scene_files:
output_path = get_output_path(input_path)
if output_path.exists() and not args.force:
print(f"Skipping (output exists): {input_path.name}")
else:
files_to_process.append((input_path, output_path))
if not files_to_process:
print("\nNo new files to process. Use --force to regenerate.")
sys.exit(0)
print(f"\nFiles to process: {len(files_to_process)}")
print("-" * 40)
# Load model
tts = load_model()
# Process each file
audio_metadata = {}
success_count = 0
for input_path, output_path in files_to_process:
try:
result = narrate_file(tts, input_path, output_path, args.speaker)
if result:
audio_metadata[output_path.name] = result
success_count += 1
except Exception as e:
print(f" Error: {e}")
print()
print("-" * 40)
print(f"Completed: {success_count}/{len(files_to_process)} files")
# Update manifest with audio info
if audio_metadata:
update_manifest(scenes_dir, audio_metadata)
if __name__ == "__main__":
main()
#!/usr/bin/env -S deno run --allow-read --allow-write
/**
* split-to-scenes.ts
*
* Parse a markdown document and analyze its structure for scene splitting.
* This script handles the mechanical aspects - the agent determines boundaries.
*
* Usage:
* # Analyze document structure
* deno run --allow-read scripts/split-to-scenes.ts input.md --analyze
*
* # Split using boundaries file (agent-generated)
* deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --boundaries boundaries.json --output ./output/
*
* # Split by headings (simple heuristic fallback)
* deno run --allow-read --allow-write scripts/split-to-scenes.ts input.md --by-headings --output ./output/
*
* # Dry run
* deno run --allow-read scripts/split-to-scenes.ts input.md --boundaries boundaries.json --dry-run
*/
// ============================================================================
// Types
// ============================================================================
interface DocumentSection {
type: 'heading' | 'paragraph' | 'list' | 'blockquote' | 'code';
level?: number; // For headings
text: string;
lineStart: number;
lineEnd: number;
wordCount: number;
}
interface DocumentAnalysis {
filename: string;
totalLines: number;
totalWords: number;
sections: DocumentSection[];
headings: { level: number; text: string; lineNumber: number }[];
suggestedBreakpoints: number[]; // Line numbers where breaks might occur
}
interface SceneBoundary {
sceneNumber: number;
slug: string;
startLine: number;
endLine: number;
content?: string; // Optional: agent can provide pre-processed content
}
interface BoundariesSpec {
source: string;
scenes: SceneBoundary[];
}
interface ManifestScene {
number: number;
slug: string;
word_count: number;
files: {
text: string;
};
source_lines: {
start: number;
end: number;
};
}
interface Manifest {
source: string;
created_at: string;
total_scenes: number;
scenes: ManifestScene[];
}
// ============================================================================
// Parsing
// ============================================================================
function parseMarkdown(content: string): DocumentSection[] {
const lines = content.split('\n');
const sections: DocumentSection[] = [];
let currentParagraph: string[] = [];
let paragraphStart = 0;
const flushParagraph = (endLine: number) => {
if (currentParagraph.length > 0) {
const text = currentParagraph.join('\n').trim();
if (text) {
sections.push({
type: 'paragraph',
text,
lineStart: paragraphStart,
lineEnd: endLine - 1,
wordCount: countWords(text),
});
}
currentParagraph = [];
}
};
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
const lineNum = i + 1;
// Heading
const headingMatch = line.match(/^(#{1,6})\s+(.+)$/);
if (headingMatch) {
flushParagraph(lineNum);
sections.push({
type: 'heading',
level: headingMatch[1].length,
text: headingMatch[2].trim(),
lineStart: lineNum,
lineEnd: lineNum,
wordCount: countWords(headingMatch[2]),
});
paragraphStart = lineNum + 1;
continue;
}
// Blockquote
if (line.startsWith('>')) {
flushParagraph(lineNum);
const text = line.replace(/^>\s*/, '');
sections.push({
type: 'blockquote',
text,
lineStart: lineNum,
lineEnd: lineNum,
wordCount: countWords(text),
});
paragraphStart = lineNum + 1;
continue;
}
// Code block start
if (line.startsWith('```')) {
flushParagraph(lineNum);
const codeStart = lineNum;
let codeEnd = lineNum;
const codeLines: string[] = [line];
// Find end of code block
for (let j = i + 1; j < lines.length; j++) {
codeLines.push(lines[j]);
if (lines[j].startsWith('```')) {
codeEnd = j + 1;
i = j;
break;
}
}
sections.push({
type: 'code',
text: codeLines.join('\n'),
lineStart: codeStart,
lineEnd: codeEnd,
wordCount: 0, // Code blocks don't count toward word count
});
paragraphStart = codeEnd + 1;
continue;
}
// List item
if (line.match(/^[\-\*\+]\s/) || line.match(/^\d+\.\s/)) {
flushParagraph(lineNum);
const text = line.replace(/^[\-\*\+]\s/, '').replace(/^\d+\.\s/, '');
sections.push({
type: 'list',
text,
lineStart: lineNum,
lineEnd: lineNum,
wordCount: countWords(text),
});
paragraphStart = lineNum + 1;
continue;
}
// Empty line
if (line.trim() === '') {
flushParagraph(lineNum);
paragraphStart = lineNum + 1;
continue;
}
// Regular paragraph content
if (currentParagraph.length === 0) {
paragraphStart = lineNum;
}
currentParagraph.push(line);
}
flushParagraph(lines.length + 1);
return sections;
}
function countWords(text: string): number {
return text
.replace(/[#*_`\[\]()]/g, '')
.split(/\s+/)
.filter((w) => w.length > 0).length;
}
function analyzeDocument(filename: string, content: string): DocumentAnalysis {
const sections = parseMarkdown(content);
const lines = content.split('\n');
const headings = sections
.filter((s) => s.type === 'heading')
.map((s) => ({
level: s.level!,
text: s.text,
lineNumber: s.lineStart,
}));
// Suggest breakpoints: after each H2 heading's content
const suggestedBreakpoints: number[] = [];
for (let i = 0; i < sections.length; i++) {
const section = sections[i];
if (section.type === 'heading' && section.level === 2) {
// Find the next H2 or end of document
let breakLine = lines.length;
for (let j = i + 1; j < sections.length; j++) {
if (sections[j].type === 'heading' && sections[j].level! <= 2) {
breakLine = sections[j].lineStart - 1;
break;
}
}
suggestedBreakpoints.push(breakLine);
}
}
return {
filename,
totalLines: lines.length,
totalWords: sections.reduce((sum, s) => sum + s.wordCount, 0),
sections,
headings,
suggestedBreakpoints,
};
}
// ============================================================================
// Scene Extraction
// ============================================================================
function extractSceneContent(
content: string,
startLine: number,
endLine: number
): string {
const lines = content.split('\n');
return lines.slice(startLine - 1, endLine).join('\n').trim();
}
function generateSlug(text: string): string {
return text
.toLowerCase()
.replace(/[^a-z0-9\s-]/g, '')
.replace(/\s+/g, '-')
.slice(0, 30)
.replace(/-+$/, '');
}
function splitByHeadings(
content: string,
analysis: DocumentAnalysis
): SceneBoundary[] {
const scenes: SceneBoundary[] = [];
const h2Headings = analysis.headings.filter((h) => h.level === 2);
if (h2Headings.length === 0) {
// No H2 headings, treat entire document as one scene
return [
{
sceneNumber: 1,
slug: generateSlug(analysis.headings[0]?.text || 'document'),
startLine: 1,
endLine: analysis.totalLines,
},
];
}
// Content before first H2 (if any)
if (h2Headings[0].lineNumber > 1) {
const firstSection = analysis.sections[0];
const slugText =
firstSection?.type === 'heading' ? firstSection.text : 'opening';
scenes.push({
sceneNumber: 1,
slug: generateSlug(slugText),
startLine: 1,
endLine: h2Headings[0].lineNumber - 1,
});
}
// Each H2 section
for (let i = 0; i < h2Headings.length; i++) {
const heading = h2Headings[i];
const nextHeading = h2Headings[i + 1];
const endLine = nextHeading
? nextHeading.lineNumber - 1
: analysis.totalLines;
scenes.push({
sceneNumber: scenes.length + 1,
slug: generateSlug(heading.text),
startLine: heading.lineNumber,
endLine,
});
}
return scenes;
}
// ============================================================================
// Output
// ============================================================================
async function writeScenes(
content: string,
boundaries: SceneBoundary[],
outputDir: string,
dryRun: boolean
): Promise<Manifest> {
const scenesDir = `${outputDir}/scenes`;
if (!dryRun) {
await Deno.mkdir(scenesDir, { recursive: true });
}
const manifestScenes: ManifestScene[] = [];
for (const boundary of boundaries) {
const paddedNum = String(boundary.sceneNumber).padStart(2, '0');
const filename = `${paddedNum}-${boundary.slug}.txt`;
const filepath = `${scenesDir}/${filename}`;
const sceneContent =
boundary.content ||
extractSceneContent(content, boundary.startLine, boundary.endLine);
const wordCount = countWords(sceneContent);
if (dryRun) {
console.log(`\n[Scene ${boundary.sceneNumber}] ${filename}`);
console.log(` Lines: ${boundary.startLine}-${boundary.endLine}`);
console.log(` Words: ${wordCount}`);
console.log(` Preview: ${sceneContent.slice(0, 100)}...`);
} else {
await Deno.writeTextFile(filepath, sceneContent);
console.log(`Wrote: ${filepath} (${wordCount} words)`);
}
manifestScenes.push({
number: boundary.sceneNumber,
slug: boundary.slug,
word_count: wordCount,
files: {
text: `scenes/${filename}`,
},
source_lines: {
start: boundary.startLine,
end: boundary.endLine,
},
});
}
const manifest: Manifest = {
source: '',
created_at: new Date().toISOString(),
total_scenes: boundaries.length,
scenes: manifestScenes,
};
if (!dryRun) {
const manifestPath = `${outputDir}/manifest.json`;
await Deno.writeTextFile(manifestPath, JSON.stringify(manifest, null, 2));
console.log(`\nWrote: ${manifestPath}`);
}
return manifest;
}
// ============================================================================
// Main
// ============================================================================
function printHelp() {
console.log(`
split-to-scenes.ts - Parse documents for scene splitting
USAGE:
deno run --allow-read scripts/split-to-scenes.ts <input.md> [options]
OPTIONS:
--analyze Analyze document structure and output JSON
--by-headings Split document by H2 headings (simple heuristic)
--boundaries <file> Use agent-generated boundaries JSON file
--output <dir> Output directory for scene files
--dry-run Preview splits without writing files
--help, -h Show this help
MODES:
1. Analyze mode: Inspect document structure
deno run --allow-read scripts/split-to-scenes.ts doc.md --analyze
2. Headings mode: Split by H2 headings
deno run --allow-read --allow-write scripts/split-to-scenes.ts doc.md --by-headings --output ./out
3. Boundaries mode: Use agent-specified boundaries
deno run --allow-read --allow-write scripts/split-to-scenes.ts doc.md --boundaries b.json --output ./out
BOUNDARIES FILE FORMAT:
{
"source": "document.md",
"scenes": [
{ "sceneNumber": 1, "slug": "opening", "startLine": 1, "endLine": 15 },
{ "sceneNumber": 2, "slug": "main-point", "startLine": 16, "endLine": 45 }
]
}
Or with pre-processed content:
{
"scenes": [
{ "sceneNumber": 1, "slug": "opening", "startLine": 1, "endLine": 15,
"content": "Adapted spoken text here..." }
]
}
`);
}
async function main() {
const args = Deno.args;
if (args.includes('--help') || args.includes('-h') || args.length === 0) {
printHelp();
Deno.exit(0);
}
// Parse arguments
const inputFile = args.find((a) => !a.startsWith('--'));
const analyze = args.includes('--analyze');
const byHeadings = args.includes('--by-headings');
const dryRun = args.includes('--dry-run');
const boundariesIdx = args.indexOf('--boundaries');
const boundariesFile = boundariesIdx >= 0 ? args[boundariesIdx + 1] : null;
const outputIdx = args.indexOf('--output');
const outputDir = outputIdx >= 0 ? args[outputIdx + 1] : null;
if (!inputFile) {
console.error('Error: No input file specified');
Deno.exit(1);
}
// Read input document
let content: string;
try {
content = await Deno.readTextFile(inputFile);
} catch {
console.error(`Error: Could not read file: ${inputFile}`);
Deno.exit(1);
}
const analysis = analyzeDocument(inputFile, content);
// Mode: Analyze
if (analyze) {
console.log(JSON.stringify(analysis, null, 2));
Deno.exit(0);
}
// Mode: Split by headings or boundaries
if (!outputDir && !dryRun) {
console.error('Error: --output directory required (or use --dry-run)');
Deno.exit(1);
}
let boundaries: SceneBoundary[];
if (boundariesFile) {
// Use agent-specified boundaries
try {
const boundariesContent = await Deno.readTextFile(boundariesFile);
const spec: BoundariesSpec = JSON.parse(boundariesContent);
boundaries = spec.scenes;
} catch (e) {
console.error(`Error: Could not read boundaries file: ${e}`);
Deno.exit(1);
}
} else if (byHeadings) {
// Simple heuristic: split by H2 headings
boundaries = splitByHeadings(content, analysis);
} else {
console.error('Error: Specify --by-headings or --boundaries <file>');
Deno.exit(1);
}
// Write scenes
const manifest = await writeScenes(
content,
boundaries,
outputDir || './output',
dryRun
);
if (!dryRun) {
manifest.source = inputFile;
const manifestPath = `${outputDir}/manifest.json`;
await Deno.writeTextFile(manifestPath, JSON.stringify(manifest, null, 2));
}
console.log(`\nTotal scenes: ${boundaries.length}`);
}
main();
#!/usr/bin/env python3
"""
transcribe-full.py - Generate word-level timestamps from audio using Whisper.
Transcribes an audio file and generates:
1. VTT file with word-level timestamps (for video players)
2. JSON file with captions (for Remotion CaptionRenderer)
Usage:
python transcribe-full.py ./narration/full-narration.wav
python transcribe-full.py ./narration/full-narration.wav --model medium
"""
import argparse
import json
import sys
from pathlib import Path
try:
import whisper
except ImportError:
print("Error: openai-whisper not installed")
print("Install with: pip install openai-whisper")
sys.exit(1)
def format_vtt_timestamp(seconds: float) -> str:
"""Convert seconds to VTT timestamp format (MM:SS.mmm)."""
minutes = int(seconds // 60)
secs = seconds % 60
return f"{minutes:02d}:{secs:06.3f}"
def transcribe_with_timestamps(audio_path: Path, model_name: str = "medium") -> dict:
"""Transcribe audio with word-level timestamps."""
print(f"Loading Whisper model '{model_name}'...")
model = whisper.load_model(model_name)
print(f"Transcribing: {audio_path}")
print("(This may take a few minutes...)")
result = model.transcribe(
str(audio_path),
word_timestamps=True,
language="en",
verbose=False,
)
return result
def to_vtt(result: dict, output_path: Path):
"""Convert whisper result to VTT format with word-level timestamps."""
with open(output_path, "w", encoding="utf-8") as f:
f.write("WEBVTT\n\n")
cue_index = 1
for segment in result.get("segments", []):
words = segment.get("words", [])
if not words:
continue
for word_info in words:
word = word_info.get("word", "").strip()
if not word:
continue
start = format_vtt_timestamp(word_info["start"])
end = format_vtt_timestamp(word_info["end"])
f.write(f"{cue_index}\n")
f.write(f"{start} --> {end}\n")
f.write(f"{word}\n\n")
cue_index += 1
print(f"Saved VTT: {output_path} ({cue_index - 1} cues)")
def to_json(result: dict, output_path: Path):
"""Convert whisper result to JSON for Remotion captions."""
captions = []
for segment in result.get("segments", []):
words = segment.get("words", [])
if not words:
continue
for word_info in words:
word = word_info.get("word", "").strip()
if not word:
continue
captions.append({
"text": word,
"startMs": int(word_info["start"] * 1000),
"endMs": int(word_info["end"] * 1000),
"confidence": word_info.get("probability", 1.0),
})
output_data = {
"segments": result.get("segments", []),
"captions": captions,
"text": result.get("text", ""),
"language": result.get("language", "en"),
}
with open(output_path, "w", encoding="utf-8") as f:
json.dump(output_data, f, indent=2, ensure_ascii=False)
print(f"Saved JSON: {output_path} ({len(captions)} words)")
def main():
parser = argparse.ArgumentParser(
description="Transcribe audio with word-level timestamps using Whisper",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
python transcribe-full.py ./narration/full-narration.wav
python transcribe-full.py ./narration/full-narration.wav --model large-v3
python transcribe-full.py audio.wav --output-dir ./output/
""",
)
parser.add_argument(
"audio_path",
type=Path,
help="Path to audio file (WAV, MP3, etc.)",
)
parser.add_argument(
"--model",
"-m",
type=str,
default="medium",
choices=["tiny", "base", "small", "medium", "large", "large-v2", "large-v3"],
help="Whisper model size (default: medium)",
)
parser.add_argument(
"--output-dir",
"-o",
type=Path,
default=None,
help="Output directory (default: same as audio file)",
)
args = parser.parse_args()
audio_path = args.audio_path.resolve()
if not audio_path.exists():
print(f"Error: Audio file not found: {audio_path}")
sys.exit(1)
# Determine output paths
if args.output_dir:
output_dir = args.output_dir.resolve()
output_dir.mkdir(parents=True, exist_ok=True)
else:
output_dir = audio_path.parent
base_name = audio_path.stem
vtt_path = output_dir / f"{base_name}.vtt"
json_path = output_dir / f"{base_name}.json"
# Transcribe
result = transcribe_with_timestamps(audio_path, args.model)
# Check if we got word-level timestamps
has_words = any(
segment.get("words")
for segment in result.get("segments", [])
)
if not has_words:
print("Warning: No word-level timestamps found in transcription")
print("This may happen with some audio files or model versions")
# Generate outputs
print("\nGenerating output files...")
to_vtt(result, vtt_path)
to_json(result, json_path)
# Summary
print("\n" + "=" * 40)
print("SUCCESS: Transcription complete!")
print(f"VTT: {vtt_path}")
print(f"JSON: {json_path}")
# Word count
word_count = sum(
len(segment.get("words", []))
for segment in result.get("segments", [])
)
print(f"Words: {word_count}")
if __name__ == "__main__":
main()
#!/usr/bin/env -S deno run --allow-read --allow-write --allow-run
/**
* transcribe-scenes.ts
*
* Transcribe audio files to get word-level timestamps using Whisper.
* Outputs VTT files and updates the manifest with caption data.
*
* Usage:
* deno run --allow-read --allow-write --allow-run scripts/transcribe-scenes.ts ./output/scenes/
* deno run --allow-read --allow-write --allow-run scripts/transcribe-scenes.ts ./output/scenes/ --model large-v3-turbo
* deno run --allow-read --allow-write --allow-run scripts/transcribe-scenes.ts ./output/scenes/ --whisper-path ./whisper.cpp
*/
// ============================================================================
// Types
// ============================================================================
interface WhisperToken {
t_dtw: number;
text: string;
timestamps: { from: string; to: string };
offsets: { from: number; to: number };
id: number;
p: number;
}
interface WhisperTranscriptionItem {
timestamps: { from: string; to: string };
offsets: { from: number; to: number };
text: string;
tokens: WhisperToken[];
}
interface WhisperOutput {
transcription: WhisperTranscriptionItem[];
}
interface Caption {
text: string;
startMs: number;
endMs: number;
confidence: number;
}
interface ManifestScene {
number: number;
slug: string;
word_count: number;
audio_duration_seconds?: number;
files: {
text: string;
audio?: string;
captions?: string;
};
captions?: Caption[];
}
interface Manifest {
source: string;
created_at: string;
total_scenes: number;
total_duration_seconds?: number;
scenes: ManifestScene[];
}
// ============================================================================
// Audio Conversion
// ============================================================================
async function convertTo16kHz(
inputPath: string,
outputPath: string
): Promise<boolean> {
console.log(` Converting to 16kHz: ${inputPath}`);
const cmd = new Deno.Command('ffmpeg', {
args: [
'-i',
inputPath,
'-ar',
'16000',
'-ac',
'1',
'-y',
outputPath,
],
stdout: 'piped',
stderr: 'piped',
});
const result = await cmd.output();
if (!result.success) {
const stderr = new TextDecoder().decode(result.stderr);
console.error(` ffmpeg error: ${stderr}`);
return false;
}
return true;
}
// ============================================================================
// Whisper Transcription
// ============================================================================
async function transcribeWithWhisper(
wavPath: string,
whisperPath: string,
model: string
): Promise<WhisperOutput | null> {
const modelPath = `${whisperPath}/models/ggml-${model}.bin`;
const mainPath = `${whisperPath}/main`;
// Check if whisper is installed
try {
await Deno.stat(mainPath);
} catch {
console.error(` Whisper not found at: ${mainPath}`);
console.error(' Install with: npx @remotion/install-whisper-cpp');
return null;
}
// Check if model exists
try {
await Deno.stat(modelPath);
} catch {
console.error(` Model not found: ${modelPath}`);
console.error(` Download with: npx @remotion/install-whisper-cpp --model ${model}`);
return null;
}
const tmpJsonPath = `${wavPath}.json`;
const cmd = new Deno.Command(mainPath, {
args: [
'-f',
wavPath,
'--output-file',
tmpJsonPath,
'--output-json',
'-ojf',
'--dtw',
model.replace('-', '.'),
'-m',
modelPath,
'-pp',
'-l',
'en',
],
cwd: whisperPath,
stdout: 'piped',
stderr: 'piped',
});
console.log(` Running Whisper transcription...`);
const result = await cmd.output();
// Read the output JSON
const jsonPath = `${tmpJsonPath}.json`;
try {
const jsonContent = await Deno.readTextFile(jsonPath);
await Deno.remove(jsonPath);
return JSON.parse(jsonContent);
} catch (e) {
if (!result.success) {
const stderr = new TextDecoder().decode(result.stderr);
console.error(` Whisper error: ${stderr}`);
}
console.error(` Could not read transcription output: ${e}`);
return null;
}
}
// ============================================================================
// Caption Processing
// ============================================================================
function whisperToCaptions(whisperOutput: WhisperOutput): Caption[] {
const captions: Caption[] = [];
for (const item of whisperOutput.transcription) {
if (item.text === '' || !item.tokens) continue;
// Get word-level timing from tokens
for (const token of item.tokens) {
const text = token.text.trim();
if (!text) continue;
// t_dtw is in centiseconds (10ms units)
const startMs = token.t_dtw >= 0 ? token.t_dtw * 10 : item.offsets.from;
const endMs = token.offsets.to;
captions.push({
text,
startMs,
endMs,
confidence: token.p,
});
}
}
return captions;
}
function captionsToVTT(captions: Caption[]): string {
const lines: string[] = ['WEBVTT', ''];
for (const caption of captions) {
const startTime = formatVTTTime(caption.startMs);
const endTime = formatVTTTime(caption.endMs);
lines.push(`${startTime} --> ${endTime}`);
lines.push(caption.text);
lines.push('');
}
return lines.join('\n');
}
function formatVTTTime(ms: number): string {
const totalSeconds = Math.floor(ms / 1000);
const minutes = Math.floor(totalSeconds / 60);
const seconds = totalSeconds % 60;
const milliseconds = ms % 1000;
return `${String(minutes).padStart(2, '0')}:${String(seconds).padStart(2, '0')}.${String(milliseconds).padStart(3, '0')}`;
}
// ============================================================================
// Main Processing
// ============================================================================
async function processScene(
wavPath: string,
whisperPath: string,
model: string
): Promise<{ vttPath: string; captions: Caption[] } | null> {
// Convert to 16kHz if needed
const wav16kPath = wavPath.replace('.wav', '_16k.wav');
if (!(await convertTo16kHz(wavPath, wav16kPath))) {
return null;
}
// Transcribe
const whisperOutput = await transcribeWithWhisper(wav16kPath, whisperPath, model);
// Clean up temp file
try {
await Deno.remove(wav16kPath);
} catch {
// Ignore cleanup errors
}
if (!whisperOutput) {
return null;
}
// Convert to captions
const captions = whisperToCaptions(whisperOutput);
// Write VTT
const vttPath = wavPath.replace('.wav', '.vtt');
const vttContent = captionsToVTT(captions);
await Deno.writeTextFile(vttPath, vttContent);
console.log(` Wrote: ${vttPath} (${captions.length} words)`);
return { vttPath, captions };
}
async function updateManifest(
manifestPath: string,
sceneResults: Map<string, Caption[]>
): Promise<void> {
let manifest: Manifest;
try {
const content = await Deno.readTextFile(manifestPath);
manifest = JSON.parse(content);
} catch {
console.error(`Warning: Could not read manifest: ${manifestPath}`);
return;
}
for (const scene of manifest.scenes) {
const audioFile = scene.files.audio;
if (!audioFile) continue;
const wavName = audioFile.split('/').pop()!;
const captions = sceneResults.get(wavName);
if (captions) {
scene.files.captions = audioFile.replace('.wav', '.vtt');
scene.captions = captions;
}
}
await Deno.writeTextFile(manifestPath, JSON.stringify(manifest, null, 2));
console.log(`\nUpdated manifest: ${manifestPath}`);
}
// ============================================================================
// Main
// ============================================================================
function printHelp() {
console.log(`
transcribe-scenes.ts - Transcribe audio for word-level timing
USAGE:
deno run --allow-read --allow-write --allow-run scripts/transcribe-scenes.ts <scenes-dir> [options]
OPTIONS:
--model <name> Whisper model (default: medium)
Options: tiny, base, small, medium, large-v2, large-v3, large-v3-turbo
--whisper-path <dir> Path to whisper.cpp installation (default: ./whisper.cpp)
--force Re-transcribe even if VTT exists
--help, -h Show this help
REQUIREMENTS:
- ffmpeg installed and in PATH
- whisper.cpp installed (use @remotion/install-whisper-cpp)
- Whisper model downloaded
EXAMPLE:
# First install whisper
npx @remotion/install-whisper-cpp --to ./whisper.cpp
npx @remotion/install-whisper-cpp --download-model --model medium --to ./whisper.cpp
# Then transcribe
deno run -A scripts/transcribe-scenes.ts ./output/scenes/ --whisper-path ./whisper.cpp
`);
}
async function main() {
const args = Deno.args;
if (args.includes('--help') || args.includes('-h') || args.length === 0) {
printHelp();
Deno.exit(0);
}
// Parse arguments
const scenesDir = args.find((a) => !a.startsWith('--'));
const force = args.includes('--force');
const modelIdx = args.indexOf('--model');
const model = modelIdx >= 0 ? args[modelIdx + 1] : 'medium';
const whisperIdx = args.indexOf('--whisper-path');
const whisperPath = whisperIdx >= 0 ? args[whisperIdx + 1] : './whisper.cpp';
if (!scenesDir) {
console.error('Error: No scenes directory specified');
Deno.exit(1);
}
// Find WAV files
const wavFiles: string[] = [];
for await (const entry of Deno.readDir(scenesDir)) {
if (entry.isFile && entry.name.endsWith('.wav')) {
const vttPath = `${scenesDir}/${entry.name.replace('.wav', '.vtt')}`;
const vttExists = await Deno.stat(vttPath).then(() => true).catch(() => false);
if (!vttExists || force) {
wavFiles.push(`${scenesDir}/${entry.name}`);
} else {
console.log(`Skipping (VTT exists): ${entry.name}`);
}
}
}
wavFiles.sort();
if (wavFiles.length === 0) {
console.log('No WAV files to process. Use --force to re-transcribe.');
Deno.exit(0);
}
console.log(`\nFiles to transcribe: ${wavFiles.length}`);
console.log(`Model: ${model}`);
console.log(`Whisper path: ${whisperPath}`);
console.log('-'.repeat(40));
// Process each file
const sceneResults = new Map<string, Caption[]>();
for (const wavPath of wavFiles) {
const filename = wavPath.split('/').pop()!;
console.log(`\nProcessing: ${filename}`);
const result = await processScene(wavPath, whisperPath, model);
if (result) {
sceneResults.set(filename, result.captions);
}
}
// Update manifest
const manifestPath = `${scenesDir}/../manifest.json`;
await updateManifest(manifestPath, sceneResults);
console.log(`\nCompleted: ${sceneResults.size}/${wavFiles.length} files`);
}
main();
# TTS Model Directory
This directory should contain the Qwen3-TTS fine-tuned voice model.
The model files are not included in git due to size (~7.8GB).
## Required Files
Place the following files here:
- config.json
- generation_config.json
- model.safetensors (main model weights)
- tokenizer_config.json
- vocab.json
- merges.txt
- preprocessor_config.json
- speech_tokenizer/ (subdirectory with tokenizer files)
## Model Source
The model was fine-tuned using qwen-tts with custom speaker embeddings.
# TTS dependencies for document-to-narration skill
qwen-tts
soundfile
torch
#!/bin/bash
# Setup Python virtual environment for TTS
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$SCRIPT_DIR"
echo "Setting up Python virtual environment for TTS..."
# Check Python version
PYTHON_CMD=""
if command -v python3.12 &> /dev/null; then
PYTHON_CMD="python3.12"
elif command -v python3 &> /dev/null; then
PYTHON_CMD="python3"
else
echo "Error: Python 3 not found"
exit 1
fi
echo "Using Python: $($PYTHON_CMD --version)"
# Create venv
if [ ! -d ".venv" ]; then
echo "Creating virtual environment..."
$PYTHON_CMD -m venv .venv
fi
# Activate and install
echo "Installing dependencies..."
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
echo ""
echo "Setup complete!"
echo ""
echo "To activate the environment:"
echo " source $SCRIPT_DIR/.venv/bin/activate"
echo ""
echo "To run TTS:"
echo " python ../scripts/narrate-scenes.py ./output/scenes/"
Related skills
FAQ
What does document-to-narration output?
document-to-narration outputs listenable narration scripts adapted from long documents, reports, or chapters. The scripts are formatted for podcasts, audiobooks, or agent voice synthesis rather than silent reading.
What input formats does document-to-narration accept?
document-to-narration works with long-form written material including documentation, reports, and book chapters. The skill rewrites prose for audio pacing without requiring the developer to manually edit each paragraph.