Whisper Transcribe

Whisper Transcribe Skill

Transcribe audio and video files to text using OpenAI's Whisper with contextual grounding from markdown files.

Purpose

Intelligent audio/video transcription that: 1. Converts media files to accurate text transcripts 2. Uses markdown context files to correct technical terms, names, and jargon 3. Handles various audio/video formats (mp3, wav, m4a, mp4, webm, etc.)

When to Use

• User asks to transcribe an audio or video file
• User wants to convert a recording to text
• User mentions "whisper" in context of transcription
• User needs meeting notes or interview transcripts
• User has media files with domain-specific terminology

Installation

macOS (Recommended for MacBook Pro)

# Install via Homebrew (recommended)
brew install ffmpeg openai-whisper

# Verify installation
whisper --version

Linux/pip Installation

# Install ffmpeg first
sudo apt install ffmpeg  # Debian/Ubuntu
# or: sudo dnf install ffmpeg  # Fedora

# Install Whisper
pip install openai-whisper

Verify Installation

whisper --version
ffmpeg -version

Transcription Workflow

Step 1: Identify Media File and Context

1. Locate the audio/video file to transcribe 2. Check for markdown files in the same directory (context files) 3. If no context files exist, optionally create one using assets/context-template.md

Step 2: Run Whisper Transcription

Basic transcription:

whisper "/path/to/audio.mp3" --output_dir "/path/to/output"

With model selection (trade-off: speed vs accuracy):

# Fast (less accurate)
whisper "audio.mp3" --model tiny

# Balanced (recommended)
whisper "audio.mp3" --model base

# High quality
whisper "audio.mp3" --model small

# Best quality (slower, requires more RAM)
whisper "audio.mp3" --model medium
whisper "audio.mp3" --model large

With language specification:

whisper "audio.mp3" --language en

Output format options:

whisper "audio.mp3" --output_format txt    # Plain text
whisper "audio.mp3" --output_format srt    # Subtitles
whisper "audio.mp3" --output_format vtt    # Web subtitles
whisper "audio.mp3" --output_format json   # Detailed JSON
whisper "audio.mp3" --output_format all    # All formats

Step 3: Apply Context Grounding

Use the scripts/transcribe_with_context.py script for automated grounding, or manually apply corrections:

# Automated approach (recommended)
python scripts/transcribe_with_context.py /path/to/audio.mp3

For manual grounding: 1. Read the transcript output 2. Read all .md files in the media file's directory 3. Extract terminology, names, and technical terms from context files 4. Search transcript for likely misrecognitions 5. Apply corrections based on context

Common corrections:

• "cooler net ease" -> "Kubernetes"
• "sequel" -> "SQL"
• "post gress" -> "Postgres"
• Names: Match phonetic variations to names in context files

Step 4: Save Corrected Transcript

Save the grounded transcript with a clear filename:

original_filename_transcript.txt
original_filename_transcript.md

Context Files

Context files are markdown files in the same directory as the media file. They provide grounding information to improve transcription accuracy.

What to Include in Context Files

• People: Names of speakers, team members, interviewees
• Technical Terms: Domain-specific vocabulary, product names
• Acronyms: Abbreviations and their expansions
• Organizations: Company names, department names
• Projects: Project codenames, feature names

Context File Example

See assets/context-template.md for a complete template.

# Meeting Context

## Speakers
- Richard Hightower (host)
- Jane Smith (engineering lead)

## Technical Terms
- Kubernetes (container orchestration)
- FastAPI (Python web framework)
- AlloyDB (Google Cloud database)

## Acronyms
- CI/CD - Continuous Integration/Continuous Deployment
- PR - Pull Request

Model Selection Guide

Use base for general use, medium for important recordings. See references/whisper-options.md for full model comparison and all available options.

Quick reference: tiny (fastest) < base (balanced) < small (better) < medium (high) < large (best accuracy)

For MacBook Pro with Apple Silicon: small or medium models recommended for best speed/accuracy balance.

Troubleshooting

"whisper: command not found"

# macOS
brew install openai-whisper

# Linux
pip install openai-whisper
export PATH="$HOME/.local/bin:$PATH"

"ffmpeg not found"

# macOS
brew install ffmpeg

# Linux
sudo apt install ffmpeg

Out of memory errors

Use a smaller model:

whisper "audio.mp3" --model tiny

Slow transcription

• Use tiny or base model for faster results
• Ensure correct architecture is being used (Apple Silicon vs Intel)

Resources

scripts/

The scripts/transcribe_with_context.py script automates the full workflow:

• Finds context files automatically
• Runs Whisper transcription
• Applies context-based corrections
• Saves the final transcript

Usage:

python scripts/transcribe_with_context.py /path/to/audio.mp3

references/

See references/whisper-options.md for complete CLI reference and advanced options.

assets/

The assets/context-template.md provides a template for creating context files to improve transcription accuracy.

Transcription Context Template

Use this template to create context files that improve transcription accuracy. Place this file (or your customized version) in the same directory as your audio/video files.

---

Meeting/Recording Information

• Date: [YYYY-MM-DD]
• Type: [Meeting / Interview / Presentation / Podcast / etc.]
• Topic: [Brief description of what was discussed]

Speakers

List all participants with their full names and roles:

• [Full Name] - [Role/Title]
• [Full Name] - [Role/Title]
• [Full Name] - [Role/Title]

Technical Terms

Domain-specific vocabulary that may be misrecognized:

Products & Services

• [Product Name] - [Brief description]
• [Service Name] - [Brief description]

Technologies

• Kubernetes - container orchestration platform
• Docker - containerization tool
• FastAPI - Python web framework
• PostgreSQL - relational database
• Redis - in-memory data store
• Terraform - infrastructure as code
• Pulumi - infrastructure as code
• AlloyDB - Google Cloud database
• Cloud Run - serverless container platform

Frameworks & Libraries

• React - JavaScript UI library
• Vue.js - JavaScript framework
• Next.js - React framework
• Django - Python web framework
• Flask - Python micro framework

Acronyms & Abbreviations

Organizations & Teams

• [Company Name]
• [Department Name]
• [Team Name]
• [Partner Organization]

Projects & Codenames

• [Project Name] - [Brief description]
• [Feature Codename] - [What it refers to]

Proper Nouns

Locations

• [Office Location]
• [City Names]
• [Building Names]

Events

• [Conference Names]
• [Meeting Series Names]

Common Phrases

Phrases frequently used in your domain:

• "[Specific phrase or jargon]"
• "[Another common phrase]"

---

Usage Notes

1. Keep this file updated - Add new terms as they come up 2. Be specific - Include exact spellings, especially for unusual names 3. Include context - Brief descriptions help with disambiguation 4. Multiple files OK - You can have separate files for different contexts:

• team-members.md - Just names and roles
• tech-glossary.md - Technical terminology
• project-context.md - Current project details

Whisper Transcribe Skill

A Claude Code skill for transcribing audio and video files using OpenAI's Whisper with context-grounding from markdown files.

  

Features

• Audio/Video Transcription: Convert media files to text using OpenAI Whisper
• Context Grounding: Uses markdown files in the same directory to improve accuracy for technical terms, names, and jargon
• Multi-format Support: Works with mp3, wav, m4a, mp4, webm, and more
• Cross-platform: Supports macOS (Homebrew) and Linux installations
• Automated Workflow: Python script handles the full transcription pipeline

Installation

Quick Install with Skilz (Recommended)

The easiest way to install this skill is using the skilz universal installer:

npx skilz install SpillwaveSolutions_whisper-transcribe/whisper-transcribe

This command automatically downloads and configures the skill for Claude Code.

View on Skilz Marketplace: whisper-transcribe

Manual Installation

Clone the repository to your Claude Code skills directory:

git clone https://github.com/SpillwaveSolutions/whisper-transcribe.git ~/.claude/skills/whisper-transcribe

Prerequisites

After installing the skill, you need to install Whisper and ffmpeg on your system.

macOS (Homebrew)

brew install ffmpeg openai-whisper

Linux

# Install ffmpeg
sudo apt install ffmpeg  # Debian/Ubuntu

# Install Whisper
pip install openai-whisper

Verify Installation

whisper --version
ffmpeg -version

Usage

Basic Transcription

whisper /path/to/audio.mp3 --output_dir /path/to/output

With Context Grounding Script

python scripts/transcribe_with_context.py /path/to/audio.mp3 --model base --language en

The script will: 1. Find markdown context files in the same directory 2. Run Whisper transcription 3. Apply corrections based on context (technical terms, names) 4. Save both original and grounded transcripts

Model Selection

For MacBook Pro with Apple Silicon: small or medium models recommended for best speed/accuracy balance.

Context Files

Create markdown files in the same directory as your audio to improve transcription accuracy.

Example Context File

# Meeting Context

## Speakers
- Richard Hightower (host)
- Jane Smith (engineering lead)

## Technical Terms
- Kubernetes (container orchestration)
- FastAPI (Python web framework)
- AlloyDB (Google Cloud database)

## Acronyms
- CI/CD - Continuous Integration/Continuous Deployment
- PR - Pull Request

See assets/context-template.md for a complete template.

Project Structure

whisper-transcribe/
├── SKILL.md                        # Skill definition
├── README.md                       # This file
├── scripts/
│   └── transcribe_with_context.py  # Automated transcription script
├── references/
│   └── whisper-options.md          # Complete Whisper CLI reference
└── assets/
    └── context-template.md         # Template for context files

Triggers

This skill activates when users mention:

• whisper, transcribe, transcription
• audio to text, video to text, speech to text
• meeting transcript, convert recording
• File extensions: .mp3, .wav, .m4a, .mp4, .webm

Troubleshooting

"whisper: command not found"

# macOS
brew install openai-whisper

# Linux
pip install openai-whisper
export PATH="$HOME/.local/bin:$PATH"

"ffmpeg not found"

# macOS
brew install ffmpeg

# Linux
sudo apt install ffmpeg

Out of memory errors

Use a smaller model:

whisper "audio.mp3" --model tiny

Slow transcription

• Use tiny or base model for faster results
• Ensure correct architecture is being used (Apple Silicon vs Intel)

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT

Whisper CLI Complete Reference

OpenAI Whisper command-line interface options and usage patterns.

Contents

• Basic Syntax
• Core Options
• Model Selection
• Language
• Output Format
• Output Directory
• Advanced Options
• Performance Options
• Common Workflows
• Supported Audio Formats
• Error Handling
• Python API
• References

Basic Syntax

whisper <audio_file> [options]

Core Options

Model Selection (--model)

Note: .en models are English-only and perform better for English audio.

# Examples
whisper audio.mp3 --model tiny      # Fastest, least accurate
whisper audio.mp3 --model base      # Good balance
whisper audio.mp3 --model small     # Better accuracy
whisper audio.mp3 --model medium    # High accuracy
whisper audio.mp3 --model large-v3  # Best accuracy

Language (--language)

Force a specific language instead of auto-detection:

whisper audio.mp3 --language en     # English
whisper audio.mp3 --language es     # Spanish
whisper audio.mp3 --language fr     # French
whisper audio.mp3 --language de     # German
whisper audio.mp3 --language ja     # Japanese
whisper audio.mp3 --language zh     # Chinese

Common language codes: en, es, fr, de, it, pt, ja, ko, zh, ru, ar, hi

Output Format (--output_format)

whisper audio.mp3 --output_format txt   # Plain text (default)
whisper audio.mp3 --output_format vtt   # WebVTT subtitles
whisper audio.mp3 --output_format srt   # SRT subtitles
whisper audio.mp3 --output_format tsv   # Tab-separated values
whisper audio.mp3 --output_format json  # Detailed JSON with timestamps
whisper audio.mp3 --output_format all   # All formats

Output Directory (--output_dir)

whisper audio.mp3 --output_dir ./transcripts
whisper audio.mp3 --output_dir /path/to/output

Advanced Options

Task (--task)

whisper audio.mp3 --task transcribe   # Transcribe in original language (default)
whisper audio.mp3 --task translate    # Translate to English

Temperature (--temperature)

Controls randomness in decoding (0.0-1.0):

whisper audio.mp3 --temperature 0     # Deterministic
whisper audio.mp3 --temperature 0.2   # Low randomness (default)

Word Timestamps (--word_timestamps)

Enable word-level timing information:

whisper audio.mp3 --word_timestamps True --output_format json

Initial Prompt (--initial_prompt)

Provide context to guide transcription style:

whisper audio.mp3 --initial_prompt "This is a technical discussion about Kubernetes and Docker."

Condition on Previous Text (--condition_on_previous_text)

whisper audio.mp3 --condition_on_previous_text True   # Use context (default)
whisper audio.mp3 --condition_on_previous_text False  # Independent segments

Compression Ratio Threshold (--compression_ratio_threshold)

Filter out segments with high compression (likely garbage):

whisper audio.mp3 --compression_ratio_threshold 2.4

No Speech Threshold (--no_speech_threshold)

Probability threshold for silence detection:

whisper audio.mp3 --no_speech_threshold 0.6

Hallucination Silence Threshold (--hallucination_silence_threshold)

Skip silent segments to reduce hallucinations:

whisper audio.mp3 --hallucination_silence_threshold 2

Performance Options

Device (--device)

whisper audio.mp3 --device cpu    # CPU only
whisper audio.mp3 --device cuda   # NVIDIA GPU
whisper audio.mp3 --device mps    # Apple Silicon (M1/M2/M3)

Threads (--threads)

CPU threads for computation:

whisper audio.mp3 --threads 4

FP16 (--fp16)

Half-precision (faster on GPU):

whisper audio.mp3 --fp16 True     # Enable (default on GPU)
whisper audio.mp3 --fp16 False    # Disable (required on CPU)

Common Workflows

Quick Draft Transcription

whisper meeting.mp3 --model tiny --output_format txt

High-Quality Transcription

whisper interview.mp3 --model medium --language en --output_format all

Generate Subtitles

whisper video.mp4 --model small --output_format srt

Translate Foreign Audio to English

whisper spanish_audio.mp3 --task translate --model medium

Technical Content with Context

whisper tech_talk.mp3 \
  --model medium \
  --language en \
  --initial_prompt "Technical discussion about cloud computing, Kubernetes, and microservices architecture." \
  --output_format json

Batch Processing

# Transcribe all MP3 files in a directory
for f in *.mp3; do
  whisper "$f" --model base --output_dir ./transcripts
done

Supported Audio Formats

Whisper uses ffmpeg and supports:

• Audio: mp3, wav, m4a, flac, ogg, opus, wma, aac
• Video: mp4, mkv, webm, avi, mov (extracts audio automatically)

Error Handling

Out of Memory

# Use smaller model
whisper audio.mp3 --model tiny

# Or disable FP16 on CPU
whisper audio.mp3 --device cpu --fp16 False

Slow Performance

# Use faster model
whisper audio.mp3 --model base

# Specify device explicitly
whisper audio.mp3 --device mps  # Apple Silicon

Poor Accuracy

# Use larger model
whisper audio.mp3 --model medium

# Specify language explicitly
whisper audio.mp3 --language en

# Provide context
whisper audio.mp3 --initial_prompt "Meeting about project planning"

Python API (Alternative)

For programmatic use:

import whisper

model = whisper.load_model("base")
result = model.transcribe("audio.mp3")
print(result["text"])

# With options
result = model.transcribe(
    "audio.mp3",
    language="en",
    task="transcribe",
    fp16=False
)

References

• OpenAI Whisper GitHub
• Whisper Model Card
• Supported Languages