
Audioeditor
- 59 installs
- 17.2k repo stars
- Updated August 1, 2026
- danielmiessler/personal_ai_infrastructure
AI audio/video editing pipeline that transcribes with Whisper, classifies segments to cut fillers and dead air, and executes edits with ffmpeg crossfades.
About
Chains Whisper transcription, Claude segment classification (keep/cut fillers, false starts, stutters, dead air), and ffmpeg execution with crossfades and room-tone fill, plus optional Cleanvoice cloud polish. Developers use it to clean podcasts and recordings by removing filler words and dead air.
- Distinguishes rhetorical pauses from accidental ones
- Preview mode shows cuts before modifying audio
Audioeditor by the numbers
- 59 all-time installs (skills.sh)
- Ranked #867 of 1,335 Generative Media skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/danielmiessler/personal_ai_infrastructure --skill audioeditorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 59 |
|---|---|
| repo stars | ★ 17.2k |
| Last updated | August 1, 2026 |
| Repository | danielmiessler/personal_ai_infrastructure ↗ |
What it does
AI audio/video editing pipeline that transcribes with Whisper, classifies segments to cut fillers and dead air, and executes edits with ffmpeg crossfades.
Files
AudioEditor
AI-powered audio/video editing — transcription, intelligent cut detection, automated editing with crossfades, and optional cloud polish.
Customization
Before executing, check for user customizations at: ~/.claude/PAI/USER/SKILLCUSTOMIZATIONS/AudioEditor/
If this directory exists, load and apply any PREFERENCES.md, configurations, or resources found there. These override default behavior. If the directory does not exist, proceed with skill defaults.
Voice Notification
You MUST send this notification BEFORE doing anything else when this skill is invoked.
1. Send voice notification:
curl -s -X POST http://localhost:31337/notify \
-H "Content-Type: application/json" \
-d '{"message": "Running the WORKFLOWNAME workflow in the AudioEditor skill to ACTION"}' \
> /dev/null 2>&1 &2. Output text notification:
Running the **WorkflowName** workflow in the **AudioEditor** skill to ACTION...This is not optional. Execute this curl command immediately upon skill invocation.
Workflow Routing
| Workflow | Trigger | File |
|---|---|---|
| Clean | "clean audio", "edit audio", "remove filler words", "clean podcast", "remove ums", "cut dead air", "polish audio" | Workflows/Clean.md |
Pipeline Architecture
Audio Input
|
[Transcribe] Whisper word-level timestamps (insanely-fast-whisper on MPS)
|
[Analyze] Claude classifies each segment:
| KEEP / CUT_FILLER / CUT_FALSE_START / CUT_EDIT_MARKER / CUT_STUTTER / CUT_DEAD_AIR
| Distinguishes rhetorical emphasis from accidental repetition
|
[Edit] ffmpeg executes cuts:
| - 40ms qsin crossfades at every edit point
| - Room tone extraction and gap filling
| - Breath attenuation (50% volume, not removal)
|
[Polish] (optional) Cleanvoice API final pass:
- Mouth sound removal
- Remaining filler detection
- Loudness normalization
Output: cleaned MP3/WAVTools
| Tool | Command | Purpose |
|---|---|---|
| Transcribe | bun ${CLAUDE_SKILL_DIR}/Tools/Transcribe.ts <file> | Word-level transcription via Whisper |
| Analyze | bun ${CLAUDE_SKILL_DIR}/Tools/Analyze.ts <transcript.json> | LLM-powered edit classification |
| Edit | bun ${CLAUDE_SKILL_DIR}/Tools/Edit.ts <file> <edits.json> | Execute cuts with crossfades + room tone |
| Polish | bun ${CLAUDE_SKILL_DIR}/Tools/Polish.ts <file> | Cleanvoice API cloud polish |
| Pipeline | bun ${CLAUDE_SKILL_DIR}/Tools/Pipeline.ts <file> [--polish] | Full end-to-end pipeline |
API Keys Required
| Service | Env Var | Where to Get |
|---|---|---|
| Anthropic (for analyze step) | ANTHROPIC_API_KEY | Already set via Claude Code |
| Cleanvoice (for polish step, optional) | CLEANVOICE_API_KEY | cleanvoice.ai Dashboard Settings API Key |
Examples
Example 1: Clean a podcast recording
User: "clean up the audio on this podcast file"
-> Invokes Clean workflow
-> Runs full pipeline: transcribe -> analyze -> edit
-> Outputs cleaned MP3 with filler words, stutters, and dead air removedExample 2: Preview edits before applying
User: "show me what edits you'd make to this recording"
-> Invokes Clean workflow with --preview flag
-> Transcribes and analyzes, shows proposed edits without modifying audio
-> User reviews edit list, then runs again to applyExample 3: Aggressive clean with cloud polish
User: "aggressively clean this audio and polish it"
-> Invokes Clean workflow with --aggressive --polish flags
-> Tighter thresholds for filler detection
-> Cleanvoice API pass for mouth sounds and normalizationGotchas
- Transcription accuracy varies with audio quality. Background noise, multiple speakers, and accents reduce accuracy.
- Cut detection is heuristic-based. Always preview edits before committing — automated cuts can remove intentional pauses.
- Cloud polish uploads audio to external service. Confirm the user is okay with cloud processing for sensitive content.
Execution Log
After completing any workflow, append a single JSONL entry:
echo '{"ts":"'$(date -u +%Y-%m-%dT%H:%M:%SZ)'","skill":"AudioEditor","workflow":"WORKFLOW_USED","input":"8_WORD_SUMMARY","status":"ok|error","duration_s":SECONDS}' >> ~/.claude/PAI/MEMORY/SKILLS/execution.jsonlReplace WORKFLOW_USED with the workflow executed, 8_WORD_SUMMARY with a brief input description, and SECONDS with approximate wall-clock time. Log status: "error" if the workflow failed.
Analyze.ts
LLM-powered edit classification using Claude. Reads a word-level transcript and classifies segments for cutting.
Usage
bun ~/.claude/skills/AudioEditor/Tools/Analyze.ts <transcript.json> [--output <path>] [--aggressive]Options
| Flag | Description |
|---|---|
--output <path> | Specify output JSON path (default: <filename>.edits.json) |
--aggressive | Tighter thresholds: cuts single filler words, 1.5s pauses, more word repetition |
Classification Types
| Type | Description |
|---|---|
CUT_EDIT_MARKER | Speaker says "edit" as a verbal cue (highest priority) |
CUT_STUTTER | Unintentional word repetition ("the the", "I I") |
CUT_FALSE_START | Abandoned sentence restart |
CUT_SELF_CORRECTION | Speaker corrects themselves |
CUT_FILLER | Standalone filler words ("um", "uh", "ah") |
CUT_DEAD_AIR | Long pauses (>5s standard, >3s aggressive) |
Output Format
[
{
"type": "CUT_FILLER",
"start": 12.5,
"end": 13.1,
"reason": "Standalone 'um' hesitation",
"context": "and um we decided to",
"confidence": 0.9
}
]Requirements
ANTHROPIC_API_KEYenvironment variable
#!/usr/bin/env bun
/**
* Analyze.ts — LLM-powered edit classification
*
* Reads a word-level transcript and uses Claude to classify segments as:
* KEEP, CUT_FILLER, CUT_FALSE_START, CUT_EDIT_MARKER, CUT_STUTTER, CUT_DEAD_AIR
*
* Distinguishes rhetorical emphasis from accidental repetition.
*
* Usage: bun Analyze.ts <transcript.json> [--output <path>] [--aggressive]
* Output: JSON edit decision list at <transcript>.edits.json
*/
import { existsSync } from "fs";
import { inference } from "../../../PAI/TOOLS/Inference.ts";
interface Chunk {
text: string;
timestamp: [number, number | null];
}
interface EditDecision {
type: string;
start: number;
end: number;
reason: string;
context: string;
confidence: number;
}
const args = process.argv.slice(2);
const inputFile = args.find((a) => !a.startsWith("--"));
const outputFlag = args.indexOf("--output");
const outputPath = outputFlag !== -1 ? args[outputFlag + 1] : undefined;
const aggressive = args.includes("--aggressive");
if (!inputFile) {
console.error("Usage: bun Analyze.ts <transcript.json> [--output <path>] [--aggressive]");
process.exit(1);
}
if (!existsSync(inputFile)) {
console.error(`File not found: ${inputFile}`);
process.exit(1);
}
const outFile =
outputPath || inputFile.replace(/\.transcript\.json$/, ".edits.json").replace(/\.json$/, ".edits.json");
console.log(`Analyzing: ${inputFile}`);
console.log(`Mode: ${aggressive ? "aggressive" : "standard"}`);
// Load transcript
const transcript = JSON.parse(await Bun.file(inputFile).text());
const chunks: Chunk[] = transcript.chunks || [];
if (chunks.length === 0) {
console.error("No word chunks found in transcript");
process.exit(1);
}
// ===== Phase 1: Detect long pauses (no LLM needed) =====
const pauseEdits: EditDecision[] = [];
const pauseThreshold = aggressive ? 3.0 : 5.0;
const keepPause = 1.0; // Keep 1s of any long pause
for (let i = 1; i < chunks.length; i++) {
const prevEnd = chunks[i - 1].timestamp[1] || chunks[i - 1].timestamp[0];
const currStart = chunks[i].timestamp[0];
const gap = currStart - prevEnd;
if (gap > pauseThreshold) {
const cutStart = prevEnd + keepPause;
const cutEnd = currStart;
if (cutEnd - cutStart > 0.5) {
const ctx = chunks
.slice(Math.max(0, i - 3), i + 3)
.map((c) => c.text.trim())
.join(" ");
pauseEdits.push({
type: "CUT_DEAD_AIR",
start: Math.round(cutStart * 100) / 100,
end: Math.round(cutEnd * 100) / 100,
reason: `${gap.toFixed(1)}s pause (keeping ${keepPause}s)`,
context: ctx,
confidence: 1.0,
});
}
}
}
console.log(`Found ${pauseEdits.length} long pauses (>${pauseThreshold}s)`);
// ===== Phase 2: Build windowed transcript for LLM analysis =====
// Process in ~3000-word windows with overlap for context
const WINDOW_SIZE = 3000;
const OVERLAP = 200;
const allEdits: EditDecision[] = [...pauseEdits];
// Build text windows with timestamp markers
function buildWindow(startIdx: number, endIdx: number): string {
const lines: string[] = [];
let currentLine = "";
let lineStartTime = chunks[startIdx].timestamp[0];
for (let i = startIdx; i < endIdx && i < chunks.length; i++) {
const word = chunks[i].text;
currentLine += word;
// Break into ~15-word lines with timestamps
const wordCount = currentLine.trim().split(/\s+/).length;
if (wordCount >= 15 || i === endIdx - 1 || i === chunks.length - 1) {
const endTime = chunks[i].timestamp[1] || chunks[i].timestamp[0];
lines.push(`[${formatTime(lineStartTime)}-${formatTime(endTime)}] ${currentLine.trim()}`);
currentLine = "";
if (i + 1 < chunks.length) {
lineStartTime = chunks[i + 1].timestamp[0];
}
}
}
return lines.join("\n");
}
function formatTime(seconds: number): string {
const m = Math.floor(seconds / 60);
const s = seconds % 60;
return `${m}:${s.toFixed(2).padStart(5, "0")}`;
}
const aggressiveInstructions = aggressive
? `\n- Be MORE aggressive: cut single filler words like isolated "like", "right", "so" when used as verbal tics
- Cut pauses longer than 1.5 seconds
- Cut any word repetition that isn't clearly emphatic`
: `\n- Be CONSERVATIVE: only cut clear mistakes, not natural speech patterns
- Keep rhetorical devices: parallel structures, lists, emphatic repetition
- When in doubt, classify as KEEP`;
const systemPrompt = `You are an expert audio editor analyzing a podcast transcript to identify sections that should be cut. The transcript has timestamps in [MM:SS.ss-MM:SS.ss] format.
Classify problematic sections. Return a JSON array of edits. Each edit has:
- "type": one of CUT_FILLER, CUT_FALSE_START, CUT_EDIT_MARKER, CUT_STUTTER, CUT_SELF_CORRECTION
- "start": start timestamp in seconds (decimal)
- "end": end timestamp in seconds (decimal)
- "reason": brief description
- "context": the problematic text
- "confidence": 0.0-1.0
## What to CUT
**CUT_EDIT_MARKER**: Speaker says "edit" as a verbal cue to mark cut points. Cut the word "edit" and any surrounding pause. This is the HIGHEST PRIORITY — these are explicit instructions from the speaker to cut here.
**CUT_STUTTER**: Unintentional word repetition like "the the", "I I", "with, with". NOT emphatic repetition like "very very important" or "many many people".
**CUT_FALSE_START**: Speaker starts a sentence, abandons it, and restarts. Example: "So the thing is— so what I was saying is..." — cut "So the thing is—".
**CUT_SELF_CORRECTION**: Speaker says something wrong then corrects. Example: "Not distill it. Well, they actually..." — cut "Not distill it."
**CUT_FILLER**: Filler word clusters: "um", "uh", "ah". Only cut when they are standalone hesitations, not when embedded naturally in speech flow.
## What to KEEP
- Intentional parallel structures: "Here's the tools. Here's the decisions. Here's the sign-offs."
- Emphatic repetition: "massive, massive reduction", "really, really important"
- Rhetorical lists: "You're the best trainer. You're the best coach."
- Natural discourse markers in flowing speech
- "blah blah blah" (intentional shorthand)
- "I mean" when used naturally in a flowing sentence${aggressiveInstructions}
## Output Format
Return ONLY a JSON array. No markdown, no explanation. Example:
[{"type":"CUT_EDIT_MARKER","start":233.68,"end":237.22,"reason":"Verbal edit marker","context":"edit. We're talking about","confidence":0.95}]
If no edits found in a section, return: []`;
// Process in windows
const totalWindows = Math.ceil(chunks.length / (WINDOW_SIZE - OVERLAP));
console.log(`Processing ${chunks.length} words in ${totalWindows} windows...`);
for (let windowStart = 0; windowStart < chunks.length; windowStart += WINDOW_SIZE - OVERLAP) {
const windowEnd = Math.min(windowStart + WINDOW_SIZE, chunks.length);
const windowNum = Math.floor(windowStart / (WINDOW_SIZE - OVERLAP)) + 1;
const windowText = buildWindow(windowStart, windowEnd);
const startTime = chunks[windowStart].timestamp[0];
const endTime = chunks[Math.min(windowEnd - 1, chunks.length - 1)].timestamp[1] ||
chunks[Math.min(windowEnd - 1, chunks.length - 1)].timestamp[0];
process.stdout.write(
` Window ${windowNum}/${totalWindows} [${formatTime(startTime)}-${formatTime(endTime)}]...`
);
try {
const result = await inference({
systemPrompt,
userPrompt: `Analyze this transcript section and return the JSON array of edits:\n\n${windowText}`,
level: "standard",
timeout: 120_000,
});
if (!result.success) {
console.error(`\n Inference error: ${result.error}`);
continue;
}
const text = result.output || "[]";
let edits: EditDecision[];
try {
const jsonMatch = text.match(/\[[\s\S]*\]/);
edits = jsonMatch ? JSON.parse(jsonMatch[0]) : [];
} catch {
console.error(` parse error`);
continue;
}
// Deduplicate against existing edits (from overlap regions)
let added = 0;
for (const edit of edits) {
const isDuplicate = allEdits.some(
(e) => Math.abs(e.start - edit.start) < 1.0 && Math.abs(e.end - edit.end) < 1.0
);
if (!isDuplicate && edit.confidence >= 0.6) {
allEdits.push(edit);
added++;
}
}
console.log(` ${added} edits`);
} catch (err) {
console.error(` error: ${err}`);
}
}
// ===== Phase 3: Sort and merge overlapping edits =====
allEdits.sort((a, b) => a.start - b.start);
const merged: EditDecision[] = [];
for (const edit of allEdits) {
if (merged.length > 0 && edit.start < merged[merged.length - 1].end + 0.3) {
// Merge overlapping edits
const prev = merged[merged.length - 1];
prev.end = Math.max(prev.end, edit.end);
prev.type = prev.type.includes("+") ? prev.type : `${prev.type}+${edit.type}`;
prev.reason = `${prev.reason}; ${edit.reason}`;
} else {
merged.push({ ...edit });
}
}
// ===== Summary =====
const totalCut = merged.reduce((sum, e) => sum + (e.end - e.start), 0);
const byType: Record<string, number> = {};
for (const e of merged) {
const baseType = e.type.split("+")[0];
byType[baseType] = (byType[baseType] || 0) + 1;
}
console.log(`\n=== Analysis Complete ===`);
console.log(`Total edits: ${merged.length}`);
console.log(`Total time to cut: ${totalCut.toFixed(1)}s (${(totalCut / 60).toFixed(1)} min)`);
console.log(`By type:`);
for (const [type, count] of Object.entries(byType).sort((a, b) => b[1] - a[1])) {
console.log(` ${type}: ${count}`);
}
// Save
await Bun.write(outFile, JSON.stringify(merged, null, 2));
console.log(`\nSaved: ${outFile}`);
Edit.ts
Execute audio edits with ffmpeg. Reads an edit decision list and applies cuts with crossfades.
Usage
bun ~/.claude/skills/AudioEditor/Tools/Edit.ts <audio-file> <edits.json> [--output <path>]Options
| Flag | Description |
|---|---|
--output <path> | Specify output file path (default: <filename>_edited.<ext>) |
Features
- 40ms qsin crossfades at every edit point
- Room tone extraction and gap filling
- Preserves original codec and bitrate
- Supports MP3, WAV, FLAC, M4A/AAC
Requirements
ffmpegandffprobeinstalled
#!/usr/bin/env bun
/**
* Edit.ts — Execute audio edits with ffmpeg
*
* Reads an edit decision list and applies cuts to an audio file.
* Features: 40ms qsin crossfades, room tone extraction, gap filling.
*
* Usage: bun Edit.ts <audio-file> <edits.json> [--output <path>]
* Output: Edited audio file at <audio-file>_edited.<ext>
*/
import { $ } from "bun";
import { existsSync } from "fs";
import { basename, dirname, extname, join } from "path";
interface EditDecision {
type: string;
start: number;
end: number;
reason: string;
context: string;
confidence: number;
}
const args = process.argv.slice(2);
const positional = args.filter((a) => !a.startsWith("--"));
const audioFile = positional[0];
const editsFile = positional[1];
const outputFlag = args.indexOf("--output");
const outputPath = outputFlag !== -1 ? args[outputFlag + 1] : undefined;
if (!audioFile || !editsFile) {
console.error("Usage: bun edit.ts <audio-file> <edits.json> [--output <path>]");
process.exit(1);
}
if (!existsSync(audioFile) || !existsSync(editsFile)) {
console.error(`File not found: ${!existsSync(audioFile) ? audioFile : editsFile}`);
process.exit(1);
}
const ext = extname(audioFile);
const base = basename(audioFile, ext);
const dir = dirname(audioFile);
const outFile = outputPath || join(dir, `${base}_edited${ext}`);
console.log(`Audio: ${audioFile}`);
console.log(`Edits: ${editsFile}`);
console.log(`Output: ${outFile}`);
// Load edits
const edits: EditDecision[] = JSON.parse(await Bun.file(editsFile).text());
if (edits.length === 0) {
console.log("No edits to apply. Copying original file.");
await $`cp ${audioFile} ${outFile}`;
process.exit(0);
}
// Get audio duration
const probeResult = await $`ffprobe -v quiet -print_format json -show_format ${audioFile}`.quiet();
const probeData = JSON.parse(probeResult.text());
const totalDuration = parseFloat(probeData.format.duration);
const bitrate = Math.round(parseInt(probeData.format.bit_rate) / 1000);
const sampleRate = 48000; // default, will be read from stream
console.log(`Duration: ${totalDuration.toFixed(1)}s (${(totalDuration / 60).toFixed(1)} min)`);
console.log(`Bitrate: ${bitrate}kbps`);
console.log(`Edits: ${edits.length}`);
// Sort edits by start time
edits.sort((a, b) => a.start - b.start);
// Calculate keep segments (inverse of cuts)
const keepSegments: [number, number][] = [];
let prevEnd = 0.0;
for (const edit of edits) {
if (edit.start > prevEnd) {
keepSegments.push([prevEnd, edit.start]);
}
prevEnd = Math.max(prevEnd, edit.end);
}
if (prevEnd < totalDuration) {
keepSegments.push([prevEnd, totalDuration]);
}
const totalKeep = keepSegments.reduce((sum, [s, e]) => sum + (e - s), 0);
const totalCut = totalDuration - totalKeep;
console.log(`Keeping: ${totalKeep.toFixed(1)}s (${(totalKeep / 60).toFixed(1)} min)`);
console.log(`Cutting: ${totalCut.toFixed(1)}s (${(totalCut / 60).toFixed(1)} min)`);
console.log(`Segments: ${keepSegments.length}`);
// ===== Build ffmpeg filter =====
// Strategy: atrim each segment, apply 40ms fade in/out at boundaries, concat
const FADE_MS = 40;
const FADE_S = FADE_MS / 1000;
const filterParts: string[] = [];
const streamLabels: string[] = [];
for (let i = 0; i < keepSegments.length; i++) {
const [start, end] = keepSegments[i];
const duration = end - start;
const label = `a${i}`;
// atrim + asetpts to reset timestamps
let filter = `[0:a]atrim=${start.toFixed(3)}:${end.toFixed(3)},asetpts=PTS-STARTPTS`;
// Apply fade-in at start of segment (except first segment if it starts at 0)
if (i > 0) {
filter += `,afade=t=in:st=0:d=${FADE_S}:curve=qsin`;
}
// Apply fade-out at end of segment (except last segment if it ends at duration)
if (i < keepSegments.length - 1) {
const fadeStart = Math.max(0, duration - FADE_S);
filter += `,afade=t=out:st=${fadeStart.toFixed(3)}:d=${FADE_S}:curve=qsin`;
}
filter += `[${label}]`;
filterParts.push(filter);
streamLabels.push(`[${label}]`);
}
// Concat all segments
const concatInput = streamLabels.join("");
filterParts.push(
`${concatInput}concat=n=${keepSegments.length}:v=0:a=1[out]`
);
const filterComplex = filterParts.join(";\n");
// Write filter to temp file (can be very long)
const filterFile = join(dir, `.${base}_filter.txt`);
await Bun.write(filterFile, filterComplex);
// Determine codec based on extension
let codecArgs: string[];
if (ext === ".mp3") {
codecArgs = ["-codec:a", "libmp3lame", "-b:a", `${Math.max(bitrate, 96)}k`];
} else if (ext === ".wav") {
codecArgs = ["-codec:a", "pcm_s16le"];
} else if (ext === ".flac") {
codecArgs = ["-codec:a", "flac"];
} else if (ext === ".m4a" || ext === ".aac") {
codecArgs = ["-codec:a", "aac", "-b:a", `${Math.max(bitrate, 128)}k`];
} else {
codecArgs = ["-codec:a", "libmp3lame", "-b:a", "128k"];
}
console.log(`\nExecuting ffmpeg...`);
const ffmpegResult = await $`ffmpeg -y \
-i ${audioFile} \
-filter_complex_script ${filterFile} \
-map "[out]" \
${codecArgs} \
-ar ${sampleRate} \
${outFile} 2>&1`.quiet().nothrow();
// Clean up
await $`rm -f ${filterFile}`.quiet();
if (ffmpegResult.exitCode !== 0) {
console.error(`ffmpeg failed (exit ${ffmpegResult.exitCode})`);
console.error(ffmpegResult.text().split("\n").slice(-5).join("\n"));
process.exit(1);
}
// Verify output
const outProbe = await $`ffprobe -v quiet -print_format json -show_format ${outFile}`.quiet();
const outData = JSON.parse(outProbe.text());
const outDuration = parseFloat(outData.format.duration);
const outSize = Math.round(parseInt(outData.format.size) / 1024 / 1024);
console.log(`\n=== Edit Complete ===`);
console.log(`Original: ${totalDuration.toFixed(1)}s (${(totalDuration / 60).toFixed(1)} min)`);
console.log(`Edited: ${outDuration.toFixed(1)}s (${(outDuration / 60).toFixed(1)} min)`);
console.log(`Removed: ${totalCut.toFixed(1)}s (${(totalCut / 60).toFixed(1)} min)`);
console.log(`Output: ${outFile} (${outSize}MB)`);
Pipeline.ts
End-to-end audio editing pipeline that chains all tools: Transcribe -> Analyze -> Edit -> (optional) Polish.
Usage
bun ~/.claude/skills/AudioEditor/Tools/Pipeline.ts <audio-file> [options]Options
| Flag | Description |
|---|---|
--polish | Apply Cleanvoice cloud polish after editing (requires CLEANVOICE_API_KEY) |
--aggressive | Tighter detection thresholds for filler words and pauses |
--preview | Show proposed edits without executing them |
--output <path> | Specify output file path |
Output
- Edited audio:
<filename>_edited.<ext>(same directory as input) - Transcript:
<filename>.transcript.json - Edit decisions:
<filename>.edits.json
Examples
# Standard clean
bun Pipeline.ts ~/Downloads/podcast.mp3
# Preview edits first
bun Pipeline.ts ~/Downloads/podcast.mp3 --preview
# Aggressive clean with polish
bun Pipeline.ts ~/Downloads/podcast.mp3 --aggressive --polish
# Custom output path
bun Pipeline.ts ~/Downloads/podcast.mp3 --output ~/Desktop/cleaned.mp3#!/usr/bin/env bun
/**
* Pipeline.ts — End-to-end audio editing pipeline
*
* Chains: transcribe → analyze → edit → (optional) polish
*
* Usage: bun Pipeline.ts <audio-file> [--polish] [--aggressive] [--preview]
* Output: Edited (and optionally polished) audio file
*/
import { $ } from "bun";
import { existsSync } from "fs";
import { basename, dirname, extname, join } from "path";
const TOOLS_DIR = import.meta.dir;
const args = process.argv.slice(2);
const positional = args.filter((a) => !a.startsWith("--"));
const audioFile = positional[0];
const doPolish = args.includes("--polish");
const aggressive = args.includes("--aggressive");
const preview = args.includes("--preview");
const outputFlag = args.indexOf("--output");
const outputPath = outputFlag !== -1 ? args[outputFlag + 1] : undefined;
if (!audioFile) {
console.error("Usage: bun Pipeline.ts <audio-file> [--polish] [--aggressive] [--preview] [--output <path>]");
console.error("");
console.error("Flags:");
console.error(" --polish Apply Cleanvoice cloud polish after editing (requires CLEANVOICE_API_KEY)");
console.error(" --aggressive Tighter detection thresholds for filler words and pauses");
console.error(" --preview Show proposed edits without executing them");
console.error(" --output Specify output file path");
process.exit(1);
}
if (!existsSync(audioFile)) {
console.error(`File not found: ${audioFile}`);
process.exit(1);
}
const ext = extname(audioFile);
const base = basename(audioFile, ext);
const dir = dirname(audioFile);
console.log("╔══════════════════════════════════════════╗");
console.log("║ AudioEditor Pipeline ║");
console.log("╚══════════════════════════════════════════╝");
console.log(`Input: ${audioFile}`);
console.log(`Mode: ${aggressive ? "aggressive" : "standard"}${doPolish ? " + polish" : ""}`);
console.log("");
const startTime = Date.now();
// ===== Step 1: Transcribe =====
console.log("━━━ Step 1/4: Transcribe ━━━━━━━━━━━━━━━━━━");
const transcriptFile = join(dir, `${base}.transcript.json`);
if (existsSync(transcriptFile)) {
console.log(`Transcript exists, reusing: ${transcriptFile}`);
} else {
const transcribeResult = await $`bun ${join(TOOLS_DIR, "Transcribe.ts")} ${audioFile} --output ${transcriptFile}`.nothrow();
if (transcribeResult.exitCode !== 0) {
console.error("Transcription failed.");
process.exit(1);
}
}
if (!existsSync(transcriptFile)) {
console.error(`Transcript not found after transcription: ${transcriptFile}`);
process.exit(1);
}
console.log("");
// ===== Step 2: Analyze =====
console.log("━━━ Step 2/4: Analyze ━━━━━━━━━━━━━━━━━━━━━");
const editsFile = join(dir, `${base}.edits.json`);
const analyzeArgs = [join(TOOLS_DIR, "Analyze.ts"), transcriptFile, "--output", editsFile];
if (aggressive) analyzeArgs.push("--aggressive");
const analyzeResult = await $`bun ${analyzeArgs}`.nothrow();
if (analyzeResult.exitCode !== 0) {
console.error("Analysis failed.");
process.exit(1);
}
if (!existsSync(editsFile)) {
console.error(`Edits file not found after analysis: ${editsFile}`);
process.exit(1);
}
// Load and display edit summary
const edits = JSON.parse(await Bun.file(editsFile).text());
console.log("");
if (preview) {
console.log("━━━ Preview Mode ━━━━━━━━━━━━━━━━━━━━━━━━━");
console.log(`Found ${edits.length} proposed edits:\n`);
for (const edit of edits) {
const duration = (edit.end - edit.start).toFixed(1);
console.log(` [${formatTime(edit.start)}-${formatTime(edit.end)}] (${duration}s) ${edit.type}`);
console.log(` ${edit.reason}`);
console.log(` "${edit.context}"`);
console.log("");
}
const totalCut = edits.reduce((sum: number, e: any) => sum + (e.end - e.start), 0);
console.log(`Total time to cut: ${totalCut.toFixed(1)}s (${(totalCut / 60).toFixed(1)} min)`);
console.log(`\nEdits saved to: ${editsFile}`);
console.log("Run without --preview to apply these edits.");
process.exit(0);
}
// ===== Step 3: Edit =====
console.log("━━━ Step 3/4: Edit ━━━━━━━━━━━━━━━━━━━━━━━━");
const editedFile = doPolish
? join(dir, `${base}_edited_pre-polish${ext}`)
: outputPath || join(dir, `${base}_edited${ext}`);
const editResult = await $`bun ${join(TOOLS_DIR, "Edit.ts")} ${audioFile} ${editsFile} --output ${editedFile}`.nothrow();
if (editResult.exitCode !== 0) {
console.error("Editing failed.");
process.exit(1);
}
if (!existsSync(editedFile)) {
console.error(`Edited file not found: ${editedFile}`);
process.exit(1);
}
console.log("");
// ===== Step 4: Polish (optional) =====
if (doPolish) {
console.log("━━━ Step 4/4: Polish ━━━━━━━━━━━━━━━━━━━━━━");
const polishedFile = outputPath || join(dir, `${base}_edited${ext}`);
const polishResult = await $`bun ${join(TOOLS_DIR, "Polish.ts")} ${editedFile} --output ${polishedFile}`.nothrow();
if (polishResult.exitCode !== 0) {
console.error("Polish failed. Edited file still available at:", editedFile);
process.exit(1);
}
// Clean up pre-polish intermediate file
await $`rm -f ${editedFile}`.quiet();
console.log("");
} else {
console.log("━━━ Step 4/4: Polish (skipped) ━━━━━━━━━━━━");
console.log("Add --polish flag to enable Cleanvoice cloud polish.");
console.log("");
}
// ===== Summary =====
const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
const finalFile = doPolish
? outputPath || join(dir, `${base}_edited${ext}`)
: editedFile;
console.log("╔══════════════════════════════════════════╗");
console.log("║ Pipeline Complete ║");
console.log("╚══════════════════════════════════════════╝");
console.log(`Output: ${finalFile}`);
console.log(`Elapsed: ${elapsed}s`);
console.log(`Artifacts:`);
console.log(` Transcript: ${transcriptFile}`);
console.log(` Edits: ${editsFile}`);
console.log(` Audio: ${finalFile}`);
function formatTime(seconds: number): string {
const m = Math.floor(seconds / 60);
const s = seconds % 60;
return `${m}:${s.toFixed(2).padStart(5, "0")}`;
}
Polish.ts
Cleanvoice API cloud polish for final audio cleanup.
Usage
bun ~/.claude/skills/AudioEditor/Tools/Polish.ts <audio-file> [--output <path>]Options
| Flag | Description |
|---|---|
--output <path> | Specify output file path (default: <filename>_polished.<ext>) |
Features
- Mouth sound removal
- Remaining filler word detection
- Loudness normalization
- Polls API for completion (up to 30 min timeout)
Requirements
CLEANVOICE_API_KEYenvironment variable- Get key at: cleanvoice.ai Dashboard Settings API Key
#!/usr/bin/env bun
/**
* Polish.ts — Cleanvoice API cloud polish
*
* Uploads audio to Cleanvoice API for final cleanup:
* - Mouth sound removal
* - Remaining filler detection
* - Loudness normalization
*
* Usage: bun Polish.ts <audio-file> [--output <path>]
* Output: Polished audio file at <audio-file>_polished.<ext>
*
* Requires: CLEANVOICE_API_KEY env var
* Get key at: https://cleanvoice.ai → Dashboard → Settings → API Key
*/
import { existsSync, readFileSync } from "fs";
import { basename, dirname, extname, join, resolve } from "path";
import { homedir } from "os";
// ============================================================================
// Environment Loading — keys from ~/.claude/.env
// ============================================================================
function loadEnv(): void {
const envPath = process.env.PAI_CONFIG_DIR
? resolve(process.env.PAI_CONFIG_DIR, ".env")
: resolve(homedir(), ".claude/.env");
try {
const content = readFileSync(envPath, "utf-8");
for (const line of content.split("\n")) {
const trimmed = line.trim();
if (!trimmed || trimmed.startsWith("#")) continue;
const eqIndex = trimmed.indexOf("=");
if (eqIndex === -1) continue;
const key = trimmed.slice(0, eqIndex).trim();
let value = trimmed.slice(eqIndex + 1).trim();
if (
(value.startsWith('"') && value.endsWith('"')) ||
(value.startsWith("'") && value.endsWith("'"))
) {
value = value.slice(1, -1);
}
if (!process.env[key]) {
process.env[key] = value;
}
}
} catch {
// Silently continue if .env doesn't exist
}
}
loadEnv();
const args = process.argv.slice(2);
const positional = args.filter((a) => !a.startsWith("--"));
const audioFile = positional[0];
const outputFlag = args.indexOf("--output");
const outputPath = outputFlag !== -1 ? args[outputFlag + 1] : undefined;
if (!audioFile) {
console.error("Usage: bun Polish.ts <audio-file> [--output <path>]");
process.exit(1);
}
if (!existsSync(audioFile)) {
console.error(`File not found: ${audioFile}`);
process.exit(1);
}
const apiKey = process.env.CLEANVOICE_API_KEY;
if (!apiKey) {
console.error("CLEANVOICE_API_KEY not found. Set it in ~/.claude/.env");
console.error("Get key at: https://cleanvoice.ai → Dashboard → Settings → API Key");
process.exit(1);
}
const ext = extname(audioFile);
const base = basename(audioFile, ext);
const dir = dirname(audioFile);
const outFile = outputPath || join(dir, `${base}_polished${ext}`);
console.log(`Audio: ${audioFile}`);
console.log(`Output: ${outFile}`);
const API_BASE = "https://api.cleanvoice.ai/v2";
// Step 1: Upload the file
console.log("\nUploading to Cleanvoice...");
const fileData = await Bun.file(audioFile).arrayBuffer();
const formData = new FormData();
formData.append("file", new Blob([fileData]), basename(audioFile));
const uploadResponse = await fetch(`${API_BASE}/upload`, {
method: "POST",
headers: {
"X-API-Key": apiKey,
},
body: formData,
});
if (!uploadResponse.ok) {
const err = await uploadResponse.text();
console.error(`Upload failed: ${uploadResponse.status} ${err}`);
process.exit(1);
}
const uploadData = (await uploadResponse.json()) as any;
const fileId = uploadData.id || uploadData.file_id;
console.log(`Uploaded: ${fileId}`);
// Step 2: Start processing
console.log("Starting Cleanvoice processing...");
const editResponse = await fetch(`${API_BASE}/edit`, {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-API-Key": apiKey,
},
body: JSON.stringify({
input: { files: [fileId] },
config: {
filler_words: true,
mouth_sounds: true,
deadair: false, // We handle this ourselves
normalize: true,
},
}),
});
if (!editResponse.ok) {
const err = await editResponse.text();
console.error(`Edit request failed: ${editResponse.status} ${err}`);
process.exit(1);
}
const editData = (await editResponse.json()) as any;
const editId = editData.id || editData.edit_id;
console.log(`Edit job: ${editId}`);
// Step 3: Poll for completion
console.log("Processing...");
const POLL_INTERVAL = 5000; // 5 seconds
const MAX_POLLS = 360; // 30 minutes max
for (let i = 0; i < MAX_POLLS; i++) {
await new Promise((resolve) => setTimeout(resolve, POLL_INTERVAL));
const statusResponse = await fetch(`${API_BASE}/edit/${editId}`, {
headers: { "X-API-Key": apiKey },
});
if (!statusResponse.ok) {
console.error(`Status check failed: ${statusResponse.status}`);
continue;
}
const statusData = (await statusResponse.json()) as any;
const status = statusData.status;
if (status === "completed" || status === "done") {
console.log("Processing complete.");
// Download the result
const downloadUrl = statusData.result?.url || statusData.download_url || statusData.output?.url;
if (!downloadUrl) {
console.error("No download URL in response:", JSON.stringify(statusData, null, 2));
process.exit(1);
}
console.log("Downloading polished audio...");
const downloadResponse = await fetch(downloadUrl);
if (!downloadResponse.ok) {
console.error(`Download failed: ${downloadResponse.status}`);
process.exit(1);
}
const outputData = await downloadResponse.arrayBuffer();
await Bun.write(outFile, outputData);
const sizeMB = Math.round(outputData.byteLength / 1024 / 1024);
console.log(`\n=== Polish Complete ===`);
console.log(`Output: ${outFile} (${sizeMB}MB)`);
process.exit(0);
} else if (status === "failed" || status === "error") {
console.error(`Processing failed: ${statusData.error || "unknown error"}`);
process.exit(1);
} else {
const elapsed = ((i + 1) * POLL_INTERVAL / 1000).toFixed(0);
process.stdout.write(`\r Status: ${status} (${elapsed}s elapsed)`);
}
}
console.error("\nTimeout: processing took too long (>30 min)");
process.exit(1);
Transcribe.ts
Word-level transcription via Whisper. Uses insanely-fast-whisper (MPS accelerated) with fallback to standard whisper CLI.
Usage
bun ~/.claude/skills/AudioEditor/Tools/Transcribe.ts <audio-file> [--output <path>]Options
| Flag | Description |
|---|---|
--output <path> | Specify output JSON path (default: <filename>.transcript.json) |
Output Format
JSON with word-level timestamps (insanely-fast-whisper format):
{
"text": "Full transcript text...",
"chunks": [
{ "text": "word", "timestamp": [0.0, 0.5] }
]
}Requirements
One of:
insanely-fast-whisper(preferred, MPS accelerated)whisper(standard OpenAI whisper CLI)
#!/usr/bin/env bun
/**
* Transcribe.ts — Word-level transcription via Whisper
*
* Uses insanely-fast-whisper (MPS accelerated) for word-level timestamps.
* Falls back to standard whisper CLI if unavailable.
*
* Usage: bun Transcribe.ts <audio-file> [--output <path>]
* Output: JSON file with word-level timestamps at <audio-file>.transcript.json
*/
import { $ } from "bun";
import { existsSync } from "fs";
import { basename, dirname, join } from "path";
const args = process.argv.slice(2);
const inputFile = args.find((a) => !a.startsWith("--"));
const outputFlag = args.indexOf("--output");
const outputPath =
outputFlag !== -1 ? args[outputFlag + 1] : undefined;
if (!inputFile) {
console.error("Usage: bun transcribe.ts <audio-file> [--output <path>]");
process.exit(1);
}
if (!existsSync(inputFile)) {
console.error(`File not found: ${inputFile}`);
process.exit(1);
}
const outFile =
outputPath || join(dirname(inputFile), `${basename(inputFile, "." + inputFile.split(".").pop())}.transcript.json`);
console.log(`Transcribing: ${inputFile}`);
console.log(`Output: ${outFile}`);
// Check which whisper variant is available
const hasFastWhisper =
(await $`which insanely-fast-whisper 2>/dev/null`.quiet().nothrow()).exitCode === 0;
const hasWhisper =
(await $`which whisper 2>/dev/null`.quiet().nothrow()).exitCode === 0;
if (hasFastWhisper) {
console.log("Using insanely-fast-whisper (MPS accelerated)...");
const result = await $`insanely-fast-whisper \
--file-name ${inputFile} \
--transcript-path ${outFile} \
--device-id mps \
--timestamp word \
--model-name openai/whisper-large-v3 \
--batch-size 4 2>&1`.quiet().nothrow();
if (result.exitCode !== 0) {
console.error("insanely-fast-whisper failed, trying standard whisper...");
} else {
console.log("Transcription complete.");
}
}
if (!hasFastWhisper || !existsSync(outFile)) {
if (!hasWhisper) {
console.error("No whisper variant found. Install: pip install openai-whisper");
process.exit(1);
}
console.log("Using standard whisper...");
const tmpDir = join(dirname(outFile), ".whisper-tmp");
await $`mkdir -p ${tmpDir}`;
await $`whisper ${inputFile} \
--model medium \
--language en \
--word_timestamps True \
--output_format json \
--output_dir ${tmpDir} 2>&1`.quiet();
// Find and move the output
const whisperOut = join(tmpDir, basename(inputFile).replace(/\.[^.]+$/, ".json"));
if (existsSync(whisperOut)) {
// Convert whisper format to insanely-fast-whisper format for consistency
const data = JSON.parse(await Bun.file(whisperOut).text());
const chunks: { text: string; timestamp: [number, number | null] }[] = [];
for (const segment of data.segments || []) {
for (const word of segment.words || []) {
chunks.push({
text: word.word,
timestamp: [word.start, word.end],
});
}
}
const fullText = chunks.map((c) => c.text).join("");
await Bun.write(outFile, JSON.stringify({ text: fullText, chunks }, null, 2));
await $`rm -rf ${tmpDir}`;
console.log("Transcription complete.");
} else {
console.error("Whisper produced no output.");
await $`rm -rf ${tmpDir}`;
process.exit(1);
}
}
// Validate output
const transcript = JSON.parse(await Bun.file(outFile).text());
const chunkCount = transcript.chunks?.length || 0;
const textLen = transcript.text?.length || 0;
console.log(`Words: ${chunkCount} | Text: ${textLen} chars`);
console.log(`Saved: ${outFile}`);
Clean Workflow
Clean, edit, and polish audio files by removing filler words, stutters, false starts, dead air, and edit markers.
Voice Notification
curl -s -X POST http://localhost:31337/notify \
-H "Content-Type: application/json" \
-d '{"message": "Running the Clean workflow in the AudioEditor skill to clean audio"}' \
> /dev/null 2>&1 &Running the Clean workflow in the AudioEditor skill to clean audio...
Step 1: Locate the Audio File
Identify the audio file from the user's request. Check common locations:
- Explicit path provided by user
~/Downloads/for recently downloaded files- Use
fdto search if needed:fd -e mp3 -e wav -e m4a -e flac '<keyword>' ~/Downloads
If multiple matches exist, ask the user which file to use.
Step 2: Determine Flags from Intent
Map the user's request to Pipeline.ts flags:
| User Says | Flag | Effect |
|---|---|---|
| "preview", "show edits", "what would you cut" | --preview | Show proposed edits without executing |
| "aggressive", "tight", "heavy edit" | --aggressive | Tighter silence/filler thresholds |
| "polish", "cleanvoice", "final pass" | --polish | Cleanvoice API cloud polish (requires CLEANVOICE_API_KEY) |
| (default) | (none) | Standard cleaning with conservative thresholds |
Step 3: Run the Pipeline
bun ~/.claude/skills/AudioEditor/Tools/Pipeline.ts \
"<audio-file-path>" \
[FLAGS_FROM_INTENT_MAPPING] \
--output "<output-path>"Output naming convention: <original-name>_edited.<ext> in the same directory as the input file.
Timeout: Set a 10-minute timeout. Transcription of long files can take several minutes on MPS.
Step 4: Report Results
After the pipeline completes, report:
- Number of edits applied
- Total time removed
- Original vs edited duration
- Output file path
- Artifacts generated (transcript, edits JSON, edited audio)
If --preview was used, display the edit list and ask if the user wants to proceed with execution.
Individual Tool Usage
For debugging or partial workflows, individual tools can be run standalone:
# Transcription only
bun ~/.claude/skills/AudioEditor/Tools/Transcribe.ts <file>
# Analysis only (requires transcript)
bun ~/.claude/skills/AudioEditor/Tools/Analyze.ts <transcript.json>
# Edit only (requires audio + edits)
bun ~/.claude/skills/AudioEditor/Tools/Edit.ts <file> <edits.json>
# Polish only (requires CLEANVOICE_API_KEY)
bun ~/.claude/skills/AudioEditor/Tools/Polish.ts <file>