
Media Use
- 211k installs
- 39.5k repo stars
- Updated August 5, 2026
- heygen-com/hyperframes
media-use is a Claude Code skill that provides comprehensive media asset management, generation, and operations for HyperFrames projects.
About
Media OS for HyperFrames projects: resolves, generates, operates on, and remembers media assets (audio, images, icons, logos, voice). Provides a single `resolve` command for BGM, SFX, images, icons, logos, and voice. Includes integrated audio engine for voiceover, transcription, captions, and background removal. Use for any media need in a HyperFrames project.
- Resolve and generate media assets (BGM, SFX, images, icons, logos, voice) with single verb
- Automatic reuse from project and global cache before fetching new assets
- Integrated audio engine for voiceover, transcription, captions, and background removal
Media Use by the numbers
- 210,774 all-time installs (skills.sh)
- +38,403 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #45 of 1,335 Generative Media skills by installs in the Skillselion catalog
- Security screen: CRITICAL risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
media-use capabilities & compatibility
- Capabilities
- resolve media · generate audio · generate images · transcribe · background remove
- Use cases
- image generation · video generation
npx skills add https://github.com/heygen-com/hyperframes --skill media-useAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 211k |
|---|---|
| repo stars | ★ 39.5k |
| Security audit | 1 / 3 scanners passed |
| Last updated | August 5, 2026 |
| Repository | heygen-com/hyperframes ↗ |
How do agents fetch and cache media assets locally?
Media OS for HyperFrames projects: resolves, generates, operates on, and remembers media assets (audio, images, icons, logos, voice). Provides a single `resolve` command for BGM, SFX, images, icons,
Who is it for?
Managing and generating all media types (audio, images, icons, logos, voice) for HyperFrames projects with automatic asset reuse
Skip if: Standalone stock-photo browsing, non-HyperFrames projects, or workflows that already ship all required media assets.
When should I use this skill?
A developer needs any audio, image, icon, voiceover, caption, or media-asset for a HyperFrames composition
What you get
Frozen local media file path, ledger manifest record, and cached asset across project and global stores.
- Frozen local media files
- Manifest records
- Transcriptions
By the numbers
- 6 media types: bgm, sfx, image, icon, logo, voice
- Free local providers: Kokoro (voice), mflux (image), LTX (video)
- Global cache at ~/.media for reuse across projects
Files
media-use
Resolve media needs into frozen local files. One verb, four types, zero context noise.
When to use
Call resolve whenever a composition needs media — background music, sound effects, images, or icons. media-use searches the HeyGen catalog, downloads the best match, freezes it locally, and registers it in a manifest. The agent gets back one line; all search noise stays on disk.
Resolve
node <SKILL_DIR>/scripts/resolve.mjs --type <type> --intent "<description>" --project <dir>Returns one line: resolved <id> → <path> (<type>, <metadata>)
Types
| Type | What it finds | Provider |
|---|---|---|
bgm | Background music | HeyGen audio catalog (10k+ tracks) |
sfx | Sound effects | Bundled 19-file library + HeyGen catalog |
image | Photos, backgrounds | HeyGen asset search (75k+ vectors) |
icon | Icons, logos | HeyGen asset search (type=icon) |
Examples
# Background music
node <SKILL_DIR>/scripts/resolve.mjs --type bgm --intent "upbeat tech launch" --project .
# → resolved bgm_001 → .media/audio/bgm/bgm_001.mp3 (bgm, 25s)
# Sound effect
node <SKILL_DIR>/scripts/resolve.mjs --type sfx --intent "whoosh" --project .
# → resolved sfx_001 → .media/audio/sfx/sfx_001.mp3 (sfx, 0.57s)
# Image
node <SKILL_DIR>/scripts/resolve.mjs --type image --intent "gradient tech background" --project .
# → resolved image_001 → .media/images/image_001.jpg (image)
# Icon
node <SKILL_DIR>/scripts/resolve.mjs --type icon --intent "rocket" --project .
# → resolved icon_001 → .media/images/icon_001.png (icon, transparent)Flags
| Flag | Description |
|---|---|
--type, -t | Media type: bgm, sfx, image, icon |
--intent, -i | What you need (natural language) |
--entity, -e | Entity name for cache matching (optional) |
--project, -p | Project directory (default: .) |
--adopt | Bulk-import existing assets/ into manifest |
--json | Output JSON instead of one-line result |
How it works
1. Check project .media/manifest.jsonl for exact-prompt match 2. Scan existing assets/ directory for unregistered files matching the need 3. Check global cache ~/.media/ for reusable asset 4. Search via provider (HeyGen audio catalog, HeyGen asset search) 5. Freeze file to .media/<type>/, register in manifest, regenerate index.md
The agent gets back one line. Candidates, scores, provenance stay on disk.
Adopt existing projects
Most HyperFrames projects already have assets in assets/. media-use adopts them:
node <SKILL_DIR>/scripts/resolve.mjs --adopt --project .
# → adopted 9 assets from assets/
# bgm_001 → assets/bgm/mango-fizz.mp3 (bgm, 146.6s)
# image_001 → assets/images/avatar.jpg (image, 400×400)ffprobe extracts real duration and dimensions. During resolve, unregistered files in assets/ matching the intent are adopted on the fly.
Reading the inventory
After resolve or adopt, read .media/index.md for the full inventory:
# .media · 4 assets
id type dur dims path description
bgm_001 bgm 25s — .media/audio/bgm/bgm_001.mp3 upbeat tech launch
sfx_001 sfx 0.6s — .media/audio/sfx/sfx_001.mp3 whoosh
image_001 image — 1920×1080 .media/images/image_001.jpg gradient tech background
icon_001 icon — 200×200 .media/images/icon_001.png rocketCross-project reuse
Assets are cached automatically on resolve. Subsequent resolves for the same prompt hit the global cache at ~/.media/ — no re-download, no provider call. Promote an asset explicitly with organize --promote <id> to make it reusable across all projects.
Files
.media/manifest.jsonl— machine SSOT, one JSON record per line.media/index.md— agent-readable table (id, type, dur, dims, path, description)~/.media/— global cross-project reuse cache (content-addressed, SHA-256)
CLI tools used
| Tool | Purpose | Required? |
|---|---|---|
ffprobe | Probe duration, dimensions, codec on adopt | Yes |
heygen | Audio catalog, asset search | For providers |
Install the heygen CLI (single static binary, no runtime) and authenticate:
curl -fsSL https://static.heygen.ai/cli/install.sh | bash # installs latest to ~/.local/bin
heygen update # if already installed: needs >= v0.1.6
export HEYGEN_API_KEY=<your-key> # or: heygen auth login --key <key>Requires heygen >= v0.1.6 — the providers tag requests with the allowlisted --headers 'X-HeyGen-Client-Source: media-use' flag, added in v0.1.6. asset search is a pre-launch command hidden from heygen --help, but it runs. Without a heygen on PATH (or a valid key) the providers print a one-line diagnostic to stderr and resolve falls through to "no provider could resolve".
eval-report.html
#!/usr/bin/env node
/**
* media-use eval — compare baseline (no media-use) vs. with media-use
* on real registry blocks. Produces an HTML report.
*/
import {
mkdtempSync,
cpSync,
rmSync,
readFileSync,
readdirSync,
existsSync,
writeFileSync,
} from "node:fs";
import { join, basename, resolve, dirname } from "node:path";
import { execFileSync } from "node:child_process";
import { tmpdir } from "node:os";
import { fileURLToPath } from "node:url";
const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = resolve(SCRIPT_DIR, "..", "..", "..");
const RESOLVE_SCRIPT = join(SCRIPT_DIR, "resolve.mjs");
const TEST_BLOCKS = [
"registry/blocks/nyc-paris-flight",
"registry/blocks/macos-tahoe-liquid-glass",
"registry/blocks/blue-sweater-intro-video",
"registry/blocks/vpn-youtube-spot",
"registry/blocks/apple-money-count",
"registry/blocks/liquid-glass-notification",
"registry/blocks/instagram-follow",
];
// Run resolve.mjs with args as a literal argv array (no shell), so values
// interpolated from manifest metadata (--intent prompt, --type) can't inject
// shell. Mirrors the execFileSync fix in probe.mjs / heygen-search.mjs.
function run(args, opts = {}) {
try {
return {
ok: true,
output: execFileSync(process.execPath, [RESOLVE_SCRIPT, ...args], {
encoding: "utf8",
timeout: 15000,
stdio: "pipe",
...opts,
}).trim(),
};
} catch (err) {
return { ok: false, output: (err.stdout || "") + (err.stderr || ""), code: err.status };
}
}
function countAssetFiles(dir) {
const assetsDir = join(dir, "assets");
if (!existsSync(assetsDir)) return { count: 0, files: [] };
const files = [];
function walk(d, base = "") {
for (const e of readdirSync(d, { withFileTypes: true })) {
const rel = base ? `${base}/${e.name}` : e.name;
if (e.isDirectory()) walk(join(d, e.name), rel);
else files.push(rel);
}
}
walk(assetsDir);
return { count: files.length, files };
}
function evalBlock(blockPath) {
const fullPath = join(REPO_ROOT, blockPath);
if (!existsSync(fullPath)) return null;
const name = basename(blockPath);
const tmp = mkdtempSync(join(tmpdir(), `mu-eval-${name}-`));
try {
cpSync(fullPath, tmp, { recursive: true });
// baseline: what the agent sees WITHOUT media-use
const baseline = countAssetFiles(tmp);
const htmlFiles = readdirSync(tmp).filter((f) => f.endsWith(".html"));
// parse compositions for asset references
const assetRefs = [];
for (const hf of htmlFiles) {
const html = readFileSync(join(tmp, hf), "utf8");
const srcMatches = html.matchAll(/src=["']([^"']+?)["']/g);
for (const m of srcMatches) {
const ref = m[1];
if (ref.startsWith("data:") || ref.startsWith("http")) continue;
assetRefs.push({ composition: hf, ref });
}
const urlMatches = html.matchAll(/url\(["']?([^"')]+?)["']?\)/g);
for (const m of urlMatches) {
const ref = m[1];
if (ref.startsWith("data:") || ref.startsWith("http") || ref.startsWith("#")) continue;
assetRefs.push({ composition: hf, ref });
}
}
// with media-use: run --adopt
const adoptResult = run(["--adopt", "--project", tmp, "--json"]);
let adopted = { ok: false, adopted: 0, assets: [] };
if (adoptResult.ok) {
try {
adopted = JSON.parse(adoptResult.output);
} catch {
/* */
}
}
// read the generated index
const indexPath = join(tmp, ".media", "index.md");
const indexContent = existsSync(indexPath)
? readFileSync(indexPath, "utf8")
: "(no index generated)";
// read manifest for detail
const manifestPath = join(tmp, ".media", "manifest.jsonl");
const manifest = existsSync(manifestPath)
? readFileSync(manifestPath, "utf8")
.trim()
.split("\n")
.map((l) => {
try {
return JSON.parse(l);
} catch {
return null;
}
})
.filter(Boolean)
: [];
// test resolve cache hit: try resolving something that was adopted
let resolveTest = null;
if (manifest.length > 0) {
const first = manifest[0];
const prompt = first.provenance?.prompt || first.description;
const r = run(["--type", first.type, "--intent", prompt, "--project", tmp, "--json"]);
if (r.ok) {
try {
resolveTest = JSON.parse(r.output);
} catch {
/* */
}
}
}
// test resolve miss: try resolving something that doesn't exist
const missResult = run([
"--type",
"bgm",
"--intent",
"nonexistent query xyz",
"--project",
tmp,
"--json",
]);
let resolveMiss = null;
if (!missResult.ok) {
try {
resolveMiss = JSON.parse(missResult.output);
} catch {
/* */
}
}
// coverage: which composition refs are covered by the manifest
const manifestPaths = new Set(manifest.map((m) => m.path));
const coverage = assetRefs.map((r) => ({
...r,
covered: manifestPaths.has(r.ref),
}));
return {
name,
baseline: { fileCount: baseline.count, files: baseline.files, htmlCount: htmlFiles.length },
compositions: htmlFiles,
assetRefs: coverage,
adopted: { count: adopted.adopted, assets: adopted.assets || [] },
index: indexContent,
manifest,
resolveTest,
resolveMiss,
};
} finally {
rmSync(tmp, { recursive: true, force: true });
}
}
function generateReport(results) {
const all = results.filter(Boolean);
const passed = all.filter((r) => r.adopted.count > 0);
const rows = results
.filter(Boolean)
.map((r) => {
const hasMetadata = r.manifest.some((m) => m.duration || m.width);
const cacheHit = r.resolveTest?._source === "cached";
const missHandled = r.resolveMiss?.ok === false;
return `<tr>
<td><strong>${r.name}</strong></td>
<td>${r.baseline.fileCount} files, ${r.baseline.htmlCount} comp${r.baseline.htmlCount === 1 ? "" : "s"}</td>
<td>${r.adopted.count} adopted</td>
<td>${hasMetadata ? "<span class='pass'>with metadata</span>" : "<span class='warn'>no metadata</span>"}</td>
<td>${cacheHit ? "<span class='pass'>cache hit</span>" : "<span class='warn'>no hit</span>"}</td>
<td>${missHandled ? "<span class='pass'>handled</span>" : "<span class='fail'>unexpected</span>"}</td>
</tr>`;
})
.join("\n");
const details = results
.filter(Boolean)
.filter((r) => r.adopted.count > 0)
.map((r) => {
const assetRows = r.manifest
.map((m) => {
const dur = m.duration != null ? `${m.duration}s` : "—";
const dims = m.width && m.height ? `${m.width}×${m.height}` : "—";
return `<tr><td>${m.id}</td><td>${m.type}</td><td>${dur}</td><td>${dims}</td><td class="path">${m.path}</td><td>${m.description || ""}</td></tr>`;
})
.join("\n");
const coveredCount = r.assetRefs.filter((c) => c.covered).length;
const totalRefs = r.assetRefs.length;
const coveragePct = totalRefs > 0 ? Math.round((coveredCount / totalRefs) * 100) : 100;
const refRows = r.assetRefs
.map(
(c) =>
`<tr><td class="path">${c.composition}</td><td class="path">${c.ref}</td><td>${c.covered ? "<span class='pass'>covered</span>" : "<span class='warn'>not in manifest</span>"}</td></tr>`,
)
.join("\n");
return `<div class="block-detail">
<h3>${r.name}</h3>
<p style="font-size:13px;color:var(--muted)">${r.compositions.length} composition${r.compositions.length === 1 ? "" : "s"}: ${r.compositions.join(", ")}</p>
<div class="comparison">
<div class="col">
<h4>Baseline (no media-use)</h4>
<p>Agent sees: ${r.baseline.fileCount} raw files in assets/<br>No metadata, no type info, no relationship to compositions.</p>
<pre class="file-list">${r.baseline.files.join("\n") || "(no assets)"}</pre>
</div>
<div class="col">
<h4>With media-use (after --adopt)</h4>
<p>Agent reads index.md — structured, typed, with metadata:</p>
<pre class="index">${escapeHtml(r.index)}</pre>
</div>
</div>
${
totalRefs > 0
? `<h4>Composition → asset coverage <span class="${coveragePct === 100 ? "pass" : "warn"}">${coveragePct}%</span> (${coveredCount}/${totalRefs} refs)</h4>
<table class="manifest">
<thead><tr><th>composition</th><th>asset reference</th><th>in manifest?</th></tr></thead>
<tbody>${refRows}</tbody>
</table>`
: ""
}
<h4>Manifest records</h4>
<table class="manifest">
<thead><tr><th>id</th><th>type</th><th>dur</th><th>dims</th><th>path</th><th>description</th></tr></thead>
<tbody>${assetRows}</tbody>
</table>
</div>`;
})
.join("\n");
return `<title>media-use eval report</title>
<style>
:root { --bg: #fafaf7; --text: #1b1b18; --muted: #7a756a; --accent: #0d7377; --good: #1a7a3a; --warn: #b45309; --fail: #dc2626; --border: #e8e5df; --surface: #fff; --mono: ui-monospace, 'SF Mono', Menlo, Consolas, monospace; --sans: system-ui, -apple-system, sans-serif; --serif: Georgia, serif }
* { box-sizing: border-box; margin: 0 } body { background: var(--bg); color: var(--text); font-family: var(--serif); line-height: 1.6; font-size: 15px; padding: 40px 24px }
.wrap { max-width: 1100px; margin: 0 auto }
h1 { font-family: var(--sans); font-size: 28px; font-weight: 700; margin-bottom: 8px; letter-spacing: -.02em }
h2 { font-family: var(--sans); font-size: 20px; font-weight: 650; margin: 32px 0 12px; letter-spacing: -.01em }
h3 { font-family: var(--sans); font-size: 17px; font-weight: 650; margin: 24px 0 8px }
h4 { font-family: var(--sans); font-size: 14px; font-weight: 600; margin: 16px 0 6px; color: var(--muted) }
p { margin-bottom: 10px }
.meta { font-family: var(--mono); font-size: 12px; color: var(--muted); margin-bottom: 24px }
.summary { display: flex; gap: 16px; margin: 16px 0; flex-wrap: wrap }
.stat { background: var(--surface); border: 1px solid var(--border); border-radius: 8px; padding: 14px 18px; flex: 1; min-width: 140px }
.stat .num { font-family: var(--sans); font-size: 28px; font-weight: 700; color: var(--accent) }
.stat .label { font-family: var(--mono); font-size: 11px; color: var(--muted); text-transform: uppercase; letter-spacing: .1em }
table { width: 100%; border-collapse: collapse; font-size: 13px; font-family: var(--sans); margin: 8px 0 }
th { text-align: left; font-family: var(--mono); font-size: 10px; letter-spacing: .08em; text-transform: uppercase; color: var(--muted); border-bottom: 2px solid var(--border); padding: 6px 8px; font-weight: 700 }
td { border-bottom: 1px solid var(--border); padding: 7px 8px; vertical-align: top }
td.path { font-family: var(--mono); font-size: 12px; color: var(--muted); max-width: 300px; overflow: hidden; text-overflow: ellipsis }
.pass { color: var(--good); font-weight: 600 } .warn { color: var(--warn); font-weight: 600 } .fail { color: var(--fail); font-weight: 600 }
.comparison { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; margin: 12px 0 }
@media(max-width:700px) { .comparison { grid-template-columns: 1fr } }
.col { background: var(--surface); border: 1px solid var(--border); border-radius: 8px; padding: 14px 16px }
.col h4 { margin-top: 0 }
pre { font-family: var(--mono); font-size: 12px; background: #1b1b18; color: #d4d0c8; border-radius: 6px; padding: 12px 14px; overflow-x: auto; margin: 6px 0; line-height: 1.5 }
pre.file-list { background: var(--bg); color: var(--muted); border: 1px solid var(--border) }
pre.index { white-space: pre; }
.block-detail { border-top: 1px solid var(--border); padding-top: 20px; margin-top: 20px }
.verdict { margin-top: 24px; padding: 16px 20px; border-radius: 8px; font-family: var(--sans); font-size: 15px }
.verdict.ship { background: #edfbf0; border: 1px solid #1a7a3a; color: #1a7a3a }
.verdict.wait { background: #fff3ec; border: 1px solid #d94f04; color: #d94f04 }
</style>
<div class="wrap">
<h1>media-use eval report</h1>
<p class="meta">${new Date().toISOString().slice(0, 10)} · ${all.length} blocks evaluated · baseline vs. media-use --adopt</p>
<div class="summary">
<div class="stat"><div class="num">${all.length}</div><div class="label">blocks tested</div></div>
<div class="stat"><div class="num">${passed.length}</div><div class="label">with assets</div></div>
<div class="stat"><div class="num">${all.reduce((s, r) => s + r.adopted.count, 0)}</div><div class="label">assets adopted</div></div>
<div class="stat"><div class="num">${all.filter((r) => r.manifest.some((m) => m.duration || m.width)).length}</div><div class="label">with ffprobe metadata</div></div>
<div class="stat"><div class="num">${(() => {
const refs = all.flatMap((r) => r.assetRefs);
const covered = refs.filter((c) => c.covered).length;
return refs.length > 0 ? Math.round((covered / refs.length) * 100) + "%" : "—";
})()}</div><div class="label">composition coverage</div></div>
</div>
<h2>Results matrix</h2>
<table>
<thead><tr><th>Block</th><th>Baseline</th><th>Adopted</th><th>Metadata</th><th>Cache hit</th><th>Miss handling</th></tr></thead>
<tbody>${rows}</tbody>
</table>
<h2>Before / after comparisons</h2>
${details}
<div class="verdict ${passed.length >= 3 ? "ship" : "wait"}">
${
passed.length >= 3
? `<strong>Ship it.</strong> ${passed.length}/${all.length} blocks adopted successfully with metadata. Resolve cache hits work. Miss handling is clean.`
: `<strong>Needs work.</strong> Only ${passed.length} blocks adopted. Check the failures above.`
}
</div>
</div>`;
}
function escapeHtml(str) {
return str.replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
}
console.log("media-use eval · running against registry blocks...\n");
const results = [];
for (const block of TEST_BLOCKS) {
const fullPath = join(REPO_ROOT, block);
if (!existsSync(fullPath)) {
console.log(` skip ${basename(block)} (not found)`);
results.push(null);
continue;
}
process.stdout.write(` ${basename(block)}...`);
const result = evalBlock(block);
if (result) {
console.log(
` ${result.adopted.count} adopted, ${result.manifest.filter((m) => m.duration || m.width).length} with metadata`,
);
} else {
console.log(" failed");
}
results.push(result);
}
const report = generateReport(results);
const outPath = join(SCRIPT_DIR, "..", "eval-report.html");
writeFileSync(outPath, report);
console.log(`\nReport: ${outPath}`);
import { readdirSync, statSync, existsSync } from "node:fs";
import { join, extname, basename } from "node:path";
import { readManifest, appendRecord, nextId } from "./manifest.mjs";
import { regenerateIndex } from "./index-gen.mjs";
import { probe } from "./probe.mjs";
const AUDIO_EXT = new Set([".mp3", ".wav", ".ogg", ".m4a", ".aac"]);
const IMAGE_EXT = new Set([".jpg", ".jpeg", ".png", ".gif", ".webp", ".svg", ".ico"]);
const VIDEO_EXT = new Set([".mp4", ".webm", ".mov"]);
function inferType(filePath) {
const ext = extname(filePath).toLowerCase();
if (AUDIO_EXT.has(ext)) {
const lower = filePath.toLowerCase();
if (lower.includes("/bgm/") || lower.includes("/music/") || lower.startsWith("bgm/"))
return "bgm";
if (lower.includes("/sfx/") || lower.includes("/sound") || lower.startsWith("sfx/"))
return "sfx";
if (lower.includes("/voice/") || lower.includes("/narrat") || lower.startsWith("voice/"))
return "voice";
return "bgm";
}
if (IMAGE_EXT.has(ext)) {
if (ext === ".svg" || ext === ".ico") return "icon";
return "image";
}
if (VIDEO_EXT.has(ext)) return "video";
return null;
}
function walkDir(dir, base = "") {
const files = [];
if (!existsSync(dir)) return files;
for (const entry of readdirSync(dir, { withFileTypes: true })) {
const rel = base ? `${base}/${entry.name}` : entry.name;
if (entry.isDirectory()) {
files.push(...walkDir(join(dir, entry.name), rel));
} else {
files.push(rel);
}
}
return files;
}
export function scanExistingAssets(projectDir) {
const assetsDir = join(projectDir, "assets");
if (!existsSync(assetsDir)) return [];
const files = walkDir(assetsDir);
const found = [];
for (const rel of files) {
const type = inferType(rel);
if (!type) continue;
const fullPath = join(assetsDir, rel);
const stat = statSync(fullPath);
const meta = probe(fullPath);
found.push({
relativePath: `assets/${rel}`,
type,
size: stat.size,
name: basename(rel, extname(rel)),
...meta,
});
}
return found;
}
export function adoptExistingAssets(projectDir) {
const existing = scanExistingAssets(projectDir);
if (existing.length === 0) return [];
const manifest = readManifest(projectDir);
const knownPaths = new Set(manifest.map((r) => r.path));
const adopted = [];
for (const asset of existing) {
if (knownPaths.has(asset.relativePath)) continue;
const id = nextId(projectDir, asset.type);
const record = {
id,
type: asset.type,
path: asset.relativePath,
source: "existing",
description: asset.name.replace(/[-_]/g, " "),
...(asset.duration != null && { duration: asset.duration }),
...(asset.width != null && { width: asset.width }),
...(asset.height != null && { height: asset.height }),
provenance: { provider: "local", adopted: true },
};
appendRecord(projectDir, record);
adopted.push(record);
}
if (adopted.length > 0) regenerateIndex(projectDir);
return adopted;
}
export function findExistingAsset(projectDir, intent, type) {
const assetsDir = join(projectDir, "assets");
if (!existsSync(assetsDir)) return null;
const lower = intent.toLowerCase();
for (const rel of walkDir(assetsDir)) {
const t = inferType(rel);
if (!t || (type && t !== type)) continue;
const name = basename(rel, extname(rel)).toLowerCase().replace(/[-_]/g, " ");
if (name.includes(lower) || lower.includes(name)) {
return { relativePath: `assets/${rel}`, type: t, name: basename(rel, extname(rel)) };
}
}
return null;
}
import { heygenSearch } from "./heygen-search.mjs";
export const bgmProvider = {
async search(intent) {
const results = heygenSearch("audio sounds list", intent, { type: "music" });
if (!results) return null;
const best = results[0];
return {
url: best.audio_url,
source: "search",
// ext derived from audio_url by resolve.mjs — catalog tracks are .mp3 or .wav
metadata: {
description: best.description || intent,
duration: best.duration || null,
provider: "heygen.audio.sounds",
provenance: { track_id: best.id, score: best.score, query: intent },
},
};
},
};
import { readFileSync, existsSync } from "node:fs";
import { join } from "node:path";
function findDesignSpec(projectDir) {
for (const name of ["frame.md", "design.md", "DESIGN.md"]) {
const p = join(projectDir, name);
if (existsSync(p)) return { path: p, name };
}
return null;
}
function parseFrontmatter(content) {
const match = content.match(/^---\n([\s\S]*?)\n---/);
if (!match) return null;
const yaml = match[1];
const tokens = {};
for (const line of yaml.split("\n")) {
const m = line.match(/^\s*(\w[\w-]*):\s*(.+)/);
if (m) tokens[m[1]] = m[2].trim().replace(/^["']|["']$/g, "");
}
return tokens;
}
function extractColors(tokens) {
const colors = [];
for (const [k, v] of Object.entries(tokens)) {
if (typeof v === "string" && /^#[0-9a-fA-F]{3,8}$/.test(v)) {
colors.push({ name: k, hex: v });
}
}
return colors;
}
export const brandProvider = {
async search(intent, { projectDir } = {}) {
if (!projectDir) return null;
const spec = findDesignSpec(projectDir);
if (!spec) return null;
const content = readFileSync(spec.path, "utf8");
const tokens = parseFrontmatter(content);
if (!tokens) return null;
const colors = extractColors(tokens);
return {
localPath: spec.path,
source: "local",
ext: ".md",
metadata: {
description: "Brand tokens from " + spec.name,
provider: "design_spec",
provenance: {
file: spec.name,
colors,
font: tokens.font || tokens.typography || null,
logo: tokens.logo || null,
},
},
};
},
};
import { readFileSync, writeFileSync, mkdirSync, existsSync, copyFileSync } from "node:fs";
import { join, basename } from "node:path";
import { createHash } from "node:crypto";
import { homedir } from "node:os";
import { readManifest, appendRecord } from "./manifest.mjs";
const SCHEMA_PREFIX = "mu-v1-";
const KEY_HEX_CHARS = 16;
const COMPLETE_SENTINEL = ".hf-complete";
export function globalMediaDir() {
return join(homedir(), ".media");
}
export function contentHash(filePath) {
const bytes = readFileSync(filePath);
return createHash("sha256").update(bytes).digest("hex");
}
function cacheEntryDir(rootDir, sha) {
return join(rootDir, SCHEMA_PREFIX + sha.slice(0, KEY_HEX_CHARS));
}
function isComplete(entryDir) {
return existsSync(join(entryDir, COMPLETE_SENTINEL));
}
function markComplete(entryDir) {
writeFileSync(join(entryDir, COMPLETE_SENTINEL), "", "utf8");
}
function readGlobalManifest() {
return readManifest(globalMediaDir());
}
function validateCacheHit(match) {
if (!match?.sha) return null;
return isComplete(cacheEntryDir(globalMediaDir(), match.sha)) ? match : null;
}
export function cacheGet(prompt, type) {
return validateCacheHit(
readGlobalManifest().find(
(r) => r.reusable && r.provenance?.prompt === prompt && (type == null || r.type === type),
),
);
}
export function cacheGetByEntity(entity) {
const lower = entity.toLowerCase();
return validateCacheHit(
readGlobalManifest().find((r) => r.reusable && r.entity && r.entity.toLowerCase() === lower),
);
}
export function cachePut(filePath, record) {
const sha = contentHash(filePath);
const dir = globalMediaDir();
const entryDir = cacheEntryDir(dir, sha);
mkdirSync(entryDir, { recursive: true });
const dest = join(entryDir, basename(filePath));
copyFileSync(filePath, dest);
markComplete(entryDir);
const globalRecord = {
...record,
sha,
reusable: true,
cached_path: dest,
};
appendRecord(globalMediaDir(), globalRecord);
return { sha, cached_path: dest };
}
export function importFromCache(cacheRecord, projectDir, localId, localPath) {
const sha = cacheRecord.sha;
const entryDir = cacheEntryDir(globalMediaDir(), sha);
if (!isComplete(entryDir)) return null;
const cachedFile = cacheRecord.cached_path;
if (!cachedFile || !existsSync(cachedFile)) return null;
mkdirSync(join(projectDir, ".media"), { recursive: true });
const fullDest = join(projectDir, localPath);
mkdirSync(join(fullDest, ".."), { recursive: true });
copyFileSync(cachedFile, fullDest);
const projectRecord = {
...cacheRecord,
id: localId,
path: localPath,
provenance: {
...cacheRecord.provenance,
imported_from: sha,
},
};
delete projectRecord.sha;
delete projectRecord.reusable;
delete projectRecord.cached_path;
return projectRecord;
}
export function promote(projectDir, id) {
const records = readManifest(projectDir);
const record = records.find((r) => r.id === id);
if (!record) throw new Error(`asset not found in project manifest: ${id}`);
const filePath = join(projectDir, record.path);
if (!existsSync(filePath)) throw new Error(`asset file not found: ${filePath}`);
return cachePut(filePath, record);
}
import { writeFileSync, copyFileSync, mkdirSync } from "node:fs";
import { dirname } from "node:path";
// ponytail: bound the download so a hostile/runaway URL can't fill the disk.
// 256MB covers any real media asset; raise if 4K video sources ever exceed it.
const MAX_FREEZE_BYTES = 256 * 1024 * 1024;
export async function freezeUrl(url, destPath) {
const res = await fetch(url);
if (!res.ok) throw new Error(`freeze failed: HTTP ${res.status} for ${String(url).slice(0, 80)}`);
const bytes = Buffer.from(await res.arrayBuffer());
if (bytes.length === 0)
throw new Error(`freeze failed: empty response for ${String(url).slice(0, 80)}`);
if (bytes.length > MAX_FREEZE_BYTES)
throw new Error(
`freeze failed: ${bytes.length} bytes exceeds ${MAX_FREEZE_BYTES} cap for ${String(url).slice(0, 80)}`,
);
mkdirSync(dirname(destPath), { recursive: true });
writeFileSync(destPath, bytes);
return bytes.length;
}
export function freezeLocalFile(srcPath, destPath) {
mkdirSync(dirname(destPath), { recursive: true });
copyFileSync(srcPath, destPath);
}
import { execFileSync } from "node:child_process";
export function heygenSearch(subcommand, query, { type, limit = 5, minScore } = {}) {
// execFileSync with an argv array (no shell), so query/type/etc. are passed as
// literal arguments — no quoting tricks, no command injection. subcommand is a
// hardcoded multi-word string (e.g. "audio sounds list"), split into tokens.
// Tag the caller via the CLI's allowlisted attribution header (heygen >= v0.1.6).
const args = [
"--headers",
"X-HeyGen-Client-Source: media-use",
...subcommand.split(" "),
"--query",
query,
];
if (type) args.push("--type", type);
args.push("--limit", String(limit));
// Server-side score floor. Honored by `audio sounds list`; the `asset search`
// backend rejects it, so only audio providers pass minScore (see image-provider).
if (minScore != null) args.push("--min-score", String(minScore));
let out;
try {
out = execFileSync("heygen", args, {
encoding: "utf8",
timeout: 15000,
stdio: ["pipe", "pipe", "pipe"],
});
} catch (err) {
// Don't swallow a broken command / auth failure as "no results" — that turns
// a typo or expired key into a silent dead end. Surface it, then give up.
const detail = err.stderr?.toString().trim() || err.stdout?.toString().trim() || err.message;
console.error(`media-use: \`heygen ${subcommand}\` failed: ${detail}`);
return null;
}
let parsed;
try {
parsed = JSON.parse(out);
} catch {
console.error(`media-use: \`heygen ${subcommand}\` returned non-JSON output`);
return null;
}
if (parsed?.error) {
const e = parsed.error;
console.error(`media-use: \`heygen ${subcommand}\` error: ${e.message ?? JSON.stringify(e)}`);
return null;
}
const data = parsed?.data;
return Array.isArray(data) && data.length > 0 ? data : null;
}
import { heygenSearch } from "./heygen-search.mjs";
export const imageProvider = {
async search(intent) {
const results = heygenSearch("asset search", intent, { type: "image" });
if (!results) return null;
const best = results[0];
return {
url: best.url,
source: "search",
// ext derived from the asset URL by resolve.mjs (.jpg/.png/.webp)
metadata: {
description: intent,
width: best.width || null,
height: best.height || null,
transparent: best.is_transparent || false,
provider: "heygen.asset.search",
provenance: { asset_id: best.id, score: best.score },
},
};
},
};
export const iconProvider = {
async search(intent) {
// No minScore: the `asset search` backend rejects --min-score and returns no score field.
const results = heygenSearch("asset search", intent, { type: "icon" });
if (!results) return null;
const best = results[0];
return {
url: best.url,
source: "search",
// ext derived from the asset URL by resolve.mjs — catalog icons are .png, not .svg
metadata: {
description: intent,
width: best.width || null,
height: best.height || null,
transparent: best.is_transparent ?? true,
provider: "heygen.asset.search",
provenance: { asset_id: best.id, score: best.score, type: "icon" },
},
};
},
};
import { writeFileSync, mkdirSync } from "node:fs";
import { dirname } from "node:path";
import { readManifest, indexPath } from "./manifest.mjs";
function pad(str, len) {
return String(str ?? "").padEnd(len);
}
function formatDur(record) {
if (record.duration == null) return "—";
return `${record.duration}s`;
}
function formatDims(record) {
if (record.width && record.height) return `${record.width}×${record.height}`;
if (record.type === "icon" && record.transparent) return "svg";
return "—";
}
export function generateIndexContent(records) {
const count = records.length;
const header = `# .media · ${count} asset${count === 1 ? "" : "s"}\n`;
if (count === 0) return header;
const cols = { id: 4, type: 5, dur: 4, dims: 5, path: 5, desc: 11 };
for (const r of records) {
cols.id = Math.max(cols.id, (r.id ?? "").length);
cols.type = Math.max(cols.type, (r.type ?? "").length);
cols.dur = Math.max(cols.dur, formatDur(r).length);
cols.dims = Math.max(cols.dims, formatDims(r).length);
cols.path = Math.max(cols.path, (r.path ?? "").length);
}
const heading =
pad("id", cols.id + 2) +
pad("type", cols.type + 2) +
pad("dur", cols.dur + 2) +
pad("dims", cols.dims + 2) +
pad("path", cols.path + 2) +
"description";
const lines = [header, heading];
for (const r of records) {
lines.push(
pad(r.id, cols.id + 2) +
pad(r.type, cols.type + 2) +
pad(formatDur(r), cols.dur + 2) +
pad(formatDims(r), cols.dims + 2) +
pad(r.path, cols.path + 2) +
(r.description ?? ""),
);
}
return lines.join("\n") + "\n";
}
export function regenerateIndex(projectDir) {
const records = readManifest(projectDir);
const content = generateIndexContent(records);
const p = indexPath(projectDir);
mkdirSync(dirname(p), { recursive: true });
writeFileSync(p, content);
return content;
}
import { readFileSync, appendFileSync, mkdirSync, existsSync } from "node:fs";
import { join } from "node:path";
const MANIFEST_FILE = "manifest.jsonl";
const INDEX_FILE = "index.md";
const TYPE_DIRS = {
bgm: "audio/bgm",
sfx: "audio/sfx",
voice: "audio/voice",
image: "images",
icon: "images",
brand: "images",
video: "video",
};
export function mediaDir(projectDir) {
return join(projectDir, ".media");
}
export function manifestPath(projectDir) {
return join(mediaDir(projectDir), MANIFEST_FILE);
}
export function indexPath(projectDir) {
return join(mediaDir(projectDir), INDEX_FILE);
}
export function typeSubdir(type) {
const sub = TYPE_DIRS[type];
if (!sub) throw new Error(`unknown media type: ${type}`);
return sub;
}
export function typeDirPath(projectDir, type) {
return join(mediaDir(projectDir), typeSubdir(type));
}
export function readManifest(projectDir) {
const p = manifestPath(projectDir);
if (!existsSync(p)) return [];
const raw = readFileSync(p, "utf8");
const records = [];
for (const line of raw.split(/\r?\n/)) {
const trimmed = line.trim();
if (!trimmed) continue;
try {
records.push(JSON.parse(trimmed));
} catch {
// ponytail: skip malformed lines, don't crash
}
}
return records;
}
export function appendRecord(projectDir, record) {
const dir = mediaDir(projectDir);
mkdirSync(dir, { recursive: true });
const typeDir = typeDirPath(projectDir, record.type);
mkdirSync(typeDir, { recursive: true });
const p = manifestPath(projectDir);
const line = JSON.stringify(record) + "\n";
appendFileSync(p, line);
}
export function findByPrompt(projectDir, prompt, type) {
const records = readManifest(projectDir);
return (
records.find((r) => r.provenance?.prompt === prompt && (type == null || r.type === type)) ||
null
);
}
export function findByEntity(projectDir, entity) {
const lower = entity.toLowerCase();
const records = readManifest(projectDir);
return records.find((r) => r.entity && r.entity.toLowerCase() === lower) || null;
}
export function nextId(projectDir, type) {
const records = readManifest(projectDir);
const prefix = type;
let max = 0;
for (const r of records) {
if (r.type !== type) continue;
const m = r.id?.match(new RegExp(`^${prefix}_(\\d+)$`));
if (m) max = Math.max(max, parseInt(m[1], 10));
}
return `${prefix}_${String(max + 1).padStart(3, "0")}`;
}
import { strict as assert } from "node:assert";
import { mkdtempSync, rmSync, readFileSync, writeFileSync, mkdirSync, existsSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import {
readManifest,
appendRecord,
findByPrompt,
findByEntity,
nextId,
manifestPath,
mediaDir,
typeDirPath,
} from "./manifest.mjs";
import { regenerateIndex, generateIndexContent } from "./index-gen.mjs";
import {
contentHash,
cachePut,
cacheGet,
cacheGetByEntity,
importFromCache,
promote,
} from "./cache.mjs";
let tmp;
function setup() {
tmp = mkdtempSync(join(tmpdir(), "mu-test-"));
}
function cleanup() {
if (tmp) rmSync(tmp, { recursive: true, force: true });
}
function makeRecord(overrides = {}) {
return {
id: "bgm_001",
type: "bgm",
path: ".media/audio/bgm/bgm_001.wav",
source: "search",
description: "soft minimal ambient",
duration: 11,
provenance: { provider: "heygen.audio.sounds", prompt: "subtle tech" },
...overrides,
};
}
function runTests() {
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
// --- manifest.mjs ---
test("readManifest returns empty array when no manifest exists", () => {
setup();
const result = readManifest(tmp);
assert.deepStrictEqual(result, []);
cleanup();
});
test("appendRecord writes valid JSONL and readManifest parses it back", () => {
setup();
const record = makeRecord();
appendRecord(tmp, record);
const records = readManifest(tmp);
assert.equal(records.length, 1);
assert.deepStrictEqual(records[0], record);
cleanup();
});
test("appendRecord creates .media/ and type subdirs on first write", () => {
setup();
appendRecord(tmp, makeRecord());
assert.ok(existsSync(mediaDir(tmp)));
assert.ok(existsSync(typeDirPath(tmp, "bgm")));
cleanup();
});
test("appendRecord appends multiple records", () => {
setup();
appendRecord(tmp, makeRecord({ id: "bgm_001" }));
appendRecord(tmp, makeRecord({ id: "bgm_002", provenance: { prompt: "energetic" } }));
const records = readManifest(tmp);
assert.equal(records.length, 2);
assert.equal(records[0].id, "bgm_001");
assert.equal(records[1].id, "bgm_002");
cleanup();
});
test("findByPrompt returns exact-match record", () => {
setup();
appendRecord(tmp, makeRecord());
const found = findByPrompt(tmp, "subtle tech", "bgm");
assert.ok(found);
assert.equal(found.id, "bgm_001");
cleanup();
});
test("findByPrompt returns null on miss", () => {
setup();
appendRecord(tmp, makeRecord());
assert.equal(findByPrompt(tmp, "nonexistent", "bgm"), null);
cleanup();
});
test("findByPrompt filters by type", () => {
setup();
appendRecord(tmp, makeRecord({ type: "sfx" }));
assert.equal(findByPrompt(tmp, "subtle tech", "bgm"), null);
assert.ok(findByPrompt(tmp, "subtle tech", "sfx"));
cleanup();
});
test("findByEntity matches case-insensitively", () => {
setup();
appendRecord(tmp, makeRecord({ entity: "GitHub", type: "icon" }));
assert.ok(findByEntity(tmp, "github"));
assert.ok(findByEntity(tmp, "GITHUB"));
assert.equal(findByEntity(tmp, "gitlab"), null);
cleanup();
});
test("nextId generates sequential ids", () => {
setup();
assert.equal(nextId(tmp, "bgm"), "bgm_001");
appendRecord(tmp, makeRecord({ id: "bgm_001" }));
assert.equal(nextId(tmp, "bgm"), "bgm_002");
appendRecord(tmp, makeRecord({ id: "bgm_002" }));
assert.equal(nextId(tmp, "bgm"), "bgm_003");
cleanup();
});
// --- index-gen.mjs ---
test("regenerateIndex produces plain-column table", () => {
setup();
appendRecord(tmp, makeRecord());
regenerateIndex(tmp);
const content = readFileSync(join(tmp, ".media", "index.md"), "utf8");
assert.ok(content.includes("# .media · 1 asset"));
assert.ok(content.includes("bgm_001"));
assert.ok(content.includes("soft minimal ambient"));
assert.ok(content.includes("11s"));
cleanup();
});
test("regenerateIndex handles empty manifest", () => {
setup();
mkdirSync(join(tmp, ".media"), { recursive: true });
writeFileSync(manifestPath(tmp), "");
regenerateIndex(tmp);
const content = readFileSync(join(tmp, ".media", "index.md"), "utf8");
assert.ok(content.includes("# .media · 0 assets"));
cleanup();
});
test("generateIndexContent includes dims for images", () => {
const records = [
makeRecord({ id: "img_001", type: "image", width: 1920, height: 1080, duration: null }),
];
const content = generateIndexContent(records);
assert.ok(content.includes("1920×1080"));
assert.ok(content.includes("img_001"));
});
test("regenerateIndex matches manifest content after multiple writes", () => {
setup();
appendRecord(tmp, makeRecord({ id: "bgm_001" }));
appendRecord(
tmp,
makeRecord({ id: "sfx_001", type: "sfx", description: "whoosh", duration: 3 }),
);
regenerateIndex(tmp);
const content = readFileSync(join(tmp, ".media", "index.md"), "utf8");
assert.ok(content.includes("# .media · 2 assets"));
assert.ok(content.includes("bgm_001"));
assert.ok(content.includes("sfx_001"));
assert.ok(content.includes("whoosh"));
cleanup();
});
// --- cache.mjs ---
test("cacheGet returns null when cache is empty", () => {
const result = cacheGet("nonexistent prompt", "bgm");
assert.equal(result, null);
});
test("cachePut + cacheGet round-trip", () => {
setup();
const filePath = join(tmp, "test.wav");
writeFileSync(filePath, "fake audio bytes for testing");
const record = makeRecord({ provenance: { prompt: "cache test" } });
const { sha } = cachePut(filePath, record);
assert.ok(sha);
assert.equal(sha.length, 64);
const found = cacheGet("cache test", "bgm");
assert.ok(found);
assert.equal(found.reusable, true);
assert.equal(found.sha, sha);
cleanup();
});
test("cacheGetByEntity finds cached asset", () => {
setup();
const filePath = join(tmp, "logo.png");
writeFileSync(filePath, "fake png bytes");
const record = makeRecord({
type: "icon",
entity: "TestCorp",
provenance: { prompt: "TestCorp logo" },
});
cachePut(filePath, record);
const found = cacheGetByEntity("testcorp");
assert.ok(found);
assert.equal(found.entity, "TestCorp");
cleanup();
});
test("contentHash is deterministic", () => {
setup();
const filePath = join(tmp, "det.bin");
writeFileSync(filePath, "deterministic content");
const h1 = contentHash(filePath);
const h2 = contentHash(filePath);
assert.equal(h1, h2);
cleanup();
});
test("promote copies project asset to global cache", () => {
setup();
const record = makeRecord();
appendRecord(tmp, record);
const filePath = join(tmp, record.path);
mkdirSync(join(filePath, ".."), { recursive: true });
writeFileSync(filePath, "promotable audio data");
const { sha } = promote(tmp, "bgm_001");
assert.ok(sha);
const cached = cacheGet("subtle tech", "bgm");
assert.ok(cached);
assert.equal(cached.sha, sha);
cleanup();
});
test("importFromCache copies cached file into project", () => {
setup();
const filePath = join(tmp, "source.wav");
writeFileSync(filePath, "importable audio");
const record = makeRecord({ provenance: { prompt: "import test" } });
const { sha } = cachePut(filePath, record);
const cached = cacheGet("import test", "bgm");
const projectDir = mkdtempSync(join(tmpdir(), "mu-import-"));
const imported = importFromCache(cached, projectDir, "bgm_001", ".media/audio/bgm/bgm_001.wav");
assert.ok(imported);
assert.equal(imported.id, "bgm_001");
assert.equal(imported.provenance.imported_from, sha);
assert.ok(existsSync(join(projectDir, ".media/audio/bgm/bgm_001.wav")));
rmSync(projectDir, { recursive: true, force: true });
cleanup();
});
// --- run ---
let passed = 0;
let failed = 0;
for (const { name, fn } of tests) {
try {
fn();
passed++;
console.log(` \x1b[32m✓\x1b[0m ${name}`);
} catch (err) {
failed++;
console.log(` \x1b[31m✗\x1b[0m ${name}`);
console.log(` ${err.message}`);
}
}
console.log(`\n${passed} passed, ${failed} failed`);
if (failed > 0) process.exit(1);
}
console.log("media-use · manifest/index/cache tests\n");
runTests();
import { execFileSync } from "node:child_process";
import { extname } from "node:path";
const IMAGE_EXT = new Set([".jpg", ".jpeg", ".png", ".gif", ".webp", ".svg", ".ico"]);
export function probe(filePath) {
const ext = extname(filePath).toLowerCase();
if (ext === ".svg") return { width: null, height: null, duration: null, codec: "svg" };
try {
// execFileSync (no shell) so a hostile filename like `"; rm -rf ~; ".png`
// can't break out of the quoting — filePath is passed as a literal argv entry.
const raw = execFileSync(
"ffprobe",
["-v", "quiet", "-print_format", "json", "-show_format", "-show_streams", filePath],
{ encoding: "utf8", timeout: 5000 },
);
const info = JSON.parse(raw);
const stream = info.streams?.[0];
const format = info.format;
const isImage = IMAGE_EXT.has(ext);
const duration = isImage
? null
: parseFloat(format?.duration) || parseFloat(stream?.duration) || null;
const width = parseInt(stream?.width, 10) || null;
const height = parseInt(stream?.height, 10) || null;
const codec = stream?.codec_name || null;
return {
duration: duration != null ? Math.round(duration * 10) / 10 : null,
width,
height,
codec,
};
} catch {
return { duration: null, width: null, height: null, codec: null };
}
}
import { test } from "node:test";
import assert from "node:assert/strict";
import { mkdtempSync, writeFileSync, existsSync, rmSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { probe } from "./probe.mjs";
// Regression for the shell-injection fix: probe() must pass the path as a literal
// argv entry, never through a shell. A filename containing shell metacharacters
// must NOT execute. Under the old execSync(`ffprobe ... "${path}"`) the embedded
// `touch` ran and created the marker; under execFileSync it cannot, regardless of
// whether ffprobe is installed (the injected command never reaches a shell).
test("probe does not execute shell metacharacters in a filename", () => {
const dir = mkdtempSync(join(tmpdir(), "probe-inject-"));
const marker = join(dir, "INJECTED");
// Slash-free basename (a real on-disk filename) that breaks out of the old
// double-quoted interpolation and would `touch INJECTED` in the cwd.
const evil = join(dir, `clip"; touch INJECTED; echo ".mp4`);
const prevCwd = process.cwd();
try {
writeFileSync(evil, "not real media");
process.chdir(dir); // so a leaked `touch INJECTED` would land next to `marker`
const meta = probe(evil);
assert.equal(existsSync(marker), false, "injected `touch` must not have run");
// Bogus/unreadable media still returns the null-shaped result, never throws.
assert.deepEqual(Object.keys(meta).sort(), ["codec", "duration", "height", "width"]);
} finally {
process.chdir(prevCwd);
rmSync(dir, { recursive: true, force: true });
}
});
import { sfxProvider } from "./sfx-provider.mjs";
import { imageProvider, iconProvider } from "./image-provider.mjs";
import { bgmProvider } from "./bgm-provider.mjs";
import { brandProvider } from "./brand-provider.mjs";
const STUB = {
async search() {
return null;
},
};
const registry = {
bgm: { ...bgmProvider, type: "bgm" },
sfx: { ...sfxProvider, type: "sfx" },
voice: { ...STUB, type: "voice" },
image: { ...imageProvider, type: "image" },
icon: { ...iconProvider, type: "icon" },
brand: { ...brandProvider, type: "brand" },
};
export function getProvider(type) {
const p = registry[type];
if (!p) throw new Error(`unknown media type: ${type}`);
return p;
}
export function listTypes() {
return Object.keys(registry);
}
import { heygenSearch } from "./heygen-search.mjs";
export const sfxProvider = {
async search(intent) {
const results = heygenSearch("audio sounds list", intent, {
type: "sound_effects",
minScore: 0.4,
});
if (!results) return null;
const best = results[0];
return {
url: best.audio_url,
source: "search",
// ext derived from audio_url by resolve.mjs — catalog SFX are .mp3 or .wav
metadata: {
description: best.description || best.name || intent,
duration: best.duration || null,
provider: "heygen.audio.sounds",
provenance: { track_id: best.id, score: best.score, query: intent },
},
};
},
};
#!/usr/bin/env node
import { existsSync } from "node:fs";
import { resolve, join, extname } from "node:path";
import { parseArgs } from "node:util";
import { appendRecord, findByPrompt, findByEntity, nextId, typeSubdir } from "./lib/manifest.mjs";
import { regenerateIndex } from "./lib/index-gen.mjs";
import { cacheGet, cacheGetByEntity, importFromCache } from "./lib/cache.mjs";
import { getProvider, listTypes } from "./lib/providers.mjs";
import { freezeUrl, freezeLocalFile } from "./lib/freeze.mjs";
import { findExistingAsset } from "./lib/adopt.mjs";
const { values: args } = parseArgs({
options: {
type: { type: "string", short: "t" },
intent: { type: "string", short: "i" },
entity: { type: "string", short: "e" },
project: { type: "string", short: "p", default: "." },
adopt: { type: "boolean", default: false },
json: { type: "boolean", default: false },
help: { type: "boolean", short: "h", default: false },
},
strict: true,
});
if (args.help) {
console.log(`media-use resolve — turn a media need into a frozen local file
Usage:
node resolve.mjs --type <type> --intent "<description>" [--project <dir>]
Types: ${listTypes().join(", ")}
Options:
--type, -t Media type (required)
--intent, -i What you need (required)
--entity, -e Entity name for cache matching (optional)
--project, -p Project directory (default: .)
--adopt Adopt all existing assets/ files into the manifest
--json Output JSON instead of one-line result
--help, -h Show this help`);
process.exit(0);
}
if (args.adopt) {
const { adoptExistingAssets } = await import("./lib/adopt.mjs");
const projectDir = resolve(args.project);
const adopted = adoptExistingAssets(projectDir);
if (args.json) {
console.log(JSON.stringify({ ok: true, adopted: adopted.length, assets: adopted }));
} else if (adopted.length === 0) {
console.log("no new assets to adopt (assets/ empty or already registered)");
} else {
console.log(`adopted ${adopted.length} asset${adopted.length === 1 ? "" : "s"} from assets/`);
for (const r of adopted) console.log(` ${r.id} → ${r.path} (${r.type})`);
}
process.exit(0);
}
if (!args.type || !args.intent) {
console.error("error: --type and --intent are required");
process.exit(2);
}
const projectDir = resolve(args.project);
const type = args.type;
const intent = args.intent;
const entity = args.entity || null;
async function run() {
// 1. project manifest — exact-prompt match
const projectHit = findByPrompt(projectDir, intent, type);
if (projectHit && existsSync(join(projectDir, projectHit.path))) {
return result(projectHit, "cached");
}
// 1b. entity match in project
if (entity) {
const entityHit = findByEntity(projectDir, entity);
if (entityHit && entityHit.type === type && existsSync(join(projectDir, entityHit.path))) {
return result(entityHit, "cached");
}
}
// 1c. scan existing assets/ directory for unregistered matches
const existingAsset = findExistingAsset(projectDir, intent, type);
if (existingAsset) {
const id = nextId(projectDir, type);
const record = {
id,
type: existingAsset.type,
path: existingAsset.relativePath,
source: "existing",
description: existingAsset.name.replace(/[-_]/g, " "),
provenance: { provider: "local", adopted: true, prompt: intent },
};
appendRecord(projectDir, record);
regenerateIndex(projectDir);
return result(record, "existing");
}
// 2. global cache — exact-prompt or entity match
const cacheHit = cacheGet(intent, type);
if (cacheHit) {
const id = nextId(projectDir, type);
const ext = extname(cacheHit.cached_path);
const localPath = `.media/${typeSubdir(type)}/${id}${ext}`;
const imported = importFromCache(cacheHit, projectDir, id, localPath);
if (imported) {
appendRecord(projectDir, imported);
regenerateIndex(projectDir);
return result(imported, "reused");
}
}
if (entity) {
const entityCacheHit = cacheGetByEntity(entity);
if (entityCacheHit && entityCacheHit.type === type) {
const id = nextId(projectDir, type);
const ext = extname(entityCacheHit.cached_path);
const localPath = `.media/${typeSubdir(type)}/${id}${ext}`;
const imported = importFromCache(entityCacheHit, projectDir, id, localPath);
if (imported) {
appendRecord(projectDir, imported);
regenerateIndex(projectDir);
return result(imported, "reused");
}
}
}
// 3. provider search
const provider = getProvider(type);
let searchResult = null;
try {
searchResult = await provider.search(intent, { entity, projectDir });
} catch {
// search failed, try generate
}
// 4. generate fallback
if (!searchResult && provider.generate) {
try {
searchResult = await provider.generate(intent, { entity, projectDir });
} catch {
// generate failed too
}
}
if (!searchResult) {
if (args.json) {
console.log(
JSON.stringify({ ok: false, error: `no provider could resolve ${type}: "${intent}"` }),
);
} else {
console.error(`error: no provider could resolve ${type}: "${intent}"`);
}
process.exit(1);
}
// 5. freeze + register
const id = nextId(projectDir, type);
const ext = searchResult.ext || extFromUrl(searchResult.url || "") || defaultExt(type);
const localPath = `.media/${typeSubdir(type)}/${id}${ext}`;
const fullPath = join(projectDir, localPath);
if (searchResult.localPath) {
freezeLocalFile(searchResult.localPath, fullPath);
} else if (searchResult.url) {
await freezeUrl(searchResult.url, fullPath);
} else {
console.error("error: provider returned no url or localPath");
process.exit(1);
}
const record = {
id,
type,
path: localPath,
source: searchResult.source || "search",
description: searchResult.metadata?.description || intent,
...(searchResult.metadata?.duration != null && { duration: searchResult.metadata.duration }),
...(searchResult.metadata?.width != null && { width: searchResult.metadata.width }),
...(searchResult.metadata?.height != null && { height: searchResult.metadata.height }),
...(searchResult.metadata?.transparent != null && {
transparent: searchResult.metadata.transparent,
}),
...(entity && { entity }),
provenance: {
provider: searchResult.metadata?.provider || "unknown",
prompt: intent,
...searchResult.metadata?.provenance,
},
};
appendRecord(projectDir, record);
regenerateIndex(projectDir);
return result(record, searchResult.source || "search");
}
function result(record, source) {
if (args.json) {
console.log(JSON.stringify({ ok: true, ...record, _source: source }));
} else {
const meta = formatMeta(record, source);
console.log(`resolved ${record.id} → ${record.path} (${meta})`);
}
}
function formatMeta(record, source) {
const parts = [record.type];
if (record.duration != null) parts.push(`${record.duration}s`);
if (record.width && record.height) parts.push(`${record.width}×${record.height}`);
if (record.transparent) parts.push("transparent");
if (source === "reused") parts.push("reused");
if (source === "generated") parts.push("generated");
return parts.join(", ");
}
function extFromUrl(url) {
try {
return extname(new URL(url).pathname) || null;
} catch {
return null;
}
}
const DEFAULT_EXT = {
bgm: ".wav",
sfx: ".mp3",
voice: ".wav",
image: ".jpg",
icon: ".svg",
brand: ".png",
};
function defaultExt(type) {
return DEFAULT_EXT[type] || ".bin";
}
run().catch((err) => {
if (args.json) {
console.log(JSON.stringify({ ok: false, error: err.message }));
} else {
console.error(`error: ${err.message}`);
}
process.exit(1);
});
import { strict as assert } from "node:assert";
import { mkdtempSync, rmSync, writeFileSync, readFileSync, mkdirSync, existsSync } from "node:fs";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { execSync } from "node:child_process";
import { appendRecord, readManifest } from "./lib/manifest.mjs";
import { regenerateIndex } from "./lib/index-gen.mjs";
import { getProvider } from "./lib/providers.mjs";
import { freezeLocalFile } from "./lib/freeze.mjs";
import { cachePut, cacheGet, importFromCache } from "./lib/cache.mjs";
const REPO_ROOT = join(import.meta.dirname, "..", "..", "..");
let tmp;
function setup() {
tmp = mkdtempSync(join(tmpdir(), "mu-resolve-test-"));
}
function cleanup() {
if (tmp) rmSync(tmp, { recursive: true, force: true });
}
function makeRecord(overrides = {}) {
return {
id: "bgm_001",
type: "bgm",
path: ".media/audio/bgm/bgm_001.wav",
source: "search",
description: "soft minimal ambient",
duration: 11,
provenance: { provider: "test", prompt: "test prompt" },
...overrides,
};
}
function resolveCmd(args) {
return `node skills/media-use/scripts/resolve.mjs ${args}`;
}
const tests = [];
function test(name, fn) {
tests.push({ name, fn });
}
// --- manifest cache hit ---
test("project manifest hit skips providers", () => {
setup();
const record = makeRecord({ provenance: { prompt: "cached query", provider: "test" } });
appendRecord(tmp, record);
const filePath = join(tmp, record.path);
mkdirSync(join(filePath, ".."), { recursive: true });
writeFileSync(filePath, "cached audio");
const out = execSync(resolveCmd(`--type bgm --intent "cached query" --project "${tmp}" --json`), {
cwd: REPO_ROOT,
encoding: "utf8",
});
const parsed = JSON.parse(out.trim());
assert.equal(parsed.ok, true);
assert.equal(parsed.id, "bgm_001");
assert.equal(parsed._source, "cached");
cleanup();
});
// --- global cache hit ---
test("global cache hit copies to project and registers", () => {
setup();
const sourceFile = join(tmp, "source.wav");
writeFileSync(sourceFile, "cached globally for resolve");
const record = makeRecord({ provenance: { prompt: "global resolve test" } });
cachePut(sourceFile, record);
const cached = cacheGet("global resolve test", "bgm");
assert.ok(cached);
const projectDir = mkdtempSync(join(tmpdir(), "mu-resolve-proj-"));
const imported = importFromCache(cached, projectDir, "bgm_001", ".media/audio/bgm/bgm_001.wav");
assert.ok(imported);
assert.ok(existsSync(join(projectDir, ".media/audio/bgm/bgm_001.wav")));
appendRecord(projectDir, imported);
regenerateIndex(projectDir);
const manifest = readManifest(projectDir);
assert.equal(manifest.length, 1);
assert.equal(manifest[0].provenance.imported_from, cached.sha);
rmSync(projectDir, { recursive: true, force: true });
cleanup();
});
// --- provider interface ---
test("getProvider returns provider with type", () => {
const p = getProvider("bgm");
assert.equal(p.type, "bgm");
assert.ok(typeof p.search === "function");
});
test("getProvider throws for unknown type", () => {
assert.throws(() => getProvider("unknown_type"), /unknown media type/);
});
// --- freeze ---
test("freezeLocalFile creates parent dirs and copies", () => {
setup();
const src = join(tmp, "src.bin");
writeFileSync(src, "freeze test data");
const dest = join(tmp, "deep/nested/dir/file.bin");
freezeLocalFile(src, dest);
assert.ok(existsSync(dest));
assert.equal(readFileSync(dest, "utf8"), "freeze test data");
cleanup();
});
// --- adopt existing assets ---
test("--adopt registers existing assets/ files", () => {
setup();
mkdirSync(join(tmp, "assets/bgm"), { recursive: true });
mkdirSync(join(tmp, "assets/icons"), { recursive: true });
writeFileSync(join(tmp, "assets/bgm/track.mp3"), "fake mp3");
writeFileSync(join(tmp, "assets/icons/logo.svg"), "fake svg");
const out = execSync(resolveCmd(`--adopt --project "${tmp}" --json`), {
cwd: REPO_ROOT,
encoding: "utf8",
});
const parsed = JSON.parse(out.trim());
assert.equal(parsed.ok, true);
assert.equal(parsed.adopted, 2);
assert.ok(parsed.assets.some((a) => a.path === "assets/bgm/track.mp3"));
assert.ok(parsed.assets.some((a) => a.path === "assets/icons/logo.svg"));
const manifest = readManifest(tmp);
assert.equal(manifest.length, 2);
cleanup();
});
test("--adopt skips already-registered assets", () => {
setup();
mkdirSync(join(tmp, "assets/bgm"), { recursive: true });
writeFileSync(join(tmp, "assets/bgm/track.mp3"), "fake mp3");
execSync(resolveCmd(`--adopt --project "${tmp}" --json`), { cwd: REPO_ROOT, encoding: "utf8" });
const out = execSync(resolveCmd(`--adopt --project "${tmp}" --json`), {
cwd: REPO_ROOT,
encoding: "utf8",
});
const parsed = JSON.parse(out.trim());
assert.equal(parsed.adopted, 0);
const manifest = readManifest(tmp);
assert.equal(manifest.length, 1);
cleanup();
});
test("resolve finds existing unregistered asset before hitting providers", () => {
setup();
mkdirSync(join(tmp, "assets/bgm"), { recursive: true });
writeFileSync(join(tmp, "assets/bgm/ambient-track.mp3"), "existing bgm");
const out = execSync(
resolveCmd(`--type bgm --intent "ambient track" --project "${tmp}" --json`),
{ cwd: REPO_ROOT, encoding: "utf8" },
);
const parsed = JSON.parse(out.trim());
assert.equal(parsed.ok, true);
assert.equal(parsed.path, "assets/bgm/ambient-track.mp3");
assert.equal(parsed._source, "existing");
cleanup();
});
// --- CLI interface ---
test("--help exits 0", () => {
const out = execSync(resolveCmd("--help"), { cwd: REPO_ROOT, encoding: "utf8" });
assert.ok(out.includes("media-use resolve"));
assert.ok(out.includes("--type"));
});
test("missing required args exits 2", () => {
try {
execSync(resolveCmd(""), { cwd: REPO_ROOT, encoding: "utf8", stdio: "pipe" });
assert.fail("should have exited");
} catch (err) {
assert.equal(err.status, 2);
}
});
test("--json returns error JSON on stub provider failure", () => {
setup();
try {
execSync(resolveCmd(`--type bgm --intent "stub fail" --project "${tmp}" --json`), {
cwd: REPO_ROOT,
encoding: "utf8",
stdio: "pipe",
});
assert.fail("should have exited");
} catch (err) {
const output = err.stdout || "";
const parsed = JSON.parse(output.trim());
assert.equal(parsed.ok, false);
assert.ok(parsed.error.includes("no provider"));
}
cleanup();
});
test("one-line output format matches contract", () => {
setup();
const record = makeRecord({ provenance: { prompt: "format test", provider: "test" } });
appendRecord(tmp, record);
const filePath = join(tmp, record.path);
mkdirSync(join(filePath, ".."), { recursive: true });
writeFileSync(filePath, "format check");
const out = execSync(resolveCmd(`--type bgm --intent "format test" --project "${tmp}"`), {
cwd: REPO_ROOT,
encoding: "utf8",
});
assert.match(out.trim(), /^resolved bgm_001 → .media\/audio\/bgm\/bgm_001\.wav \(bgm/);
cleanup();
});
// --- run ---
async function main() {
console.log("media-use · resolve engine tests\n");
let passed = 0;
let failed = 0;
for (const { name, fn } of tests) {
try {
await fn();
passed++;
console.log(` \x1b[32m✓\x1b[0m ${name}`);
} catch (err) {
failed++;
console.log(` \x1b[31m✗\x1b[0m ${name}`);
console.log(` ${err.message}`);
}
}
console.log(`\n${passed} passed, ${failed} failed`);
if (failed > 0) process.exit(1);
}
main();
Related skills
How it compares
Use media-use inside HyperFrames builds for catalog-to-disk resolution; use a general stock-media API integration when not composing with HyperFrames.
FAQ
What media types does media-use resolve?
media-use resolves four types: background music (BGM), sound effects (SFX), images, and icons. Each type uses the same resolve verb with a --type flag and natural-language intent string.
How does media-use keep agent context small?
media-use runs resolve.mjs to search the HeyGen catalog and freeze matches locally. The agent receives one file path per resolve call while catalog search results and intermediates stay on disk.
Is Media Use safe to install?
skills.sh reports 1 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.