
Qiaomu Mtv Creator
- 9 installs
- 7 repo stars
- Updated June 1, 2026
- joeseesun/qiaomu-mtv-creator
Generate music video subtitles and lyric synchronization from audio files for MTV-style video creation.
About
Creates time-synced lyric files (LRC/SRT) from audio for music video production. Integrates with Suno music generation for automated video asset creation.
- Generates LRC/SRT subtitle files with precise time synchronization
- Integrates with Suno-generated audio for end-to-end video production
Qiaomu Mtv Creator by the numbers
- 9 all-time installs (skills.sh)
- Ranked #1,065 of 1,335 Generative Media skills by installs in the Skillselion catalog
- Data as of Aug 2, 2026 (Skillselion catalog sync)
npx skills add https://github.com/joeseesun/qiaomu-mtv-creator --skill qiaomu-mtv-creatorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 9 |
|---|---|
| repo stars | ★ 7 |
| Last updated | June 1, 2026 |
| Repository | joeseesun/qiaomu-mtv-creator ↗ |
What it does
Generate music video subtitles and lyric synchronization from audio files for MTV-style video creation.
Files
Qiaomu MTV Creator - AI 音乐视频制作
将 Suno 生成的歌曲自动转换为两类 MV:稳定快速的传统配图字幕版,以及更像音乐视频的 HyperFrames/GSAP 动态歌词版。
V2.4 升级要点
- MTV 审美规则沉淀:新增“画面、字幕、转场、字体、验收”的设计原则。未来不要把 Hyper MV 做成固定底部字幕的幻灯片;每首歌都要有和歌词意象一致的 motion grammar。
- 字幕系统升级:默认 Hyper 模板要使用多安全区 anchor、轻微倾斜、词级 stagger、回声/描边/玻璃质感等可控变化。字幕可以不规则,但必须服务歌曲气质;避免逐帧抖动、过度缩放、粗糙旋转和一直居中。
- 转场系统升级:图片之间不只做淡入淡出。默认组合柔和溶解、玻璃擦拭、光扫、纹理遮罩、轻微场景分层,让镜头推进保留,同时让 scene change 有记忆点。
- 字体本地化:Hyper 模板默认打包
Anton+Space Grotesk本地 woff2,避免渲染时回退到系统字体导致质感变差或不同机器表现不一致。 - 去模板痕迹:片头必须使用真实歌曲标题,优先从 LRC
[ti:]、项目元数据或字幕文件名读取,并清理 clip/hash 后缀;motion profile、debug label、URL slug、占位文字不能出现在成片里。 - 固定作者署名:所有歌曲作者/artist/author 默认写
向阳乔木。不要从 Suno、文件名、URL、LRC[by:]或模型输出里猜作者;片头需要署名时也使用向阳乔木。 - 音频 UI 默认关闭:波形、频谱、进度条、节拍脉冲不再作为默认装饰。只有当歌曲概念明确需要“播放器/电台/信号/电子可视化”时才打开;冷感、叙事、梦幻歌曲默认不出现这些界面化元素。
- 渲染脚本稳定化:
render_hyper_mtv.py默认使用本机 Puppeteer headless shell、streaming encode、长超时、缓存限制和 SDR 输出,并按可回收内存选择 worker,减少手动拼环境变量和系统 Chrome 崩溃概率。
V2.3 升级要点
- 新名字:主 skill 名为
qiaomu-mtv-creator;旧名mtv-creator不再保留,避免别名误触发。 - 单源目录:skill 实体只放在
/Users/joe/.agents/skills/qiaomu-mtv-creator;不要再创建.claude兼容软链或mtv-creator旧名入口。 - 双模式:
classic保留传统 FFmpeg 配图字幕工作流;hyper用 HyperFrames + GSAP 制作动态歌词、场景内排版、音频响应和更丰富转场。 - GSAP 官方技能吸收:已吸收
greensock/gsap-skills的 core/timeline/plugins/utils/performance 规则,落到references/gsap-mtv-motion.md,并由scripts/create_hyper_mtv.py自动生成更丰富的 HyperFrames/GSAP MTV 工程。 - 动效谱系:Hyper 模式新增
cinematic、poster、glitch、dream、minimal五个 motion profile,用 GSAP timeline labels、position parameter、stagger、transform aliases、autoAlpha、gsap.utils.wrap()等模式组织场景、歌词、粒子、光扫和节拍脉冲。 - 渲染加速策略:新增
scripts/render_hyper_mtv.py,根据可用内存选择 HyperFramesquality/fps/workers;低内存时先提示清理,不再默认硬开 auto workers。 - SRT/LRC 统一:自动识别 Suno 同版本
.srt/.lrc,LRC 会转换为项目内可烧录 SRT。 - 项目化输出:每次生成
project.json、timeline.json、preflight_report.json、storyboard.html。 - 可恢复阶段:通过
--stage preflight|prepare|storyboard|images|compose|hyper|all分步运行。 - Codex 生图优先:
--image-provider auto在 Codex 环境中解析为codex-plan,优先让 Codex 内置生图接管关键帧;只有 Codex 生图不可用或非 Codex 运行时才回退即梦。 - 字幕 fallback:如果本机 FFmpeg 没有
subtitles/drawtextfilter,会自动用 Pillow 渲染透明字幕层再 overlay。
核心流程
Suno 歌曲(含 .srt / .lrc 精准歌词)
↓
解析为统一 timeline.json → preflight 检查 → storyboard.html
↓
Claude/Codex 读歌词 → 分析歌曲结构 → 生成 visual_config.json
↓
生图:优先 Codex 内置生图计划;失败或不可用时才用即梦批量落盘 fallback
↓
合成 MV:classic 用 FFmpeg;hyper 用 HyperFrames + GSAP kinetic timeline
↓
输出: MTV 视频 (MP4)⚠️ 重要原则
1. 用 Suno SRT,不用 Whisper:Suno 生成歌曲时已自动生成精准 SRT,路径与 mp3 同目录,优先使用 2. Claude 直接生成描述:Claude Code 本身就是大模型,读 SRT 后直接生成视觉描述,不调第三方 API 3. 音频选版本:Suno 会生成两个版本(.mp3 和 -1.mp3),选时长更长、覆盖所有 SRT 时间戳的版本 4. skill 单源在 .agents:所有新增和修改都落在 /Users/joe/.agents/skills/qiaomu-mtv-creator;不要恢复 .claude 兼容软链或旧 mtv-creator 别名 5. Codex 生图优先,不先跑即梦:Codex 内置生图质量通常更适合关键帧;脚本生成 codex_image_requests.md 供 Codex 接管。只有 Codex 生图不可用、用户明确要求 fallback、或非 Codex 自动运行时,才使用即梦。 6. Suno 资料先走 qiaomu-suno-master:遇到 Suno URL、clip ID、下载音频、导出 LRC/SRT,先使用 /Users/joe/.agents/skills/qiaomu-suno-master,不要先手写抓页面。 7. Hyper 渲染先看内存:hyperframes render --workers auto 在可用内存很低时会同时启动多个浏览器并失败;最终渲染前先跑 render_hyper_mtv.py --dry-run,低于阈值时请用户清理内存,除非用户接受慢速 --allow-low-memory。 8. GSAP 只用可渲染时间线:HyperFrames 里 GSAP 必须同步创建 paused timeline,注册到 window.__timelines.main;不要 tl.play()、不要 Date.now() / Math.random() / 网络 fetch / 无限 repeat。 9. MV 不是歌词 PPT:默认避免“整首歌固定底部居中字幕 + 每张图同一种淡入淡出”。字幕、转场、镜头和氛围层都要有歌曲自己的节奏,但不能抢歌词。 10. 字幕稳定优先:字幕动效可以丰富,但不能抖。优先使用 autoAlpha、y、低幅度 rotation、词级 stagger 和长一点的 sine/power ease;少用大幅 scale、rotationX、短促抖动和频繁 filter 切换。 11. 从歌词意象生成 motion grammar:玻璃、雨、盐、霓虹、纸张、广播、海、风等意象要变成对应的视觉动效语言,例如 glass wipe、rain streak、salt grain、signal scan、paper slip,而不是套同一套模板。 12. 不要让模板身份露出:片头可以有设计标题,但不能出现 Poster Motion、cinematic motion、placeholder、URL slug、clip id/hash、脚本调试名。真实歌名优先从 LRC [ti:] 获取,例如 Salt On Glass,不要显示 salt-on-glass-ba258447。 13. 不要把 MV 做成播放器界面:波形、频谱、进度条、均衡器、底部节拍线默认关闭。除非歌曲主题是广播、电台、信号、俱乐部、电子设备或用户明确要求,否则这些元素会破坏沉浸感。 14. 作者固定为向阳乔木:项目元数据、HyperFrames mtv-data.json、片头署名、发布说明中的歌曲作者统一写 向阳乔木,不要使用 Suno AI、下载账号、文件名、URL slug 或 LRC [by:] 作为作者。
MTV 设计原则与风格原则
做 Hyper MTV 时先定一个 4 层视觉系统,而不是直接套模板:
1. 画面层:关键帧要有统一摄影/插画风格、稳定色彩、重复母题。镜头推进可以慢,但每个段落的推进方向、焦点和速度要略有差别。 2. 字幕层:字幕是 MV 的表演者,不是外挂字幕。每行歌词可以在 4-7 个安全 anchor 中切换:左下、右上、右中、左墙、低位桌面、居中宽屏等。位置变化要和画面负空间匹配,不能遮住主要主体。 3. 字体层:每支 MV 至少有明确字体气质。优先选择凝练、有音乐感的 display sans / condensed sans;不要默认系统粗体一直居中。温柔歌用窄体、细腻阴影、低对比回声;强烈歌再用红色描边、glitch、切片。 4. 转场层:每个 scene change 至少叠加两种转场语言,例如 cross-dissolve + glass sweep、soft bloom + texture wipe、rain scan + parallax drift。避免所有图片之间同一种 xfade。 5. 氛围层:粒子、波形、光扫、噪声、进度条要低调服务节奏。不要为了“动”而动;能表达歌词母题的动效优先。 6. 片头层:片头是作品的一部分,不是模板封面。只显示真实歌名或经过设计的短标题;作者署名统一写 向阳乔木。不要显示 motion profile、工程名、哈希后缀、下载文件名、调试标签。
字幕风格要按歌曲气质调节:
- 梦幻/怀旧/冷感:位置有轻微漂移但不跳;文字可偏左/偏右,带玻璃回声、柔和描边、低透明度重复影;转场用长溶解、光扫、雾化遮罩。
- 摇滚/脏感/强节拍:可使用更大字号、upper-case、红色/暖色强调词、切片、扫描线和短促冲击,但要控制在副歌或强拍。
- 民谣/叙事/安静:减少字词拆分,使用低位、侧边或画面负空间的排版;用纸张、影子、窗光、慢变焦表达段落。
- 电子/赛博/不稳定:允许 glitch、scanline、短促位移和硬切,但要避免全程抖动造成阅读疲劳。
质量门槛:
- 抽帧时不能整首歌都像同一个字幕位置的重复截图。
- 相邻两三帧字幕边缘不应出现肉眼可见抖动。
- 转场不能黑屏,不能只有硬淡出;至少几处 scene change 要能看出歌曲母题。
- 标题/水印式模板文字默认只用于片头,不能整首歌常驻左上角干扰歌词;片头也不能出现 profile label、slug 或 hash。
- 波形、频谱、进度条、节拍线这类播放器 UI 默认不出现;抽帧看到它们时,需要能说清楚它们为什么属于这首歌。
标准工作流(Claude Code 操作)
Step 1:准备项目与 Storyboard
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio "/path/to/song-1.mp3" \
--style ink-painting \
--ratio 16:9 \
--stage prepare检查输出:
project.json
timeline.json
preflight_report.json
storyboard.html
visual_config.json
codex_image_requests.mdStep 2A:Codex 内置生图接管关键帧(默认优先)
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio "/path/to/song-1.mp3" \
--style ink-painting \
--ratio 16:9 \
--stage prepare然后由 Codex 读取 codex_image_requests.md 逐张生成 images/scene_XX.png。完成后:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio "/path/to/song-1.mp3" \
--visual-config "/path/to/project/visual_config.json" \
--stage composeStep 2B:即梦 fallback(仅 Codex 生图不可用时)
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio "/path/to/song-1.mp3" \
--style ink-painting \
--ratio 16:9 \
--output ~/Videos/MTV/项目名/ \
--skip-transcribe \
--subtitle "/path/to/song.srt" \
--visual-config "/path/to/project/visual_config.json" \
--stage images \
--image-provider jimeng \
--workers 5Step 2C:HyperFrames / GSAP 动态歌词 MV(推荐模式)
适合用户明确要求“不像幻灯片”“歌词更酷”“画面随音乐动”“参考 HyperFrames”的任务。传统 classic 模式仍然保留;hyper 是更高表现力、更慢但更像 MV 的模式。
工作流:
1. 先用本 skill 的 prepare 生成 timeline.json、visual_config.json、关键帧生图计划。 2. 用 Codex 内置生图生成 images/scene_XX.png。 3. 运行 --stage hyper 自动生成 hyper-mv/:复制音频/关键帧,写入 assets/mtv-data.js,创建本地 assets/gsap.min.js 和 GSAP-rich index.html。 4. 先运行 npm run check 和 npx hyperframes inspect --samples 24,确认没有 console error、对比度问题、歌词越界。 5. 快速预览用 draft profile,最终版用 final profile。
生成 HyperFrames 工程:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio "/path/to/song-1.mp3" \
--subtitle "/path/to/song.srt" \
--visual-config "/path/to/project/visual_config.json" \
--stage hyper \
--motion-profile cinematic动效谱系:
| Profile | 适用 | 特点 |
|---|---|---|
cinematic | 流行、摇滚、叙事歌 | 慢推拉、稳重歌词入场、克制节拍脉冲 |
poster | 复古、民谣、插画风 | 海报纸感、轻微错位、块面化歌词 |
glitch | 电子、愤怒、赛博 | 扫描线、短促抖动、锐利切换 |
dream | 氛围、怀旧、柔和歌曲 | 长溶解、漂浮粒子、柔和 blur-in |
minimal | 细腻歌词、spoken 段落 | 少动效、安静淡入淡出 |
快速预览:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/render_hyper_mtv.py \
--project "/path/to/project/hyper-mv" \
--profile review最终渲染:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/render_hyper_mtv.py \
--project "/path/to/project/hyper-mv" \
--profile final \
--check优先使用 render_hyper_mtv.py 而不是手写 npx hyperframes render。脚本会自动:
- 选择本机 Puppeteer
chrome-headless-shell,避免系统 Chrome 版本/权限导致渲染不稳。 - 开启 streaming encode,长视频不先堆满帧缓存。
- 设置 Puppeteer 启动/协议超时、帧缓存上限和
--sdr。 - 按内存自动选择 worker;需要更快时先看
--dry-run输出,再决定是否显式--workers 2/3。
如果脚本提示 LOW MEMORY,先请用户清理内存或关闭重型应用,再重新渲染。只有用户明确接受“慢一点也行”时,才使用:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/render_hyper_mtv.py \
--project "/path/to/project/hyper-mv" \
--profile final \
--allow-low-memory刚才实测教训:hyperframes render --workers auto 在 0.3GB immediate free memory 时会校准后尝试多 worker,随后多个浏览器进程同时启动失败;--workers 1 可以完成,但 220 秒 1080p 视频耗时约 7 分 39 秒。因此默认策略是:内存足够时多 worker 换速度,内存低时先问用户清理,而不是静默退到慢速。
参数说明
| 参数 | 短参数 | 默认值 | 说明 |
|---|---|---|---|
--audio | -a | (必填) | 音频文件路径 |
--lyrics | -l | 自动查找 | 歌词文件路径(.lyrics.json) |
--style | -s | newyorker | 配图风格(见风格列表) |
--ratio | -r | 9:16 | 视频比例(9:16/16:9/1:1) |
--output | -o | ~/Videos/MTV/ | 输出目录 |
--workers | -w | 3 | 并发生成配图数(推荐 5) |
--stage | all | 分阶段运行:preflight/prepare/storyboard/images/compose/hyper/all | |
--image-provider | auto | 生图后端:auto/codex-plan/jimeng/none;auto 在 Codex 中优先 codex-plan,非 Codex 才回退 jimeng | |
--motion-profile | cinematic | HyperFrames/GSAP 动效谱系:cinematic/poster/glitch/dream/minimal | |
--hyper-dir | 项目目录/hyper-mv | HyperFrames 工程输出目录 | |
--preflight-only | false | 只检查输入和时间轴 | |
--storyboard-only | false | 只生成 storyboard 预览 | |
--no-subtitle | false | 不烧录字幕 | |
--skip-transcribe | false | 跳过转录(现在会自动优先使用 Suno SRT/LRC,通常不必手动指定) | |
--skip-images | false | 跳过生图(已有图片时使用) | |
--subtitle | 自动查找 | 指定 SRT/LRC 文件路径 | |
--images-dir | 自动 | 指定已有图片目录 | |
--visual-config | - | 指定配图配置文件 |
风格推荐
| 歌曲类型 | 推荐风格 |
|---|---|
| 摇滚/硬核 | newyorker, pen-sketch, woodcut |
| 民谣/抒情 | watercolor, ink-painting, morandi |
| 电子/流行 | flat-illustration, isometric, low-poly |
| 古风/中国风 | ink-painting, woodcut, retro-poster |
| 儿歌/轻松 | children-book, cartoon, paper-cut |
完整风格列表见 qiaomu-image-generator skill。
工作流详解
Step 1: 字幕与时间轴
优先使用 Suno 同目录字幕:
- 精确匹配
歌名.mp3→歌名.srt/歌名.lrc - 精确匹配
歌名-1.mp3→歌名-1.srt/歌名-1.lrc - LRC 自动转换为项目内 SRT
- 只有找不到 Suno 字幕时才调用
whisper-transcribe - 输出
timeline.json和preflight_report.json
Step 2: 分析歌词生成配图描述
Claude 分析歌词,为每个段落生成视觉描述:
- 提取核心意象和情感
- 转换为纯视觉语言(无文字)
- 生成
visual_config.json
示例:
歌词: "午夜的终端闪烁着光,键盘就是我的战场"
描述: "深夜办公室,一个人影坐在发光的屏幕前,手指悬停在键盘上,周围是代码的光影"Step 3: 生成配图
默认优先 Codex 内置生图:
--image-provider auto在 Codex 环境中生成codex_image_requests.md- 审阅
codex_image_requests.md - 用 Codex 内置生图生成
images/scene_XX.png
即梦只作为 fallback:
- Codex 生图不可用、用户明确接受 fallback,或非 Codex 环境中
auto运行时,才调用qiaomu-image-generator - fallback 时显式使用
--image-provider jimeng
Step 4: 合成视频
classic 模式调用内置 FFmpeg 合成:
- 音频 + 图片 + 字幕 → MP4
- 平滑转场效果
- 专业字幕烧录
- 自动修正
xfade重叠造成的时长缩短
hyper 模式调用 scripts/create_hyper_mtv.py:
- 读取
project.json、timeline.json、visual_config.json - 复制音频和
images/scene_XX.*到hyper-mv/assets/ - 安装并复制本地
gsap.min.js,避免 CDN 在渲染时失败 - 生成一个 paused GSAP master timeline:scene layers、kinetic lyrics、beat pulse、wave bars、particles、progress
- 后续用
scripts/render_hyper_mtv.py做 review/final 渲染
输出结构
~/Videos/MTV/代码丛林_20260125/
├── 代码丛林.mp4 # 最终 MTV
├── project.json # MTV 项目元数据
├── timeline.json # 统一字幕/场景时间轴
├── preflight_report.json # 输入、依赖、字幕覆盖率检查
├── storyboard.html # 可审阅分镜预览
├── codex_image_requests.md # Codex 内置生图 prompt 清单
├── audio/
│ └── 代码丛林.mp3 # 原始音频
├── subtitles/
│ ├── 代码丛林.srt # 字幕文件
│ └── 代码丛林.json # 时间轴 JSON
├── images/
│ ├── scene_01.png # 配图 1
│ ├── scene_02.png # 配图 2
│ └── ...
├── visual_config.json # 配图配置
├── metadata.json # 项目元数据
└── hyper-mv/ # HyperFrames/GSAP 动态歌词工程(--stage hyper)
├── index.html
├── package.json
└── assets/
├── mtv-data.js
├── gsap.min.js
├── song.mp3
└── images/与其他 Skills 的关系
| Skill | 角色 |
|---|---|
suno-music-creator | 上游:生成歌曲和歌词 |
whisper-transcribe | 依赖:转录时间轴 |
qiaomu-image-generator | 依赖:生成配图(即梦 API) |
ffmpeg | 依赖:合成视频、烧录字幕、社交平台兼容编码 |
podcast-to-video | 参考:字幕和合成风格最佳实践 |
gsap / greensock/gsap-skills | Hyper 模式动效规则来源:timeline、stagger、plugins、utils、performance |
GSAP / HyperFrames 动效规则
详细规则见 references/gsap-mtv-motion.md。做 Hyper MTV 时至少遵守:
- 用
gsap.timeline({ paused: true }),注册window.__timelines.main。 - 用 labels 和 position parameter 编排场景、歌词、节拍,不用散乱
delay。 - 优先 transform aliases 和
autoAlpha;少动画 layout 属性。 gsap.utils.wrap()/clamp()/toArray()用于确定性映射,不用Math.random()。- 默认核心 GSAP 足够;SplitText、ScrambleText、MotionPath、DrawSVG、MorphSVG 等插件只在本地 asset 可用且显式注册时使用。
推荐工作流(Claude 生成描述)
最佳实践:让 Claude 分析歌词并生成视觉描述,而不是使用简单的关键词匹配。
Step 1: 准备阶段
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio ~/乔木新知识库/04.素材/AI音乐/代码丛林.mp3 \
--prepare-only \
--scenes 8这会输出每个场景的歌词,等待 Claude 生成视觉描述。
Step 2: Claude 生成描述
在 Claude Code 对话中:
用户:为以下歌词场景生成视觉描述(纯视觉语言,不要文字):
场景 1: "午夜的终端闪烁着光,键盘就是我的战场"
场景 2: "代码在眼前不停旋转,这个 Bug 藏得太深太远"
...
Claude:
场景 1: 深夜办公室,一个人影坐在发光的屏幕前,手指悬停在键盘上,周围是代码的光影
场景 2: 迷宫般的电路板,一个人在其中寻找出路,远处有微弱的光点
...Step 3: 更新配置并生成
将 Claude 生成的描述更新到 visual_config.json,然后:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio ~/乔木新知识库/04.素材/AI音乐/代码丛林.mp3 \
--visual-config ~/Videos/MTV/代码丛林_20260125/visual_config.json \
--skip-transcribe完整示例
从 Suno 歌曲到 MTV
# 1. 生成歌曲(suno-music-creator)
python /Users/joe/.agents/skills/suno-music-creator/scripts/generate_music.py \
'{"title":"代码丛林","prompt":"[Verse 1]...","tags":"rock"}' \
--download
# 2. 一键生成 MTV(本 skill)
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio ~/乔木新知识��/04.素材/AI音乐/代码丛林.mp3 \
--style newyorker \
--ratio 9:16自定义配图数量
# 指定生成 6 张配图
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio song.mp3 \
--scenes 6 \
--style watercolor使用已有素材
# 跳过转录,使用已有 SRT
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio song.mp3 \
--skip-transcribe \
--subtitle existing.srt
# 跳过生图,使用已有图片
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio song.mp3 \
--skip-images \
--images-dir ./my_images/常见问题
| 问题 | 原因 | 解决方案 |
|---|---|---|
| 转录效果差 | 音乐人声混合 | 确保使用 --no-vad |
| 配图风格不对 | 描述包含风格词 | 检查 visual_config.json |
| 视频比例错误 | 图片比例不匹配 | 确保 --ratio 与图片一致 |
| 字幕不同步 | 时间轴不准 | 使用 --auto-correct |
| Twitter/X 上传失败 "宽高比太小" | 视频格式不兼容 | 已自动修复(yuv420p + High profile) |
字幕位置优化
根据各平台 UI 安全区域研究:
9:16 竖屏(TikTok/抖音)
- 底部安全区域:250px(被 UI 按钮覆盖)
- 字幕位置:距离底部 280px
- 字体大小:32px
- 右侧边距:120px(避开点赞/评论按钮)
16:9 横屏(YouTube/B站)
- 底部安全区域:60px
- 字幕位置:距离底部 60px
- 字体大小:28px
视频格式兼容性
为确保社交媒体平台兼容,视频自动使用:
- 编码:H.264 High Profile Level 4.0
- 像素格式:yuv420p(不是 yuv444p)
- 宽高比:SAR 1:1 + 正确的 DAR
依��安装
# Whisper
pip install faster-whisper
# FFmpeg (macOS)
brew install ffmpeg
# 图片处理
pip install Pillow注意事项
1. 音乐转录:必须禁用 VAD(--no-vad),否则唱歌会被过滤 2. 配图描述:使用纯视觉语言,不要包含文字 3. 视频比例:抖音/TikTok 用 9:16,B站/YouTube 用 16:9 4. 生成时间:完整流程约 5-10 分钟(取决于配图数量) 5. SRT 版本必须匹配:Suno 为每个版本生成独立 SRT(歌名.srt 对应 歌名.mp3,歌名-1.srt 对应 歌名-1.mp3),两者时间轴不同,混用会导致画面和歌词错位约10秒。使用 将进酒-1.mp3 时必须用 将进酒-1.srt。 6. Suno 可能生成重叠时间戳:部分条目 start 时间相同(如同时演唱),脚本已自动去重,不影响效果。
.DS_Store
.env
.env.*
node_modules/
__pycache__/
*.pyc
*.pyo
*.mp3
*.mp4
*.mov
*.wav
*.aac
work-*/
*.tmp
MIT License
Copyright (c) 2026 joeseesun
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
qiaomu-mtv-creator
把 Suno 歌曲、LRC/SRT 歌词和分镜图,变成更像真正音乐视频的 MTV。
Turn Suno songs, timed lyrics, and generated keyframes into polished music videos.
<p align="center"> <a href="https://github.com/joeseesun/qiaomu-mtv-creator/stargazers"><img alt="Stars" src="https://img.shields.io/github/stars/joeseesun/qiaomu-mtv-creator?style=for-the-badge&logo=github" /></a> <a href="https://github.com/joeseesun/qiaomu-mtv-creator/network/members"><img alt="Forks" src="https://img.shields.io/github/forks/joeseesun/qiaomu-mtv-creator?style=for-the-badge&logo=github" /></a> <a href="https://github.com/joeseesun/qiaomu-mtv-creator/issues"><img alt="Issues" src="https://img.shields.io/github/issues/joeseesun/qiaomu-mtv-creator?style=for-the-badge&logo=github" /></a> <a href="https://github.com/joeseesun/qiaomu-mtv-creator/commits/main"><img alt="Last commit" src="https://img.shields.io/github/last-commit/joeseesun/qiaomu-mtv-creator?style=for-the-badge&logo=git" /></a> <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-blue.svg?style=for-the-badge" /></a> </p>
npx skills add joeseesun/qiaomu-mtv-creator中文 | English
---
中文
很多 AI 歌曲不是缺画面,而是缺一套稳定的音乐视频工作流。
qiaomu-mtv-creator 把下载好的 Suno 音频、LRC/SRT 歌词、分镜和关键帧组织成一个可恢复的 MTV 项目。它既能走传统 FFmpeg 配图字幕版,也能生成 HyperFrames + GSAP 的动态歌词 MV:多锚点字幕、玻璃/雨/信号等歌曲母题转场、本地字体、稳定渲染脚本,以及默认不会露出模板占位文字的片头。
样例输出
一次 Hyper MTV 生成会产出类似下面的结构:
project.json
timeline.json
visual_config.json
storyboard.html
codex_image_requests.md
images/scene_01.png ...
hyper-mv/index.html
hyper-mv/assets/mtv-data.json
Salt-On-Glass-poster.mp4生成的 mtv-data.json 会保留真实歌名和固定作者署名:
{
"title": "Salt On Glass",
"author": "向阳乔木",
"artist": "向阳乔木",
"titleKicker": "向阳乔木",
"motionProfile": "poster"
}核心能力
- Suno 歌曲项目化:优先使用 Suno 自带 SRT/LRC,不用 Whisper 重新猜歌词时间轴。
- Codex 生图优先:默认生成
codex_image_requests.md,由 Codex 内置生图接管关键帧;不可用时再走即梦 fallback。 - 两种合成模式:
classic用 FFmpeg 快速生成配图字幕版;hyper用 HyperFrames + GSAP 制作动态歌词 MV。 - 更像 MV 的动效规则:字幕不固定死在底部,转场不只淡入淡出;从歌词意象生成 motion grammar。
- 去模板痕迹:真实歌名来自 LRC
[ti:]或项目元数据,自动清理 URL slug / clip hash;音频波形、进度条、节拍线默认关闭。 - 稳定渲染:
render_hyper_mtv.py默认使用 Puppeteer headless shell、streaming encode、长超时、SDR 输出和内存感知 worker。
安装
npx skills add joeseesun/qiaomu-mtv-creator前置条件
- [ ] 已安装支持 Agent Skills 的 Codex / Claude Code / OpenCode 等工具。
- [ ] 本机有 Python 3。
- [ ] 本机可用 FFmpeg。
- [ ] Hyper MV 需要 Node.js / npm,用于运行 HyperFrames、GSAP 和本地字体依赖。
- [ ] Suno 歌曲下载建议配合
qiaomu-suno-masterskill 使用。 - [ ] Codex 生图不可用时,才需要配置即梦 fallback。
自然语言用法
用 qiaomu-mtv-creator 为这首 Suno 歌做 MTV:https://suno.com/s/...Salt On Glass 用 HyperFrames 再做一个冷感海报风 MTV。把这首歌生成 MV,字幕不要固定底部,转场多一点玻璃和雨的感觉。典型工作流
准备项目:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio "/path/to/song.mp3" \
--subtitle "/path/to/song.lrc" \
--style "coldwave cinematic stills" \
--ratio 16:9 \
--stage prepare生成 HyperFrames 工程:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/create_mtv.py \
--audio "/path/to/song.mp3" \
--subtitle "/path/to/song.lrc" \
--visual-config "/path/to/project/visual_config.json" \
--stage hyper \
--motion-profile poster渲染预览与终版:
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/render_hyper_mtv.py \
--project "/path/to/project/hyper-mv" \
--profile review \
--check
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/render_hyper_mtv.py \
--project "/path/to/project/hyper-mv" \
--profile finalMotion Profiles
| Profile | 适合 | 风格 |
|---|---|---|
cinematic | 流行、摇滚、叙事歌 | 慢推拉、稳重歌词入场、克制节拍脉冲 |
poster | 复古、民谣、插画风 | 海报纸感、轻微错位、块面化歌词 |
glitch | 电子、愤怒、赛博 | 扫描线、短促抖动、锐利切换 |
dream | 氛围、怀旧、柔和歌曲 | 长溶解、漂浮粒子、柔和 blur-in |
minimal | 细腻歌词、spoken 段落 | 更少动效、安静淡入淡出 |
设计原则
- MTV 不是歌词 PPT:字幕、转场、镜头和氛围层要服务歌曲气质。
- 字幕稳定优先:避免抖动、大幅缩放、粗糙旋转和频繁 filter 切换。
- 标题只显示真实歌名;作者统一为
向阳乔木。 - 不默认显示音频波形、进度条、均衡器和底部节拍线。
- 最终输出前必须跑
npm run check、抽帧和ffmpeg -v error -i final.mp4 -f null -。
Troubleshooting
| 问题 | 处理 |
|---|---|
No valid skills found | 检查 SKILL.md YAML frontmatter,description 建议使用块标量。 |
| HyperFrames 渲染时 Chrome 崩溃 | 使用 render_hyper_mtv.py,它会优先选择 Puppeteer headless shell 并开启 streaming encode。 |
| 终版渲染太慢 | 先跑 render_hyper_mtv.py --dry-run 看内存判断;内存足够时再显式 --workers 2 或 --workers 3。 |
| 字幕越界或遮主体 | 重新运行 npm run check / hyperframes inspect,调整 anchor 或降低字号。 |
| 片头出现 slug/hash | 检查 LRC 是否有 [ti:真实歌名],脚本也会从文件名中清理常见 hash 后缀。 |
隐私与限制
- 音频、歌词、分镜和渲染默认在本机项目目录内处理。
- Codex 生图、即梦 fallback、Suno 下载和 GitHub 发布会涉及对应外部服务。
- 本 skill 带有乔木本地工作流约定,例如作者署名固定为
向阳乔木。
---
<a name="english"></a>
English
qiaomu-mtv-creator turns a Suno song plus timed lyrics into a recoverable music-video project. It can produce a fast FFmpeg image-subtitle video or a richer HyperFrames + GSAP kinetic lyric MV with local fonts, scene transitions, and deterministic render timelines.
Install
npx skills add joeseesun/qiaomu-mtv-creatorWhat It Does
- Uses Suno SRT/LRC timestamps instead of re-transcribing with Whisper.
- Generates a storyboard, timeline, visual config, and Codex image request plan.
- Prioritizes Codex image generation; falls back to Jimeng only when needed.
- Builds HyperFrames projects with GSAP timelines and local assets.
- Avoids template leakage: no profile labels, URL slugs, clip hashes, or player-style waveform UI by default.
- Writes author/artist metadata as
向阳乔木.
Example Prompts
Use qiaomu-mtv-creator to make an MTV for this Suno song: https://suno.com/s/...Make a cold poster-style HyperFrames MV for Salt On Glass.Requirements
- [ ] An Agent Skills compatible coding agent.
- [ ] Python 3.
- [ ] FFmpeg.
- [ ] Node.js / npm for HyperFrames and GSAP renders.
- [ ] Optional: Suno download workflow via
qiaomu-suno-master.
Limitations
This is a practical production workflow rather than a generic hosted service. Some paths and defaults are optimized for the Qiaomu local music workflow, especially the fixed author credit 向阳乔木.
License
MIT
GSAP Motion Grammar for Qiaomu MTV
This reference adapts the official greensock/gsap-skills repository for HyperFrames MTV work. It is intentionally narrower than the upstream skills: use it when building timed music videos, kinetic lyrics, scene transitions, and beat-reactive overlays.
Source absorbed: https://github.com/greensock/gsap-skills Checked on: 2026-05-30 GSAP npm version verified: 3.15.0
HyperFrames Contract
HyperFrames must own playback. Build the GSAP timeline synchronously, keep it paused, and register it under the composition id.
<script src="assets/gsap.min.js"></script>
<script>
window.__timelines = window.__timelines || {};
const tl = gsap.timeline({ paused: true });
window.__timelines.main = tl;
</script>Rules:
- Do not call
tl.play()in render-critical compositions. - Do not use
Date.now(),Math.random(), timers, network fetches, or async timeline construction. - Use finite tweens only. Never use
repeat: -1in a rendered MTV. - The registry key must match
data-composition-id.
GSAP Core Rules To Carry Into MTV
- Use timelines, labels, and the position parameter instead of chained
delay. - Prefer
autoAlphaover rawopacitywhen elements should disappear. - Prefer transform aliases:
x,y,xPercent,yPercent,scale,rotation,rotationX,rotationY. - Use documented eases:
power2.out,power3.inOut,expo.out,sine.inOut,back.out(1.7). - Set
immediateRender: falseon laterfrom()orfromTo()tweens that target the same property. - Put repeated defaults on the timeline:
const tl = gsap.timeline({
paused: true,
defaults: { duration: 0.55, ease: "power2.out" }
});Utilities For Deterministic Music Motion
Use gsap.utils instead of ad hoc math when mapping timeline data:
clamp()for safe lyric enter/exit durations.mapRange()andnormalize()for scene progress to camera drift.wrap()for deterministic cycling through directions, color accents, and lyric variants.snap()for stepped waveform or beat-grid motion.toArray()for converting generated node lists before staggered tweens.distribute()for particle fields and repeated line accents.
Avoid gsap.utils.random() in HyperFrames unless seeded upstream and serialized into data. The render should be deterministic.
Plugin Policy
Official GSAP plugins are available from the public gsap npm package. No Club membership, auth token, private registry, or .npmrc is needed.
Default MTV templates should remain core-only unless the plugin adds visible value and the local asset is copied into assets/:
- SplitText: rich character/word lyric reveals. Use only when the plugin file is local and registered.
- ScrambleText: glitch/reveal text for electronic, cyber, or unstable moods.
- CustomEase: song-specific easing curves when built-ins feel generic.
- MotionPath: route/travel motifs, moving lights, orbiting objects.
- DrawSVG/MorphSVG: hand-drawn lines, logo-free symbols, waveform strokes.
Always call gsap.registerPlugin(...) before plugin use.
MTV Motion Layers
Use five layered systems, all driven by one master timeline:
1. Scene layer
- Crossfade or wipe between keyframes.
- Animate camera with slow
scale,xPercent,yPercent, and tinyrotation. - Use labels like
scene_01at the scene start.
2. Lyric layer
- Generate one lyric DOM node per timed line.
- Split into word spans manually by default; stagger words for readable kinetic lyrics.
- Keep subtitles in a safe bottom or center region and avoid overflow.
3. Beat accent layer
- Pulse bars, scanlines, frame flashes, and small scale changes on a finite beat grid.
- Derive the grid from song duration or explicit beat markers; avoid infinite loops.
4. Atmosphere layer
- Use deterministic particles, grain, vignette, paper texture, film scratches, or light sweeps.
- Animate transforms and
autoAlpha; keep filters sparse.
5. Progress layer
- A tiny progress mark or scene counter helps long music videos feel paced.
- Animate
scaleXfrom 0 to 1 over the full duration withease: "none".
MTV Aesthetic Grammar
The goal is a music video, not an illustrated lyric slide deck. A generated MTV should feel like one coherent editorial motion system:
- Define a recurring visual motif from the lyrics: glass, rain, salt, paper, radio, neon, ocean,
rust, shadow, etc.
- Translate the motif into motion vocabulary. Examples: glass uses soft reflection wipes and
doubled echoes; rain uses vertical streaks and downward light drag; paper uses slips, folds, and print misregistration; radio uses scanlines and signal drift.
- Keep the main camera move simple enough to preserve generated still-image quality. Let transition
overlays and lyric layout provide variety.
- Avoid one-note templates. If every scene uses the same centered subtitle and same crossfade, the
result is unfinished even when technically correct.
Lyric Design Rules
Kinetic lyrics must stay readable and stable. Richness should come from layout, typography, and phrasing, not from jitter.
- Use 4-7 safe lyric anchors and cycle them deterministically according to song section and image
negative space: lower-left, upper-right, center-right, left-wall, lower-right, center-wide, table-low.
- Use restrained irregularity: slight tilt, small x/y drift, word-level stagger, faint echo layers,
subtle outline, or a soft panel. Do not shake every line.
- Enter with
autoAlpha+y+ optional tiny rotation. Avoid largescale,rotationX, fast
alternating filters, and transform values that cause visible subtitle edge jitter.
- Hold each line calmly for most of its duration. The line should not keep moving after the entrance
unless the song explicitly calls for it.
- Let important words get a small style difference: warm tint, faint underline, outline, or delayed
word reveal. Do not over-highlight every word.
- For quiet or dreamlike songs, prefer condensed or editorial sans typography with glassy shadows
and low-contrast echo. For aggressive songs, allow chunkier type, hard outlines, red accents, and cut-up word placement.
Scene Transition Rules
Every scene change should carry some intentional language:
- Do not rely only on crossfade. Layer at least two transition cues: dissolve + light sweep, wipe +
texture, bloom + parallax, glass reflection + fog, scanline + cut.
- Match transitions to the song motif. A glass song should use reflection panes, salt grain, and
frosted wipes; a radio song can use static, signal bands, and channel switching; a paper song can use print slips and edge shadows.
- Keep transitions short enough to avoid muddying the still image. For soft songs, 0.8-1.6s is
usually enough; for sharp songs, 0.16-0.6s.
- Avoid black frames unless explicitly designed as a musical rest.
- Do not let persistent title marks or debug-like labels sit over the whole video. Title typography
belongs in an intro/outro or as a deliberate lyric moment.
- Never expose internal profile names such as
poster motion, template labels, URL slugs, or clip
hashes. Opening titles must come from real metadata, preferably LRC [ti:]; sanitize filenames before displaying them.
- Treat song authorship as fixed metadata: author/artist credits are always
向阳乔木. Do not use
Suno AI, the download account, LRC [by:], a URL slug, or a model guess as the song author.
- Audio visualizers are opt-in. Waveforms, equalizer bars, progress lines, and beat pulses read as
player UI unless the song itself is about radio, signal, clubs, machines, or electronic interfaces. For quiet, cold, dreamlike, or narrative songs, spend that motion budget on scene transitions, camera drift, glass/reflection/rain/paper motifs, and lyric typography instead.
Motion Profiles
Use these defaults unless the song suggests otherwise:
| Profile | Best for | Motion language |
|---|---|---|
cinematic | pop, rock, ballad, story songs | slow camera drift, confident lyric reveals, restrained pulses |
poster | retro, folk, illustration-heavy | paper slide, print misregistration, low rotation, blocky lyric entrances |
glitch | electronic, cyber, anger | short jitter, scanline flashes, sharp lyric slices, tighter beat grid |
dream | ambient, soft pop, nostalgia | long dissolves, low contrast pulses, floating particles, blur-in lyrics |
minimal | delicate lyrics, spoken sections | fewer scene moves, quiet fades, no aggressive beat effects |
Performance Rules
- Animate transforms and
autoAlphafirst. Avoidwidth,height,top,left,margin, andpadding. - Use
will-changeonly on elements that actually animate. - Use
staggerinstead of creating many separate equivalent tweens. - Keep simultaneous filtered layers low; image
filteris acceptable at transition boundaries, not everywhere. - Kill nothing manually in a static HyperFrames render; create the finite timeline once.
- Prefer the skill renderer wrapper over raw
npx hyperframes render; it sets the stable browser,
streaming encode, long Puppeteer timeouts, cache limits, and SDR defaults.
- Choose workers by reclaimable memory, but cap final renders conservatively. A single worker is
slow and steady; 2-3 workers are faster only when memory is actually available.
- Remove decorative DOM that does not serve the song. Continuous visualizer bars, progress fills,
and beat loops add render work and can make the MV feel like software UI.
MTV QA Checklist
Before rendering final:
npm run checkpasses in the HyperFrames project.npx hyperframes inspect --samples 24has no layout issues.- Lyrics stay inside safe bounds on representative frames.
- No black frames during scene transitions.
- The MP4 passes
ffmpeg -v error -i final.mp4 -f null -. - The local project uses
assets/gsap.min.js; no CDN dependency during render.
#!/usr/bin/env python3
"""Create a GSAP-rich HyperFrames MTV project from qiaomu MTV assets."""
import argparse
import html
import json
import re
import shutil
import subprocess
import sys
from datetime import datetime
from pathlib import Path
from string import Template
HYPERFRAMES_VERSION = "0.6.60"
DEFAULT_GSAP_VERSION = "3.15.0"
DEFAULT_SONG_AUTHOR = "向阳乔木"
IMAGE_SUFFIXES = {".png", ".jpg", ".jpeg", ".webp"}
RATIO_SIZES = {
"16:9": (1920, 1080),
"9:16": (1080, 1920),
"1:1": (1080, 1080),
}
MOTION_PROFILES = {
"cinematic": {
"scene_overlap": 1.1,
"scene_push": 7,
"lyric_in": 0.34,
"lyric_out": 0.36,
"word_stagger": 0.035,
"beat_step": 1.15,
"pulse_strength": 0.82,
"particle_count": 30,
"accent": "#ff5638",
"warm": "#ffe3ad",
"show_waveform": False,
"show_progress": False,
"show_pulse": False,
},
"poster": {
"scene_overlap": 0.85,
"scene_push": 4,
"lyric_in": 0.28,
"lyric_out": 0.30,
"word_stagger": 0.026,
"beat_step": 1.35,
"pulse_strength": 0.62,
"particle_count": 20,
"accent": "#ef4738",
"warm": "#f7d28b",
"show_waveform": False,
"show_progress": False,
"show_pulse": False,
},
"glitch": {
"scene_overlap": 0.48,
"scene_push": 12,
"lyric_in": 0.18,
"lyric_out": 0.22,
"word_stagger": 0.018,
"beat_step": 0.72,
"pulse_strength": 1.0,
"particle_count": 36,
"accent": "#42f5d7",
"warm": "#ff4c77",
"show_waveform": False,
"show_progress": False,
"show_pulse": False,
},
"dream": {
"scene_overlap": 1.55,
"scene_push": 5,
"lyric_in": 0.48,
"lyric_out": 0.46,
"word_stagger": 0.045,
"beat_step": 1.6,
"pulse_strength": 0.46,
"particle_count": 34,
"accent": "#9fd6ff",
"warm": "#ffe3d4",
"show_waveform": False,
"show_progress": False,
"show_pulse": False,
},
"minimal": {
"scene_overlap": 0.75,
"scene_push": 2,
"lyric_in": 0.30,
"lyric_out": 0.30,
"word_stagger": 0.025,
"beat_step": 1.8,
"pulse_strength": 0.34,
"particle_count": 12,
"accent": "#f2efe6",
"warm": "#ffffff",
"show_waveform": False,
"show_progress": False,
"show_pulse": False,
},
}
INDEX_TEMPLATE = Template(
"""<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=$width, height=$height" />
<title>$title</title>
<script src="assets/gsap.min.js"></script>
<script src="assets/mtv-data.js"></script>
<style>
@font-face {
font-family: "Anton";
src: url("assets/fonts/anton-latin-400-normal.woff2") format("woff2");
font-weight: 400;
font-style: normal;
font-display: block;
}
@font-face {
font-family: "Space Grotesk";
src: url("assets/fonts/space-grotesk-latin-400-normal.woff2") format("woff2");
font-weight: 400;
font-style: normal;
font-display: block;
}
@font-face {
font-family: "Space Grotesk";
src: url("assets/fonts/space-grotesk-latin-700-normal.woff2") format("woff2");
font-weight: 700;
font-style: normal;
font-display: block;
}
* {
box-sizing: border-box;
margin: 0;
padding: 0;
}
html,
body {
width: ${width}px;
height: ${height}px;
overflow: hidden;
background: #050606;
color: #fff7e5;
font-family: "Space Grotesk", sans-serif;
}
#root {
position: relative;
width: ${width}px;
height: ${height}px;
overflow: hidden;
isolation: isolate;
background:
radial-gradient(circle at 70% 26%, color-mix(in srgb, var(--accent) 28%, transparent), transparent 34%),
linear-gradient(135deg, #050707 0%, #11191d 48%, #040404 100%);
--accent: $accent;
--warm: $warm;
}
#root[data-show-waveform="false"] .wave,
#root[data-show-progress="false"] .progress,
#root[data-show-pulse="false"] .pulse {
display: none;
}
.backgrounds,
.image-layer,
.image-layer img,
.grade,
.grain,
.vignette,
.scanlines,
.light-sweep,
.transition-pane,
.reflection-sweep,
.texture-wipe,
.edge-glow,
.particles,
.lyric-stage {
position: absolute;
inset: 0;
}
.image-layer {
opacity: 0;
transform-origin: center;
will-change: transform, opacity;
}
.image-layer img {
width: 100%;
height: 100%;
object-fit: cover;
filter: saturate(1.08) contrast(1.05);
}
.grade {
z-index: 4;
pointer-events: none;
background:
linear-gradient(90deg, rgba(0, 0, 0, 0.58), transparent 30%, transparent 70%, rgba(0, 0, 0, 0.52)),
linear-gradient(0deg, rgba(3, 4, 5, 0.76), transparent 38%, rgba(3, 4, 5, 0.24)),
radial-gradient(circle at 52% 78%, color-mix(in srgb, var(--accent) 18%, transparent), transparent 42%);
mix-blend-mode: multiply;
}
.vignette {
z-index: 10;
pointer-events: none;
box-shadow: inset 0 0 min(11vw, 210px) rgba(0, 0, 0, 0.84);
}
.grain {
z-index: 8;
pointer-events: none;
opacity: 0.17;
background-image:
repeating-linear-gradient(0deg, rgba(255, 255, 255, 0.045) 0 1px, transparent 1px 3px),
repeating-linear-gradient(90deg, rgba(255, 255, 255, 0.025) 0 1px, transparent 1px 4px);
mix-blend-mode: overlay;
}
.scanlines {
z-index: 9;
pointer-events: none;
opacity: 0;
background: repeating-linear-gradient(
0deg,
rgba(255, 255, 255, 0.0) 0 8px,
color-mix(in srgb, var(--accent) 24%, transparent) 8px 9px,
rgba(255, 255, 255, 0.0) 9px 16px
);
mix-blend-mode: screen;
}
.light-sweep {
z-index: 7;
pointer-events: none;
width: 38%;
background: linear-gradient(100deg, transparent, rgba(255, 249, 230, 0.18), transparent);
transform-origin: center;
will-change: transform;
}
.transition-pane {
z-index: 12;
pointer-events: none;
opacity: 0;
background:
linear-gradient(96deg, transparent 0 22%, rgba(255, 249, 230, 0.22) 42%, rgba(150, 216, 255, 0.14) 52%, transparent 72%),
repeating-linear-gradient(90deg, transparent 0 18px, rgba(255, 255, 255, 0.045) 18px 20px);
mix-blend-mode: screen;
transform-origin: center;
will-change: transform, opacity, clip-path;
}
.reflection-sweep {
z-index: 11;
pointer-events: none;
opacity: 0;
background:
linear-gradient(110deg, transparent 0 28%, rgba(255, 255, 255, 0.20) 44%, transparent 58%),
radial-gradient(circle at 50% 72%, color-mix(in srgb, var(--accent) 18%, transparent), transparent 38%);
mix-blend-mode: screen;
filter: blur(0.5px);
will-change: transform, opacity;
}
.texture-wipe {
z-index: 5;
pointer-events: none;
opacity: 0;
background:
repeating-linear-gradient(0deg, rgba(255,255,255,0.055) 0 1px, transparent 1px 6px),
repeating-linear-gradient(90deg, rgba(255,255,255,0.035) 0 1px, transparent 1px 11px);
mix-blend-mode: overlay;
will-change: opacity, transform;
}
.edge-glow {
z-index: 12;
pointer-events: none;
opacity: 0;
background:
radial-gradient(circle at 8% 50%, color-mix(in srgb, var(--warm) 26%, transparent), transparent 26%),
radial-gradient(circle at 92% 48%, color-mix(in srgb, var(--accent) 22%, transparent), transparent 28%);
mix-blend-mode: screen;
will-change: opacity, transform;
}
.particles {
z-index: 6;
pointer-events: none;
}
.particle {
position: absolute;
width: 6px;
height: 6px;
border-radius: 999px;
background: var(--warm);
opacity: 0;
box-shadow: 0 0 16px color-mix(in srgb, var(--accent) 60%, transparent);
will-change: transform, opacity;
}
.wave {
position: absolute;
left: 7%;
right: 7%;
bottom: 8.1%;
z-index: 14;
height: 64px;
display: flex;
align-items: end;
justify-content: center;
gap: 8px;
pointer-events: none;
opacity: 0.68;
}
.bar {
width: 5px;
height: 18px;
border-radius: 999px;
background: linear-gradient(180deg, var(--warm), var(--accent));
transform-origin: center bottom;
opacity: 0.48;
will-change: transform, opacity;
}
.pulse {
position: absolute;
left: 12%;
right: 12%;
bottom: 10.2%;
z-index: 13;
height: 3px;
border-radius: 999px;
background: linear-gradient(90deg, transparent, var(--warm), var(--accent), transparent);
opacity: 0;
transform-origin: center;
will-change: transform, opacity;
}
.lyric-stage {
z-index: 18;
pointer-events: none;
}
.lyric-line {
position: absolute;
display: flex;
flex-wrap: wrap;
align-items: center;
justify-content: flex-start;
gap: 0.22em;
font-family: "Space Grotesk", sans-serif;
color: #fff7df;
font-size: var(--lyric-size, 72px);
line-height: 0.96;
font-weight: 760;
letter-spacing: 0;
text-align: left;
text-transform: uppercase;
text-shadow:
0 3px 0 rgba(0, 0, 0, 0.78),
0 0 22px color-mix(in srgb, var(--accent) 32%, transparent),
0 0 68px rgba(0, 0, 0, 0.86);
opacity: 0;
transform: translate3d(0, 18px, 0) rotate(var(--tilt, 0deg));
transform-origin: var(--origin, 50% 50%);
backface-visibility: hidden;
-webkit-font-smoothing: antialiased;
will-change: transform, opacity;
}
.lyric-line.short {
--lyric-size: 88px;
}
.lyric-line.chorus {
color: var(--warm);
text-shadow:
0 4px 0 rgba(0, 0, 0, 0.88),
0 0 38px color-mix(in srgb, var(--accent) 72%, transparent),
0 0 88px color-mix(in srgb, var(--accent) 42%, transparent);
}
.word {
display: inline-block;
margin-right: 0.02em;
transform-origin: 50% 64%;
backface-visibility: hidden;
will-change: transform, opacity, color;
}
.word.emphasis {
color: color-mix(in srgb, var(--warm) 86%, white);
}
.anchor-lower-left {
left: max(68px, 5.4vw);
bottom: max(118px, 10.2vh);
width: min(860px, 48vw);
--tilt: -1.4deg;
--origin: 0% 100%;
}
.anchor-upper-right {
right: max(74px, 5.8vw);
top: max(92px, 9vh);
width: min(830px, 46vw);
justify-content: flex-end;
text-align: right;
--tilt: 1.2deg;
--origin: 100% 0%;
}
.anchor-center-right {
right: max(80px, 6vw);
top: 34%;
width: min(840px, 46vw);
justify-content: flex-end;
text-align: right;
--tilt: -0.8deg;
--origin: 100% 50%;
}
.anchor-left-wall {
left: max(72px, 5.8vw);
top: 27%;
width: min(780px, 43vw);
--tilt: -2.2deg;
--origin: 0% 50%;
}
.anchor-lower-right {
right: max(70px, 5.6vw);
bottom: max(122px, 10.6vh);
width: min(880px, 48vw);
justify-content: flex-end;
text-align: right;
--tilt: 1deg;
--origin: 100% 100%;
}
.anchor-center-wide {
left: max(110px, 7vw);
right: max(110px, 7vw);
top: 42%;
justify-content: center;
text-align: center;
--tilt: -0.4deg;
--origin: 50% 50%;
}
.anchor-table-low {
left: 13%;
right: 13%;
bottom: max(86px, 7.6vh);
justify-content: center;
text-align: center;
--tilt: 0.7deg;
--origin: 50% 100%;
}
.style-whisper {
color: rgba(255, 247, 223, 0.88);
}
.style-glass .word {
text-shadow:
0 2px 0 rgba(0, 0, 0, 0.86),
0 0 20px color-mix(in srgb, var(--accent) 34%, transparent);
}
.style-glass::after,
.style-echo::after {
content: attr(data-text);
position: absolute;
inset: 0;
z-index: -1;
color: color-mix(in srgb, var(--accent) 28%, transparent);
transform: translate(14px, 10px);
opacity: 0.34;
filter: blur(0.3px);
}
.style-salt {
text-shadow:
0 3px 0 rgba(0, 0, 0, 0.82),
0 0 28px rgba(255, 255, 255, 0.18);
}
.style-salt .word:nth-child(odd) {
color: color-mix(in srgb, var(--warm) 82%, white);
}
.style-frame {
padding: 18px 22px;
margin: -18px -22px;
background: linear-gradient(90deg, rgba(255, 247, 229, 0.12), rgba(159, 214, 255, 0.035));
box-shadow: inset 0 -1px 0 rgba(255, 255, 255, 0.18);
}
.style-pulse .word.emphasis,
.style-pulse.chorus .word {
color: color-mix(in srgb, var(--warm) 92%, white);
}
.title-mark {
position: absolute;
left: max(54px, 4.4vw);
top: max(44px, 5.4vh);
z-index: 20;
display: grid;
gap: 8px;
max-width: min(640px, 58vw);
font-family: "Anton", sans-serif;
color: rgba(255, 247, 229, 0.94);
text-transform: uppercase;
}
.title-mark .kicker {
color: var(--accent);
font-size: clamp(16px, 1.2vw, 24px);
font-weight: 850;
letter-spacing: 0.16em;
}
.title-mark .title {
font-size: clamp(30px, 3.2vw, 56px);
line-height: 0.96;
font-weight: 900;
letter-spacing: 0;
text-shadow: 0 3px 0 rgba(0, 0, 0, 0.72);
}
.progress {
position: absolute;
left: max(54px, 4.4vw);
right: max(54px, 4.4vw);
bottom: max(44px, 5.1vh);
z-index: 20;
height: 4px;
border-radius: 999px;
overflow: hidden;
background: rgba(255, 247, 229, 0.18);
}
.progress-fill {
width: 100%;
height: 100%;
transform: scaleX(0);
transform-origin: left;
background: linear-gradient(90deg, var(--warm), var(--accent), var(--warm));
will-change: transform;
}
</style>
</head>
<body>
<div
id="root"
data-composition-id="main"
data-start="0"
data-duration="$duration"
data-width="$width"
data-height="$height"
data-show-waveform="$show_waveform"
data-show-progress="$show_progress"
data-show-pulse="$show_pulse"
>
<audio
id="song"
src="$audio_src"
data-start="0"
data-duration="$duration"
data-track-index="20"
data-volume="1"
></audio>
<div class="backgrounds"></div>
<div class="grade"></div>
<div class="light-sweep"></div>
<div class="texture-wipe"></div>
<div class="reflection-sweep"></div>
<div class="transition-pane"></div>
<div class="edge-glow"></div>
<div class="particles"></div>
<div class="grain"></div>
<div class="scanlines"></div>
<div class="vignette"></div>
<div class="pulse"></div>
<div class="wave"></div>
<div class="lyric-stage"></div>
<div class="title-mark">
$kicker_markup
<div class="title">$display_title</div>
</div>
<div class="progress"><div class="progress-fill"></div></div>
</div>
<script>
const data = window.MTV_DATA;
const motion = data.motion || {};
const root = document.getElementById("root");
const backgrounds = root.querySelector(".backgrounds");
const lyricStage = root.querySelector(".lyric-stage");
const particles = root.querySelector(".particles");
const wave = root.querySelector(".wave");
const showWaveform = motion.show_waveform === true;
const showProgress = motion.show_progress === true;
const showPulse = motion.show_pulse === true;
function seconds(value, fallback) {
const parsed = Number(value);
return Number.isFinite(parsed) ? parsed : fallback;
}
function safeDuration(start, end, fallback) {
return Math.max(0.18, seconds(end, start + fallback) - start);
}
function escapeText(value) {
return String(value || "").replace(/[&<>]/g, (char) => ({
"&": "&",
"<": "<",
">": ">"
})[char]);
}
const emphasisWords = new Set([
"glass", "salt", "rain", "light", "door", "heart", "hands", "cold",
"pulse", "shape", "name", "night", "still", "back", "soft"
]);
function wordSpans(text) {
return String(text || "")
.split(/\\s+/)
.filter(Boolean)
.map((word) => {
const key = word.toLowerCase().replace(/[^a-z0-9']/g, "");
const className = emphasisWords.has(key) ? "word emphasis" : "word";
return "<span class=\\"" + className + "\\">" + escapeText(word) + "</span>";
})
.join(" ");
}
const lyricAnchors = gsap.utils.wrap([
"anchor-lower-left",
"anchor-upper-right",
"anchor-center-right",
"anchor-left-wall",
"anchor-lower-right",
"anchor-center-wide",
"anchor-table-low"
]);
const lyricStyles = gsap.utils.wrap([
"style-whisper",
"style-glass",
"style-salt",
"style-echo",
"style-frame",
"style-pulse"
]);
function lyricSize(text, kind) {
const length = String(text || "").length;
if (kind === "chorus") return length > 30 ? 86 : 98;
if (length <= 22) return 88;
if (length >= 42) return 62;
return 74;
}
data.images.forEach((image, index) => {
const layer = document.createElement("div");
layer.className = "image-layer";
layer.dataset.index = String(index + 1);
const img = document.createElement("img");
img.src = image.src;
img.alt = "";
layer.appendChild(img);
backgrounds.appendChild(layer);
});
data.lyrics.forEach((line, index) => {
const el = document.createElement("div");
const kind = line.kind || "line";
const anchor = kind === "chorus" ? lyricAnchors(index + 2) : lyricAnchors(index);
const style = kind === "short" ? lyricStyles(index + 1) : lyricStyles(index);
el.className = ["lyric-line", kind, anchor, style].join(" ");
el.dataset.index = String(index + 1);
el.dataset.text = String(line.text || "").toUpperCase();
el.style.setProperty("--lyric-size", lyricSize(line.text, kind) + "px");
el.innerHTML = wordSpans(line.text);
lyricStage.appendChild(el);
});
const particleCount = Number(motion.particle_count || 24);
for (let i = 0; i < particleCount; i += 1) {
const p = document.createElement("div");
p.className = "particle";
p.style.left = ((i * 37) % 100) + "%";
p.style.top = (12 + ((i * 53) % 78)) + "%";
p.style.width = (3 + (i % 5)) + "px";
p.style.height = p.style.width;
particles.appendChild(p);
}
if (showWaveform && wave) {
for (let i = 0; i < 42; i += 1) {
const bar = document.createElement("div");
bar.className = "bar";
bar.style.height = (12 + ((i * 17) % 46)) + "px";
wave.appendChild(bar);
}
}
window.__timelines = window.__timelines || {};
gsap.defaults({ overwrite: "auto" });
const tl = gsap.timeline({
paused: true,
defaults: { ease: "power2.out" }
});
const layers = gsap.utils.toArray(".image-layer");
const lyricLines = gsap.utils.toArray(".lyric-line");
const bars = gsap.utils.toArray(".bar");
const particleNodes = gsap.utils.toArray(".particle");
const direction = gsap.utils.wrap([
{ x: -1, y: 0.25, r: -0.28 },
{ x: 1, y: -0.18, r: 0.24 },
{ x: 0.36, y: 1, r: 0.18 },
{ x: -0.28, y: -1, r: -0.18 }
]);
const transitionMode = gsap.utils.wrap([
{ paneFrom: -120, paneTo: 140, sweepFrom: -90, sweepTo: 120, y: 0 },
{ paneFrom: 120, paneTo: -140, sweepFrom: 90, sweepTo: -120, y: 0 },
{ paneFrom: -90, paneTo: 110, sweepFrom: -60, sweepTo: 100, y: -10 },
{ paneFrom: 90, paneTo: -110, sweepFrom: 60, sweepTo: -100, y: 10 }
]);
gsap.set(layers, { autoAlpha: 0, scale: 1.1, transformOrigin: "50% 50%" });
gsap.set(lyricLines, { autoAlpha: 0, y: 18, scale: 1, force3D: false });
gsap.set(".pulse", { autoAlpha: 0, scaleX: 0.2 });
gsap.set(".scanlines", { autoAlpha: 0 });
gsap.set(".light-sweep", { xPercent: -130, skewX: -14 });
gsap.set(".transition-pane", { autoAlpha: 0, xPercent: -120, skewX: -10 });
gsap.set(".reflection-sweep", { autoAlpha: 0, xPercent: -90, skewX: -8 });
gsap.set(".texture-wipe", { autoAlpha: 0, yPercent: -8 });
gsap.set(".edge-glow", { autoAlpha: 0, scale: 1.04 });
gsap.set(particleNodes, { autoAlpha: 0, y: 28, scale: 0.4 });
const totalDuration = seconds(data.duration, $duration);
const imageCount = Math.max(1, data.images.length);
const defaultSceneDuration = totalDuration / imageCount;
const overlap = Number(motion.scene_overlap || 1);
const push = Number(motion.scene_push || 5);
data.images.forEach((image, index) => {
const layer = layers[index];
const vector = direction(index);
const start = seconds(image.start, index * defaultSceneDuration);
const end = seconds(image.end, Math.min(totalDuration, (index + 1) * defaultSceneDuration));
const duration = safeDuration(start, end, defaultSceneDuration);
const fade = Math.min(overlap, duration * 0.42);
tl.addLabel("scene_" + String(index + 1).padStart(2, "0"), start);
if (index > 0) {
const mode = transitionMode(index);
const transitionStart = Math.max(0, start - Math.min(0.72, fade * 0.52));
const transitionDuration = Math.min(1.35, Math.max(0.52, fade * 0.86));
tl.fromTo(".transition-pane",
{
autoAlpha: 0,
xPercent: mode.paneFrom,
yPercent: mode.y,
skewX: mode.paneFrom < 0 ? -12 : 12,
clipPath: "inset(0 46% 0 46%)"
},
{
autoAlpha: 0.62,
xPercent: mode.paneTo,
yPercent: mode.y * -0.35,
skewX: mode.paneFrom < 0 ? -4 : 4,
clipPath: "inset(0 0% 0 0%)",
duration: transitionDuration,
ease: "sine.inOut",
immediateRender: false
},
transitionStart
);
tl.to(".transition-pane",
{ autoAlpha: 0, duration: transitionDuration * 0.42, ease: "sine.out" },
transitionStart + transitionDuration * 0.58
);
tl.fromTo(".reflection-sweep",
{ autoAlpha: 0, xPercent: mode.sweepFrom, skewX: mode.sweepFrom < 0 ? -8 : 8 },
{
autoAlpha: 0.42,
xPercent: mode.sweepTo,
skewX: mode.sweepFrom < 0 ? -2 : 2,
duration: transitionDuration * 1.08,
ease: "power2.inOut",
immediateRender: false
},
transitionStart + 0.04
);
tl.to(".reflection-sweep",
{ autoAlpha: 0, duration: transitionDuration * 0.36, ease: "sine.out" },
transitionStart + transitionDuration * 0.68
);
tl.fromTo(".texture-wipe",
{ autoAlpha: 0, yPercent: mode.y * -0.6 },
{
autoAlpha: 0.24,
yPercent: mode.y * 0.5,
duration: transitionDuration,
ease: "sine.inOut",
immediateRender: false
},
transitionStart
);
tl.to(".texture-wipe", { autoAlpha: 0, duration: transitionDuration * 0.5, ease: "sine.out" }, transitionStart + transitionDuration * 0.5);
tl.fromTo(".edge-glow",
{ autoAlpha: 0, scale: 1.02 },
{ autoAlpha: 0.22, scale: 1, duration: transitionDuration * 0.44, ease: "power2.out", immediateRender: false },
transitionStart + transitionDuration * 0.18
);
tl.to(".edge-glow", { autoAlpha: 0, duration: transitionDuration * 0.52, ease: "sine.out" }, transitionStart + transitionDuration * 0.52);
}
tl.fromTo(layer,
{
autoAlpha: 0,
scale: 1.08,
xPercent: vector.x * -push,
yPercent: vector.y * -push,
rotation: vector.r * -1.6,
filter: "blur(7px) saturate(0.86)"
},
{
autoAlpha: 1,
scale: 1.025,
xPercent: vector.x * -1.2,
yPercent: vector.y * -1.2,
rotation: vector.r,
filter: "blur(0px) saturate(1.06)",
duration: fade,
ease: "power3.out",
immediateRender: false
},
start
);
tl.to(layer,
{
scale: 1.16 + (index % 3) * 0.018,
xPercent: vector.x * push,
yPercent: vector.y * push,
rotation: vector.r * 2.6,
duration: duration,
ease: "none"
},
start
);
tl.to(layer,
{
autoAlpha: 0,
duration: fade,
ease: "power2.inOut"
},
Math.max(start, end - fade)
);
});
const lyricIn = Number(motion.lyric_in || 0.32);
const lyricOut = Number(motion.lyric_out || 0.32);
const wordStagger = Number(motion.word_stagger || 0.03);
data.lyrics.forEach((line, index) => {
const el = lyricLines[index];
const words = gsap.utils.toArray(el.querySelectorAll(".word"));
const start = seconds(line.start, 0);
const duration = Math.max(0.24, seconds(line.duration, 1.8));
const exitAt = Math.max(start + lyricIn + 0.08, start + duration - lyricOut);
const yIn = line.kind === "chorus" ? 24 : 18;
const yOut = line.kind === "chorus" ? -22 : -16;
const xIn = index % 2 === 0 ? -12 : 12;
const xOut = index % 2 === 0 ? 8 : -8;
tl.addLabel("lyric_" + String(index + 1).padStart(3, "0"), start);
tl.fromTo(el,
{ autoAlpha: 0, x: xIn, y: yIn, filter: "blur(3px)" },
{
autoAlpha: 1,
x: 0,
y: 0,
filter: "blur(0px)",
duration: Math.min(lyricIn, duration * 0.42),
ease: "sine.out",
force3D: false,
immediateRender: false
},
start
);
tl.fromTo(words,
{ autoAlpha: 0, y: 12 },
{
autoAlpha: 1,
y: 0,
duration: Math.min(0.48, duration * 0.34),
ease: "sine.out",
stagger: { each: wordStagger, from: line.kind === "chorus" ? "center" : "start" },
force3D: false,
immediateRender: false
},
start + 0.06
);
tl.to(el,
{
autoAlpha: 0,
x: xOut,
y: yOut,
filter: "blur(2px)",
duration: Math.min(lyricOut, duration * 0.34),
ease: "sine.inOut",
force3D: false
},
exitAt
);
});
const beatStep = Number(motion.beat_step || 1.2);
const pulseStrength = Number(motion.pulse_strength || 0.65);
if (showPulse || showWaveform) {
for (let t = 0; t < totalDuration; t += beatStep) {
const accent = 0.72 + (Math.sin(t * 1.618) + 1) * 0.14;
if (showPulse) {
tl.fromTo(".pulse",
{ autoAlpha: 0, scaleX: 0.16 },
{
autoAlpha: Math.min(1, pulseStrength * accent),
scaleX: 0.92 + pulseStrength * 0.16,
duration: 0.12,
ease: "power2.out",
immediateRender: false
},
t
);
tl.to(".pulse", { autoAlpha: 0, scaleX: 1.12, duration: 0.42, ease: "power3.out" }, t + 0.12);
}
if (showWaveform && bars.length) {
tl.to(bars, {
scaleY: (i) => 0.5 + ((i * 13 + Math.round(t * 10)) % 21) / 10,
autoAlpha: 0.38 + pulseStrength * 0.36,
duration: 0.13,
ease: "power2.out",
stagger: { amount: 0.12, from: "center" }
}, t);
tl.to(bars, { scaleY: 0.72, autoAlpha: 0.42, duration: 0.38, ease: "sine.out" }, t + 0.13);
}
}
}
particleNodes.forEach((node, index) => {
const lane = direction(index + 1);
const start = (index / Math.max(1, particleNodes.length)) * Math.min(totalDuration, 8);
const travel = Math.max(8, totalDuration - start);
tl.fromTo(node,
{
autoAlpha: 0,
xPercent: lane.x * -180,
yPercent: lane.y * 120,
scale: 0.35
},
{
autoAlpha: 0.42,
xPercent: lane.x * 280,
yPercent: lane.y * -220,
scale: 1.1,
duration: travel,
ease: "none",
immediateRender: false
},
start
);
tl.to(node, { autoAlpha: 0, duration: 1.4, ease: "sine.inOut" }, Math.max(start, totalDuration - 1.4));
});
tl.fromTo(".title-mark",
{ autoAlpha: 0, y: -18 },
{ autoAlpha: 0.72, y: 0, duration: 1.1, ease: "power3.out", immediateRender: false },
0.2
);
tl.to(".title-mark", { autoAlpha: 0, y: -10, duration: 1.2, ease: "sine.inOut" }, 7.5);
if (showProgress) {
tl.to(".progress-fill", { scaleX: 1, duration: totalDuration, ease: "none" }, 0);
}
tl.fromTo(".light-sweep", { xPercent: -130, skewX: -14 }, { xPercent: 360, skewX: -14, duration: totalDuration, ease: "none" }, 0);
tl.to(".grain", { autoAlpha: 0.23, duration: totalDuration, ease: "sine.inOut" }, 0);
if (data.motionProfile === "glitch") {
for (let t = 0.4; t < totalDuration; t += Math.max(1.4, beatStep * 3)) {
tl.fromTo(".scanlines",
{ autoAlpha: 0, y: -18 },
{ autoAlpha: 0.38, y: 0, duration: 0.08, ease: "none", immediateRender: false },
t
);
tl.to(".scanlines", { autoAlpha: 0, duration: 0.16, ease: "none" }, t + 0.08);
tl.to("#root", { x: 5, duration: 0.04, yoyo: true, repeat: 3, ease: "none" }, t);
}
}
tl.addLabel("end", totalDuration);
window.__timelines["main"] = tl;
</script>
</body>
</html>
"""
)
def load_json(path: Path) -> dict:
with open(path, "r", encoding="utf-8") as f:
return json.load(f)
def write_json(path: Path, data: dict):
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
json.dump(data, f, ensure_ascii=False, indent=2)
def write_text(path: Path, text: str):
path.parent.mkdir(parents=True, exist_ok=True)
with open(path, "w", encoding="utf-8") as f:
f.write(text)
def compact_inline_scripts(document: str) -> str:
"""Keep generated composition line count under HyperFrames lint limits."""
output = []
script_buffer = []
in_inline_script = False
for line in document.splitlines():
stripped = line.strip()
if stripped == "<script>":
output.append(line)
in_inline_script = True
script_buffer = []
continue
if in_inline_script and stripped == "</script>":
output.append(" ".join(part.strip() for part in script_buffer if part.strip()))
output.append(line)
in_inline_script = False
continue
if in_inline_script:
script_buffer.append(line)
continue
if not stripped:
continue
output.append(line)
return "\n".join(output) + "\n"
def format_duration(value: float) -> str:
if abs(value - round(value)) < 0.001:
return str(int(round(value)))
return f"{value:.3f}".rstrip("0").rstrip(".")
def is_marker_text(text: str) -> bool:
stripped = text.strip()
return bool(stripped.startswith("[") and stripped.endswith("]"))
def clean_package_name(value: str) -> str:
chars = []
for ch in value:
if ch.isascii() and ch.isalnum():
chars.append(ch.lower())
else:
chars.append("-")
cleaned = "".join(chars)
cleaned = "-".join(part for part in cleaned.split("-") if part)
return cleaned[:64] or "qiaomu-hyper-mv"
def collect_images(images_dir: Path) -> list[Path]:
if not images_dir.exists():
return []
images = [p for p in images_dir.iterdir() if p.is_file() and p.suffix.lower() in IMAGE_SUFFIXES]
scene_images = sorted([p for p in images if p.stem.startswith("scene_")])
return scene_images or sorted(images)
def resolve_project(project_dir: Path) -> dict:
project_path = project_dir / "project.json"
timeline_path = project_dir / "timeline.json"
if not project_path.exists():
raise SystemExit(f"Missing project.json in {project_dir}")
if not timeline_path.exists():
raise SystemExit(f"Missing timeline.json in {project_dir}")
project = load_json(project_path)
timeline = load_json(timeline_path)
paths = project.get("paths", {})
visual_config_path = Path(paths.get("visual_config") or project_dir / "visual_config.json").expanduser()
visual_config = load_json(visual_config_path) if visual_config_path.exists() else {}
audio_path = Path(project.get("audio", "")).expanduser()
if not audio_path.exists():
fallback_audio = project_dir / "audio" / audio_path.name
if fallback_audio.exists():
audio_path = fallback_audio
if not audio_path.exists():
raise SystemExit(f"Audio file not found: {audio_path}")
images_dir = Path(paths.get("images") or visual_config.get("output_dir") or project_dir / "images").expanduser()
images = collect_images(images_dir)
if not images:
raise SystemExit(f"No images found in {images_dir}")
return {
"project": project,
"timeline": timeline,
"visual_config": visual_config,
"audio_path": audio_path,
"images_dir": images_dir,
"images": images,
}
def lyric_kind(text: str) -> str:
lowered = text.lower()
if "looking back" in lowered or "never relax" in lowered:
return "chorus"
if len(text) <= 22:
return "short"
return "line"
def build_lyrics(timeline: dict) -> list[dict]:
lyrics = []
for segment in timeline.get("segments", []):
text = str(segment.get("text", "")).strip()
if not text or is_marker_text(text):
continue
start = float(segment.get("start_seconds", 0))
end = float(segment.get("end_seconds", start + 1.6))
if end <= start:
end = start + 1.0
lyrics.append({
"id": int(segment.get("id", len(lyrics) + 1)),
"start": round(start, 3),
"duration": round(end - start, 3),
"text": text,
"kind": lyric_kind(text),
})
return lyrics
def build_image_records(images: list[Path], timeline: dict, visual_config: dict) -> list[dict]:
duration = float(timeline.get("audio_duration", 0) or 0)
visual_items = {
int(item.get("index", idx + 1)): item
for idx, item in enumerate(visual_config.get("illustrations", []))
if str(item.get("index", idx + 1)).isdigit()
}
scenes = timeline.get("scenes", [])
count = max(1, len(images))
records = []
for idx, image_path in enumerate(images):
visual_item = visual_items.get(idx + 1, {})
scene = scenes[min(idx, len(scenes) - 1)] if scenes else {}
start = visual_item.get("start_seconds", scene.get("start_seconds"))
end = visual_item.get("end_seconds", scene.get("end_seconds"))
if start is None or end is None:
start = duration * idx / count if duration else idx * 6.0
end = duration * (idx + 1) / count if duration else start + 6.0
start = float(start)
end = float(end)
if end <= start:
end = start + (duration / count if duration else 6.0)
records.append({
"id": idx + 1,
"src": f"assets/images/{image_path.name}",
"start": round(start, 3),
"end": round(end, 3),
"lyrics": visual_item.get("lyrics") or scene.get("text", ""),
"motion": visual_item.get("motion", ""),
"mood": visual_item.get("mood", ""),
})
if records and duration:
records[0]["start"] = 0
records[-1]["end"] = max(records[-1]["end"], duration)
return records
def extract_lrc_tag(path: Path, tag: str) -> str:
if not path.exists():
return ""
pattern = re.compile(r"^\[" + re.escape(tag) + r":(.+)\]$")
try:
for line in path.read_text(encoding="utf-8", errors="ignore").splitlines()[:40]:
match = pattern.match(line.strip())
if match:
return match.group(1).strip()
except OSError:
return ""
return ""
def strip_machine_suffix(value: str) -> str:
text = str(value or "").strip()
if not text:
return ""
text = re.sub(r"\.[A-Za-z0-9]+$", "", text)
text = re.sub(r"[-_ ]+[A-Za-z]{0,3}[0-9A-Fa-f]{5,}$", "", text)
text = re.sub(r"[-_ ]+[A-Za-z0-9]{8,}$", lambda m: "" if re.search(r"\d", m.group(0)) else m.group(0), text)
if "-" in text or "_" in text:
text = re.sub(r"[-_]+", " ", text)
text = re.sub(r"\s+", " ", text).strip(" -_")
if text and text == text.lower():
text = text.title()
return text
def resolve_song_title(project_dir: Path, resolved: dict) -> str:
project = resolved["project"]
timeline = resolved["timeline"]
visual_config = resolved["visual_config"]
candidates = [
project.get("title"),
project.get("song_title"),
project.get("name"),
project.get("metadata", {}).get("title") if isinstance(project.get("metadata"), dict) else "",
timeline.get("title"),
timeline.get("song_title"),
visual_config.get("title"),
visual_config.get("song_title"),
]
for key in ("subtitle", "srt"):
path_value = project.get(key) or timeline.get(key)
if path_value:
tag_title = extract_lrc_tag(Path(path_value).expanduser(), "ti")
if tag_title:
candidates.append(tag_title)
candidates.append(Path(path_value).expanduser().stem)
candidates.extend([
resolved["audio_path"].stem,
project_dir.name,
])
for candidate in candidates:
title = strip_machine_suffix(str(candidate or ""))
if title and title.lower() not in {"untitled", "placeholder", "qiaomu mtv"}:
return title
return "Qiaomu MTV"
def copy_project_assets(resolved: dict, hyper_dir: Path):
assets_dir = hyper_dir / "assets"
image_out = assets_dir / "images"
image_out.mkdir(parents=True, exist_ok=True)
audio_path = resolved["audio_path"]
audio_target = assets_dir / f"song{audio_path.suffix.lower() or '.mp3'}"
shutil.copy2(audio_path, audio_target)
copied_images = []
for image in resolved["images"]:
target = image_out / image.name
shutil.copy2(image, target)
copied_images.append(target)
return audio_target, copied_images
def ensure_gsap_asset(hyper_dir: Path, version: str, skip_install: bool):
assets_dir = hyper_dir / "assets"
target = assets_dir / "gsap.min.js"
if target.exists():
return
if not skip_install:
result = subprocess.run(["npm", "install"], cwd=hyper_dir)
if result.returncode != 0:
raise SystemExit(result.returncode)
source = hyper_dir / "node_modules" / "gsap" / "dist" / "gsap.min.js"
if not source.exists():
print(f"Warning: gsap.min.js not found. Run `npm install gsap@{version} --save-exact` in {hyper_dir}.", file=sys.stderr)
return
shutil.copy2(source, target)
def ensure_font_assets(hyper_dir: Path, skip_install: bool):
assets_dir = hyper_dir / "assets"
font_dir = assets_dir / "fonts"
required = {
"anton-latin-400-normal.woff2": hyper_dir / "node_modules" / "@fontsource" / "anton" / "files" / "anton-latin-400-normal.woff2",
"space-grotesk-latin-400-normal.woff2": hyper_dir / "node_modules" / "@fontsource" / "space-grotesk" / "files" / "space-grotesk-latin-400-normal.woff2",
"space-grotesk-latin-700-normal.woff2": hyper_dir / "node_modules" / "@fontsource" / "space-grotesk" / "files" / "space-grotesk-latin-700-normal.woff2",
}
if all((font_dir / name).exists() for name in required):
return
if not skip_install and not all(source.exists() for source in required.values()):
result = subprocess.run(["npm", "install"], cwd=hyper_dir)
if result.returncode != 0:
raise SystemExit(result.returncode)
font_dir.mkdir(parents=True, exist_ok=True)
for name, source in required.items():
if source.exists():
shutil.copy2(source, font_dir / name)
else:
print(f"Warning: font asset not found: {source}", file=sys.stderr)
def build_data(project_dir: Path, resolved: dict, audio_target: Path, image_targets: list[Path], motion_profile: str) -> dict:
project = resolved["project"]
timeline = resolved["timeline"]
visual_config = resolved["visual_config"]
duration = float(timeline.get("audio_duration", 0) or 0)
if duration <= 0:
duration = 1.0
title = resolve_song_title(project_dir, resolved)
author = DEFAULT_SONG_AUTHOR
return {
"title": title,
"author": author,
"artist": author,
"titleKicker": author,
"duration": round(duration, 3),
"ratio": project.get("ratio", "16:9"),
"style": project.get("style", ""),
"audio": f"assets/{audio_target.name}",
"motionProfile": motion_profile,
"motion": MOTION_PROFILES[motion_profile],
"images": build_image_records(image_targets, timeline, visual_config),
"lyrics": build_lyrics(timeline),
}
def write_hyperframes_project(hyper_dir: Path, data: dict, gsap_version: str):
ratio = data.get("ratio", "16:9")
width, height = RATIO_SIZES.get(ratio, RATIO_SIZES["16:9"])
motion = data.get("motion", {})
duration = format_duration(float(data.get("duration", 1)))
title = html.escape(data.get("title", "Qiaomu MTV"))
display_title = html.escape(data.get("title", "Qiaomu MTV").upper())
title_kicker = str(data.get("titleKicker", "") or "").strip()
kicker_markup = f'<div class="kicker">{html.escape(title_kicker.upper())}</div>' if title_kicker else ""
show_waveform = "true" if motion.get("show_waveform") is True else "false"
show_progress = "true" if motion.get("show_progress") is True else "false"
show_pulse = "true" if motion.get("show_pulse") is True else "false"
html_document = INDEX_TEMPLATE.substitute(
width=width,
height=height,
duration=duration,
title=title,
display_title=display_title,
kicker_markup=kicker_markup,
show_waveform=show_waveform,
show_progress=show_progress,
show_pulse=show_pulse,
audio_src=html.escape(data.get("audio", "assets/song.mp3")),
accent=motion.get("accent", "#ff5638"),
warm=motion.get("warm", "#ffe3ad"),
)
write_text(hyper_dir / "index.html", compact_inline_scripts(html_document))
write_text(
hyper_dir / "assets" / "mtv-data.js",
"window.MTV_DATA = " + json.dumps(data, ensure_ascii=False, indent=2) + ";\n",
)
write_json(hyper_dir / "assets" / "mtv-data.json", data)
write_json(
hyper_dir / "hyperframes.json",
{
"$schema": "https://hyperframes.heygen.com/schema/hyperframes.json",
"registry": "https://raw.githubusercontent.com/heygen-com/hyperframes/main/registry",
"paths": {
"blocks": "compositions",
"components": "compositions/components",
"assets": "assets",
},
},
)
write_json(
hyper_dir / "meta.json",
{
"id": "qiaomu-hyper-mtv",
"name": "qiaomu-hyper-mtv",
"createdAt": datetime.now().isoformat(),
},
)
write_json(
hyper_dir / "package.json",
{
"name": clean_package_name(data.get("title", "qiaomu-hyper-mtv")),
"private": True,
"type": "module",
"scripts": {
"dev": f"npx --yes hyperframes@{HYPERFRAMES_VERSION} preview",
"check": f"npx --yes hyperframes@{HYPERFRAMES_VERSION} lint && npx --yes hyperframes@{HYPERFRAMES_VERSION} validate && npx --yes hyperframes@{HYPERFRAMES_VERSION} inspect",
"render": f"npx --yes hyperframes@{HYPERFRAMES_VERSION} render",
"publish": f"npx --yes hyperframes@{HYPERFRAMES_VERSION} publish",
},
"dependencies": {
"@fontsource/anton": "5.2.7",
"@fontsource/space-grotesk": "5.2.10",
"gsap": gsap_version,
},
},
)
write_text(
hyper_dir / "AGENTS.md",
"""# Qiaomu Hyper MTV
Run checks before rendering:
```bash
npm run check
python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/render_hyper_mtv.py --project . --profile review --check
```
GSAP rules:
- Register a paused timeline on `window.__timelines.main`.
- Keep all render motion deterministic: no `Date.now()`, `Math.random()`, timers, or network fetches.
- Prefer transform aliases and `autoAlpha`; keep repeats finite.
- Use the local `assets/gsap.min.js` file, not a CDN.
- Treat lyrics as designed typography, not fixed subtitles: use safe anchors, stable word reveals, and no visible jitter.
- Scene transitions should carry the song motif; avoid reducing the whole MV to one repeated crossfade.
""",
)
def main():
parser = argparse.ArgumentParser(description="Create a GSAP-rich HyperFrames MTV project.")
parser.add_argument("--project-dir", required=True, help="qiaomu MTV project directory with project.json and timeline.json")
parser.add_argument("--hyper-dir", help="Output HyperFrames project directory (default: project-dir/hyper-mv)")
parser.add_argument("--motion-profile", choices=sorted(MOTION_PROFILES), default="cinematic")
parser.add_argument("--gsap-version", default=DEFAULT_GSAP_VERSION)
parser.add_argument("--skip-npm-install", action="store_true", help="Do not run npm install; useful when assets/gsap.min.js already exists")
parser.add_argument("--force", action="store_true", help="Overwrite an existing hyper-dir")
args = parser.parse_args()
project_dir = Path(args.project_dir).expanduser().resolve()
hyper_dir = Path(args.hyper_dir).expanduser().resolve() if args.hyper_dir else project_dir / "hyper-mv"
if hyper_dir.exists() and args.force:
shutil.rmtree(hyper_dir)
elif hyper_dir.exists() and any(hyper_dir.iterdir()):
raise SystemExit(f"HyperFrames directory already exists. Use --force to overwrite: {hyper_dir}")
hyper_dir.mkdir(parents=True, exist_ok=True)
resolved = resolve_project(project_dir)
write_hyperframes_project(
hyper_dir,
{
"title": "placeholder",
"duration": float(resolved["timeline"].get("audio_duration", 1) or 1),
"ratio": resolved["project"].get("ratio", "16:9"),
"motionProfile": args.motion_profile,
"motion": MOTION_PROFILES[args.motion_profile],
"audio": "assets/song.mp3",
"images": [],
"lyrics": [],
},
args.gsap_version,
)
audio_target, image_targets = copy_project_assets(resolved, hyper_dir)
data = build_data(project_dir, resolved, audio_target, image_targets, args.motion_profile)
write_hyperframes_project(hyper_dir, data, args.gsap_version)
ensure_gsap_asset(hyper_dir, args.gsap_version, args.skip_npm_install)
ensure_font_assets(hyper_dir, args.skip_npm_install)
print("HyperFrames MTV project created")
print(f" project: {hyper_dir}")
print(f" profile: {args.motion_profile}")
print(f" images: {len(image_targets)}")
print(f" lyrics: {len(data.get('lyrics', []))}")
print("Next:")
print(f" cd {hyper_dir}")
print(" npm run check")
print(f" python /Users/joe/.agents/skills/qiaomu-mtv-creator/scripts/render_hyper_mtv.py --project {hyper_dir} --profile review --check")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
豆包歌词转录模块 v2
使用豆包录音文件识别模型2.0 (volc.seedasr.auc)
启用 enable_music_fc 提升音乐歌词识别准确度
"""
import json
import os
import requests
import time
import base64
import uuid
from pathlib import Path
from typing import Optional, Dict, Any, List
# API 配置 - 模型 2.0
BASE_URL = 'https://openspeech.bytedance.com/api/v3/auc/bigmodel'
APP_ID = '4039285844'
ACCESS_TOKEN = '4ZEZNpJBEMIHfUPYizD693UWMH_dT2N6'
RESOURCE_ID = 'volc.seedasr.auc'
def format_time_srt(milliseconds: int) -> str:
"""将毫秒转换为 SRT 时间格式: 00:00:00,000"""
seconds = milliseconds / 1000
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = int(seconds % 60)
millis = int(milliseconds % 1000)
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
def submit_audio_file(
audio_path: Path,
language: str = 'en-US',
) -> str:
"""
提交音频文件到豆包 API v2
Args:
audio_path: 音频文件路径
language: 语言代码 (en-US, zh-CN)
Returns:
task_id: 任务ID
"""
if not audio_path.exists():
raise FileNotFoundError(f"音频文件不存在: {audio_path}")
# 读取音频文件并 base64 编码
with open(audio_path, 'rb') as f:
audio_data = f.read()
audio_base64 = base64.b64encode(audio_data).decode('utf-8')
print(f" 📤 提交音频到豆包 v2: {audio_path.name} ({len(audio_data)/1024/1024:.2f} MB)")
# 生成请求 ID
request_id = str(uuid.uuid4())
# 构建请求体
payload = {
"user": {
"uid": "mtv_creator_user"
},
"audio": {
"data": audio_base64,
"format": audio_path.suffix.lstrip('.').lower(),
"rate": 16000,
"channel": 1
},
"request": {
"model_name": "bigmodel",
"enable_music_fc": True, # 关键:启用音乐功能,提升歌词识别
"enable_itn": True,
"enable_punc": True,
"language": language
}
}
# 发送请求
response = requests.post(
f'{BASE_URL}/submit',
json=payload,
headers={
'X-Api-App-Key': APP_ID,
'X-Api-Access-Key': ACCESS_TOKEN,
'X-Api-Resource-Id': RESOURCE_ID,
'X-Api-Request-Id': request_id,
'Content-Type': 'application/json'
},
timeout=60
)
if response.status_code != 200:
raise Exception(f"提交失败: HTTP {response.status_code}\n{response.text}")
# v2 API 的状态码在 HTTP Header 中
api_status = response.headers.get('X-Api-Status-Code')
api_message = response.headers.get('X-Api-Message')
if api_status != '20000000':
raise Exception(f"API 错误: {api_message} (code: {api_status})")
# 任务 ID 就是我们提交的 Request ID
task_id = request_id
print(f" ✅ 任务ID: {task_id}")
return task_id
def query_results(task_id: str, timeout: int = 300) -> Dict[str, Any]:
"""
查询识别结果(阻塞模式,自动轮询)
Args:
task_id: 任务ID (即提交时的 X-Api-Request-Id)
timeout: 超时时间(秒)
Returns:
识别结果
"""
print(f" ⏳ 等待豆包识别结果...")
start_time = time.time()
while True:
response = requests.post(
f'{BASE_URL}/query',
json={}, # v2 API query 的 body 为空
headers={
'X-Api-App-Key': APP_ID,
'X-Api-Access-Key': ACCESS_TOKEN,
'X-Api-Resource-Id': RESOURCE_ID,
'X-Api-Request-Id': task_id, # 使用提交时的 request_id
'Content-Type': 'application/json'
},
timeout=60
)
if response.status_code != 200:
raise Exception(f"查询失败: HTTP {response.status_code}\n{response.text}")
# v2 API 的状态码在 HTTP Header 中
api_status = response.headers.get('X-Api-Status-Code')
api_message = response.headers.get('X-Api-Message')
result = response.json() if response.text and response.text != '{}' else {}
if api_status == '20000000':
# 成功完成
elapsed = time.time() - start_time
print(f" ✅ 识别完成! 耗时 {elapsed:.1f}s")
return result
elif api_status == '20000001':
# 处理中
elapsed = time.time() - start_time
if elapsed > timeout:
raise TimeoutError(f"识别超时 ({timeout}s)")
print(f" ⏳ 处理中... ({elapsed:.0f}s)")
time.sleep(2)
else:
raise Exception(f"识别失败: {api_message} (code: {api_status})")
def generate_srt(result: Dict[str, Any], output_path: Path) -> int:
"""
从豆包 v2 结果生成 SRT 字幕文件
Args:
result: 豆包 API 返回结果
output_path: 输出 SRT 文件路径
Returns:
字幕段落数量
"""
def is_music_tag(text):
"""检查是否为音乐标记"""
stripped = text.strip()
return stripped.startswith('[') and stripped.endswith(']')
def is_suffix_of_previous(current: str, previous: str) -> bool:
"""检查当前文本是否为前一句的后缀(重复尾部)"""
if not previous:
return False
current = current.strip().lower()
previous = previous.strip().lower()
if len(current) < len(previous) * 0.5:
if previous.endswith(current):
return True
return False
# 从 v2 API 响应中提取 utterances
utterances = result.get('result', {}).get('utterances', [])
with open(output_path, 'w', encoding='utf-8') as f:
index = 1
previous_text = ""
for utt in utterances:
text = utt.get('text', '')
# 跳过音乐标记
if is_music_tag(text):
continue
# 跳过重复尾部
if is_suffix_of_previous(text, previous_text):
continue
# v2 API 的时间戳字段
start_time = format_time_srt(utt.get('start_time', 0))
end_time = format_time_srt(utt.get('end_time', 0))
f.write(f"{index}\n")
f.write(f"{start_time} --> {end_time}\n")
f.write(f"{text}\n\n")
previous_text = text
index += 1
print(f" 💾 字幕已保存: {output_path}")
return index - 1
def correct_lyrics_with_original(
result: Dict[str, Any],
original_lyrics: str
) -> Dict[str, Any]:
"""
用原始歌词校正豆包识别结果(保留时间戳)
Args:
result: 豆包 API 返回结果
original_lyrics: 原始歌词文本
Returns:
校正后的结果
"""
from difflib import SequenceMatcher
def is_music_tag(line):
"""检查是否为音乐标记:[Verse]、[Chorus]、[Bridge] 等"""
stripped = line.strip()
return stripped.startswith('[') and stripped.endswith(']')
utterances = result.get('result', {}).get('utterances', [])
# 过滤掉音乐标记行
original_lines = [
line.strip()
for line in original_lyrics.strip().split('\n')
if line.strip() and not is_music_tag(line)
]
print(f" 🔍 用原始歌词校正识别结果...")
print(f" 识别段落: {len(utterances)}, 原始歌词行: {len(original_lines)}")
corrected_utterances = []
last_matched_time = 0
for line_idx, original_line in enumerate(original_lines):
best_score = 0
best_utt = None
# 只在 last_matched_time 之后的 utterances 中寻找
for utt in utterances:
text = utt.get('text', '')
if is_music_tag(text):
continue
# 关键约束:只考虑时间更晚的片段
if utt.get('start_time', 0) < last_matched_time:
continue
score = SequenceMatcher(None, text.lower(), original_line.lower()).ratio()
if score > best_score:
best_score = score
best_utt = utt
# 只保留相似度 > 0.5 的
if best_score > 0.5 and best_utt:
corrected_utt = best_utt.copy()
corrected_utt['text'] = original_line # 使用原始歌词文本
corrected_utterances.append(corrected_utt)
last_matched_time = best_utt.get('start_time', 0)
result['result']['utterances'] = corrected_utterances
print(f" ✅ 校正完成,保留 {len(corrected_utterances)} / {len(original_lines)} 行")
return result
def transcribe_audio(
audio_path: Path,
output_dir: Path,
language: str = 'en-US',
original_lyrics: Optional[str] = None
) -> Path:
"""
完整的音频转录流程(使用 v2 API)
Args:
audio_path: 音频文件路径
output_dir: 输出目录
language: 语言代码
original_lyrics: 原始歌词(用于校正)
Returns:
生成的 SRT 文件路径
"""
output_dir.mkdir(parents=True, exist_ok=True)
# 检测语言
if original_lyrics:
has_chinese = any('\u4e00' <= c <= '\u9fff' for c in original_lyrics)
language = 'zh-CN' if has_chinese else 'en-US'
print(f" 🌍 自动检测语言: {language}")
# 提交识别任务
task_id = submit_audio_file(
audio_path=audio_path,
language=language
)
# 查询结果
result = query_results(task_id, timeout=300)
# 如果有原始歌词,进行校正
if original_lyrics:
result = correct_lyrics_with_original(result, original_lyrics)
# 生成 SRT 文件
srt_path = output_dir / f"{audio_path.stem}.srt"
num_segments = generate_srt(result, srt_path)
# 保存完整 JSON 结果(用于调试)
json_path = output_dir / f"{audio_path.stem}_doubao_v2.json"
with open(json_path, 'w', encoding='utf-8') as f:
json.dump(result, f, ensure_ascii=False, indent=2)
print(f" 📊 识别段落: {num_segments}")
return srt_path
def main():
"""命令行测试入口"""
import argparse
parser = argparse.ArgumentParser(description="豆包音乐歌词转录 v2")
parser.add_argument("audio", help="音频文件路径")
parser.add_argument("--output", "-o", default=".", help="输出目录")
parser.add_argument("--language", "-l", default="en-US", help="语言代码")
parser.add_argument("--lyrics", help="原始歌词文件(用于校正)")
args = parser.parse_args()
audio_path = Path(args.audio).expanduser()
output_dir = Path(args.output).expanduser()
# 读取原始歌词
original_lyrics = None
if args.lyrics:
lyrics_path = Path(args.lyrics).expanduser()
if lyrics_path.suffix == '.json':
with open(lyrics_path, 'r', encoding='utf-8') as f:
data = json.load(f)
original_lyrics = data.get('prompt', '')
else:
with open(lyrics_path, 'r', encoding='utf-8') as f:
original_lyrics = f.read()
# 转录
srt_path = transcribe_audio(
audio_path=audio_path,
output_dir=output_dir,
language=args.language,
original_lyrics=original_lyrics
)
print(f"\n✅ 完成! SRT 文件: {srt_path}")
if __name__ == '__main__':
main()
#!/usr/bin/env python3
"""
豆包歌词转录模块 v3
优化版:启用 enable_channel_split、show_speech_rate
实现单词级时间轴,交给大模型智能生成SRT
"""
import json
import os
import requests
import time
import base64
import uuid
from pathlib import Path
from typing import Optional, Dict, Any, List
# API 配置 - 模型 2.0
BASE_URL = 'https://openspeech.bytedance.com/api/v3/auc/bigmodel'
APP_ID = '4039285844'
ACCESS_TOKEN = '4ZEZNpJBEMIHfUPYizD693UWMH_dT2N6'
RESOURCE_ID = 'volc.seedasr.auc'
def format_time_srt(milliseconds: int) -> str:
"""将毫秒转换为 SRT 时间格式: 00:00:00,000"""
seconds = milliseconds / 1000
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = int(seconds % 60)
millis = int(milliseconds % 1000)
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
def submit_audio_file(
audio_path: Path,
language: str = 'en-US',
) -> str:
"""
提交音频文件到豆包 API v3(优化版)
Args:
audio_path: 音频文件路径
language: 语言代码 (en-US, zh-CN)
Returns:
task_id: 任务ID
"""
if not audio_path.exists():
raise FileNotFoundError(f"音频文件不存在: {audio_path}")
# 读取音频文件并 base64 编码
with open(audio_path, 'rb') as f:
audio_data = f.read()
audio_base64 = base64.b64encode(audio_data).decode('utf-8')
print(f" 📤 提交音频到豆包 v3: {audio_path.name} ({len(audio_data)/1024/1024:.2f} MB)")
# 生成请求 ID
request_id = str(uuid.uuid4())
# 构建请求体(v3 优化版)
payload = {
"user": {
"uid": "mtv_creator_user"
},
"audio": {
"data": audio_base64,
"format": audio_path.suffix.lstrip('.').lower(),
"rate": 16000,
"channel": 1
},
"request": {
"model_name": "bigmodel",
"enable_music_fc": True, # 启用音乐功能
"enable_itn": True, # 启用逆文本标准化
"enable_punc": True, # 启用标点
"enable_channel_split": True, # 🆕 启用声道分离
"show_speech_rate": True, # 🆕 显示语速
"words_per_line": 10, # 🆕 每行词数(控制粒度)
"language": language
}
}
# 发送请求
response = requests.post(
f'{BASE_URL}/submit',
json=payload,
headers={
'X-Api-App-Key': APP_ID,
'X-Api-Access-Key': ACCESS_TOKEN,
'X-Api-Resource-Id': RESOURCE_ID,
'X-Api-Request-Id': request_id,
'Content-Type': 'application/json'
},
timeout=60
)
if response.status_code != 200:
raise Exception(f"提交失败: HTTP {response.status_code}\n{response.text}")
# v2 API 的状态码在 HTTP Header 中
api_status = response.headers.get('X-Api-Status-Code')
api_message = response.headers.get('X-Api-Message')
if api_status != '20000000':
raise Exception(f"API 错误: {api_message} (code: {api_status})")
task_id = request_id
print(f" ✅ 任务ID: {task_id}")
return task_id
def query_results(task_id: str, timeout: int = 300) -> Dict[str, Any]:
"""
查询识别结果(阻塞模式,自动轮询)
Args:
task_id: 任务ID (即提交时的 X-Api-Request-Id)
timeout: 超时时间(秒)
Returns:
识别结果(包含 word-level timestamps)
"""
print(f" ⏳ 等待豆包识别结果...")
start_time = time.time()
while True:
response = requests.post(
f'{BASE_URL}/query',
json={},
headers={
'X-Api-App-Key': APP_ID,
'X-Api-Access-Key': ACCESS_TOKEN,
'X-Api-Resource-Id': RESOURCE_ID,
'X-Api-Request-Id': task_id,
'Content-Type': 'application/json'
},
timeout=60
)
if response.status_code != 200:
raise Exception(f"查询失败: HTTP {response.status_code}\n{response.text}")
api_status = response.headers.get('X-Api-Status-Code')
api_message = response.headers.get('X-Api-Message')
result = response.json() if response.text and response.text != '{}' else {}
if api_status == '20000000':
elapsed = time.time() - start_time
print(f" ✅ 识别完成! 耗时 {elapsed:.1f}s")
return result
elif api_status == '20000001':
elapsed = time.time() - start_time
if elapsed > timeout:
raise TimeoutError(f"识别超时 ({timeout}s)")
print(f" ⏳ 处理中... ({elapsed:.0f}s)")
time.sleep(2)
else:
raise Exception(f"识别失败: {api_message} (code: {api_status})")
def extract_word_level_timestamps(result: Dict[str, Any]) -> List[Dict[str, Any]]:
"""
从豆包结果提取单词级时间戳
Args:
result: 豆包 API 返回结果
Returns:
单词列表,每个包含 {word, start_time, end_time}
"""
utterances = result.get('result', {}).get('utterances', [])
all_words = []
for utt in utterances:
text = utt.get('text', '').strip()
# 跳过音乐标记
if text.startswith('[') and text.endswith(']'):
continue
# 提取 words 字段(豆包v2可能返回)
words = utt.get('words', [])
if words:
# 如果有word-level timestamps
for word_info in words:
all_words.append({
'word': word_info.get('text', ''),
'start_time': word_info.get('start_time', 0),
'end_time': word_info.get('end_time', 0)
})
else:
# 如果没有words,按空格分割并平均分配时间
start = utt.get('start_time', 0)
end = utt.get('end_time', 0)
duration = end - start
word_list = text.split()
word_count = len(word_list)
if word_count > 0:
time_per_word = duration / word_count
for i, word in enumerate(word_list):
word_start = start + i * time_per_word
word_end = start + (i + 1) * time_per_word
all_words.append({
'word': word,
'start_time': int(word_start),
'end_time': int(word_end)
})
return all_words
def build_llm_prompt_for_srt(
words: List[Dict[str, Any]],
original_lyrics: Optional[str] = None
) -> str:
"""
构建给LLM的提示词,让其智能生成SRT
Args:
words: 单词级时间戳列表
original_lyrics: 原始歌词(可选)
Returns:
LLM提示词
"""
words_json = json.dumps(words[:50], indent=2, ensure_ascii=False) # 只显示前50个词作为示例
prompt = f"""你是一个专业的字幕时间轴编辑师。我有一首歌的单词级时间戳数据,需要你帮我生成精准的SRT字幕文件。
**单词级时间戳数据**(共{len(words)}个词):
```json
{words_json}
... (还有 {len(words)-50} 个词)
```
**时间戳说明**:
- start_time/end_time 单位为毫秒
- 每个词都有精确的开始和结束时间
"""
if original_lyrics:
prompt += f"""
**原始歌词**(ground truth):
```
{original_lyrics}
```
**任务要求**:
1. 对比单词时间戳和原始歌词,智能合并成合理的字幕行
2. 每行字幕长度适中(5-15个词,或1-2句歌词)
3. 使用原始歌词的准确文本(而不是识别的词)
4. 保留所有实际歌词行,跳过标记行(如[Verse 1]、[Chorus]等)
5. 字幕行的时间边界应该对齐到单词的时间戳
"""
else:
prompt += f"""
**任务要求**:
1. 智能合并单词成合理的字幕行
2. 每行字幕长度适中(5-15个词)
3. 考虑语义完整性(不要在句子中间断行)
4. 保持时间精准度
"""
prompt += """**输出格式**(JSON数组):
```json
[
{
"start_time": 30000,
"end_time": 36720,
"text": "The tide is rising, can you feel it too?"
},
{
"start_time": 36720,
"end_time": 43440,
"text": "The future's golden, shining bright and new"
}
]
```
**规则**:
- start_time/end_time 必须为整数(毫秒)
- text 为完整的歌词行
- 时间边界精准对齐到词的边界
- 相邻字幕之间可以有间隙,但不要重叠
请直接输出JSON数组,不要其他文字。"""
return prompt
def transcribe_audio(
audio_path: Path,
output_dir: Path,
language: str = 'en-US',
original_lyrics: Optional[str] = None
) -> Path:
"""
完整的音频转录流程(使用 v3 优化版)
Args:
audio_path: 音频文件路径
output_dir: 输出目录
language: 语言代码
original_lyrics: 原始歌词(用于校正)
Returns:
生成的文件路径(JSON格式,需要LLM处理)
"""
output_dir.mkdir(parents=True, exist_ok=True)
# 检测语言
if original_lyrics:
has_chinese = any('\u4e00' <= c <= '\u9fff' for c in original_lyrics)
language = 'zh-CN' if has_chinese else 'en-US'
print(f" 🌍 自动检测语言: {language}")
# 提交识别任务
task_id = submit_audio_file(
audio_path=audio_path,
language=language
)
# 查询结果
result = query_results(task_id, timeout=300)
# 提取单词级时间戳
words = extract_word_level_timestamps(result)
print(f" 📊 提取到 {len(words)} 个词的时间戳")
# 保存完整结果(包含words)
json_path = output_dir / f"{audio_path.stem}_doubao_v3_words.json"
with open(json_path, 'w', encoding='utf-8') as f:
json.dump({
'words': words,
'original_result': result
}, f, ensure_ascii=False, indent=2)
print(f" 💾 单词时间戳已保存: {json_path}")
# 生成LLM提示词
llm_prompt = build_llm_prompt_for_srt(words, original_lyrics)
prompt_path = output_dir / f"{audio_path.stem}_llm_prompt.txt"
with open(prompt_path, 'w', encoding='utf-8') as f:
f.write(llm_prompt)
print(f" 📝 LLM提示词已保存: {prompt_path}")
print("\n" + "="*70)
print("🤖 接下来请将提示词发送给Claude,获取智能生成的SRT字幕")
print("="*70)
return json_path
def generate_srt_from_llm_output(
llm_json: str,
output_path: Path
) -> int:
"""
从LLM生成的JSON创建SRT文件
Args:
llm_json: LLM返回的JSON字符串
output_path: 输出SRT文件路径
Returns:
字幕段落数量
"""
subtitles = json.loads(llm_json)
with open(output_path, 'w', encoding='utf-8') as f:
for i, sub in enumerate(subtitles, 1):
start_time = format_time_srt(sub['start_time'])
end_time = format_time_srt(sub['end_time'])
text = sub['text']
f.write(f"{i}\n")
f.write(f"{start_time} --> {end_time}\n")
f.write(f"{text}\n\n")
print(f" 💾 SRT字幕已保存: {output_path}")
return len(subtitles)
def main():
"""命令行测试入口"""
import argparse
parser = argparse.ArgumentParser(description="豆包音乐歌词转录 v3(优化版)")
parser.add_argument("audio", help="音频文件路径")
parser.add_argument("--output", "-o", default=".", help="输出目录")
parser.add_argument("--language", "-l", default="en-US", help="语言代码")
parser.add_argument("--lyrics", help="原始歌词文件(用于校正)")
args = parser.parse_args()
audio_path = Path(args.audio).expanduser()
output_dir = Path(args.output).expanduser()
# 读取原始歌词
original_lyrics = None
if args.lyrics:
lyrics_path = Path(args.lyrics).expanduser()
if lyrics_path.suffix == '.json':
with open(lyrics_path, 'r', encoding='utf-8') as f:
data = json.load(f)
original_lyrics = data.get('prompt', '')
else:
with open(lyrics_path, 'r', encoding='utf-8') as f:
original_lyrics = f.read()
# 转录(生成words JSON + LLM prompt)
json_path = transcribe_audio(
audio_path=audio_path,
output_dir=output_dir,
language=args.language,
original_lyrics=original_lyrics
)
print(f"\n✅ 完成! 单词时间戳: {json_path}")
print("\n下一步:")
print("1. 查看生成的 _llm_prompt.txt 文件")
print("2. 将提示词发送给Claude")
print("3. 用Claude返回的JSON调用 generate_srt_from_llm_output() 生成SRT")
if __name__ == '__main__':
main()
#!/usr/bin/env python3
"""Render a HyperFrames MTV project with memory-aware worker selection."""
import argparse
import os
import re
import glob
import subprocess
import sys
from pathlib import Path
HYPERFRAMES_VERSION = "0.6.60"
DEFAULT_MIN_FREE_GB = 4.0
def run(cmd, cwd: Path = None, check: bool = False):
result = subprocess.run(cmd, cwd=cwd, text=True, capture_output=True)
if check and result.returncode != 0:
if result.stdout:
print(result.stdout)
if result.stderr:
print(result.stderr, file=sys.stderr)
raise SystemExit(result.returncode)
return result
def parse_gb(value: str) -> float:
try:
return float(value)
except (TypeError, ValueError):
return 0.0
def mac_memory_info() -> dict:
"""Return immediate and reclaimable memory on macOS without extra deps."""
total = 0.0
total_result = run(["sysctl", "-n", "hw.memsize"])
if total_result.returncode == 0:
total = int(total_result.stdout.strip()) / (1024 ** 3)
vm_result = run(["vm_stat"])
if vm_result.returncode != 0:
return {"total_gb": total, "free_gb": 0.0, "available_gb": 0.0}
first = vm_result.stdout.splitlines()[0] if vm_result.stdout else ""
page_match = re.search(r"page size of (\d+) bytes", first)
page_size = int(page_match.group(1)) if page_match else 4096
pages = {}
for line in vm_result.stdout.splitlines()[1:]:
match = re.match(r"([^:]+):\s+([0-9,]+)\.", line.strip())
if match:
pages[match.group(1).strip()] = int(match.group(2).replace(",", ""))
free_pages = pages.get("Pages free", 0) + pages.get("Pages speculative", 0)
reclaimable_pages = free_pages + pages.get("Pages inactive", 0)
return {
"total_gb": total,
"free_gb": free_pages * page_size / (1024 ** 3),
"available_gb": reclaimable_pages * page_size / (1024 ** 3),
}
def find_headless_shell() -> str:
configured = os.environ.get("PRODUCER_HEADLESS_SHELL_PATH")
if configured and Path(configured).exists():
return configured
patterns = [
str(Path.home() / ".cache/puppeteer/chrome-headless-shell/*/chrome-headless-shell-*/chrome-headless-shell"),
str(Path.home() / ".cache/puppeteer/chrome/*/chrome-*/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing"),
]
candidates = []
for pattern in patterns:
candidates.extend(glob.glob(pattern))
existing = [path for path in candidates if Path(path).exists()]
return sorted(existing)[-1] if existing else ""
def choose_render_settings(profile: str, requested_workers: str, memory: dict, allow_low_memory: bool) -> dict:
profiles = {
"review": {"quality": "draft", "fps": 24},
"final": {"quality": "standard", "fps": 30},
"high": {"quality": "high", "fps": 30},
}
settings = profiles[profile].copy()
free_gb = memory["free_gb"]
usable_gb = max(memory["free_gb"], memory["available_gb"] * 0.65)
if requested_workers != "auto":
settings["workers"] = int(requested_workers)
return settings
if usable_gb >= 24:
workers = 6
elif usable_gb >= 16:
workers = 4
elif usable_gb >= 10:
workers = 3
elif usable_gb >= 6:
workers = 2
else:
workers = 1
workers = min(workers, {"review": 4, "final": 3, "high": 2}[profile])
if profile in ("final", "high") and memory["available_gb"] < DEFAULT_MIN_FREE_GB and not allow_low_memory:
settings["workers"] = workers
settings["blocked_by_memory"] = True
return settings
settings["workers"] = workers
settings["usable_gb"] = usable_gb
return settings
def default_output(project_dir: Path, profile: str) -> Path:
parent = project_dir.parent
base_name = parent.name or project_dir.name
variant = project_dir.name
if variant == "hyper-mv":
suffix = "hyper-review" if profile == "review" else "hyper"
else:
for prefix in ("hyper-mv-", "hyper-"):
if variant.startswith(prefix):
variant = variant[len(prefix):]
break
suffix = f"{variant}-review" if profile == "review" else variant
return parent / f"{base_name}-{suffix}.mp4"
def main():
parser = argparse.ArgumentParser(
description="Memory-aware HyperFrames renderer for qiaomu MTV projects."
)
parser.add_argument("--project", default=".", help="HyperFrames project directory, e.g. ./hyper-mv")
parser.add_argument("--output", help="Output MP4 path")
parser.add_argument("--profile", choices=["review", "final", "high"], default="final")
parser.add_argument("--workers", default="auto", help="auto or an explicit worker count")
parser.add_argument("--min-free-gb", type=float, default=DEFAULT_MIN_FREE_GB)
parser.add_argument("--allow-low-memory", action="store_true", help="Render anyway with 1 worker when memory is low")
parser.add_argument("--check", action="store_true", help="Run npm run check before rendering")
parser.add_argument("--no-sdr", action="store_true", help="Do not pass HyperFrames --sdr")
parser.add_argument("--quiet", action="store_true", help="Pass HyperFrames --quiet")
parser.add_argument("--dry-run", action="store_true", help="Print the render command without running it")
args = parser.parse_args()
project_dir = Path(args.project).expanduser().resolve()
if not (project_dir / "index.html").exists():
raise SystemExit(f"Not a HyperFrames project: {project_dir}")
memory = mac_memory_info()
settings = choose_render_settings(args.profile, args.workers, memory, args.allow_low_memory)
output_path = Path(args.output).expanduser().resolve() if args.output else default_output(project_dir, args.profile)
print("HyperFrames render preflight")
print(f" project: {project_dir}")
print(f" output: {output_path}")
print(f" memory: {memory['free_gb']:.1f} GB immediate free, {memory['available_gb']:.1f} GB reclaimable, {memory['total_gb']:.1f} GB total")
if settings.get("blocked_by_memory") and memory["available_gb"] < args.min_free_gb:
print()
print(f"LOW MEMORY: final render wants at least {args.min_free_gb:.1f} GB reclaimable memory.")
print("Ask the user to close heavy apps or clear memory, then rerun this command.")
print("If they accept a slower render, rerun with --allow-low-memory; it will use --workers 1.")
raise SystemExit(75)
if args.check:
run(["npm", "run", "check"], cwd=project_dir, check=True)
cmd = [
"npx",
"--yes",
f"hyperframes@{HYPERFRAMES_VERSION}",
"render",
"--output",
str(output_path),
"--quality",
settings["quality"],
"--fps",
str(settings["fps"]),
"--workers",
str(settings["workers"]),
]
if not args.no_sdr:
cmd.append("--sdr")
if args.quiet:
cmd.append("--quiet")
print(f" profile: {args.profile}")
print(f" command: {' '.join(cmd)}")
if args.dry_run:
return
env = os.environ.copy()
env.setdefault("NO_COLOR", "1")
headless_shell = find_headless_shell()
if headless_shell:
env.setdefault("PRODUCER_HEADLESS_SHELL_PATH", headless_shell)
print(f" browser: {env['PRODUCER_HEADLESS_SHELL_PATH']}")
env.setdefault("PRODUCER_ENABLE_STREAMING_ENCODE", "true")
env.setdefault("PRODUCER_STREAMING_ENCODE_MAX_DURATION_SECONDS", "9999")
env.setdefault("PRODUCER_FRAME_DATA_URI_CACHE_LIMIT", "32")
env.setdefault("PRODUCER_FRAME_DATA_URI_CACHE_BYTES_MB", "128")
env.setdefault("PRODUCER_PUPPETEER_PROTOCOL_TIMEOUT_MS", "900000")
env.setdefault("PRODUCER_PUPPETEER_LAUNCH_TIMEOUT_MS", "300000")
completed = subprocess.run(cmd, cwd=project_dir, env=env)
raise SystemExit(completed.returncode)
if __name__ == "__main__":
main()