Framevideo Voiceover Ssml

FrameVideo Voiceover SSML

Use this skill when the voice track should come from project subtitles, transcript words, or AI analysis text instead of a hand-written script.

Workflow

1. Find the source text in this order:

• project subtitles / transcript
• AI analysis / extracted scene copy
• existing narration draft

2. Turn that source into a speakable voice script. 3. Keep SSML-style markup in the authoring layer:

• phoneme for pronunciation fixes
• break for pauses
• ttnumber for custom number reading

4. Send the marked script to a TTS provider only if it supports the tags. 5. Save both:

• the marked source script
• the provider-safe fallback text when needed

6. Write the generated audio back into the project voiceover flow, then sync transcript/captions from the result.

Use the right source

• Prefer project subtitles or transcript first.
• If subtitles are too terse, expand them with the AI analysis text that already exists for the project.
• Keep the spoken wording close to the visible content unless the user asks for a stronger ad-style rewrite.

Markup rules

• Use phoneme only for words that TTS misreads.
• Use break for intentional pauses, not every comma.
• Use ttnumber for product numbers, dates, pricing, or counts that should be spoken differently from how they appear.
• Keep the original text readable; markup should clarify delivery, not obscure meaning.

Provider behavior

• If the TTS backend supports SSML-like tags, pass them through unchanged.
• If the backend does not support them, strip the tags into a clean fallback script before generation.
• Keep the original marked script next to the fallback so the audio can be regenerated later.

Project writeback

• Treat generated narration as a project asset, not a one-off export.
• Reuse the project's existing voiceover, transcript, and caption paths.
• Keep the access-token header contract used by the Chanjing voice APIs.

Read these references

• references/workflow.md
• references/ssml.md

SSML Notes

Supported tags

phoneme

Use for pronunciation correction.

Example:

<phoneme alphabet="py" ph="xi1">茜</phoneme>

break

Use for deliberate pauses.

Example:

<break time="0.5s"/>

ttnumber

Use for custom number reading.

Example:

<ttnumber pronounce="一九五五">1955</ttnumber>

Rules

• Keep the child text plain and short.
• Use markup sparingly.
• Preserve the source script alongside the generated fallback text.
• If the provider cannot read these tags, strip them before synthesis.

Voiceover Workflow

Goal

Turn project subtitles or AI analysis into a narration script, generate audio, then feed the result back into transcript and caption assets.

Recommended flow

1. Locate the text source. 2. Normalize it into a narration draft. 3. Add SSML-style markup only where it improves delivery. 4. Decide whether the current TTS provider can preserve the markup. 5. Generate audio. 6. Save the marked script, the fallback script if needed, and the final audio. 7. Regenerate transcript/captions from the audio and keep them aligned with the project content.

Source priority

• Project subtitles and transcript are the primary source.
• AI analysis copy is the next fallback when subtitles are too thin.
• Manual narration is the last resort.

Fallback rule

When SSML is not supported by the provider:

• remove markup
• keep the spoken text natural
• preserve the original marked version for future regeneration

Writeback rule

Write generated assets back into the same project structure used by the existing voice workflow so later steps can find them without extra wiring.