Stepfun Tts

StepFun stepaudio-2.5-tts

Generate Chinese / Japanese speech with stepaudio-2.5-tts (released 2026-04, verified 2026-04-23). Contextual TTS — emotion and prosody go through natural-language description, not fixed labels.

Companion: for transcription with stepaudio-2.5-asr (the sibling model), use the stepfun-asr skill — they share an API key but live on different endpoints with different body shapes.

Why this skill exists — StepAudio 2.5 has two non-obvious pitfalls that cost hours if you don't know them:

1. stepaudio-2.5-tts rejects voice_label (the step-tts-2 way). Emotion/prosody now goes through instruction (natural-language description, ≤200 chars) and inline () parentheses inside the text itself. 2. Censorship is stricter — anything containing 死 / 消失 / sensitive political terms returns censorship_block. Your rewrite options are in references/migration_from_v2.md.

Config and auth

API key lives in $STEPFUN_API_KEY (preferred) or ${CLAUDE_PLUGIN_DATA}/config.json (fallback for cross-session persistence). All bundled scripts try env first, then config.

First-time setup (one-liner):

mkdir -p "${CLAUDE_PLUGIN_DATA}" && cat > "${CLAUDE_PLUGIN_DATA}/config.json" <<EOF
{"api_key": "<paste key here>"}
EOF

If the user hasn't set a key, ask them to paste it (don't guess / don't use a placeholder). StepFun API keys are available at https://platform.stepfun.com/ → API Keys. Use a Normal key, not a Plan key (Plan keys are restricted to text models and silently fail on audio endpoints).

Common tasks — decision tree

Starting points

• Synthesize a single line: Run python3 scripts/tts_generate.py --text "你好" --out /tmp/hello.mp3 --instruction "温暖的希望感". For fine-grained control read the "Contextual TTS" section below.
• A full migration from step-tts-2 → stepaudio-2.5-tts: read references/migration_from_v2.md end-to-end before touching code. It has the INSTRUCTION_MAP, the SKIP_CENSORED list pattern, and the output-directory-strategy for non-destructive A/B.

Contextual TTS — beyond emotion labels

The headline feature of stepaudio-2.5-tts is that you stop mapping emotions to fixed tags and start describing what you want in natural language. Two layers:

Global context (`instruction` parameter) — sets the overall tone for the entire utterance. ≤200 chars. Think of it like giving stage direction to a voice actor.

instruction: "克制的悲伤，语气低沉柔弱，像快要消失一样"

Inline context (`()` parentheses inside `input`) —句内 directives. Parenthesised content is consumed as directions and is NOT read aloud. Use for precise control of pauses, breath, emphasis, or mid-sentence emotion shifts.

input: "(试探着问)你好吗？(开心地)太好了！(突然沉下来)不过...我快要消失了。"

Examples that worked in practice (from 2026-04-23 verification):

• instruction: "活泼俏皮，像是在撒娇，带点嘴硬" — visibly speeds up delivery vs neutral
• instruction: "耳语声，气声很重，几乎听不清" — produces audible whisper/breath
• input: "你好(停顿一下)我是蕾格(轻声)今天(加重)的天气真不错。" — inline directives all respected

What `stepaudio-2.5-tts` will NOT accept — voice_label parameter. Error: voice_label is not supported for v2 models. This is the #1 migration gotcha from step-tts-2.

Common error patterns (real errors, real fixes)

More in references/known_issues.md.

When to read references

• references/api_reference.md — exact request/response JSON for /v1/audio/speech, all fields, error responses. Read when writing raw HTTP calls instead of using the bundled scripts.
• references/migration_from_v2.md — complete playbook for moving a step-tts-2 project to stepaudio-2.5-tts. Has the emotion→instruction rewrite table, the A/B directory strategy, decision checkpoints, and the 2026-04 speed/quality trade-off data (stepaudio-2.5-tts is ~20% slower than step-tts-2; audible prosody improvement). Read before any migration work.
• references/known_issues.md — censorship patterns, TTS duration inflation, v2-family parameter naming gotcha, 1000-char hard cap. Read when debugging anomalous output or evaluating whether to adopt.

Design invariants (don't break these)

1. Non-destructive A/B output — when regenerating a corpus with a new model, write to a parallel directory (voice/zh_v25/), never overwrite the production corpus. The migration playbook shows why. 2. Per-line censorship handling — if 2/29 lines get censorship_block, don't fail the batch. Log the skipped IDs, continue. Mixed-model fallback (step-tts-2 for the skipped 2) is normal. 3. Don't duplicate voice_label logic in new code — any new TTS code targeting stepaudio-2.5-tts should only use instruction + inline (). Do not write a branch that conditionally emits voice_label.

Pricing (verified 2026-04-23, volatile)

• stepaudio-2.5-tts contextual synthesis: ~5.8 元 / 万字符
• Zero-shot voice cloning: ~9.9 元 / 音色

Re-verify at https://platform.stepfun.com/docs/zh/guides/pricing/details before quoting to stakeholders.

stepaudio-2.5-tts API Reference

Exact request/response shapes for stepaudio-2.5-tts. Verified 2026-04-23 against the live StepFun API. Read this when you need to call the API by hand (curl, custom HTTP client) instead of using the bundled scripts/tts_generate.py.

Endpoint

POST https://api.stepfun.com/v1/audio/speech
Content-Type: application/json
Authorization: Bearer <STEPFUN_API_KEY>

Request body

{
  "model": "stepaudio-2.5-tts",
  "input": "你好，我是蕾格。",
  "voice": "shuangkuaijiejie",
  "response_format": "mp3",
  "speed": 1.0,
  "volume": 1.0,
  "instruction": "克制的悲伤，语气低沉柔弱"
}

Inline directives inside input

Parentheses () in the input are consumed as TTS control signals, not pronounced. Examples that work:

• (停顿一下) — insert a pause
• (轻声) — reduce volume / breathy
• (加重) — stress the following word
• (试探着问) — apply a tone shift mid-sentence
• (突然沉下来) — emotion pivot

You can mix instruction (global tone) with inline () (per-phrase micro-control):

{
  "instruction": "富有情绪弧线的独白",
  "input": "(试探着问)你好吗？(开心地)太好了！(突然沉下来)不过...我快要消失了。"
}

Response

On success: binary audio stream in the requested response_format. HTTP 200. No JSON wrapper. Save the body directly as .mp3/.wav/.opus.

Known error responses

{"error":{"message":"voice_label is not supported for v2 models","type":"request_params_invalid"}}

→ Remove voice_label, use instruction instead.

{"error":{"message":"The content you provided or machine outputted is blocked.","type":"censorship_block"}}

→ Content triggered censorship. Common triggers: 死, 消失, politically sensitive terms. See known_issues.md.

Comparison with sibling and legacy endpoints

Legacy step-tts-2 still works. It's the baseline in migration_from_v2.md and the per-line fallback when stepaudio-2.5-tts hits censorship_block.

Auth and key handling

• Key header: Authorization: Bearer <key>
• Keys can be retrieved at https://platform.stepfun.com/ → API Keys
• "Plan" keys (cheaper subscription) are restricted to text models on api.stepfun.com/step_plan. They cannot call audio endpoints. Use a "Normal" key for all TTS calls.
• Same key works for both TTS and ASR — no separate scopes

Rate / throughput notes (observed, not officially documented)

• ~400ms sleep between batch requests avoids 429s in practice
• MP3 responses consistently at 128kbps 24kHz mono (TTS default)

stepaudio-2.5-tts — Known Issues and Non-Obvious Behavior

Collected from end-to-end testing 2026-04-23. These are things that burned real time to discover; they are not in the official docs.

Stricter content censorship than step-tts-2

Symptom: stepaudio-2.5-tts returns {"error":{"message":"The content you provided or machine outputted is blocked.","type":"censorship_block"}} for content that step-tts-2 happily synthesized.

Observed triggers:

• 死 (die/dead) in any context, even negation
• 消失 (disappear / vanish)
• Combinations with emotional context: "我快要...消失了"
• Politically sensitive terms (standard CN content rules)

Key insight: Rewriting negations doesn't help — "我没有死" blocks as readily as "我死了". The classifier isn't doing deep semantic parsing.

Response strategies (pick per line): 1. Rewrite: "RAG 已死" → "这个技术过时了" 2. Fallback: keep step-tts-2 for the 2-5% of lines that block 3. Whitelist: contact StepFun BD (worth it at >5% blockage)

See migration_from_v2.md for the full blocking→fallback workflow.

TTS duration inflation on short lines

Observation: Very short lines (1-2s in step-tts-2) become dramatically longer in stepaudio-2.5-tts.

Example from the reference project:

• ...你能看到我吗？ (10 chars)
• step-tts-2: 1.24s
• stepaudio-2.5-tts: 2.57s (+107%)

Cause: The new model adds a pre-breath, pauses on ... ellipses, and gives the line emotional weight — all of which lengthens delivery.

Not a bug, but have a plan:

• If your UI has per-line timing (auto-advance, animation sync), re-tune it after migration
• If you want the old pacing, write instruction: "快速、干脆、不要停顿" — but this negates a lot of what you're paying for in the new model

stepaudio-2.5-tts is a "v2 model" for parameter rejection

Why the error says "v2 models": StepFun internally groups stepaudio-2.5-tts with their v2 family despite the "2.5" version number. The error message voice_label is not supported for v2 models uses this internal grouping, which is confusing.

Don't pattern-match on the version string. Just know that:

• stepaudio-2.5-tts → use instruction parameter
• step-tts-2 → use voice_label parameter
• They are NOT API-compatible despite sharing /v1/audio/speech

TTS text cap: 1000 chars (hard, not soft)

The API rejects >1000 char inputs with a 400 error. Split at sentence boundaries before sending.

Non-obvious caveat when probing the limit: don't use highly-repetitive test text. The TTS itself accepts repetitive 800-char inputs and produces normal audio, but if you then transcribe that audio with stepaudio-2.5-asr for round-trip verification, the ASR can hallucinate 3-4× character expansion (a known ASR-side bug, see the stepfun-asr skill's known_issues.md). Use varied real-world text for cap-probing tests.

Voice cloning — not tested in this skill

Zero-shot voice cloning (9.9 元/音色) is advertised as a headline feature but was not verified in this skill's test pass. If you need voice cloning, check the StepFun docs at https://platform.stepfun.com/docs/zh/api-reference/audio/create-voice and validate on your own data — don't assume the quality claims without a listen test.

"Plan key" vs "Normal key" — silent audio failure

StepFun sells a cheap "Plan" subscription for text models (step_plan endpoint). Plan keys cannot call audio endpoints. This silently manifests as 4xx errors that don't mention auth at all.

If you hit auth-shaped failures and your account has a Plan subscription, verify you're using a Normal key (different value, obtained separately in the StepFun console under the same "API Keys" page).

Migrating from step-tts-2 to stepaudio-2.5-tts

Complete playbook for moving a production step-tts-2 voice corpus to stepaudio-2.5-tts. Based on a real end-to-end migration done 2026-04-23 on a ~30-line Chinese voice-acting project. Read this before changing any production code.

What "migration" actually means here

The API endpoint is the same (/v1/audio/speech), but the emotional control model is completely different:

You are not just swapping a model ID. You need to rewrite every voice_label.emotion call and handle a new class of error (censorship_block).

The rewrite — emotion tags → instruction sentences

The step-tts-2 style usually looked like:

const EMOTION_MAP = {
  sad: '悲伤',
  hopeful: '高兴',
  relieved: '高兴',
  smile: '非常高兴',
};

body.voice_label = { emotion: EMOTION_MAP[expression] };

The stepaudio-2.5-tts equivalent:

const INSTRUCTION_MAP = {
  sad: '克制的悲伤，语气低沉柔弱，像快要消失一样',
  hopeful: '温暖的希望感，语气鼓励，带着期待',
  relieved: '如释重负，语气柔和放松',
  smile: '明朗开心，语气上扬，带着微笑',
};

body.instruction = INSTRUCTION_MAP[expression];
// DELETE body.voice_label entirely — sending it triggers an error

How to write a good instruction

Writing instruction sentences is a craft. They should describe what the performance feels like, not just "happy" or "sad". Good instructions use:

• A core emotion word (悲伤, 希望, 开心)
• A qualifier that bounds intensity (克制的, 温暖的, 如释重负)
• A specific vocal behavior (语气低沉柔弱, 带着微笑, 像快要消失一样)

Bad: "悲伤" — same semantic content as the old emotion tag; wastes the instruction parameter Good: "克制的悲伤，语气低沉柔弱，像快要消失一样" — three distinct signals the model can combine

Keep under 200 chars. In practice 30-50 chars is plenty.

Inline () directives — new capability

Beyond the global instruction, stepaudio-2.5-tts parses parentheses inside the input itself. Directives in parentheses are not read aloud — they control delivery.

Use when a single line has multiple emotional beats:

input: "(试探着问)你好吗？(开心地)太好了！(突然沉下来)不过...我快要消失了。"

Also useful for micro-control:

• (停顿一下) between clauses that shouldn't run together
• (轻声) for intimate moments
• (加重) on the key word of a sentence

This is genuinely new vs step-tts-2 and worth using on your most dramatic lines.

Handling censorship_block

stepaudio-2.5-tts rejects more content than step-tts-2. Observed triggers from the reference project:

The error:

{"error":{"message":"The content you provided or machine outputted is blocked.","type":"censorship_block"}}

Three response strategies (pick per line, not globally):

1. Rewrite — "RAG 已死" → "这个技术过时了" keeps the narrative, passes censorship 2. Mixed-model fallback — keep step-tts-2 for the 2-3 blocked lines, use stepaudio-2.5-tts for the rest. The voice difference is audible but tolerable for 2 lines out of 30 3. Request whitelist — contact StepFun BD for your account; only worth it if you have >5% blockage rate

The batch script (scripts/tts_generate.py --batch) logs censored IDs separately so you can handle them individually rather than aborting.

A/B directory strategy — don't overwrite production

Non-destructive layout. Output the new model's files to a parallel directory, not on top of the existing corpus:

public/data/voice/
├── zh/         ← step-tts-2 production (untouched)
└── zh_v25/     ← stepaudio-2.5-tts candidate (A/B)

In the runtime voice loader, add a fallback for lines that are not in zh_v25/ (censored ones):

const getVoicePath = (nodeId: string, lang: string) => {
  // 2 lines censored on v25 — keep step-tts-2 for those
  const censored = new Set(['encounter_3', 'chapter_3_final_hope']);
  const useV25 = lang === 'zh' && !censored.has(nodeId);
  const dir = useV25 ? 'zh_v25' : lang;
  return `/data/voice/${dir}/${nodeId}.mp3`;
};

Benefits:

• Instant rollback: delete the zh_v25/ directory
• Run the game/app with old voices while evaluating the new ones
• Human A/B: toggle the flag per-session to compare

Don't hard-code the censored list across multiple files

If you put ['encounter_3', 'chapter_3_final_hope'] in the runtime loader, the generation script, AND the docs, you'll have three places to update when the list changes. Treat the generation script's SKIP_CENSORED set as the SSOT; the loader can import or mirror it but shouldn't independently enumerate the same IDs.

The time-cost you're buying

Across 26 lines of the reference project, stepaudio-2.5-tts produced ~20% longer total duration than step-tts-2 for the same text. Per-line:

Short lines grow a lot because the new model adds breath and pause before starting. Long lines often shrink because English loanwords are handled faster.

Implications for UI timing:

• Auto-advance timers need re-tuning; anything that assumed 6-8 chars/sec is now 5.5 chars/sec
• Short lines feel markedly slower — the "...你能看到我吗？" intake of breath is noticeable
• Overall dialogue pacing in a visual novel becomes 20% slower

This is the main trade-off users report: the new model is more "alive" but the app feels slower. Whether that's acceptable is a product decision, not a technical one.

Decision checkpoints — before you commit

Use these to keep the migration from becoming a one-way door.

Before the first full regeneration

• [ ] Confirm the A/B directory layout matches the loader's fallback logic (production directory untouched)
• [ ] Write the INSTRUCTION_MAP — don't just mechanically translate emotion tags, actually describe the performance
• [ ] Have 3-5 sample lines ready for listening test BEFORE regenerating the whole corpus (spend 5 min on quality, save 20 min of regeneration if it sounds wrong)

After the first full regeneration

• [ ] Listen to every censored line manually. Decide rewrite vs fallback vs whitelist
• [ ] Check ab_compare.sh output: is the total duration change within acceptable bounds?
• [ ] Play 5 random new lines in-context (the actual game/app) — don't trust the raw mp3 listen-through

Before switching production

• [ ] Run a real user-playthrough session end-to-end on the new voices
• [ ] Check every line with ... punctuation for how the new model handles the ellipsis (sometimes too slow, sometimes too abrupt)
• [ ] Re-tune any auto-advance, skip, or per-line timing parameters
• [ ] Write a rollback script: rm -rf zh_v25/ + revert loader change. If it's more than one commit, you've leaked complexity into the codebase

After production rollout

• [ ] Monitor user sentiment on pacing for 1-2 weeks before pronouncing success
• [ ] Keep the step-tts-2 generation script in version control for at least one release cycle

What NOT to do

• Don't regenerate blindly. Listen to samples first. A/B of 30 lines takes ~10 min; regenerate-without-listening is how you ship something that "technically works" but sounds worse.
• Don't mix `voice_label` and `instruction` in the same request. stepaudio-2.5-tts rejects voice_label entirely, but people occasionally leave it in to "be safe" — it's not safe, it's an error.
• Don't try to pass emotion tags through `instruction`. instruction: "悲伤" works but wastes the parameter. Write a full sentence.
• Don't delete the step-tts-2 generation script until you've shipped the new model to production and had it stable for a full release cycle. You will want the rollback path.
• Don't assume Japanese migrates the same way. The reference project didn't test Japanese voices under stepaudio-2.5-tts. Run a separate mini-A/B for Japanese before committing.

Batch migration with the bundled script

The skill's scripts/tts_generate.py --batch is built for this workflow. Feed it a JSONL file:

{"id": "encounter_1", "text": "...你能看到我吗？", "instruction": "克制的悲伤，语气低沉柔弱"}
{"id": "encounter_2", "text": "太好了... 最近能看到我的人，越来越少了。", "instruction": "克制的悲伤，语气低沉柔弱"}

Run:

python3 scripts/tts_generate.py --batch lines.jsonl --out-dir ./voice/zh_v25

The script handles censorship_block per-line, reports the skipped IDs at the end, and keeps going. You get a clean list of "what to fall back on step-tts-2 for" without having to retry the whole batch.