Speech Build

Speech Skill (TTS & STT)

Use this skill to implement audio generation and transcription workflows using the google-genai and google-cloud-speech SDKs.

Quick Start Setup

from google import genai
from google.genai import types
# For STT: from google.cloud import speech_v2

client = genai.Client()

Reference Materials

• [Text-to-Speech (TTS)](references/tts.md): Gemini-TTS, Chirp 3 HD, Instant Custom Voice.
• [Speech-to-Text (STT)](references/stt.md): Chirp 3 Transcription, Diarization, Streaming.
• [Voices & Locales](references/voices.md): Available voices (Aoede, Puck...) and languages.
• [Prompting Guide](references/prompting.md): How to control style, accent, and pacing in Gemini-TTS.
• [Source Code](references/source_code.md): Deep inspection of SDK internals.

Common Workflows

1. Generate Speech (Gemini-TTS)

response = client.models.generate_content(
    model="gemini-2.5-flash-preview-tts",
    contents="Hello, world!",
    config=types.GenerateContentConfig(
        response_modalities=["AUDIO"],
        speech_config=types.SpeechConfig(
            voice_config=types.VoiceConfig(
                prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name='Kore')
            )
        )
    )
)

2. Transcribe Audio (Chirp 3)

# Requires google-cloud-speech
from google.cloud import speech_v2
# ... (See stt.md for full setup)
response = speech_client.recognize(...)

Prompting for Gemini-TTS

Gemini-TTS understands natural language instructions for how to speak.

Structure

1. Audio Profile: Persona/Archetype (e.g., "Radio DJ"). 2. Scene: Environment/Vibe (e.g., "Late night studio"). 3. Director's Notes: Specifics on Style, Pacing, Accent. 4. Transcript: The actual text to read.

Examples

Single Speaker

Say in a spooky whisper:
"By the pricking of my thumbs... Something wicked this way comes"

Multi-Speaker

Make Speaker1 sound tired and bored, and Speaker2 sound excited and happy:

Speaker1: So... what's on the agenda today?
Speaker2: You're never going to guess!

Advanced Prompt

# AUDIO PROFILE: Jaz R. (Radio DJ)
## THE SCENE: The London Studio
It is 10:00 PM... upbeat atmosphere.

### DIRECTOR'S NOTES
Style: The "Vocal Smile". Bright, sunny.
Pace: Fast, bouncing cadence. No dead air.
Accent: Estuary accent from Brixton, London.

#### TRANSCRIPT
Yes, massive vibes in the studio! ...

Google GenAI SDK Source Code (Speech)

Use web_fetch to retrieve raw code for deep inspection.

Base URL: https://raw.githubusercontent.com/googleapis/python-genai/main/google/genai/

Key Modules

Models (Generation)

• File: models.py
• URL: https://raw.githubusercontent.com/googleapis/python-genai/main/google/genai/models.py
• Purpose: generate_content logic for TTS.

Types (Configuration)

• File: types.py
• URL: https://raw.githubusercontent.com/googleapis/python-genai/main/google/genai/types.py
• Purpose: Definitions for SpeechConfig, VoiceConfig, PrebuiltVoiceConfig, MultiSpeakerVoiceConfig.

Client

• File: client.py
• URL: https://raw.githubusercontent.com/googleapis/python-genai/main/google/genai/client.py

Speech-to-Text (STT) - Chirp 3

Chirp 3 offers state-of-the-art multilingual transcription and speaker diarization.

Transcribe Audio (Synchronous)

For audio < 1 minute.

from google.cloud import speech_v2
from google.cloud.speech_v2.types import cloud_speech

client = speech_v2.SpeechClient(...)

config = cloud_speech.RecognitionConfig(
    auto_decoding_config=cloud_speech.AutoDetectDecodingConfig(),
    model="chirp_3",
    language_codes=["auto"], # Language identification
)

request = cloud_speech.RecognizeRequest(
    recognizer="projects/.../locations/.../recognizers/_",
    config=config,
    content=audio_bytes, # or uri="gs://..."
)

response = client.recognize(request=request)

Batch Recognition

For long audio files.

request = cloud_speech.BatchRecognizeRequest(
    recognizer=recognizer,
    config=config,
    files=[cloud_speech.BatchRecognizeFileMetadata(uri="gs://...")],
    recognition_output_config=cloud_speech.RecognitionOutputConfig(
        gcs_output_config=cloud_speech.GcsOutputConfig(uri="gs://output-bucket")
    ),
)
operation = client.batch_recognize(request=request)
result = operation.result()

Speaker Diarization

Identify different speakers.

config = cloud_speech.RecognitionConfig(
    features=cloud_speech.RecognitionFeatures(
        diarization_config=cloud_speech.SpeakerDiarizationConfig(),
    ),
    model="chirp_3",
    # ...
)

Streaming STT

Real-time transcription.

# Create generator yielding StreamingRecognizeRequest
requests = create_streaming_requests(audio_file)
responses = client.streaming_recognize(requests=requests)
for response in responses:
    print(response.results[0].alternatives[0].transcript)

Text-to-Speech (TTS)

Overview

Google Cloud and Vertex AI offer multiple TTS solutions: 1. Gemini-TTS: Advanced, controllable speech generation using Gemini 2.5 models. 2. Chirp 3 HD: High-fidelity, natural-sounding voices. 3. Instant Custom Voice (Chirp 3): Create custom voices from short audio samples (allowlist required).

Gemini-TTS (Preview)

Use gemini-2.5-flash-preview-tts for controllable speech.

Single Speaker

from google import genai
from google.genai import types

client = genai.Client()
response = client.models.generate_content(
    model="gemini-2.5-flash-preview-tts",
    contents="Say cheerfully: Have a wonderful day!",
    config=types.GenerateContentConfig(
        response_modalities=["AUDIO"],
        speech_config=types.SpeechConfig(
            voice_config=types.VoiceConfig(
                prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name='Kore')
            )
        ),
    )
)
# Save response.candidates[0].content.parts[0].inline_data.data to .wav

Multi-Speaker

config = types.GenerateContentConfig(
    response_modalities=["AUDIO"],
    speech_config=types.SpeechConfig(
        multi_speaker_voice_config=types.MultiSpeakerVoiceConfig(
            speaker_voice_configs=[
                types.SpeakerVoiceConfig(
                    speaker='Joe',
                    voice_config=types.VoiceConfig(
                        prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name='Kore')
                    )
                ),
                types.SpeakerVoiceConfig(
                    speaker='Jane',
                    voice_config=types.VoiceConfig(
                        prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name='Puck')
                    )
                ),
            ]
        )
    )
)

Chirp 3 HD (Vertex AI)

High-fidelity voices for general use.

response = client.models.generate_content(
    model="gemini-2.5-flash-tts", # or chirp-3-hd via speech client
    contents="Hello world",
    config=types.GenerateContentConfig(
        speech_config=types.SpeechConfig(
            language_code="en-US",
            voice_config=types.VoiceConfig(
                prebuilt_voice_config=types.PrebuiltVoiceConfig(voice_name="Aoede")
            )
        )
    )
)

Instant Custom Voice (Chirp 3)

Requires Allowlist. Uses voices:generateVoiceCloningKey to create a key, then gemini-2.5-flash-tts to synthesize.

1. Create Cloning Key (REST API)

The SDK does not yet support key generation directly. Use the REST API.

Endpoint: POST https://texttospeech.googleapis.com/v1beta1/voices:generateVoiceCloningKey

Body:

{
  "reference_audio": {
    "content": "BASE64_ENCODED_WAV",
    "audio_config": {"audio_encoding": "LINEAR16", "sample_rate_hertz": 24000}
  },
  "voice_talent_consent": {
    "content": "BASE64_ENCODED_WAV",
    "audio_config": {"audio_encoding": "LINEAR16", "sample_rate_hertz": 24000}
  },
  "consent_script": "I am the owner of this voice and I consent to Google using this voice to create a synthetic voice model.",
  "language_code": "en-US"
}

2. Synthesize (SDK)

Use the key in VoiceConfig.

response = client.models.generate_content(
    model="gemini-2.5-flash-tts",
    contents="This is my cloned voice.",
    config=types.GenerateContentConfig(
        response_modalities=["AUDIO"],
        speech_config=types.SpeechConfig(
            voice_config=types.VoiceConfig(
                voice_clone=types.VoiceClone(
                    voice_cloning_key="YOUR_GENERATED_KEY"
                )
            )
        )
    )
)

Voices & Locales

Gemini-TTS Voices

30 options available. Use these names in voice_name.

Chirp 3 HD Voices

Currently 8 distinct voices (4 male, 4 female) available in 31 languages. Common examples: Aoede, Puck, Charon, Kore, Fenrir, Leda, Orus, Zephyr.

Locales (Gemini-TTS)

Supports 24 languages including:

• en-US, en-GB, en-IN
• es-US, es-ES
• fr-FR, de-DE
• ja-JP, ko-KR
• pt-BR
• hi-IN
• And more.