Video Generation

Video Generation with Gemini (Veo 3.1)

Use this skill when the user asks to generate or extend videos with Gemini using the Python SDK. Default to veo-3.1-fast-generate-preview, resolution="720p", and duration_seconds=4, unless the user asks otherwise or the task requires different settings (e.g., extension, interpolation, reference images, 1080p/4k).

Workflow

1) Identify the task type: text-to-video, image-to-video, reference images, first/last frames (interpolation), or video extension. 2) Ensure GEMINI_API_KEY is available (env or local .env), then use the Python SDK. 3) When using images, pass types.Image(imageBytes=..., mimeType=...) (not PIL.Image or types.Part) to avoid input type errors. 4) Call client.models.generate_videos(...) with the correct inputs/config (see references). 5) Poll the operation until done, then download and save the video. 6) If no videos are returned, surface a clear error and suggest checking the API key, model, and config.

Use these references (by task type)

• Common setup and workflow: references/overview.md
• Parameters and constraints: references/parameters.md
• Model versions and limits: references/model-versions-and-limitations.md
• Prompting guidance: references/prompt-guide.md

Task types

• Text-to-video: examples/text-to-video.md
• Image-to-video: examples/image-to-video.md
• Reference images: examples/reference-images.md
• First/last frames (interpolation): examples/first-last-frames.md
• Video extension: examples/video-extension.md

Tuning examples

• Aspect ratio: examples/aspect-ratio.md
• Resolution (4k): examples/resolution.md
• Negative prompt: examples/negative-prompt.md

Defaults and notes

• Default model: veo-3.1-fast-generate-preview.
• Default output: 720p, 4 seconds.
• For image inputs, always provide imageBytes + mimeType via types.Image to prevent INVALID_ARGUMENT errors.
• 1080p/4k, reference images, interpolation, and video extension require duration_seconds=8.
• Video extension is limited to 720p inputs and requires a video from a previous Veo generation.
• Video generation can take minutes; allow longer timeouts when running commands.

Example: Aspect ratio (portrait)

from google import genai
from google.genai import types

client = genai.Client()

prompt = (
    "A montage of pizza making: a chef tossing and flattening the floury dough, "
    "ladling rich red tomato sauce in a spiral, sprinkling mozzarella cheese and pepperoni, "
    "and a final shot of the bubbling golden-brown pizza, upbeat electronic music with a "
    "rhythmical beat is playing, high energy professional video."
)

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt=prompt,
    config=types.GenerateVideosConfig(
        aspect_ratio="9:16",
        resolution="720p",
        duration_seconds=4,
    ),
)

while not operation.done:
    operation = client.operations.get(operation)

for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"pizza_making_{i}.mp4")

Example: First and last frames (interpolation)

from google import genai
from google.genai import types
from io import BytesIO

client = genai.Client()

first_frame = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=(
        "A woman stands in a field of tall grass, her long dark hair blowing in the wind. "
        "The grass sways gently, and she seems to be at peace."
    ),
    config=types.GenerateContentConfig(response_modalities=["IMAGE"]),
).parts[0].as_image()

last_frame = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents=(
        "A woman walks along a cliff edge overlooking the sea. Her hair blows in the wind, "
        "and the sky is a stormy gray."
    ),
    config=types.GenerateContentConfig(response_modalities=["IMAGE"]),
).parts[0].as_image()

def pil_to_types_image(pil_image) -> types.Image:
    buf = BytesIO()
    pil_image.save(buf, format="PNG")
    return types.Image(imageBytes=buf.getvalue(), mimeType="image/png")

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    image=pil_to_types_image(first_frame),
    prompt=(
        "A ghostly woman walks slowly through a field, then she walks along a cliff "
        "overlooking the ocean."
    ),
    config=types.GenerateVideosConfig(
        last_frame=pil_to_types_image(last_frame),
        resolution="720p",
        duration_seconds=8,
    ),
)

while not operation.done:
    operation = client.operations.get(operation)

for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"interpolation_{i}.mp4")

Example: Image-to-video

from google import genai
from google.genai import types
from io import BytesIO

client = genai.Client()

# First create a source image
image_response = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents="A cute baby monkey, wearing a tiny cowboy hat.",
    config=types.GenerateContentConfig(response_modalities=["IMAGE"]),
)

image_pil = image_response.parts[0].as_image()
buf = BytesIO()
image_pil.save(buf, format="PNG")
image = types.Image(imageBytes=buf.getvalue(), mimeType="image/png")

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    image=image,
    prompt="Panning wide shot of a calico kitten sleeping in the sunshine",
    config=types.GenerateVideosConfig(
        resolution="720p",
        duration_seconds=4,
    ),
)

while not operation.done:
    operation = client.operations.get(operation)

for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"image_to_video_{i}.mp4")

Example: Negative prompt

from google import genai
from google.genai import types

client = genai.Client()

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt="A cinematic shot of a majestic lion in the savannah.",
    config=types.GenerateVideosConfig(
        negative_prompt="cartoon, drawing, low quality",
        resolution="720p",
        duration_seconds=4,
    ),
)

while not operation.done:
    operation = client.operations.get(operation)

for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"lion_{i}.mp4")

Example: Reference images

from google import genai
from google.genai import types
from io import BytesIO

client = genai.Client()

dress_image = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents="A beautiful red dress in a clothing store.",
    config=types.GenerateContentConfig(response_modalities=["IMAGE"]),
).parts[0].as_image()

glasses_image = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents="A very small and thin gold frame, in front of a blue wall.",
    config=types.GenerateContentConfig(response_modalities=["IMAGE"]),
).parts[0].as_image()

woman_image = client.models.generate_content(
    model="gemini-2.5-flash-image",
    contents="A woman with shoulder length brown hair.",
    config=types.GenerateContentConfig(response_modalities=["IMAGE"]),
).parts[0].as_image()

def pil_to_types_image(pil_image) -> types.Image:
    buf = BytesIO()
    pil_image.save(buf, format="PNG")
    return types.Image(imageBytes=buf.getvalue(), mimeType="image/png")

reference_images = [
    types.VideoGenerationReferenceImage(image=pil_to_types_image(dress_image), reference_type="asset"),
    types.VideoGenerationReferenceImage(image=pil_to_types_image(glasses_image), reference_type="asset"),
    types.VideoGenerationReferenceImage(image=pil_to_types_image(woman_image), reference_type="asset"),
]

prompt = (
    "A woman in a red dress, white sneakers and sunglasses walks down a street in a city "
    "with a sidewalk."
)

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt=prompt,
    config=types.GenerateVideosConfig(
        reference_images=reference_images,
        resolution="720p",
        duration_seconds=8,
    ),
)

while not operation.done:
    operation = client.operations.get(operation)

for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"reference_{i}.mp4")

Example: Resolution (4k)

from google import genai
from google.genai import types

client = genai.Client()

prompt = "A drone flies over the Grand Canyon at sunset, 4k resolution."

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt=prompt,
    config=types.GenerateVideosConfig(
        resolution="4k",
        duration_seconds=8,
    ),
)

while not operation.done:
    operation = client.operations.get(operation)

for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"4k_grand_canyon_{i}.mp4")

Example: Text-to-video (dialogue)

from google import genai
from google.genai import types

client = genai.Client()

prompt = (
    "A close up of two people staring at a cryptic drawing on a wall, torchlight flickering. "
    "A man murmurs, 'This must be it. That's the secret code.' The woman looks at him and "
    "whispering excitedly, 'What did you find?'"
)

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt=prompt,
    config=types.GenerateVideosConfig(
        resolution="720p",
        duration_seconds=4,
    ),
)

while not operation.done:
    operation = client.operations.get(operation)

for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"dialogue_example_{i}.mp4")

Example: Video extension

from google import genai
from google.genai import types

client = genai.Client()

# Generate an initial video
initial_op = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt="A butterfly flutters through a meadow, landing on a flower.",
    config=types.GenerateVideosConfig(
        resolution="720p",
        duration_seconds=8,
    ),
)

while not initial_op.done:
    initial_op = client.operations.get(initial_op)

initial_video = initial_op.response.generated_videos[0].video

# Extend the video
extend_op = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt="Track the butterfly into the garden, keep the camera movement smooth.",
    config=types.GenerateVideosConfig(
        video=initial_video,
        resolution="720p",
        duration_seconds=8,
    ),
)

while not extend_op.done:
    extend_op = client.operations.get(extend_op)

for i, video in enumerate(extend_op.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"extended_{i}.mp4")

Model versions, features, and limitations

Model versions (Gemini API)

• veo-3.1-generate-preview: Veo 3.1 preview model.
• veo-3.1-fast-generate-preview: Veo 3.1 fast preview model (speed-optimized).

Both support text and image inputs and output video with audio.

Model features (Veo 3.1)

• Output video includes native audio.
• Input modalities: text-to-video, image-to-video, video-to-video (extension). Only Veo 3.1 supports video extension.
• Resolution: 720p, 1080p (8s only), 4k (8s only).
• Frame rate: 24 fps.
• Video duration: 4s, 6s, 8s (8s required for 1080p/4k or reference images).
• Videos per request: 1.

Video extension specifics

• Extends a video by 7 seconds per request, up to 20 extensions.
• Input video length limit: 141 seconds; output limit: 148 seconds.
• Video extension is limited to 720p input videos.

Limitations

• Request latency can range from 11 seconds to 6 minutes.
• Regional limits apply to person_generation (EU/UK/CH/MENA).
• Generated videos are stored for 2 days; download within 2 days (derived videos re-reference the storage window).
• Videos are watermarked with SynthID.
• Audio errors: videos can be blocked due to safety filters or other audio-processing issues.

Overview (Python SDK)

Requirements

• A Gemini API key is required; check the environment or a local ./.env file. The SDK reads GEMINI_API_KEY from the environment by default.
• Install the Python SDK (Python 3.9+):

pip install -q -U google-genai

Base workflow (text-to-video)

from google import genai
from google.genai import types

client = genai.Client()

prompt = "A cinematic close-up of a cup of coffee with steam swirling"

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    prompt=prompt,
    config=types.GenerateVideosConfig(
        resolution="720p",
        duration_seconds=4,
    ),
)

# Poll until done
while not operation.done:
    operation = client.operations.get(operation)

# Download and save
for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"video_{i}.mp4")

Base workflow (image-to-video)

from google import genai
from google.genai import types

client = genai.Client()

# First load a source image (local file or URL)
import mimetypes
import requests

def load_image(path_or_url: str) -> types.Image:
    if path_or_url.startswith(("http://", "https://")):
        resp = requests.get(path_or_url, timeout=30)
        data = resp.content
        mime_type = resp.headers.get("Content-Type", "image/png")
    else:
        with open(path_or_url, "rb") as f:
            data = f.read()
        mime_type = mimetypes.guess_type(path_or_url)[0] or "image/png"
    return types.Image(imageBytes=data, mimeType=mime_type)

image = load_image("input.png")

operation = client.models.generate_videos(
    model="veo-3.1-fast-generate-preview",
    image=image,
    prompt="Panning wide shot of a calico kitten sleeping in the sunshine.",
    config=types.GenerateVideosConfig(
        resolution="720p",
        duration_seconds=4,
    ),
)

# Poll until done
while not operation.done:
    operation = client.operations.get(operation)

# Download and save
for i, video in enumerate(operation.response.generated_videos):
    client.files.download(file=video.video)
    video.video.save(f"image_to_video_{i}.mp4")

Notes

• generate_videos returns a long-running operation; always poll until done.
• Use duration_seconds=8 for 1080p/4k, reference images, interpolation, or video extension.
• See references/parameters.md for available config fields and constraints.

Veo 3.1 parameters (Python config)

Use types.GenerateVideosConfig(...) with snake_case field names.

Fields

• prompt (str): Main text prompt for video generation.
• negative_prompt (str): What to avoid in the output.
• image (types.Image): Input image for image-to-video. Must include imageBytes and mimeType.
• last_frame (types.Image): Last frame for interpolation (use with image as the first frame). Must include imageBytes and mimeType.
• reference_images (list): Reference images for appearance consistency. Use types.VideoGenerationReferenceImage with reference_type="asset" and types.Image instances.
• video (Video): Input video for extension. Must be a prior Veo output.
• aspect_ratio (str): "16:9" or "9:16".
• resolution (str): "720p", "1080p", or "4k".
• duration_seconds (int): 4, 6, or 8. Use 8 seconds when required (see constraints).
• person_generation (str): "allow_all" or "allow_adult". "allow_adult" only with image-to-video, interpolation, or reference images. "allow_all" only with text-to-video or video extension.
• seed (int): Random seed for reproducibility.
• number_of_videos (int): For Veo 3.1, only 1 is supported.

Constraints

• 1080p or 4k requires duration_seconds=8.
• Reference images, interpolation (first/last frames), and video extension require duration_seconds=8.
• Video extension is limited to 720p input videos and requires a Veo-generated video as input.
• person_generation is only supported in certain regions; requests can fail if unsupported.

Veo prompt guide (summary + examples)

Prompt basics

Include as many of these elements as needed:

• Subject: "An astronaut in space."
• Context: "An astronaut floating in a space station with Earth visible through the window."
• Action: "A cat chases a laser pointer across the floor."
• Style: "A black and white film noir scene of a detective smoking in the rain."
• Camera: "A POV shot driving through a tunnel with neon lights."
• Composition: "A wide shot of a desert landscape with a lone cactus."
• Ambiance: "A girl holding a puppy in a field at sunset, warm light, gentle breeze."

Tips

• Be explicit about subject, action, and camera when you want precision.
• Use negative prompts to avoid unwanted elements.
• If the model struggles, add more constraints or specificity.

Examples

Short prompts

• "Icicles melting in the sun"

Less vs more detail

• Less detail: "Man on the phone"
• More detail: "A shot following a man from behind as he walks down a city street while talking on a phone. The camera follows at waist height from behind. We see the city street with cars and pedestrians. The man is wearing a black jacket and jeans, and carries a leather satchel over his shoulder. The street is lit by late afternoon sunlight. The man looks serious as he speaks."

Simple vs detailed (cartoon style)

• Simple: "A cute creature with snow leopard-like fur is walking in winter forest, 3D cartoon style render."
• Detailed: "Create a short 3D animated scene in a joyful cartoon style. A cute creature with snow leopard-like fur, large expressive eyes, and a friendly, rounded form happily prances through a whimsical winter forest. The scene should feature rounded, snow-covered trees, gentle falling snowflakes, and warm sunlight filtering through the branches. The creature's bouncy movements and wide smile should convey pure delight. Aim for an upbeat, heartwarming tone with bright, cheerful colors and playful animation."

Examples by writing elements

• Subject + context: "An architectural rendering of a white concrete apartment building with flowing organic shapes, seamlessly blending with lush greenery and futuristic elements."
• Subject + context: "A satellite floating through outer space with the moon and some stars in the background."
• Action: "A wide shot of a woman walking along the beach, looking content and relaxed towards the horizon at sunset."
• Style: "Film noir style, man and woman walk on the street, mystery, cinematic, black and white."
• Camera motion + composition: "A POV shot from a vintage car driving in the rain, Canada at night, cinematic."
• Composition (close-up): "Extreme close-up of a an eye with city reflected in it."
• Ambiance: "A close-up of a girl holding adorable golden retriever puppy in the park, sunlight."
• Ambiance: "Cinematic close-up shot of a sad woman riding a bus in the rain, cool blue tones, sad mood."

Negative prompt example

• Prompt: "Generate a short, stylized animation of a large, solitary oak tree with leaves blowing vigorously in a strong wind."
• Negative prompt: "urban background, man-made structures, dark, stormy, or threatening atmosphere."
• Tip: avoid instructive phrases like "no" or "don't"; describe unwanted elements directly (for example, "no frame, no wall" becomes "frame, wall").

Aspect ratio prompts

• Widescreen (16:9): "Create a video with a tracking drone view of a man driving a red convertible car in Palm Springs, 1970s, warm sunlight, long shadows."
• Portrait (9:16): "Create a video highlighting the smooth motion of a majestic Hawaiian waterfall within a lush rainforest. Focus on realistic water flow, detailed foliage, and natural lighting to convey tranquility. Capture the rushing water, misty atmosphere, and dappled sunlight filtering through the dense canopy. Use smooth, cinematic camera movements to showcase the waterfall and its surroundings. Aim for a peaceful, realistic tone, transporting the viewer to the serene beauty of the Hawaiian rainforest."

Prompting with reference images

• "A small surfer riding a wave inside a sink, miniature scale."
• "An angler fish wearing a clown costume, deep sea, dramatic lighting."

Prompting with first and last frames

• "A ginger cat jumping from a tall cliff, slow motion."

Prompting with video extension

• "A paraglider soaring through a mountain valley, golden hour."