
Video Generation
- 2.5k installs
- 79.3k repo stars
- Updated August 5, 2026
- bytedance/deer-flow
video-generation creates AIGC clips from JSON prompt files and generate.py with Gemini Veo or MiniMax backends.
About
The video-generation skill creates AIGC videos using structured JSON prompts and a bundled Python generate.py script. Workflow step one gathers subject, style, technical specs, aspect ratio, and optional reference images without scanning user folders under /mnt/user-data. Step two writes a descriptive JSON prompt file to /mnt/user-data/workspace/. Step three optionally creates a reference image via the image-generation skill; a single image guides the first or last frame. Step four runs generate.py with --prompt-file, optional --reference-images, --output-file, and --aspect-ratio defaulting to 16:9. The skill forbids reading the Python source; call it with parameters only. Provider selection is automatic: GEMINI_API_KEY enables Gemini Veo; only MINIMAX_API_KEY routes to MiniMax async video with first_frame_image; VIDEO_GENERATION_PROVIDER can force gemini or minimax. Outputs land in /mnt/user-data/outputs/ and should be shared via present_files with brief result notes and iteration offers. Prompts must be English regardless of user language.
- Structured JSON prompts written to /mnt/user-data/workspace/ before generation.
- generate.py called with prompt file, reference images, output path, aspect ratio.
- Do not read generate.py; invoke with CLI parameters only.
- GEMINI_API_KEY or MINIMAX_API_KEY auto-selects provider; MiniMax ignores aspect-ratio.
- Optional reference image from image-generation skill guides first or last frame.
Video Generation by the numbers
- 2,508 all-time installs (skills.sh)
- +61 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #141 of 1,335 Generative Media skills by installs in the Skillselion catalog
- Security screen: CRITICAL risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
video-generation capabilities & compatibility
- Capabilities
- json prompt authoring for scene, camera, dialogu · generate.py cli execution with reference images · gemini veo and minimax provider auto selection · output delivery via present_files with iteration
- Works with
- openai
- Use cases
- video generation · image generation · orchestration
- Runs
- Runs locally
- Pricing
- Bring your own API key
What video-generation says it does
Do NOT read the python file, instead just call it with the parameters.
Always use English for prompts regardless of user's language
npx skills add https://github.com/bytedance/deer-flow --skill video-generationAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 2.5k |
|---|---|
| repo stars | ★ 79.3k |
| Security audit | 0 / 3 scanners passed |
| Last updated | August 5, 2026 |
| Repository | bytedance/deer-flow ↗ |
How do I generate a short video clip from a structured scene description with optional reference frames?
Generate AIGC videos from structured JSON prompts and optional reference images via generate.py with Gemini Veo or MiniMax providers.
Who is it for?
Agent sessions that need scripted video clips with camera, dialogue, and audio blocks in JSON.
Skip if: Skip for static image-only requests; use image-generation instead.
When should I use this skill?
User asks to generate, create, or imagine videos with structured prompts or reference images.
What you get
An MP4 in /mnt/user-data/outputs/ produced from a validated JSON prompt and optional reference image.
- MP4 video file
- Python integration function
Files
Video Generation Skill
Overview
This skill generates high-quality videos using structured prompts and a Python script. The workflow includes creating JSON-formatted prompts and executing video generation with optional reference image.
Core Capabilities
- Create structured JSON prompts for AIGC video generation
- Support reference image as guidance or the first/last frame of the video
- Generate videos through automated Python script execution
Workflow
Step 1: Understand Requirements
When a user requests video generation, identify:
- Subject/content: What should be in the image
- Style preferences: Art style, mood, color palette
- Technical specs: Aspect ratio, composition, lighting
- Reference image: Any image to guide generation
- You don't need to check the folder under
/mnt/user-data
Step 2: Create Structured Prompt
Generate a structured JSON file in /mnt/user-data/workspace/ with naming pattern: {descriptive-name}.json
Step 3: Create Reference Image (Optional when image-generation skill is available)
Generate reference image for the video generation.
- If only 1 image is provided, use it as the guided frame of the video
Step 3: Execute Generation
Call the Python script:
python /mnt/skills/public/video-generation/scripts/generate.py \
--prompt-file /mnt/user-data/workspace/prompt-file.json \
--reference-images /path/to/ref1.jpg \
--output-file /mnt/user-data/outputs/generated-video.mp4 \
--aspect-ratio 16:9Parameters:
--prompt-file: Absolute path to JSON prompt file (required)--reference-images: Absolute paths to reference image (optional)--output-file: Absolute path to output image file (required)--aspect-ratio: Aspect ratio of the generated image (optional, default: 16:9)
[!NOTE] Do NOT read the python file, instead just call it with the parameters.
Video Generation Example
User request: "Generate a short video clip depicting the opening scene from "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe"
Step 1: Search for the opening scene of "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe" online
Step 2: Create a JSON prompt file with the following content:
{
"title": "The Chronicles of Narnia - Train Station Farewell",
"background": {
"description": "World War II evacuation scene at a crowded London train station. Steam and smoke fill the air as children are being sent to the countryside to escape the Blitz.",
"era": "1940s wartime Britain",
"location": "London railway station platform"
},
"characters": ["Mrs. Pevensie", "Lucy Pevensie"],
"camera": {
"type": "Close-up two-shot",
"movement": "Static with subtle handheld movement",
"angle": "Profile view, intimate framing",
"focus": "Both faces in focus, background soft bokeh"
},
"dialogue": [
{
"character": "Mrs. Pevensie",
"text": "You must be brave for me, darling. I'll come for you... I promise."
},
{
"character": "Lucy Pevensie",
"text": "I will be, mother. I promise."
}
],
"audio": [
{
"type": "Train whistle blows (signaling departure)",
"volume": 1
},
{
"type": "Strings swell emotionally, then fade",
"volume": 0.5
},
{
"type": "Ambient sound of the train station",
"volume": 0.5
}
]
}Step 3: Use the image-generation skill to generate the reference image
Load the image-generation skill and generate a single reference image narnia-farewell-scene-01.jpg according to the skill.
Step 4: Use the generate.py script to generate the video
python /mnt/skills/public/video-generation/scripts/generate.py \
--prompt-file /mnt/user-data/workspace/narnia-farewell-scene.json \
--reference-images /mnt/user-data/outputs/narnia-farewell-scene-01.jpg \
--output-file /mnt/user-data/outputs/narnia-farewell-scene-01.mp4 \
--aspect-ratio 16:9Do NOT read the python file, just call it with the parameters.
Output Handling
After generation:
- Videos are typically saved in
/mnt/user-data/outputs/ - Share generated videos (come first) with user as well as generated image if applicable, using
present_filestool - Provide brief description of the generation result
- Offer to iterate if adjustments needed
Notes
- Always use English for prompts regardless of user's language
- JSON format ensures structured, parsable prompts
- Reference image enhance generation quality significantly
- Iterative refinement is normal for optimal results
Providers (Gemini / MiniMax)
Auto-selected by environment variables (CLI unchanged):
GEMINI_API_KEYset → Gemini Veo (default, unchanged).- Only
MINIMAX_API_KEYset → MiniMax video (/v1/video_generation, async 3-step poll/download). - Force with
VIDEO_GENERATION_PROVIDER=gemini|minimax.
MiniMax overrides: MINIMAX_API_HOST (default https://api.minimaxi.com), MINIMAX_VIDEO_MODEL (default MiniMax-Hailuo-2.3). The first reference image is used as MiniMax first_frame_image. MiniMax ignores --aspect-ratio (it uses resolution/duration).
import base64
import os
import time
import requests
MINIMAX_DEFAULT_HOST = "https://api.minimaxi.com"
def _resolve_provider(override_env: str, existing_provider: str, has_existing_creds: bool) -> str:
"""Pick the provider: <SKILL>_PROVIDER override > existing creds > MiniMax fallback."""
override = os.getenv(override_env)
if override:
return override.strip().lower()
if has_existing_creds:
return existing_provider
if os.getenv("MINIMAX_API_KEY"):
return "minimax"
raise ValueError(
f"No credentials found. Set GEMINI_API_KEY for {existing_provider}, "
f"or MINIMAX_API_KEY for minimax (optionally force with {override_env})."
)
def _minimax_host() -> str:
return os.getenv("MINIMAX_API_HOST", MINIMAX_DEFAULT_HOST).rstrip("/")
def _ensure_output_dir(output_file: str) -> None:
"""Create the output file's parent directory so nested paths don't fail."""
output_dir = os.path.dirname(output_file)
if output_dir:
os.makedirs(output_dir, exist_ok=True)
def _check_base_resp(payload: dict) -> None:
base = payload.get("base_resp") or {}
if base.get("status_code", 0) != 0:
raise Exception(f"MiniMax error {base.get('status_code')}: {base.get('status_msg')}")
def _guess_mime(image_path: str) -> str:
ext = os.path.splitext(image_path)[1].lower()
return {
".png": "image/png",
".webp": "image/webp",
".gif": "image/gif",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
}.get(ext, "image/jpeg")
def _to_data_url(image_path: str) -> str:
with open(image_path, "rb") as f:
b64 = base64.b64encode(f.read()).decode("utf-8")
return f"data:{_guess_mime(image_path)};base64,{b64}"
def _poll_video_task(host: str, auth: str, task_id: str,
max_attempts: int = 120, interval: int = 3) -> str:
for _ in range(max_attempts):
response = requests.get(
f"{host}/v1/query/video_generation",
headers={"Authorization": auth},
params={"task_id": task_id},
timeout=30,
)
response.raise_for_status()
payload = response.json()
status = payload.get("status")
if status == "Success":
return payload["file_id"]
if status == "Fail":
base = payload.get("base_resp") or {}
raise Exception(
f"MiniMax video task {task_id} failed: "
f"{base.get('status_code')} {base.get('status_msg')}"
)
# Surface query-level errors (bad task_id, auth) that arrive as a non-zero
# base_resp without a terminal status, then keep polling.
_check_base_resp(payload)
time.sleep(interval)
raise Exception(f"MiniMax video task {task_id} timed out after {max_attempts} polls")
def _retrieve_file_url(host: str, auth: str, file_id: str) -> str:
response = requests.get(
f"{host}/v1/files/retrieve",
headers={"Authorization": auth},
params={"file_id": file_id},
timeout=30,
)
response.raise_for_status()
payload = response.json()
_check_base_resp(payload)
return payload["file"]["download_url"]
def _download(url: str, output_file: str) -> None:
response = requests.get(url, timeout=300)
response.raise_for_status()
_ensure_output_dir(output_file)
with open(output_file, "wb") as f:
f.write(response.content)
def _generate_video_minimax(
prompt: str, reference_images: list[str], output_file: str
) -> str:
api_key = os.getenv("MINIMAX_API_KEY")
if not api_key:
return "MINIMAX_API_KEY is not set"
host = _minimax_host()
auth = f"Bearer {api_key}"
body = {"model": os.getenv("MINIMAX_VIDEO_MODEL", "MiniMax-Hailuo-2.3"), "prompt": prompt}
if reference_images:
body["first_frame_image"] = _to_data_url(reference_images[0])
response = requests.post(
f"{host}/v1/video_generation",
headers={"Authorization": auth, "Content-Type": "application/json"},
json=body,
timeout=60,
)
response.raise_for_status()
payload = response.json()
_check_base_resp(payload)
task_id = payload["task_id"]
file_id = _poll_video_task(host, auth, task_id)
download_url = _retrieve_file_url(host, auth, file_id)
_download(download_url, output_file)
return f"The video has been generated successfully to {output_file}"
def download(url: str, output_file: str) -> None:
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
raise ValueError("GEMINI_API_KEY is not set")
response = requests.get(url, headers={"x-goog-api-key": api_key}, timeout=300)
response.raise_for_status()
_ensure_output_dir(output_file)
with open(output_file, "wb") as f:
f.write(response.content)
def _generate_video_gemini(
prompt: str, reference_images: list[str], output_file: str
) -> str:
reference_payload = []
request_json = {"instances": [{"prompt": prompt}]}
for reference_image in reference_images:
with open(reference_image, "rb") as f:
image_b64 = base64.b64encode(f.read()).decode("utf-8")
reference_payload.append(
{"image": {"mimeType": "image/jpeg", "bytesBase64Encoded": image_b64},
"referenceType": "asset"}
)
if reference_payload:
request_json["instances"][0]["referenceImages"] = reference_payload
api_key = os.getenv("GEMINI_API_KEY")
if not api_key:
return "GEMINI_API_KEY is not set"
response = requests.post(
"https://generativelanguage.googleapis.com/v1beta/models/veo-3.1-generate-preview:predictLongRunning",
headers={"x-goog-api-key": api_key, "Content-Type": "application/json"},
json=request_json,
timeout=60,
)
response.raise_for_status()
data = response.json()
operation_name = data["name"]
while True:
response = requests.get(
f"https://generativelanguage.googleapis.com/v1beta/{operation_name}",
headers={"x-goog-api-key": api_key},
timeout=30,
)
response.raise_for_status()
data = response.json()
if data.get("done", False):
sample = data["response"]["generateVideoResponse"]["generatedSamples"][0]
download(sample["video"]["uri"], output_file)
break
time.sleep(3)
return f"The video has been generated successfully to {output_file}"
def generate_video(
prompt_file: str,
reference_images: list[str],
output_file: str,
aspect_ratio: str = "16:9",
) -> str:
with open(prompt_file, "r", encoding="utf-8") as f:
prompt = f.read()
provider = _resolve_provider(
"VIDEO_GENERATION_PROVIDER", "gemini", bool(os.getenv("GEMINI_API_KEY"))
)
if provider == "minimax":
# MiniMax video uses resolution/duration, not aspect_ratio; aspect_ratio ignored.
return _generate_video_minimax(prompt, reference_images, output_file)
if provider in ("gemini", "google"):
return _generate_video_gemini(prompt, reference_images, output_file)
raise ValueError(f"Unknown video provider: {provider!r} (use 'gemini' or 'minimax')")
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser(description="Generate videos using Gemini or MiniMax API")
parser.add_argument("--prompt-file", required=True, help="Absolute path to JSON prompt file")
parser.add_argument("--reference-images", nargs="*", default=[],
help="Absolute paths to reference images (space-separated)")
parser.add_argument("--output-file", required=True, help="Output path for generated video")
parser.add_argument("--aspect-ratio", required=False, default="16:9",
help="Aspect ratio of the generated video (Gemini only)")
args = parser.parse_args()
try:
print(generate_video(args.prompt_file, args.reference_images,
args.output_file, args.aspect_ratio))
except Exception as e:
print(f"Error while generating video: {e}")
Related skills
How it compares
Choose this when you already use Deer Flow and need a minimal Python wrapper around Veo instead of building the REST payload manually.
FAQ
Which API keys select the video provider?
GEMINI_API_KEY selects Gemini Veo; only MINIMAX_API_KEY selects MiniMax. Override with VIDEO_GENERATION_PROVIDER.
Should I read the generate.py source?
No. Call the script with --prompt-file, --reference-images, --output-file, and --aspect-ratio only.
What language should JSON prompts use?
Always English regardless of the user's language.
Is Video Generation safe to install?
skills.sh reports 0 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.