
Alibabacloud Video Translation
- 150 installs
- 208 repo stars
- Updated August 4, 2026
- aliyun/alibabacloud-aiops-skills
Add Alibaba Cloud video translation pipelines that localize spoken or captioned media, enabling multilingual content products, marketing clips, or training videos via agent-guided API integration.
About
Skill for integrating Alibaba Cloud video translation services through agents. It helps teams submit translation jobs, manage multilingual media workflows, and embed localized video output into content platforms, education products, or marketing automation without building speech models in-house.
- Video localization APIs
- Speech and caption translation
- Async media job flows
- Multilingual content pipelines
- Agent-guided media integration
Alibabacloud Video Translation by the numbers
- 150 all-time installs (skills.sh)
- Ranked #709 of 1,335 Generative Media skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/aliyun/alibabacloud-aiops-skills --skill alibabacloud-video-translationAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 150 |
|---|---|
| repo stars | ★ 208 |
| Last updated | August 4, 2026 |
| Repository | aliyun/alibabacloud-aiops-skills ↗ |
What it does
Add Alibaba Cloud video translation pipelines that localize spoken or captioned media, enabling multilingual content products, marketing clips, or training videos via agent-guided API integration.
Files
Video Translation Skill
One-click video translation powered by Alibaba Cloud IMS, supporting subtitle-level and speech-level translation.
---
Input Format Requirements
IMPORTANT: Different APIs use different address formats!
API Address Format Reference
| API | Address Format | Example |
|---|---|---|
SubmitIProductionJob (subtitle extraction) | `oss://` format | oss://my-bucket/videos/test.mp4 |
SubmitVideoTranslationJob (video translation) | HTTP URL format | https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4 |
Key: Subtitle extraction uses oss://, video translation uses HTTP URL!User Input Handling
| User Input Type | Processing Method |
|---|---|
| HTTP URL format | Use directly for video translation; convert to oss:// if subtitle extraction needed |
oss:// format | Use directly for subtitle extraction; convert to HTTP URL for video translation |
| Local video | MUST ask for OSS upload path, save both formats after upload |
Format Conversion Rules
oss:// format ⇄ HTTP URL format
oss://my-bucket/videos/test.mp4
⇄
https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4Conversion Formula:
oss://<bucket>/<path>→https://<bucket>.oss-<region>.aliyuncs.com/<path>- HTTP URL does not require signing, use Bucket domain format directly
Local Video Processing Flow
User provides local video path
│
├─ AskUserQuestion: "Please provide OSS upload path (format: oss://<bucket>/<path>/<filename>.mp4)"
│
├─ User specifies upload path
│ ├─ Check if Bucket exists
│ ├─ Upload file: aliyun oss cp <local_path> <oss_path>
│ ├─ Save oss:// format → for subtitle extraction
│ └─ Save HTTP URL format → for video translation
│
└─ User does not specify path → STOP, user MUST provide upload pathUpload Command:
aliyun oss cp <local_path> oss://<bucket>/<path>/<filename>.mp4Save both formats after upload:
Local: /Users/demo/videos/test.mp4
Uploaded to: oss://my-bucket/videos/test.mp4
├─ oss:// format: oss://my-bucket/videos/test.mp4 (for subtitle extraction)
└─ HTTP URL: https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4 (for video translation)---
Execution Gate Checklist
Strict Requirement: Agent MUST execute in phase order, cannot proceed without passing current phase!
Phase 0: Environment and Credential Check (HARD-GATE)
| Check Item | Command | Pass Condition | Failure Handling |
|---|---|---|---|
| CLI version | aliyun version | >= 3.3.1 | STOP, see cli-installation-guide.md |
| Credential status | aliyun configure list | Valid status | STOP, guide configuration |
| Plugin installation | aliyun configure set --auto-plugin-install true | Set | Auto-set |
HARD-GATE: Cannot proceed with any subsequent operations without passing!
---
Phase 1: Translation Mode Confirmation (BLOCKING)
AskUserQuestion: "Do you need subtitle translation (translate subtitles only) or speech translation (translate subtitles + replace voiceover)?"
┌─ Subtitle translation → NeedSpeechTranslate: false
└─ Speech translation → NeedSpeechTranslate: true
⚠️ No reply received → STOP, cannot proceed!DO NOT infer translation mode from input type!
---
Phase 2: Subtitle Processing Confirmation (BLOCKING)
AskUserQuestion: "Do you need to erase original subtitles from the video? Do you need to burn-in translated subtitles?"
⚠️ No reply received → STOP, cannot proceed!Parameter Mapping:
| Feature | Parameter | Value |
|---|---|---|
| Erase original subtitles | DetextArea | "Auto" / coordinates / not set (no erasure) |
| Burn-in new subtitles | SubtitleConfig | config object / not set (no burn-in) |
---
Phase 3: Output Path Confirmation (Non-blocking)
| Condition | Processing Method |
|---|---|
| User explicitly specifies | Use user's path |
| User does not specify | Use default path and inform user |
Default Output Rules:
- Bucket: Same bucket as input video
- Directory: Same directory as input video
- Filename:
{source}_translated_{random8}.mp4 - Example:
oss://bucket/videos/demo.mp4→oss://bucket/videos/demo_translated_a1b2c3d4.mp4
DO NOT use shell variables, use Python: python3 -c "import random; print(''.join(random.choices('abcdefghijkmnpqrstuvwxyz23456789', k=8)))"---
Phase 4: Subtitle Review Confirmation (Conditional Blocking)
| Trigger Condition | Processing Method |
|---|---|
| User chooses to review subtitles | BLOCKING, MUST wait for user confirmation of review result |
| User does not need review | Non-blocking, proceed |
CRITICAL: After subtitle extraction, MUST output content as-is for user review, DO NOT change format!
---
Scenario Entry Selector
Key Points:
1. When user inputs local video, MUST first upload to OSS and get HTTP URL
2. When user does not provide subtitle, MUST ask if subtitle extraction and review is needed
User inputs video
│
├─ Local video?
│ └─ Yes → AskUserQuestion: "Please provide OSS upload path"
│ ├─ User provides path → Upload to OSS → Convert to HTTP URL → Continue
│ └─ User does not provide → STOP
│
├─ oss:// format?
│ └─ Yes → Inform user to convert to HTTP URL format
│
└─ HTTP URL format? → Continue
│
├─ User provides SRT file?
│ ├─ Yes → Input type = with_subtitle
│ │ ├─ Translation mode = speech → 【Scenario 4】 ⚠️ MUST ask CustomSrtType
│ │ └─ Translation mode = subtitle → 【Scenario 3】
│ │
│ └─ No → Input type = only_video ⚠️ MUST ask if review needed
│ │
│ ├─ AskUserQuestion: "Do you need to extract subtitles for review first, or translate directly?"
│ │
│ ├─ Need review → 【Scenario 2】 ⚠️ Phase 4 blocking
│ │
│ └─ Direct translation → 【Scenario 1】 (TextSource=OCR_ASR)| Scenario | Name | Blocking Point | TextSource | Flow |
|---|---|---|---|---|
| 0 | Local video upload | OSS upload path inquiry | - | Upload→HTTP URL→Subsequent scenario |
| 1 | Direct translation | Phase 1, 2 | OCR_ASR | Submit translation directly |
| 2 | Subtitle review | Phase 1, 2, Subtitle review inquiry, Phase 4 | SubtitleFile | Extract subtitle→Review→Translate |
| 3 | Subtitle translation + user subtitle | Phase 1, 2 | SubtitleFile | Use user SRT to translate directly |
| 4 | Speech translation + user subtitle | Phase 1, 2 + CustomSrtType confirmation | SubtitleFile | Confirm subtitle language then translate |
Scenario 0 (Local video) detailed flow:
1. AskUserQuestion: "Please provide OSS upload path (format: oss://<bucket>/<path>/<filename>.mp4)"
2. After user specifies path, execute aliyun oss cp <local_path> <oss_path>3. Convert to HTTP URL: https://<bucket>.oss-<region>.aliyuncs.com/<path>/<filename>.mp44. Continue with subsequent scenario flow
Scenario 2 detailed flow:
1. Ask for subtitle detection region (roi parameter)
2. Call CaptionExtraction to extract subtitles, input and output use oss:// format3. Output subtitle content as-is for user review
4. After user confirmation, use reviewed SRT to submit translation
---
Parameter Decision Table
Decision Rules: Clearly define handling for each parameter, DO NOT assume arbitrarily!
| Parameter | Trigger Condition | Handling Method | Default Value | Prohibited Behavior |
|---|---|---|---|---|
NeedSpeechTranslate | Always | MUST ask | None | DO NOT infer from input |
NeedFaceTranslate | Always | Fixed value | false | DO NOT set to true |
DetextArea | User chooses erasure | MUST ask | None | DO NOT set to Auto arbitrarily |
SubtitleConfig | User chooses burn-in | Can use default | Standard style | DO NOT skip confirmation |
TextSource | Scenario decides | Scenario rules | See scenario mapping | DO NOT choose arbitrarily |
CustomSrtType | Scenario 4 | MUST ask | None | DO NOT infer arbitrarily |
OutputConfig.MediaURL | Output path | Can use default | Default rules | DO NOT use shell variables |
JobParams.roi | Subtitle extraction | MUST ask | [[0.5,1],[0,1]] | DO NOT set default arbitrarily |
SourceLanguage | User specifies or inferable | Can use default | Auto detect | Use zh for Chinese only |
TargetLanguage | User specifies | Can use default | en | Ask for other languages |
TextSource Scenario Mapping:
| Scenario | Value | Description |
|---|---|---|
| 1 | OCR_ASR | Auto-detect subtitles |
| 2 | SubtitleFile | Reviewed SRT |
| 3, 4 | SubtitleFile | User-provided SRT |
CustomSrtType Trigger Rules:
| Condition | Value |
|---|---|
| CaptionExtraction extracted | SourceSrt |
| User provides subtitle (Scenario 4) | MUST ask: SourceSrt / TargetSrt |
---
Failure Protection Mechanism
HARD-GATE: After speech translation fails, DO NOT auto-switch to subtitle translation!
API Error Handling
| ErrorCode | Handling Action |
|---|---|
Forbidden.SubscriptionRequired | See ram-policies.md |
InvalidParameter | See api-parameters.md |
InputConfig.Subtitle is invalid | See troubleshooting.md |
JobFailed | Record JobId, ask user if retry needed |
SRT Format Repair Flow
Detect empty subtitle entries → Delete empty entries → Renumber → Upload repaired file → Inform userSee troubleshooting.md for details.
---
CLI Command Templates
IMPORTANT: Before submitting API, MUST reference [api-parameters.md](references/api-parameters.md) to confirm parameter format!
See cli-commands.md for details.
Core Commands:
# Register media asset
aliyun ice register-media-info --input-url "oss://<bucket>/<object>" --media-type video --user-agent AlibabaCloud-Agent-Skills
# Submit subtitle extraction (use OSS path)
aliyun ice submit-iproduction-job \
--function-name CaptionExtraction \
--input "Media=oss://<bucket>/<object> Type=OSS" \
--biz-output "Media=oss://<bucket>/<output>.srt Type=OSS" \
--job-params '{"lang":"ch","roi":[[0.5,1],[0,1]]}' \
--force \
--user-agent AlibabaCloud-Agent-Skills
# Submit video translation
aliyun ice submit-video-translation-job \
--user-agent AlibabaCloud-Agent-SkillsCLI Format Key Points:
- Subtitle extraction uses command namesubmit-iproduction-job(lowercase,-separator)
---inputand--biz-outputformat: space-separated string"Media=... Type=OSS", NOT JSON
- --job-params format: JSON string- MUST add --force to skip plugin parameter validation- All ICE commands MUST add `--user-agent AlibabaCloud-Agent-Skills`
---
Documentation Reference
| Document | Content |
|---|---|
| workflow-details.md | Detailed execution flow for 4 scenarios |
| cli-commands.md | CLI command template library |
| troubleshooting.md | Error handling details |
| api-parameters.md | Complete API parameter documentation |
| ram-policies.md | RAM permission requirements |
| cli-installation-guide.md | CLI installation guide |
---
Key Constraints
- Before submitting API, MUST reference [api-parameters.md](references/api-parameters.md) to confirm parameter format
- All ICE CLI commands MUST add `--user-agent AlibabaCloud-Agent-Skills`
- Subtitle extraction (SubmitIProductionJob) uses `oss://` format
- Video translation (SubmitVideoTranslationJob) uses HTTP URL format, no signing needed
- Local videos MUST first be uploaded to OSS, user MUST provide upload path
NeedFaceTranslateMUST befalseSpeechTranslateandSubtitleTranslateare mutually exclusiveInputConfig.SubtitleMUST use HTTPS format, DO NOT useoss://- Speech translation + SRT input requires
SpeechTranslate.CustomSrtType - DO NOT infer translation mode from input type
---
Task Polling
Mandatory: MUST continuously poll task status until completion (State=Finished) or failure (State=Failed), DO NOT exit early!
| Task Type | Query Command | Interval | Timeout |
|---|---|---|---|
| Subtitle extraction | QueryIProductionJob | 30 seconds | 5 minutes |
| Video translation | get-smart-handle-job | 30 seconds | 30 minutes |
Polling Logic:
Loop polling until:
- State == "Finished" → Return result
- State == "Failed" → Report error
- Exceeds 30 minutes → Report TimeoutError
Prohibited: Return after single query / Skip polling and return JobId directlyTime Reference (3-minute video):
- Subtitle-level translation: 3-5 minutes
- Speech-level translation: 10-20 minutes
---
Result Retrieval
# Get media asset info
aliyun ice get-media-info --media-id "<MediaId>"
# Generate signed URL (for private Bucket)
aliyun oss sign oss://<bucket>/<object> --timeout 3600---
End of Document
API Parameters for Video Translation
This document provides detailed parameter configuration for video translation related APIs.
---
SubmitIProductionJob (Subtitle Extraction)
Basic Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
| FunctionName | String | Yes | Fixed as CaptionExtraction |
| Name | String | No | Task name |
| Input | Object | Yes | Input configuration, OSS address only |
| Output | Object | Yes | Output configuration, OSS address only |
| JobParams | String | No | Algorithm parameters JSON |
Input Configuration
{
"Type": "OSS",
"Media": "oss://<bucket>/<object>"
}| Field | Description |
|---|---|
| Type | Media type: OSS or Media |
| Media | OSS address or Media ID |
Output Configuration
{
"Type": "OSS",
"Media": "oss://<bucket>/{source}-{timestamp}-{sequenceId}.srt"
}Output Format: SRT subtitle file
Supported Placeholders:
{source}: Input filename{timestamp}: Unix timestamp{sequenceId}: Sequence number
JobParams (CaptionExtraction)
{
"fps": 5,
"roi": [[0.5, 1], [0, 1]],
"lang": "ch",
"track": "main"
}All parameters are optional:
| Parameter | Type | Description |
|---|---|---|
| fps | Integer | Sampling frame rate, range [2,10], default 5 |
| roi | Array | Subtitle detection region [[top, bottom], [left, right]], normalized values, default bottom 1/4 of video |
| lang | String | Recognition language: ch(Chinese) / en(English) / ch_ml(Chinese-English mixed) |
| track | String | "main" extract main subtitle track only |
---
SubmitVideoTranslationJob Parameters
InputConfig
Input parameters for video translation task, JSON format.
{
"Type": "Video",
"Video": "<mediaId_or_ossUrl>",
"Subtitle": "<srt_url_or_content>"
}| Field | Type | Required | Description |
|---|---|---|---|
| Type | String | Yes | Input type, fixed as "Video" |
| Video | String | Yes | Video media ID or OSS URL |
| Subtitle | String | No | SRT subtitle file URL or content |
OutputConfig
Output parameters for video translation task, JSON format.
{
"MediaURL": "https://<bucket>.oss-<region>.aliyuncs.com/<object>.mp4",
"Width": null,
"Height": null
}| Field | Type | Required | Description |
|---|---|---|---|
| MediaURL | String | Yes | OSS URL for output video (must use https:// format) |
| Width | Integer | No | Output video width, null means same as source |
| Height | Integer | No | Output video height, null means same as source |
Default Output Path Rules:
>
If user does not specify output path, use the following defaults:
- Bucket: Same bucket as input video
- Path: Same directory as input video
- Filename: {source}_translated_{timestamp}.mp4>
Example:
- Input: oss://my-bucket/videos/demo.mp4- Default output: https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/demo_translated_1711440000.mp4EditingConfig
Configuration parameters for video translation task, JSON format.
{
"SourceLanguage": "zh",
"TargetLanguage": "en",
"NeedSpeechTranslate": false,
"NeedFaceTranslate": false,
"BilingualSubtitle": false,
"SupportEditing": true,
"TextSource": "OCR_ASR",
"DetextArea": null,
"SubtitleTranslate": {
"OcrArea": "Auto",
"SubtitleConfig": {
"Type": "Text",
"FontSize": 95,
"FontColor": "#ffffff",
"Font": "Alibaba PuHuiTi",
"Y": 0.15,
"TextWidth": 0.9,
"Alignment": "Center",
"BorderStyle": 1
}
}
}Main Field Descriptions
| Field | Type | Required | Description |
|---|---|---|---|
| SourceLanguage | String | Yes | Source language code |
| TargetLanguage | String | Yes | Target language code |
| NeedSpeechTranslate | Boolean | No | Whether to use speech-level translation, must be false for subtitle-level |
| NeedFaceTranslate | Boolean | No | Must be set to false, this Skill does not enable face translation |
| BilingualSubtitle | Boolean | No | Whether to use bilingual subtitles |
| SupportEditing | Boolean | No | Whether to support editing |
| TextSource | String | No | Subtitle source, see TextSource details below |
| CustomSrtType | String | Conditionally Required | External subtitle type, required when TextSource=SubtitleFile |
| DetextArea | String | No | Subtitle erasure area, see DetextArea details below |
| SubtitleTranslate | Object | Conditionally Required | Subtitle-level translation config (required when NeedSpeechTranslate=false) |
| SpeechTranslate | Object | Conditionally Required | Speech-level translation config (required when NeedSpeechTranslate=true) |
DetextArea Parameter Details
Used to erase original subtitles from video.
| Value | Description |
|---|---|
| Not set/null | No subtitle erasure |
Auto | Auto-detect erasure area |
[[x, y, width, height]] | Custom erasure range, supports multiple areas |
Custom Erasure Area Parameters:
x: Horizontal distance ratio of subtitle box top-left corner from video top-left, range [0, 1]y: Vertical distance ratio of subtitle box top-left corner from video top-left, range [0, 1]width: Subtitle box width ratio relative to video width, range [0, 1]height: Subtitle box height ratio relative to video height, range [0, 1]
Example (erase bottom 10% of video):
"DetextArea": "[[0, 0.9, 1, 0.1]]"TextSource Parameter Details
| Value | Applicable Scenario | Description |
|---|---|---|
OCR_ASR | User does not need subtitle review | OCR preferred, fallback to ASR if failed (default value) |
SubtitleFile | User provides SRT / reviewed SRT | Subtitle source is external SRT file, requires InputConfig.Subtitle |
ASR | ASR only | Recognize subtitles via speech only |
OCR | OCR only | Recognize subtitles via image only |
ALL | ASR+OCR fusion | ASR primary, OCR correction (subtitle-level only) |
CustomSrtType Parameter Details
Valid only when TextSource=SubtitleFile| Value | Description |
|---|---|
SourceSrt | Subtitle is in source language, needs translation |
TargetSrt | Subtitle is in target language (already translated), burn-in directly |
SubtitleConfig Fields
| Field | Type | Description | Example |
|---|---|---|---|
| Type | String | Subtitle type | "Text" |
| FontSize | Integer | Font size | 60/95/130 |
| FontColor | String | Font color | "#ffffff" |
| Font | String | Font name | "Alibaba PuHuiTi" |
| Y | Float | Vertical position (0=bottom, 1=top) | 0.15 (bottom) |
| TextWidth | Float | Text width ratio | 0.9 |
| Alignment | String | Alignment | "Center" |
| BorderStyle | Integer | Border style | 1 |
---
Translation Mode Configuration
Important Constraints:
- This Skill only supports subtitle-level and speech-level translation, NeedFaceTranslate must be set to false
- SpeechTranslate and SubtitleTranslate cannot be set simultaneously, choose one based on translation mode
- When speech-level translation uses SRT input, must fill `SpeechTranslate.CustomSrtType`
| Mode | NeedSpeechTranslate | NeedFaceTranslate | Config Parameter | Description |
|---|---|---|---|---|
| Subtitle-level (subtitle) | false | false | SubtitleTranslate | Translate and replace subtitles in video |
| Speech-level (speech) | true | false | SpeechTranslate | Speech synthesis of translated content |
Speech-level Translation CustomSrtType Required Rules
Mandatory: When speech-level translation (NeedSpeechTranslate=true) uses SRT file as input, must fill `SpeechTranslate.CustomSrtType`.>
Reference: https://help.aliyun.com/zh/ims/use-cases/introduction-and-examples-of-video-translation-parameters
| SRT Source | CustomSrtType | Description |
|---|---|---|
| CaptionExtraction extracted | SourceSrt | Default, subtitle is in source language, needs translation |
| User provided | Must confirm | Ask user whether subtitle is source or target language |
---
Subtitle Style Mapping
Font Size
| User Option | FontSize Value |
|---|---|
| small | 60 |
| medium | 95 |
| large | 130 |
Subtitle Position
| User Option | Y Value |
|---|---|
| top | 0.15 |
| center | 0.5 |
| bottom | 0.85 |
---
Language Codes
Common Languages
| Language | Code |
|---|---|
| Chinese | zh |
| English | en |
| Japanese | ja |
| Korean | ko |
| French | fr |
| German | de |
| Spanish | es |
| Russian | ru |
| Portuguese | pt |
| Arabic | ar |
For complete language list, see: Alibaba Cloud Video Translation Supported Languages
---
Complete Examples
Scenario A: Direct Translation - Subtitle-level Translation (TextSource=OCR_ASR)
{
"InputConfig": "{\"Type\":\"Video\",\"Video\":\"oss://my-bucket.oss-cn-shanghai.aliyuncs.com/input.mp4\"}",
"OutputConfig": "{\"MediaURL\":\"https://my-bucket.oss-cn-shanghai.aliyuncs.com/output.mp4\"}",
"EditingConfig": "{\"SourceLanguage\":\"zh\",\"TargetLanguage\":\"en\",\"NeedSpeechTranslate\":false,\"NeedFaceTranslate\":false,\"TextSource\":\"OCR_ASR\",\"SubtitleTranslate\":{\"OcrArea\":\"Auto\",\"SubtitleConfig\":{\"Type\":\"Text\",\"FontSize\":95,\"FontColor\":\"#ffffff\",\"Font\":\"Alibaba PuHuiTi\",\"Y\":0.15}}}"
}Scenario B: Using External SRT File (TextSource=SubtitleFile)
{
"InputConfig": "{\"Type\":\"Video\",\"Video\":\"oss://my-bucket.oss-cn-shanghai.aliyuncs.com/input.mp4\",\"Subtitle\":\"https://my-bucket.oss-cn-shanghai.aliyuncs.com/subtitle.srt\"}",
"OutputConfig": "{\"MediaURL\":\"https://my-bucket.oss-cn-shanghai.aliyuncs.com/output.mp4\"}",
"EditingConfig": "{\"SourceLanguage\":\"zh\",\"TargetLanguage\":\"en\",\"NeedSpeechTranslate\":false,\"NeedFaceTranslate\":false,\"TextSource\":\"SubtitleFile\",\"CustomSrtType\":\"SourceSrt\",\"SubtitleTranslate\":{\"OcrArea\":\"Auto\",\"SubtitleConfig\":{\"Type\":\"Text\",\"FontSize\":95,\"FontColor\":\"#ffffff\",\"Font\":\"Alibaba PuHuiTi\",\"Y\":0.15}}}"
}Speech-level Translation
{
"InputConfig": "{\"Type\":\"Video\",\"Video\":\"oss://my-bucket.oss-cn-shanghai.aliyuncs.com/input.mp4\"}",
"OutputConfig": "{\"MediaURL\":\"https://my-bucket.oss-cn-shanghai.aliyuncs.com/output.mp4\"}",
"EditingConfig": "{\"SourceLanguage\":\"zh\",\"TargetLanguage\":\"en\",\"NeedSpeechTranslate\":true,\"NeedFaceTranslate\":false,\"SpeechTranslate\":{\"VoiceConfig\":{\"Voice\":\"zhiyan_emo\"}}}"
}Speech-level Translation + SRT Input (Must fill CustomSrtType)
{
"InputConfig": "{\"Type\":\"Video\",\"Video\":\"oss://my-bucket.oss-cn-shanghai.aliyuncs.com/input.mp4\",\"Subtitle\":\"https://my-bucket.oss-cn-shanghai.aliyuncs.com/subtitle.srt\"}",
"OutputConfig": "{\"MediaURL\":\"https://my-bucket.oss-cn-shanghai.aliyuncs.com/output.mp4\"}",
"EditingConfig": "{\"SourceLanguage\":\"zh\",\"TargetLanguage\":\"en\",\"NeedSpeechTranslate\":true,\"NeedFaceTranslate\":false,\"TextSource\":\"SubtitleFile\",\"SpeechTranslate\":{\"CustomSrtType\":\"SourceSrt\",\"VoiceConfig\":{\"Voice\":\"zhiyan_emo\"}}}"
}Note:
- All modes must set NeedFaceTranslate: false- When usingTextSource=SubtitleFile, must specifySubtitlefield in InputConfig
- `InputConfig.Subtitle` must use HTTPS format, `oss://` prefix is prohibited: https://<bucket>.oss-<region>.aliyuncs.com/<object>- Speech-level translation + SRT input requires filling `SpeechTranslate.CustomSrtType`
---
Subtitle Extraction Result Format
Subtitle extraction task outputs SRT format file directly to specified OSS path.
SRT Format Example
1
00:00:00,000 --> 00:00:02,500
First sentence
2
00:00:02,500 --> 00:00:05,000
Second sentenceSRT Format Validation and Auto Repair
Mandatory: Before submitting speech translation task, must check and fix empty subtitle entries in SRT file!
Problem Cause: InputConfig.Subtitle is invalid error is usually caused by SRT containing empty subtitle entries:
3
00:00:08,000 --> 00:00:10,600
4
00:00:10,600 --> 00:00:12,000
This is a normal subtitleEntry 3 above has only sequence number and timeline, no content, does not conform to SRT specification.
Processing Flow: 1. Check SRT file, find all empty subtitle entries 2. Auto-delete empty subtitle entries, renumber 3. Inform user: "Detected X empty subtitle entries, auto-deleted and renumbered" 4. Upload repaired SRT file
---
Error Handling
Common Error Codes
| ErrorCode | Description | Solution |
|---|---|---|
InvalidParameter | Parameter format error | Check parameter format |
Forbidden | Insufficient permissions | Check RAM permissions |
QuotaExceeded | Resource quota exceeded | Contact customer service to increase quota |
InputConfig.Subtitle is invalid | SRT format error | Check if contains empty subtitle entries |
Subtitle Extraction Failed
Extraction failed → Request user to provide SRT fileLanguage Detection Failed
Analyze extracted result characters:
- Contains Chinese characters → zh
- Contains Japanese characters → ja
- Contains Korean characters → ko
- Mainly English → en
- Cannot determine → Ask userSpeech Translation Failed Handling
Mandatory: After speech translation fails, DO NOT auto-switch to subtitle translation, must confirm with user first!
>
Use AskUserQuestion tool to ask user:- "Speech translation failed, error message: {ErrorMessage}. Do you want to try switching to subtitle translation mode?"
---
Limits and Constraints
| Limit Item | Description |
|---|---|
| Video size | Recommended not to exceed 2GB |
| Video duration | Recommended not to exceed 2 hours |
| Supported formats | mp4, mov, avi and other common formats |
CLI Command Templates
This document provides ready-to-use CLI command templates.
---
Environment Check
# Check CLI version
aliyun version
# Check credential configuration
aliyun sts GetCallerIdentity
# Check ICE plugin
aliyun ice help---
Media Registration
aliyun ice register-media-info \
--input-url "oss://<bucket>/<object>" \
--media-type video \
--user-agent AlibabaCloud-Agent-Skills| Parameter | Required | Description |
|---|---|---|
--input-url | Yes | OSS address or VOD media address |
--media-type | No | video/audio/image |
--title | No | Title |
--overwrite | No | Overwrite registered media |
Returns: MediaId
---
Subtitle Extraction (CaptionExtraction)
CLI Format Key: Use command namesubmit-iproduction-job+--force
Method 1: Using OSS Path
aliyun ice submit-iproduction-job \
--function-name CaptionExtraction \
--input "Media=oss://<bucket>/<object> Type=OSS" \
--biz-output "Media=oss://<bucket>/<output>.srt Type=OSS" \
--job-params '{"lang":"ch","roi":[[0.5,1],[0,1]]}' \
--name "<task_name>" \
--force \
--user-agent AlibabaCloud-Agent-SkillsMethod 2: Using MediaId (Registered Media)
aliyun ice submit-iproduction-job \
--function-name CaptionExtraction \
--input "Media=<mediaId> Type=MediaId" \
--biz-output "Media=oss://<bucket>/<output>.srt Type=OSS" \
--job-params '{"lang":"ch","roi":[[0.5,1],[0,1]]}' \
--name "<task_name>" \
--force \
--user-agent AlibabaCloud-Agent-SkillsCLI Format Notes:
- Use command name:
submit-iproduction-job(lowercase,-separator) - Use lowercase parameter names:
--function-name,--input,--biz-output,--job-params - `--input` and `--biz-output` format: space-separated string
"Media=... Type=OSS", NOT JSON - `--job-params` format: JSON string
- Add
--forceto skip plugin parameter validation - `--job-params` is required, must include
roiparameter
JobParams Parameters:
{
"fps": 5,
"roi": [[0.5, 1], [0, 1]],
"lang": "ch",
"track": "main"
}| Parameter | Required | Description |
|---|---|---|
roi | Yes | Subtitle detection region [[top,bottom],[left,right]] |
lang | No | ch(Chinese) / en(English) / ch_ml(Chinese-English mixed) |
fps | No | Sampling frame rate [2,10], default 5 |
track | No | "main" extract main subtitle track only |
---
Query Subtitle Extraction Task
aliyun ice QueryIProductionJob \
--JobId "<job_id>" \
--force \
--user-agent AlibabaCloud-Agent-Skills| Status | Description |
|---|---|
| Init | Initializing |
| Queuing | In queue |
| Analysing | Analyzing |
| Processing | Processing |
| Success | Success |
| Fail | Failed |
---
Submit Video Translation Task
CLI Format Key: JSON format parameters + --region to specify service regionMode 1: Subtitle-level Translation
aliyun ice submit-video-translation-job \
--input-config '{"Type":"Video","Video":"https://<bucket>.oss-<region>.aliyuncs.com/<object>.mp4"}' \
--output-config '{"MediaURL":"https://<bucket>.oss-<region>.aliyuncs.com/<output>.mp4"}' \
--editing-config '{"SourceLanguage":"zh","TargetLanguage":"en","NeedSpeechTranslate":false,"NeedFaceTranslate":false,"TextSource":"OCR_ASR","SubtitleTranslate":{"OcrArea":"Auto","SubtitleConfig":{"Type":"Text","FontSize":48,"FontColor":"#ffffff","Font":"STHeiti","Y":0.15}}}' \
--title "<task_title>" \
--region <region> \
--user-agent AlibabaCloud-Agent-SkillsMode 2: Speech-level Translation
aliyun ice submit-video-translation-job \
--input-config '{"Type":"Video","Video":"https://<bucket>.oss-<region>.aliyuncs.com/<object>.mp4"}' \
--output-config '{"MediaURL":"https://<bucket>.oss-<region>.aliyuncs.com/<output>.mp4"}' \
--editing-config '{"SourceLanguage":"zh","TargetLanguage":"en","NeedSpeechTranslate":true,"NeedFaceTranslate":false,"SpeechTranslate":{"VoiceConfig":{"Voice":"zhiyan_emo"}}}' \
--title "<task_title>" \
--region <region> \
--user-agent AlibabaCloud-Agent-SkillsMode 3: Using External SRT File
Key:InputConfig.Subtitlemust use HTTPS format,oss://prefix is prohibited
aliyun ice submit-video-translation-job \
--input-config '{"Type":"Video","Video":"https://<bucket>.oss-<region>.aliyuncs.com/<object>.mp4","Subtitle":"https://<bucket>.oss-<region>.aliyuncs.com/<subtitle>.srt"}' \
--output-config '{"MediaURL":"https://<bucket>.oss-<region>.aliyuncs.com/<output>.mp4"}' \
--editing-config '{"SourceLanguage":"zh","TargetLanguage":"en","NeedSpeechTranslate":false,"NeedFaceTranslate":false,"TextSource":"SubtitleFile","CustomSrtType":"SourceSrt","SubtitleConfig":{"Type":"Text","FontSize":48,"FontColor":"#ffffff","Font":"STHeiti","Y":0.15}}' \
--title "<task_title>" \
--region <region> \
--user-agent AlibabaCloud-Agent-SkillsEditingConfig Core Fields:
| Field | Description |
|---|---|
NeedSpeechTranslate | false=subtitle-level, true=speech-level |
NeedFaceTranslate | Must be false |
TextSource | OCR_ASR(default) / SubtitleFile(use SRT) |
CustomSrtType | Required when using SRT: SourceSrt(source language) / TargetSrt(already translated) |
---
Query Video Translation Task
aliyun ice get-smart-handle-job \
--job-id "<job_id>" \
--user-agent AlibabaCloud-Agent-Skills| State | Description |
|---|---|
| Created | Created |
| Executing | Executing |
| Finished | Completed |
| Failed | Failed |
Polling Recommendation: Query every 30 seconds, timeout 30 minutes
---
OSS Commands
Note: OSS commands do not support --user-agentLocal Video Upload
# Upload local video to OSS
aliyun oss cp <local_path> oss://<bucket>/<path>/<filename>.mp4
# Example
aliyun oss cp /Users/demo/videos/test.mp4 oss://my-bucket/videos/test.mp4OSS URL Format Reference
Important: Different APIs use different address formats!
| API | Address Format | Example |
|---|---|---|
SubmitIProductionJob (subtitle extraction) | `oss://` format | oss://my-bucket/videos/test.mp4 |
SubmitVideoTranslationJob (video translation) | HTTP URL format | https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4 |
Key: Subtitle extraction uses oss://, video translation uses HTTP URL!Format Conversion Rules
oss:// format ⇄ HTTP URL format
oss://my-bucket/videos/test.mp4
⇄
https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/test.mp4Conversion Formula: oss://<bucket>/<path> → https://<bucket>.oss-<region>.aliyuncs.com/<path>
| Parameter | Description |
|---|---|
<bucket> | OSS Bucket name |
<region> | Region, e.g., cn-shanghai, cn-beijing |
<path> | File path |
Other OSS Commands
# List files
aliyun oss ls oss://<bucket>/<prefix>
# Download file
aliyun oss cp oss://<bucket>/<object> <local_path>
# Generate signed URL (for result sharing, required for private Bucket)
aliyun oss sign oss://<bucket>/<object> --timeout 3600---
Command Quick Reference
| Purpose | Command |
|---|---|
| Environment check | aliyun version |
| Credential check | aliyun sts GetCallerIdentity |
| Register media | aliyun ice register-media-info |
| Submit subtitle extraction | aliyun ice SubmitIProductionJob --FunctionName CaptionExtraction --force |
| Query subtitle extraction | aliyun ice QueryIProductionJob --force |
| Submit video translation | aliyun ice submit-video-translation-job |
| Query video translation | aliyun ice get-smart-handle-job |
| Query media info | aliyun ice get-media-info |
| List OSS | aliyun oss ls |
| Download from OSS | aliyun oss cp (from oss) |
| Upload to OSS | aliyun oss cp (to oss) |
| Sign URL | aliyun oss sign |
Aliyun CLI Installation & Configuration Guide
Complete guide for installing and configuring Aliyun CLI.
Aliyun CLI 3.3.1+: Supports installing and using all published Alibaba Cloud product plugins. Make sure to upgrade to 3.3.1 or later for full plugin ecosystem coverage.
Installation
macOS
Using Homebrew (Recommended)
brew install aliyun-cli
# Upgrade to latest
brew upgrade aliyun-cli
# Verify version (>= 3.3.1)
aliyun versionUsing Binary
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz
# Extract
tar -xzf aliyun-cli-macosx-latest-amd64.tgz
# Move to PATH
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionLinux
Debian/Ubuntu
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionCentOS/RHEL
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionARM64 Architecture
# Download ARM64 version
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-arm64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-arm64.tgz
sudo mv aliyun /usr/local/bin/Windows
Using Binary 1. Download from: https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip 2. Extract the ZIP file 3. Add the directory to your PATH environment variable 4. Open new Command Prompt or PowerShell 5. Verify: aliyun version
Using PowerShell
# Download
Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip"
# Extract
Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli
# Add to PATH (requires admin privileges)
$env:Path += ";C:\aliyun-cli"
[Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::Machine)
# Verify
aliyun versionConfiguration
Quick Start
aliyun configure set \
--mode AK \
--access-key-id <your-access-key-id> \
--access-key-secret <your-access-key-secret> \
--region cn-hangzhouAll aliyun configure commands support non-interactive flags, which is the recommended approach — it works in scripts, CI/CD pipelines, and agent-driven automation without hanging on stdin prompts.
Where to Get Access Keys
1. Log in to Aliyun Console: https://ram.console.aliyun.com/ 2. Navigate to: AccessKey Management 3. Create a new AccessKey pair 4. Save the secret immediately — it's only shown once
Configuration Modes
Aliyun CLI supports 6 authentication modes. All examples below use non-interactive flags.
1. AK Mode (Access Key)
Most common mode for personal accounts and scripts.
aliyun configure set \
--mode AK \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--region cn-hangzhouConfiguration is stored in ~/.aliyun/config.json:
{
"current": "default",
"profiles": [
{
"name": "default",
"mode": "AK",
"access_key_id": "LTAI5tXXXXXXXX",
"access_key_secret": "8dXXXXXXXXXXXXXXXXXXXXXXXX",
"region_id": "cn-hangzhou",
"output_format": "json",
"language": "en"
}
]
}2. StsToken Mode (Temporary Credentials)
For short-lived access (tokens expire in 1-12 hours).
aliyun configure set \
--mode StsToken \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--sts-token v1.0:XXXXXXXXXXXXXXXX \
--region cn-hangzhouUse cases: CI/CD pipelines, temporary access for external contractors, cross-account access.
3. RamRoleArn Mode (Assume RAM Role)
Assume a RAM role for elevated or cross-account access.
aliyun configure set \
--mode RamRoleArn \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--ram-role-arn acs:ram::123456789012:role/AdminRole \
--role-session-name my-session \
--region cn-hangzhouUse cases: cross-account resource access, temporary elevated privileges, role-based access control.
4. EcsRamRole Mode (ECS Instance RAM Role)
Use the RAM role attached to an ECS instance — no credentials needed.
aliyun configure set \
--mode EcsRamRole \
--ram-role-name MyEcsRole \
--region cn-hangzhouRequirements: must be running on an ECS instance with a RAM role attached.
Use cases: scripts and automation running on ECS instances.
5. RsaKeyPair Mode (RSA Key Pair)
Use RSA key pair for authentication (generate key pair in Aliyun Console first).
aliyun configure set \
--mode RsaKeyPair \
--private-key /path/to/private-key.pem \
--key-pair-name my-key-pair \
--region cn-hangzhou6. RamRoleArnWithEcs Mode (ECS + RAM Role)
Combine ECS instance role with RAM role assumption for cross-account access from ECS.
aliyun configure set \
--mode RamRoleArnWithEcs \
--ram-role-name MyEcsRole \
--ram-role-arn acs:ram::123456789012:role/TargetRole \
--role-session-name my-session \
--region cn-hangzhouEnvironment Variables
Highest priority - overrides config file
Access Key Mode
export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_REGION_ID=cn-hangzhouSTS Token Mode
export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_SECURITY_TOKEN=your_sts_token
export ALIBABA_CLOUD_REGION_ID=cn-hangzhouECS RAM Role Mode
export ALIBABA_CLOUD_ECS_METADATA=role_nameUse Case:
- CI/CD pipelines
- Docker containers
- Temporary credential override
Managing Multiple Profiles
Create Named Profiles
aliyun configure set --profile projectA \
--mode AK \
--access-key-id LTAI5tAAAAAAAA \
--access-key-secret 8dAAAAAAAAAAAAAAAAAAAAAAAA \
--region cn-hangzhou
aliyun configure set --profile projectB \
--mode AK \
--access-key-id LTAI5tBBBBBBBB \
--access-key-secret 8dBBBBBBBBBBBBBBBBBBBBBBBB \
--region cn-shanghaiUse Specific Profile
aliyun ecs describe-instances --profile projectA
export ALIBABA_CLOUD_PROFILE=projectA
aliyun ecs describe-instances # Uses projectAList and Switch Profiles
aliyun configure list # List all profiles
aliyun configure set --current projectA # Switch default profileCredential Priority
Credentials are loaded in this order (first found wins):
1. Command-line flag: --profile <name> 2. Environment variable: ALIBABA_CLOUD_PROFILE 3. Environment credentials: ALIBABA_CLOUD_ACCESS_KEY_ID, etc. 4. Configuration file: ~/.aliyun/config.json (current profile) 5. ECS Instance RAM Role: If running on ECS with attached role
Verification
Test Authentication
# Basic test - list regions
aliyun ecs describe-regions
# Expected output: JSON array of regionsIf successful, you'll see:
{
"Regions": {
"Region": [
{
"RegionId": "cn-hangzhou",
"RegionEndpoint": "ecs.cn-hangzhou.aliyuncs.com",
"LocalName": "华东 1(杭州)"
},
...
]
},
"RequestId": "..."
}If failed, you'll see error messages:
InvalidAccessKeyId.NotFound- Wrong Access Key IDSignatureDoesNotMatch- Wrong Access Key SecretInvalidSecurityToken.Expired- STS token expired (for StsToken mode)Forbidden.RAM- Insufficient permissions
Debug Configuration
# Show current configuration
aliyun configure get
# Test with debug logging
aliyun ecs describe-regions --log-level=debug
# Check credential provider
aliyun configure get modeSecurity Best Practices
1. Use RAM Users (Not Root Account)
❌ Don't: Use Aliyun root account credentials ✅ Do: Create RAM users with specific permissions
# Create RAM user in console
# Attach only necessary policies
# Use RAM user's access keys2. Principle of Least Privilege
Grant only the minimum permissions needed:
# Example: Read-only ECS access
# Attach policy: AliyunECSReadOnlyAccess3. Rotate Access Keys Regularly
# Create new access key in RAM Console, then update configuration
aliyun configure set --access-key-id NEW_KEY --access-key-secret NEW_SECRET
# Delete old access key from console4. Use STS Tokens for Temporary Access
aliyun configure set --mode StsToken \
--access-key-id XXXX --access-key-secret XXXX \
--sts-token XXXX --region cn-hangzhou5. Use ECS RAM Roles When Possible
aliyun configure set --mode EcsRamRole --ram-role-name MyRole --region cn-hangzhou6. Never Commit Credentials
# Add to .gitignore
echo "~/.aliyun/config.json" >> .gitignore
# Use environment variables in CI/CD instead7. Secure Config File
# Restrict permissions
chmod 600 ~/.aliyun/config.jsonTroubleshooting
Issue: Command Not Found
# Check installation
which aliyun
# Check PATH
echo $PATH
# Reinstall or add to PATHIssue: Authentication Failed
# Verify configuration
aliyun configure get
# Test with debug
aliyun ecs describe-regions --log-level=debug
# Check credentials in console
# Verify access key is activeIssue: Permission Denied
# Error: Forbidden.RAM
# Check RAM user permissions
# Attach necessary policies in RAM console
# Example: AliyunECSFullAccess for ECS operationsIssue: STS Token Expired
# Error: InvalidSecurityToken.Expired
# Reconfigure with new token
aliyun configure set --mode StsToken \
--access-key-id XXXX --access-key-secret XXXX \
--sts-token NEW_TOKEN --region cn-hangzhouIssue: Wrong Region
# Some resources may not exist in the specified region
# Check available regions
aliyun ecs describe-regions
# Update default region
aliyun configure set region cn-shanghaiAdvanced Configuration
Custom Endpoint
# Use custom or private endpoint
export ALIBABA_CLOUD_ECS_ENDPOINT=ecs-vpc.cn-hangzhou.aliyuncs.comProxy Settings
# HTTP proxy
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080
# No proxy for specific domains
export NO_PROXY=localhost,127.0.0.1,.aliyuncs.comTimeout Settings
# Connection timeout (default: 10s)
export ALIBABA_CLOUD_CONNECT_TIMEOUT=30
# Read timeout (default: 10s)
export ALIBABA_CLOUD_READ_TIMEOUT=30Next Steps
After installation and configuration:
1. Install plugins for services you need (v3.3.1+ supports all published product plugins):
aliyun plugin install --names ecs vpc rds
# List all available plugins
aliyun plugin list-remote2. Explore commands:
aliyun ecs --help
aliyun fc --help3. Read documentation:
- Command Syntax Guide
- Global Flags Reference
- Common Scenarios
References
- Official Documentation: https://help.aliyun.com/zh/cli/
- RAM Console: https://ram.console.aliyun.com/
- Access Key Management: https://ram.console.aliyun.com/manage/ak
- Plugin Repository: https://github.com/aliyun/aliyun-cli
RAM Policies for Video Translation Skill
This document lists all RAM permissions required for the Video Translation Skill.
Permission List
ICE (Intelligent Cloud Editing) Permissions
| Permission | Description | Usage |
|---|---|---|
ice:RegisterMediaInfo | Register media info | Get MediaId |
ice:SubmitIProductionJob | Submit intelligent production job | Subtitle extraction (CaptionExtraction) |
ice:QueryIProductionJob | Query intelligent production job | Query subtitle extraction job status |
ice:GetSmartHandleJob | Query intelligent job result | Query video translation job status |
ice:SubmitVideoTranslationJob | Submit video translation job | Video translation core functionality |
ice:GetMediaInfo | Query media info | Get final video URL |
OSS Permissions
| Permission | Description | Usage |
|---|---|---|
oss:GetObject | Read OSS object | Read input video |
oss:PutObject | Write OSS object | Write translation result |
oss:ListObjects | List OSS objects | Verify file existence |
Complete RAM Policy JSON
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ice:RegisterMediaInfo",
"ice:SubmitIProductionJob",
"ice:QueryIProductionJob",
"ice:GetSmartHandleJob",
"ice:SubmitVideoTranslationJob",
"ice:GetMediaInfo"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"oss:GetObject",
"oss:PutObject",
"oss:ListObjects"
],
"Resource": [
"acs:oss:*:*:<your-input-bucket>/*",
"acs:oss:*:*:<your-output-bucket>/*"
]
}
]
}Permission Configuration Steps
1. Login to RAM Console 2. Create a custom permission policy, paste the JSON above (replace bucket names) 3. Grant the policy to the RAM user or role that needs to use video translation
Important Notes
- Principle of Least Privilege: Recommend limiting OSS permissions to specific buckets
- Region Restriction: If further restriction is needed, specify region in Resource
- Subscription Purchase: Even with
AliyunICEFullAccessgranted, if you encounter "not subscribed" error, you need to purchase a subscription package at IMS Video Translation Product Page before using the service
# Video Translation Skill Dependencies
# Install: pip3 install -r requirements.txt
alibabacloud_tea_openapi==0.3.4
alibabacloud_credentials==0.3.5
alibabacloud_tea_util==0.3.4
alibabacloud_openapi_util==0.1.1Troubleshooting Guide for Video Translation Skill
This document provides detailed error handling procedures for video translation task execution.
---
1. API Error Handling
1.1 Forbidden.SubscriptionRequired
Error Meaning: User has not enabled video translation service or has insufficient permissions.
Handling Flow:
Detect error → Output permission requirements → Guide user to activate serviceRequired Permission List:
| Service | Permission |
|---|---|
| ICE | ice:RegisterMediaInfo |
| ICE | ice:SubmitIProductionJob |
| ICE | ice:QueryIProductionJob |
| ICE | ice:GetSmartHandleJob |
| ICE | ice:SubmitVideoTranslationJob |
| OSS | oss:GetObject |
| OSS | oss:PutObject |
| OSS | oss:ListObjects |
Standard Output:
## Insufficient Permissions
You need to enable the video translation service and configure the corresponding RAM permissions.
**Required Permissions**:
- ice:RegisterMediaInfo
- ice:SubmitIProductionJob
- ice:QueryIProductionJob
- ice:GetSmartHandleJob
- ice:SubmitVideoTranslationJob
- oss:GetObject
- oss:PutObject
- oss:ListObjects
**Solution**:
1. Login to [RAM Console](https://ram.console.aliyun.com/)
2. Create a custom permission policy with the above permissions
3. Grant the policy to the current RAM user or role
**Next Step**: Re-execute the task after configuration is complete---
1.2 InvalidParameter
Error Meaning: API parameter format error.
Handling Flow:
Detect error → Parse error details → Output correction suggestions based on table belowCommon Parameter Errors:
| Parameter | Common Error | Correction Method |
|---|---|---|
InputConfig | JSON format error | Ensure JSON string is properly escaped, use single quotes to wrap |
InputConfig.Media | Used HTTP URL | SubmitIProductionJob uses oss:// format |
OutputConfig.MediaURL | Used oss:// prefix | SubmitVideoTranslationJob uses HTTP URL format |
EditingConfig | Missing required fields | Add SourceLanguage, TargetLanguage |
DetextArea | Format error | Use string format: "[[0, 0.9, 1, 0.1]]" |
SubtitleConfig | Missing configuration | Add Type, FontSize, FontColor fields |
CustomSrtType | Not filled | Required for speech translation + SRT input: SourceSrt or TargetSrt |
Standard Output:
## Parameter Format Error
**Error Message**: {original error}
**Possible Causes**:
- {cause 1}
- {cause 2}
**Correction Suggestions**:
- {suggestion 1}
- {suggestion 2}
**Next Step**: Re-execute after correcting parameters---
1.3 InputConfig.Subtitle is invalid
Error Meaning: SRT subtitle file format is invalid, usually contains empty subtitle entries.
Handling Flow: See SRT Format Repair Flow
---
1.4 JobFailed
Error Meaning: Task execution failed.
Handling Flow:
Detect error → Record JobId → Analyze failure reason → Ask user to retry or switch modeStandard Output:
## Task Execution Failed
**JobId**: {JobId}
**Failure Reason**: {failure reason}
**Suggested Solutions**:
1. Retry task
2. Check input video format
3. Contact technical support
**Next Step**: Confirm whether to retry the task?---
2. SRT Format Repair Flow
2.1 Problem Identification
Empty subtitle entry example:
1
00:00:00,000 --> 00:00:02,500
First sentence
2
00:00:02,500 --> 00:00:05,000
Second sentence
3
00:00:08,000 --> 00:00:10,600
4
00:00:10,600 --> 00:00:12,000
This is normal subtitleProblem: Entry 3 has only sequence number and timeline, no subtitle content.
2.2 Repair Steps
Detect empty subtitle entries → Delete empty entries → Renumber → Upload repaired file → Inform user2.3 Repair Command
Use Python script to auto-repair:
import re
def fix_srt(content: str) -> tuple[str, int]:
"""
Fix empty subtitle entries in SRT file.
Args:
content: SRT file content
Returns:
(repaired content, count of removed empty entries)
"""
# Match SRT entries: sequence + timeline + content
pattern = r'(\d+)\n(\d{2}:\d{2}:\d{2},\d{3} --> \d{2}:\d{2}:\d{2},\d{3})\n(.*?)(?=\n\d+\n|\Z)'
matches = re.findall(pattern, content, re.DOTALL)
# Filter out entries with empty content
valid_entries = [(idx, timecode, text.strip())
for idx, timecode, text in matches
if text.strip()]
# Renumber and build new content
result = []
for new_idx, (_, timecode, text) in enumerate(valid_entries, 1):
result.append(f"{new_idx}\n{timecode}\n{text}\n")
removed_count = len(matches) - len(valid_entries)
return '\n'.join(result), removed_count2.4 Standard Output
## SRT File Repair
**Detection Result**: Found {count} empty subtitle entries
**Action Taken**: Auto-deleted empty entries and renumbered
**Repaired File**: {repaired OSS path}
**Next Step**: Continue task with repaired SRT file---
3. Speech Translation Failed Handling
HARD-GATE: After speech translation fails, DO NOT auto-switch to subtitle translation mode, must ask user first!
3.1 Handling Flow
Speech translation failed → Output error message → AskUserQuestion to ask about switching → Execute after user confirmation3.2 Prohibited Actions
- DO NOT auto-switch to subtitle translation mode
- DO NOT assume user's choice
- DO NOT skip user confirmation
3.3 Standard Inquiry
Use AskUserQuestion tool:
## Speech Translation Failed
**Error Message**: {original error}
**Available Options**:
1. Switch to subtitle translation mode (translate subtitles only, no voiceover replacement)
2. Retry speech translation
3. Terminate task
**Please Select**: How would you like to proceed?3.4 User Choice Handling
| User Choice | Follow-up Action |
|---|---|
| Switch to subtitle translation | Set NeedSpeechTranslate: false, resubmit task |
| Retry speech translation | Resubmit task with same parameters |
| Terminate task | Output termination info, record completed steps |
---
4. AskUserQuestion No Response Handling
4.1 Timeout Handling Rules
| Timeout | Action |
|---|---|
| 30 seconds | Output waiting prompt, continue waiting |
| 60 seconds | Output pause info, retain step records |
4.2 30 Second Timeout Output
## Waiting for Your Response
Waiting for your confirmation to continue the task...
**Current Status**: Waiting for user to confirm translation mode
**Time Waited**: 30 seconds
Please reply when ready.4.3 60 Second Timeout Output
## Task Paused
Task has been paused due to no response for an extended period.
**Completed Steps**:
- step 1: {step 1 description}
- step 2: {step 2 description}
**Waiting Confirmation**: {question waiting for confirmation}
**Resume**: Please reply to your question, I will continue from the current step.---
5. Failure Message Output Template
5.1 General Template
## Task Execution Failed
**Failure Phase**: {phase name}
**Failure Type**: {type: API call failed / User confirmation timeout / Parameter error / Other}
**JobId**: {if applicable}
**Error Message**:{original error}
**Suggested Solutions**:
- {solution 1}
- {solution 2}
**Completed Steps**:
- step 1: {step 1 description}
- step 2: {step 2 description}
**Next Step**: Re-execute after confirmation5.2 Phase Name Reference
| Phase | Name |
|---|---|
| 0 | Environment and Credential Check |
| 1 | Translation Mode Confirmation |
| 2 | Subtitle Processing Confirmation |
| 3 | Output Path Confirmation |
| 4 | Subtitle Review Confirmation |
| 5 | Task Submission |
| 6 | Task Polling |
| 7 | Result Retrieval |
---
6. Common Issue Troubleshooting
6.1 Video Cannot Be Processed
| Symptom | Possible Cause | Solution |
|---|---|---|
| Video format not supported | Non-standard format | Convert to mp4/mov format |
| Video too large | Exceeds 2GB | Compress or split video |
| Video duration too long | Exceeds 2 hours | Process in segments |
6.2 Subtitle Extraction Failed
| Symptom | Possible Cause | Solution |
|---|---|---|
| No subtitles detected | Video has no subtitles or subtitles are blurry | Provide external SRT file |
| Language detection error | Mixed languages | Manually specify SourceLanguage |
| Empty extraction result | ROI area setting error | Adjust ROI parameter |
6.3 Translation Quality Issues
| Symptom | Possible Cause | Solution |
|---|---|---|
| Inaccurate translation | Technical terms | Provide terminology list or use reviewed subtitles |
| Incorrect subtitle position | SubtitleConfig settings | Adjust Y value and TextWidth |
| Subtitles blocking image | Y value setting improper | Adjust subtitle position parameter |
---
7. Error Recovery Strategy
7.1 Retryable Errors
The following errors can be auto-retried:
| Error Type | Max Retries | Retry Interval |
|---|---|---|
| Network timeout | 3 | 5 seconds |
| Service temporarily unavailable | 3 | 10 seconds |
| Resource quota exceeded (transient) | 1 | 30 seconds |
7.2 Non-Retryable Errors
The following errors require user intervention:
- Insufficient permissions (Forbidden.SubscriptionRequired)
- Parameter error (InvalidParameter)
- Video format not supported
- SRT format error
7.3 Retry Flow
Detect retryable error → Wait interval → Retry → Success/Failure
↓
Max retries reached → Output failure info → Ask user---
End of Document
Workflow Details
This document provides detailed execution flow and timing for 4 scenarios.
---
Scenario Overview
| Scenario | Name | Entry Condition | TextSource |
|---|---|---|---|
| 1 | Direct Translation | User provides video only, no review needed | OCR_ASR |
| 2 | Subtitle Review | User provides video only, needs subtitle review first | SubtitleFile |
| 3 | Subtitle Translation + User Subtitle | User provides video + SRT, subtitle translation mode | SubtitleFile |
| 4 | Speech Translation + User Subtitle | User provides video + SRT, speech translation mode | SubtitleFile |
---
Scenario 1: Direct Translation
Execution Flow
Phase 0: Environment Check → Phase 1: Translation Mode Confirmation → Phase 2: Subtitle Processing Confirmation →
Phase 3: Output Path Confirmation → Phase 5: Task Submission → Phase 6: Task Polling → Phase 7: Result RetrievalDetailed Steps
Step 1: Environment Check (Phase 0)
- Check CLI version >= 3.3.1
- Check credential status (aliyun configure list)
- Check input video OSS file exists
Duration: ~10 seconds
Step 2: Register Media
aliyun ice register-media-info --input-url "oss://<bucket>/<object>" --media-type videoDuration: ~3-5 seconds (async task)
Step 3: Translation Mode Confirmation (Phase 1) ⚠️ BLOCKING
Use AskUserQuestion to ask:
- "Do you need subtitle translation (translate subtitles only) or speech translation (translate subtitles + replace voiceover)?"
Must wait for user response!
Step 4: Subtitle Processing Confirmation (Phase 2) ⚠️ BLOCKING
Use AskUserQuestion to ask:
- "Do you need to erase original subtitles from the video?"
- "Do you need to burn-in translated subtitles?"
Must wait for user response!
Step 5: Output Path Confirmation (Phase 3)
- User specifies path → Use user's path
- User does not specify → Use default path and inform user
Default Path Rule: {source}_translated_{random8}.mp4
Step 6: Task Submission (Phase 5)
aliyun ice submit-video-translation-job \
--input-config '{"Type":"Video","Video":"<mediaId>"}' \
--output-config '{"MediaURL":"https://<output_url>"}' \
--editing-config '{"TextSource":"OCR_ASR",...}'Duration: ~2 seconds
Step 7: Task Polling (Phase 6)
Query every 30 seconds until task completes or fails.
Expected Duration:
- Subtitle-level translation: 3-5 minutes
- Speech-level translation: 10-20 minutes
Step 8: Result Retrieval (Phase 7)
- Get output video URL
- Generate signed URL (private Bucket)
- Output result to user
---
Scenario 2: Subtitle Review
Key: When user does not provide subtitle, MUST ask if subtitle extraction and review is needed!
Execution Flow
Phase 0 → Phase 1 → Phase 2 → 【MUST ASK】 Need subtitle review? →
User chooses review → Phase 4: Subtitle Review Confirmation → Phase 5 → Phase 6 → Phase 7Detailed Steps
Step 1-5: Same as Scenario 1
Execute Phase 0-3, same flow.
Step 6: Need Subtitle Review? ⚠️ BLOCKING
Must Ask: When user does not provide subtitle, must confirm if extraction and review is needed!
Use AskUserQuestion to ask:
- "Do you need to extract subtitles for review first, or translate directly?"
| User Answer | Follow-up Action |
|---|---|
| Need review | Enter subtitle extraction flow |
| Direct translation | Switch to Scenario 1 (TextSource=OCR_ASR) |
Step 7: Subtitle Detection Region Confirmation ⚠️ BLOCKING
After user chooses review, ask subtitle detection region:
Use AskUserQuestion to ask:
- "Where are the subtitles roughly located in the video? Bottom 1/4, bottom 1/2?"
ROI Mapping Table:
| User Answer | ROI Parameter |
|---|---|
| Bottom 1/4 | [[0.75, 1], [0, 1]] |
| Bottom 1/2 | [[0.5, 1], [0, 1]] |
| Bottom 1/3 | [[0.67, 1], [0, 1]] |
| Full screen detection | [[0, 1], [0, 1]] |
Step 8: Subtitle Extraction (CaptionExtraction)
CLI Format Key:--Input,--Output,--JobParamsmust use JSON string format!
# Using MediaId (registered media)
aliyun ice SubmitIProductionJob \
--FunctionName CaptionExtraction \
--Input '{"Type":"MediaId","Media":"<mediaId>"}' \
--Output '{"Type":"OSS","Media":"oss://<bucket>/<output>.srt"}' \
--JobParams '{"lang":"ch","roi":[[0.5,1],[0,1]]}' \
--force
# Or using OSS path
aliyun ice SubmitIProductionJob \
--FunctionName CaptionExtraction \
--Input '{"Type":"OSS","Media":"oss://<bucket>/<object>"}' \
--Output '{"Type":"OSS","Media":"oss://<bucket>/<output>.srt"}' \
--JobParams '{"lang":"ch","roi":[[0.5,1],[0,1]]}' \
--forceDuration: 1-2 minutes
Step 9: Query Subtitle Extraction Result
Query every 30 seconds until Success or Fail.
Step 10: Subtitle Review Confirmation (Phase 4) ⚠️ BLOCKING
CRITICAL: Must execute strictly!
1. Get extracted subtitle content 2. Output subtitle content as-is to user (DO NOT change format) 3. AskUserQuestion: "Subtitle extraction complete, please check if content is correct, need modifications?" 4. Must wait for user confirmation
Step 11: Task Submission (Phase 5)
After user confirmation, submit translation task using reviewed SRT file:
aliyun ice submit-video-translation-job \
--input-config '{"Type":"Video","Video":"<mediaId>","Subtitle":"<srt_https_url>"}' \
--output-config '{"MediaURL":"https://<output_url>"}' \
--editing-config '{"TextSource":"SubtitleFile",...}'Step 12-13: Same as Scenario 1 Phase 6-7
---
Scenario 3: Subtitle Translation + User Subtitle
Execution Flow
Phase 0 → Phase 1 → Phase 2 → Phase 3 → Phase 5 → Phase 6 → Phase 7Detailed Steps
Step 1-3: Same as Scenario 1
Execute Phase 0-2.
Step 4: Check Subtitle File
- Check user-provided SRT file OSS exists
- Validate SRT format (fix empty subtitle entries)
Step 5: Task Submission
aliyun ice submit-video-translation-job \
--input-config '{"Type":"Video","Video":"<mediaId>","Subtitle":"<srt_https_url>"}' \
--output-config '{"MediaURL":"https://<output_url>"}' \
--editing-config '{"TextSource":"SubtitleFile","NeedSpeechTranslate":false,...}'Key: NeedSpeechTranslate: false (subtitle translation mode)---
Scenario 4: Speech Translation + User Subtitle
Execution Flow
Phase 0 → Phase 1 → Phase 2 → CustomSrtType Confirmation ⚠️ BLOCKING →
Phase 3 → Phase 5 → Phase 6 → Phase 7Detailed Steps
Step 1-3: Same as Scenario 1
Execute Phase 0-2.
Step 4: CustomSrtType Confirmation ⚠️ BLOCKING
Must Ask: For speech translation + user subtitle, must confirm subtitle language type!
Use AskUserQuestion to ask:
- "Is the subtitle file you provided in source language or target language?"
| User Answer | CustomSrtType |
|---|---|
| Source language | SourceSrt |
| Target language (already translated) | TargetSrt |
Step 5: Task Submission
aliyun ice submit-video-translation-job \
--input-config '{"Type":"Video","Video":"<mediaId>","Subtitle":"<srt_https_url>"}' \
--output-config '{"MediaURL":"https://<output_url>"}' \
--editing-config '{"TextSource":"SubtitleFile","NeedSpeechTranslate":true,"SpeechTranslate":{"CustomSrtType":"SourceSrt",...},...}'Key:NeedSpeechTranslate: true+SpeechTranslate.CustomSrtType
---
Blocking Point Summary
| Phase | Blocking Type | Trigger Condition |
|---|---|---|
| 0 | HARD-GATE | CLI version or credential not passed |
| 1 | BLOCKING | Translation mode not confirmed |
| 2 | BLOCKING | Subtitle processing not confirmed |
| 3 | Non-blocking | Output path not specified (default available) |
| Need subtitle review? | BLOCKING | User does not provide subtitle, must ask |
| 4 | BLOCKING | Subtitle review not confirmed (when user chooses review) |
| CustomSrtType | BLOCKING | Speech translation + user subtitle |
---
Polling Strategy
Subtitle Extraction Task
- Interval: 30 seconds
- Timeout: 5 minutes
- States: Init → Queuing → Analysing → Processing → Success/Fail
Video Translation Task
- Interval: 30 seconds
- Timeout: 30 minutes
- States: Created → Executing → Finished/Failed
---
Time Estimation
| Task Type | 3-minute Video | 10-minute Video |
|---|---|---|
| Subtitle extraction | 1-2 minutes | 3-5 minutes |
| Subtitle-level translation | 3-5 minutes | 8-12 minutes |
| Speech-level translation | 10-20 minutes | 30-50 minutes |
---
End of Document
# Video Translation Script Dependencies
# Install: pip3 install -r requirements.txt
alibabacloud_tea_openapi==0.3.4
alibabacloud_credentials==0.3.5
alibabacloud_tea_util==0.3.4
alibabacloud_openapi_util==0.1.1#!/usr/bin/env python3
"""
Video Translation Skill - Python SDK Implementation
This script provides Python SDK implementation for video translation operations
when CLI is not available or for more complex scenarios.
Dependencies:
pip3 install -r requirements.txt
Usage:
python video_translation.py submit-extract --input-media "oss://bucket/video.mp4" --output-media "oss://bucket/{source}-{timestamp}.srt" --region cn-shanghai
python video_translation.py get-job --job-id "xxx" --region cn-shanghai
python video_translation.py submit-translation --input-file "oss://bucket/video.mp4" --output-url "oss://bucket/output.mp4" --source-lang zh --target-lang en --region cn-shanghai
"""
import argparse
import hashlib
import json
import os
import re
import sys
import time
import uuid
from typing import Optional, Dict, Any
from urllib.parse import urlparse
from alibabacloud_tea_openapi.client import Client as OpenApiClient
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_tea_util import models as util_models
from alibabacloud_openapi_util.client import Client as OpenApiUtilClient
# =============================================================================
# Input Validation Functions
# =============================================================================
class ValidationError(ValueError):
"""Raised when input validation fails."""
pass
def validate_oss_url(url: str, param_name: str) -> str:
"""
Validate OSS URL format.
Allowed formats:
- oss://bucket/path/object.ext
- https://bucket.oss-region.aliyuncs.com/path/object.ext
Args:
url: URL to validate
param_name: Parameter name for error messages
Returns:
Validated URL
Raises:
ValidationError: If URL format is invalid
"""
if not url:
raise ValidationError(f"{param_name} cannot be empty")
# Check for dangerous characters that could be used for injection
dangerous_chars = ['`', '$', '|', ';', '&', '<', '>', '\n', '\r']
for char in dangerous_chars:
if char in url:
raise ValidationError(f"{param_name} contains invalid character: {repr(char)}")
# Validate oss:// format
if url.startswith("oss://"):
# oss://bucket/path/object
pattern = r'^oss://[a-z0-9][a-z0-9-]{1,61}[a-z0-9](?:/.*)?$'
if not re.match(pattern, url):
raise ValidationError(f"{param_name} invalid oss:// format. Expected: oss://bucket/path/object")
return url
# Validate https:// format
if url.startswith("https://") or url.startswith("http://"):
try:
parsed = urlparse(url)
if not parsed.netloc:
raise ValidationError(f"{param_name} invalid URL: missing host")
if not parsed.path or parsed.path == '/':
raise ValidationError(f"{param_name} invalid URL: missing path")
return url
except Exception as e:
raise ValidationError(f"{param_name} invalid URL format: {e}")
raise ValidationError(
f"{param_name} must start with 'oss://' or 'https://' (got: {url[:20]}...)"
)
def validate_http_url(url: str, param_name: str) -> str:
"""
Validate HTTP URL format (for video translation API).
Args:
url: URL to validate
param_name: Parameter name for error messages
Returns:
Validated URL
Raises:
ValidationError: If URL format is invalid
"""
if not url:
raise ValidationError(f"{param_name} cannot be empty")
# Check for dangerous characters
dangerous_chars = ['`', '$', '|', ';', '&', '<', '>', '\n', '\r']
for char in dangerous_chars:
if char in url:
raise ValidationError(f"{param_name} contains invalid character: {repr(char)}")
# Must be http or https
if not url.startswith("http://") and not url.startswith("https://"):
raise ValidationError(f"{param_name} must start with 'http://' or 'https://'")
try:
parsed = urlparse(url)
if not parsed.netloc:
raise ValidationError(f"{param_name} invalid URL: missing host")
return url
except Exception as e:
raise ValidationError(f"{param_name} invalid URL format: {e}")
def validate_detext_area(area: Optional[str]) -> Optional[str]:
"""
Validate DetextArea parameter to prevent injection.
Allowed values:
- None or empty (no text erasure)
- "Auto" (automatic detection)
- "[[x, y, w, h]]" format (custom coordinates, all values 0-1)
Args:
area: DetextArea value to validate
Returns:
Validated value
Raises:
ValidationError: If value is invalid
"""
if area is None or area == "":
return None
# Check for dangerous characters
dangerous_chars = ['`', '$', '|', ';', '&', '<', '>', '\n', '\r', '\\', '"']
for char in dangerous_chars:
if char in area:
raise ValidationError(f"DetextArea contains invalid character: {repr(char)}")
# Allow "Auto"
if area == "Auto":
return area
# Validate coordinate format: [[x, y, w, h]] or [[x1,y1], [x2,y2]]
# Only allow digits, decimal points, brackets, commas, and spaces
allowed_pattern = r'^[\[\]\s\.\d,]+$'
if not re.match(allowed_pattern, area):
raise ValidationError(
f"DetextArea must be 'Auto' or coordinate format like '[[0, 0.9, 1, 0.1]]'"
)
# Try to parse as JSON to validate structure
try:
coords = json.loads(area)
# Validate it's a list
if not isinstance(coords, list):
raise ValidationError("DetextArea must be a list")
# Could add more validation for coordinate values (0-1 range)
for item in coords if isinstance(coords[0], list) else [coords]:
if isinstance(item, (int, float)):
if not 0 <= item <= 1:
raise ValidationError("DetextArea coordinates must be between 0 and 1")
except json.JSONDecodeError:
raise ValidationError(f"DetextArea invalid JSON format: {area}")
return area
def validate_job_id(job_id: str) -> str:
"""
Validate Job ID format.
Args:
job_id: Job ID to validate
Returns:
Validated Job ID
Raises:
ValidationError: If Job ID is invalid
"""
if not job_id:
raise ValidationError("Job ID cannot be empty")
# Job IDs are typically alphanumeric with hyphens/underscores
# Allow only safe characters
allowed_pattern = r'^[a-zA-Z0-9_-]+$'
if not re.match(allowed_pattern, job_id):
raise ValidationError(f"Job ID contains invalid characters. Only alphanumeric, hyphen, and underscore allowed.")
return job_id
def validate_language_code(code: str) -> str:
"""
Validate language code.
Args:
code: Language code (e.g., 'zh', 'en', 'ja')
Returns:
Validated language code
Raises:
ValidationError: If language code is invalid
"""
if not code:
raise ValidationError("Language code cannot be empty")
# Language codes are typically 2-4 lowercase letters
allowed_pattern = r'^[a-z]{2,4}$'
if not re.match(allowed_pattern, code):
raise ValidationError(f"Language code must be 2-4 lowercase letters (got: {code})")
return code
def validate_region(region: str) -> str:
"""
Validate region ID.
Args:
region: Region ID (e.g., 'cn-shanghai', 'cn-beijing')
Returns:
Validated region ID
Raises:
ValidationError: If region is invalid
"""
if not region:
raise ValidationError("Region cannot be empty")
# Region format: cn-city, us-city, etc.
allowed_pattern = r'^[a-z]{2}-[a-z]+$'
if not re.match(allowed_pattern, region):
raise ValidationError(f"Invalid region format. Expected: 'cn-shanghai' (got: {region})")
return region
def generate_client_token(
input_media: str,
output_media: str,
extra: Optional[str] = None
) -> str:
"""
生成确定性幂等键 (ClientToken)。
基于输入输出路径生成 SHA256 哈希,确保相同参数产生相同 token。
这可以防止因网络超时或错误重试导致的重复任务创建。
Args:
input_media: 输入媒体 URL
output_media: 输出媒体 URL
extra: 额外的区分参数 (如翻译配置等)
Returns:
64 字符的 SHA256 哈希字符串
"""
# 构建确定性字符串
content = f"{input_media}|{output_media}"
if extra:
content += f"|{extra}"
# 生成 SHA256 哈希
return hashlib.sha256(content.encode('utf-8')).hexdigest()
def generate_default_output_path(
input_url: str,
region: str,
suffix: str = "translated",
extension: str = ".mp4"
) -> str:
"""
根据输入路径生成默认输出路径。
Args:
input_url: 输入视频的 OSS 地址 (oss://bucket/path/file.mp4)
region: 服务区域 (cn-shanghai)
suffix: 文件名后缀 (translated)
extension: 文件扩展名 (.mp4 或 .srt)
Returns:
https:// 格式的输出路径
Example:
input: oss://my-bucket/videos/demo.mp4
output: https://my-bucket.oss-cn-shanghai.aliyuncs.com/videos/demo_translated_1711440000.mp4
"""
timestamp = int(time.time())
# 解析 oss:// URL
if input_url.startswith("oss://"):
# oss://bucket/path/file.mp4
path = input_url[6:] # 移除 "oss://"
parts = path.split("/", 1)
bucket = parts[0]
object_path = parts[1] if len(parts) > 1 else ""
elif input_url.startswith("https://") or input_url.startswith("http://"):
# https://bucket.oss-region.aliyuncs.com/path/file.mp4
parsed = urlparse(input_url)
bucket = parsed.netloc.split(".")[0]
object_path = parsed.path.lstrip("/")
else:
raise ValueError(f"不支持的 URL 格式: {input_url}")
# 获取目录和文件名
dir_path = os.path.dirname(object_path)
filename = os.path.basename(object_path)
name_without_ext = os.path.splitext(filename)[0]
# 生成新文件名
new_filename = f"{name_without_ext}_{suffix}_{timestamp}{extension}"
# 拼接新路径
if dir_path:
new_object_path = f"{dir_path}/{new_filename}"
else:
new_object_path = new_filename
# 返回 https:// 格式
return f"https://{bucket}.oss-{region}.aliyuncs.com/{new_object_path}"
def create_client(region: str) -> OpenApiClient:
"""Create ICE OpenAPI client."""
credential = CredentialClient()
config = open_api_models.Config(credential=credential)
config.endpoint = f"ice.{region}.aliyuncs.com"
config.user_agent = "AlibabaCloud-Agent-Skills"
return OpenApiClient(config)
def submit_subtitle_extraction(
client: OpenApiClient,
input_media: str,
output_media: str,
name: Optional[str] = None,
lang: Optional[str] = None,
fps: Optional[int] = None,
roi: Optional[list] = None,
track: Optional[str] = None,
client_token: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit CaptionExtraction job for subtitle extraction.
Args:
client: OpenAPI client
input_media: Input video OSS URL
output_media: Output SRT file OSS URL
name: Job name
lang: Recognition language
fps: Sampling frame rate
roi: Region of interest
track: Track mode
client_token: 幂等键,用于防止重复创建任务。
如果不提供,将基于 input_media 和 output_media 自动生成。
Returns:
API response dict
"""
params = open_api_models.Params(
action="SubmitIProductionJob",
version="2020-11-09",
protocol="HTTPS",
method="POST",
auth_type="AK",
style="RPC",
pathname="/",
req_body_type="json",
body_type="json",
)
# Build job_params - direct parameters without FunctionName/Config wrapper
job_params = {}
if fps is not None:
job_params["fps"] = fps
if roi is not None:
job_params["roi"] = roi
if lang is not None:
job_params["lang"] = lang
if track is not None:
job_params["track"] = track
queries = {
"FunctionName": "CaptionExtraction",
"Input.Type": "OSS",
"Input.Media": input_media,
"Output.Type": "OSS",
"Output.Media": output_media,
}
if job_params:
queries["JobParams"] = json.dumps(job_params)
if name:
queries["Name"] = name
# 幂等性支持: 自动生成或使用提供的 ClientToken
if client_token is None:
client_token = generate_client_token(input_media, output_media)
queries["ClientToken"] = client_token
request = open_api_models.OpenApiRequest(query=OpenApiUtilClient.query(queries))
runtime = util_models.RuntimeOptions(
connect_timeout=10, # 连接超时 10 秒
read_timeout=30, # 读取超时 30 秒
)
response = client.call_api(params, request, runtime)
return response
def query_iproduction_job(client: OpenApiClient, job_id: str) -> Dict[str, Any]:
"""Query intelligent production job status (for CaptionExtraction etc.)."""
params = open_api_models.Params(
action="QueryIProductionJob",
version="2020-11-09",
protocol="HTTPS",
method="POST",
auth_type="AK",
style="RPC",
pathname="/",
req_body_type="json",
body_type="json",
)
queries = {"JobId": job_id}
request = open_api_models.OpenApiRequest(query=OpenApiUtilClient.query(queries))
runtime = util_models.RuntimeOptions(
connect_timeout=10,
read_timeout=30,
)
response = client.call_api(params, request, runtime)
return response
def get_smart_handle_job(client: OpenApiClient, job_id: str) -> Dict[str, Any]:
"""Get video translation job result (for SubmitVideoTranslationJob)."""
params = open_api_models.Params(
action="GetSmartHandleJob",
version="2020-11-09",
protocol="HTTPS",
method="POST",
auth_type="AK",
style="RPC",
pathname="/",
req_body_type="json",
body_type="json",
)
queries = {"JobId": job_id}
request = open_api_models.OpenApiRequest(query=OpenApiUtilClient.query(queries))
runtime = util_models.RuntimeOptions(
connect_timeout=10,
read_timeout=30,
)
response = client.call_api(params, request, runtime)
return response
def submit_video_translation_job(
client: OpenApiClient,
input_config: str,
output_config: str,
editing_config: str,
title: Optional[str] = None,
description: Optional[str] = None,
client_token: Optional[str] = None,
) -> Dict[str, Any]:
"""
Submit video translation job.
Args:
client: OpenAPI client
input_config: Input config JSON string
output_config: Output config JSON string
editing_config: Editing config JSON string
title: Job title
description: Job description
client_token: 幂等键,用于防止重复创建任务。
如果不提供,将基于 input_config 和 output_config 自动生成。
Returns:
API response dict
"""
params = open_api_models.Params(
action="SubmitVideoTranslationJob",
version="2020-11-09",
protocol="HTTPS",
method="POST",
auth_type="AK",
style="RPC",
pathname="/",
req_body_type="json",
body_type="json",
)
queries = {
"InputConfig": input_config,
"OutputConfig": output_config,
"EditingConfig": editing_config,
}
if title:
queries["Title"] = title
if description:
queries["Description"] = description
# 幂等性支持: 自动生成或使用提供的 ClientToken
if client_token is None:
# 基于 input_config 和 output_config 生成确定性 token
client_token = generate_client_token(input_config, output_config, editing_config)
queries["ClientToken"] = client_token
request = open_api_models.OpenApiRequest(query=OpenApiUtilClient.query(queries))
runtime = util_models.RuntimeOptions(
connect_timeout=10,
read_timeout=30,
)
response = client.call_api(params, request, runtime)
return response
def wait_for_job(client: OpenApiClient, job_id: str, timeout: int = 3600, interval: int = 10) -> Dict[str, Any]:
"""Wait for job to complete."""
start_time = time.time()
while True:
if time.time() - start_time > timeout:
raise TimeoutError(f"Job {job_id} timed out after {timeout} seconds")
result = get_smart_handle_job(client, job_id)
body = result.get("body", {})
state = body.get("State", "")
print(f"Job {job_id} state: {state}")
if state == "Finished":
return result
elif state == "Failed":
error_msg = body.get("ErrorMessage", "Unknown error")
raise RuntimeError(f"Job {job_id} failed: {error_msg}")
time.sleep(interval)
def asr_result_to_srt(asr_result: str) -> str:
"""Convert ASR result JSON to SRT format."""
try:
items = json.loads(asr_result)
except json.JSONDecodeError:
return asr_result
srt_lines = []
for i, item in enumerate(items, 1):
content = item.get("content", "")
from_time = item.get("from", 0)
to_time = item.get("to", 0)
# Convert seconds to SRT timestamp format
from_ts = seconds_to_srt_timestamp(from_time)
to_ts = seconds_to_srt_timestamp(to_time)
srt_lines.append(f"{i}")
srt_lines.append(f"{from_ts} --> {to_ts}")
srt_lines.append(content)
srt_lines.append("")
return "\n".join(srt_lines)
def seconds_to_srt_timestamp(seconds: float) -> str:
"""Convert seconds to SRT timestamp format (HH:MM:SS,mmm)."""
hours = int(seconds // 3600)
minutes = int((seconds % 3600) // 60)
secs = int(seconds % 60)
millis = int((seconds % 1) * 1000)
return f"{hours:02d}:{minutes:02d}:{secs:02d},{millis:03d}"
def build_input_config(video_url: str, subtitle_url: Optional[str] = None) -> str:
"""Build InputConfig JSON."""
config = {"Type": "Video", "Video": video_url}
if subtitle_url:
config["Subtitle"] = subtitle_url
return json.dumps(config)
def build_output_config(media_url: str, width: Optional[int] = None, height: Optional[int] = None) -> str:
"""Build OutputConfig JSON."""
config = {"MediaURL": media_url}
if width:
config["Width"] = width
if height:
config["Height"] = height
return json.dumps(config)
def build_editing_config(
source_language: str,
target_language: str,
translation_mode: str = "subtitle",
bilingual: bool = False,
font_size: str = "medium",
position: str = "bottom",
text_source: str = "OCR_ASR",
custom_srt_type: Optional[str] = None,
detext_area: Optional[str] = None,
) -> str:
"""
Build EditingConfig JSON.
Args:
text_source: 字幕来源 - OCR_ASR(自动识别), SubtitleFile(外部SRT文件), ASR, OCR, ALL
custom_srt_type: 当 text_source=SubtitleFile 时必填 - SourceSrt(原语种), TargetSrt(目标语种)
detext_area: 字幕擦除区域 - None(不擦除), "Auto"(自动识别), "[[x,y,w,h]]"(自定义)
Raises:
ValidationError: If detext_area contains invalid characters
"""
# Validate detext_area to prevent injection
detext_area = validate_detext_area(detext_area)
# Translation mode mapping
need_speech = translation_mode == "speech"
# Font size mapping
font_size_map = {"small": 60, "medium": 95, "large": 130}
font_size_value = font_size_map.get(font_size, 95)
# Position mapping
position_map = {"top": 0.15, "center": 0.5, "bottom": 0.85}
y_value = position_map.get(position, 0.85)
config = {
"SourceLanguage": source_language,
"TargetLanguage": target_language,
"NeedSpeechTranslate": need_speech,
"NeedFaceTranslate": False, # 面容翻译明确不开启
"BilingualSubtitle": bilingual,
"SupportEditing": True,
"TextSource": text_source,
}
# 当使用外部 SRT 文件时,需要指定 CustomSrtType
if text_source == "SubtitleFile" and custom_srt_type:
config["CustomSrtType"] = custom_srt_type
# 字幕擦除配置
if detext_area:
config["DetextArea"] = detext_area
if need_speech:
# 语音级翻译: 使用 SpeechTranslate 配置
config["SpeechTranslate"] = {
"VoiceConfig": {
"Voice": "zhiyan_emo"
}
}
else:
# 字幕级翻译: 使用 SubtitleTranslate 配置
config["SubtitleTranslate"] = {
"OcrArea": "Auto",
"SubtitleConfig": {
"Type": "Text",
"FontSize": font_size_value,
"FontColor": "#ffffff",
"Font": "Alibaba PuHuiTi",
"Y": y_value,
"TextWidth": 0.9,
"Alignment": "Center",
"BorderStyle": 1,
},
}
return json.dumps(config)
def main():
parser = argparse.ArgumentParser(description="Video Translation Skill")
subparsers = parser.add_subparsers(dest="command", help="Commands")
# Submit subtitle extraction job (CaptionExtraction)
extract_parser = subparsers.add_parser("submit-extract", help="Submit subtitle extraction job (ASR+OCR combined)")
extract_parser.add_argument("--input-media", required=True, help="Input video OSS URL")
extract_parser.add_argument("--output-media", required=True, help="Output SRT file OSS URL (supports placeholders: {source}, {timestamp}, {sequenceId})")
extract_parser.add_argument("--region", default="cn-shanghai", help="Region ID")
extract_parser.add_argument("--name", help="Job name")
extract_parser.add_argument("--lang", help="Recognition language: ch/en/ch_ml (optional)")
extract_parser.add_argument("--fps", type=int, help="Sampling frame rate (optional)")
extract_parser.add_argument("--track", help="Track mode: 'main' for main subtitle only (optional)")
extract_parser.add_argument("--client-token", help="Idempotent key for preventing duplicate jobs (auto-generated if not provided)")
extract_parser.add_argument("--wait", action="store_true", help="Wait for job completion")
# Get job result
get_parser = subparsers.add_parser("get-job", help="Get job result")
get_parser.add_argument("--job-id", required=True, help="Job ID")
get_parser.add_argument("--region", default="cn-shanghai", help="Region ID")
# Submit translation job
trans_parser = subparsers.add_parser("submit-translation", help="Submit video translation job")
trans_parser.add_argument("--input-file", required=True, help="Input video OSS URL or media ID")
trans_parser.add_argument("--output-url", required=True, help="Output video OSS URL")
trans_parser.add_argument("--source-lang", required=True, help="Source language code")
trans_parser.add_argument("--target-lang", required=True, help="Target language code")
trans_parser.add_argument("--region", default="cn-shanghai", help="Region ID")
trans_parser.add_argument("--mode", default="subtitle", choices=["subtitle", "speech"], help="Translation mode")
trans_parser.add_argument("--bilingual", action="store_true", help="Enable bilingual subtitles")
trans_parser.add_argument("--font-size", default="medium", choices=["small", "medium", "large"], help="Subtitle font size")
trans_parser.add_argument("--position", default="bottom", choices=["top", "center", "bottom"], help="Subtitle position")
trans_parser.add_argument("--subtitle-url", help="Custom subtitle file URL")
trans_parser.add_argument("--client-token", help="Idempotent key for preventing duplicate jobs (auto-generated if not provided)")
trans_parser.add_argument("--wait", action="store_true", help="Wait for job completion")
args = parser.parse_args()
if not args.command:
parser.print_help()
sys.exit(1)
# =================================================================
# Input Validation
# =================================================================
try:
# Validate region
args.region = validate_region(args.region)
if args.command == "submit-extract":
args.input_media = validate_oss_url(args.input_media, "--input-media")
args.output_media = validate_oss_url(args.output_media, "--output-media")
elif args.command == "get-job":
args.job_id = validate_job_id(args.job_id)
elif args.command == "submit-translation":
# input-file can be OSS URL or media ID (alphanumeric)
if args.input_file.startswith("oss://") or args.input_file.startswith("http"):
args.input_file = validate_oss_url(args.input_file, "--input-file")
# Otherwise treat as media ID (validated by API)
args.output_url = validate_http_url(args.output_url, "--output-url")
args.source_lang = validate_language_code(args.source_lang)
args.target_lang = validate_language_code(args.target_lang)
if args.subtitle_url:
args.subtitle_url = validate_http_url(args.subtitle_url, "--subtitle-url")
except ValidationError as e:
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
# =================================================================
# Execute Commands
# =================================================================
client = create_client(args.region)
if args.command == "submit-extract":
result = submit_subtitle_extraction(
client,
args.input_media,
args.output_media,
name=args.name,
lang=args.lang,
fps=args.fps,
track=args.track,
client_token=getattr(args, 'client_token', None),
)
print(json.dumps(result, indent=2, ensure_ascii=False))
if args.wait and result.get("body", {}).get("JobId"):
job_id = result["body"]["JobId"]
print(f"\nWaiting for extraction job {job_id}...")
final_result = wait_for_job(client, job_id)
print(json.dumps(final_result, indent=2, ensure_ascii=False))
print(f"\nSRT file generated at: {args.output_media}")
elif args.command == "get-job":
result = get_smart_handle_job(client, args.job_id)
print(json.dumps(result, indent=2, ensure_ascii=False))
elif args.command == "submit-translation":
input_config = build_input_config(args.input_file, args.subtitle_url)
output_config = build_output_config(args.output_url)
editing_config = build_editing_config(
args.source_lang,
args.target_lang,
translation_mode=args.mode,
bilingual=args.bilingual,
font_size=args.font_size,
position=args.position,
)
result = submit_video_translation_job(
client,
input_config,
output_config,
editing_config,
client_token=getattr(args, 'client_token', None),
)
print(json.dumps(result, indent=2, ensure_ascii=False))
if args.wait and result.get("body", {}).get("Data", {}).get("JobId"):
job_id = result["body"]["Data"]["JobId"]
print(f"\nWaiting for translation job {job_id}...")
final_result = wait_for_job(client, job_id)
print(json.dumps(final_result, indent=2, ensure_ascii=False))
if __name__ == "__main__":
main()