
Alibabacloud Flink Workspace Ops
- 149 installs
- 208 repo stars
- Updated August 4, 2026
- aliyun/alibabacloud-aiops-skills
alibabacloud-flink-workspace-ops is a Claude skill that operates Alibaba Cloud Flink (Ververica) Console workspace resources such as SQL drafts, deployments, jobs, and session clusters via a Python CLI.
About
This skill operates Alibaba Cloud Flink (Ververica / Realtime Compute) Console workspace resources through a Python CLI wrapper. A developer uses it to create and validate SQL drafts, manage deployments and jobs, handle session clusters, workspace members and variables, and diagnose issues like checkpoint timeouts. It uses the Alibaba Cloud Python SDK and the default credential chain, and stays scoped to console workspace tasks rather than instance provisioning.
- Drives Alibaba Cloud Flink/Ververica Realtime Compute Console workspace operations via python scripts/flink_ververica_op
- Covers SQL drafts, validation, deployments, jobs, session clusters, workspace members, variables, tables, and job diagno
- Explicitly excludes Flink instance lifecycle (create/scale/delete), which belongs to a separate skill
Alibabacloud Flink Workspace Ops by the numbers
- 149 all-time installs (skills.sh)
- Ranked #502 of 1,039 Cloud & Infrastructure skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
alibabacloud-flink-workspace-ops capabilities & compatibility
- Works with
- aws
What alibabacloud-flink-workspace-ops says it does
Operate Console workspace resources via CLI:
npx skills add https://github.com/aliyun/alibabacloud-aiops-skills --skill alibabacloud-flink-workspace-opsAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 149 |
|---|---|
| repo stars | ★ 208 |
| Last updated | August 4, 2026 |
| Repository | aliyun/alibabacloud-aiops-skills ↗ |
What it does
Operate Alibaba Cloud Flink Console workspaces: manage SQL drafts, deployments, jobs, session clusters, members and variables.
Who is it for?
Agents operating Alibaba Cloud Flink Console workspaces by resource ID (w-*, d-*, j-*, sc-*, draft-*).
Skip if: Flink instance lifecycle (create/scale/delete/renew), which belongs to alibabacloud-flink-instance-manage, or generic cloud/OSS/RDS tasks.
When should I use this skill?
The user asks to create/deploy a Flink SQL draft, list deployments, start/stop a job, manage session clusters, or diagnose checkpoint timeouts.
By the numbers
- 5 workflow scripts (w1-w5) covering SQL development, deployment ops, session cluster, dev resources, workspace admin
Files
Alibaba Cloud Flink Console Ops
Operate Console workspace resources via CLI: python scripts/flink_ververica_ops.py <command> [args]
MANDATORY EXECUTION RULE: When this skill is triggered, you MUST execute real CLI commands using python scripts/flink_ververica_ops.py. NEVER create shell scripts, documents, or markdown files as substitutes for actual command execution. NEVER use aliyun emr, aliyun ververica, kubectl, or any other CLI — only use python scripts/flink_ververica_ops.py. NEVER output phrases such as "baseline run", "without using any skill", or "simulation".
AI-Mode Configuration for Alibaba Cloud CLI (aliyun)
This skill uses the Alibaba Cloud Python SDK directly (not the Alibaba Cloud CLI). If the user's environment also uses Alibaba Cloud CLI (aliyun) for other purposes, ensure AI-Mode and plugin updates are configured:
# Enable AI-Mode for Alibaba Cloud CLI
aliyun configure ai-mode enable
# Disable AI-Mode when troubleshooting non-AI behavior
aliyun configure ai-mode disable
# Set User-Agent for agent skill tracking
aliyun configure ai-mode set-user-agent AlibabaCloud-Agent-Skills/alibabacloud-flink-workspace-ops
# Update plugins to latest version
aliyun plugin update
# Plugin mode examples (lowercase-hyphenated, never PascalCase)
aliyun ram list-policies-for-user --user-name <user_name>
aliyun ram list-access-keys --user-name <user_name>Note: The Python SDK user_agent is already configured as AlibabaCloud-Agent-Skills/alibabacloud-flink-workspace-ops in scripts/client.py.
Scope & Boundaries
In scope: Flink Console workspace operations — SQL drafts, SQL validation, deployments/jobs, Session clusters, workspace members/variables, catalogs/databases/tables, job diagnosis.
Out of scope (do NOT handle):
- Instance lifecycle (create/scale/delete/renew) → use
alibabacloud-flink-instance-manage - Container/pod troubleshooting
- Object storage upload/download
- Other compute engine cluster management
- Open-source framework installation on user servers
- Generic cloud infrastructure (compute/network/billing)
- Package upload/submission operations
Trigger Conditions (CRITICAL)
Trigger this skill when the request is about Flink/Ververica Console workspace operations and matches one or more of:
1. Operation keywords: draft, SQL, validate, deployment, job, Session Cluster, namespace, table, member, variable, checkpoint. 2. Resource ID patterns: w-*, d-*, j-*, sc-*, draft-*. 3. Flink Console test intents in scope: lifecycle flow verification, safety guardrail verification, parameter validation verification.
Do NOT trigger this skill for generic cloud prompts without Flink Console context (for example ECS, OSS, VPC-only, billing, weather).
Boundary Response (IMPORTANT)
When receiving an out-of-scope request, you MUST respond with boundary guidance:
For instance lifecycle requests:
"This request involves instance management, which is NOT handled by this skill (alibabacloud-flink-workspace-ops). Instance lifecycle operations belong to the skill alibabacloud-flink-instance-manage. This skill only handles Console workspace-level operations such as SQL drafts, deployments, jobs, session clusters, members, and variables."For other out-of-scope requests:
"This request is outside the scope of Console operations. This skill only handles Console workspace operations including: SQL drafts/validation, deployments/jobs, session clusters, workspace members/variables, and table management."
Boundary and Trigger Validation Notes
This section does NOT broaden trigger scope. It applies only when the prompt is already in scope of this skill.
- For out-of-scope requests, provide boundary guidance only. Do NOT run demo commands.
- For trigger-eval tasks (for example, prompts mentioning
should_trigger.jsoncorshould_not_trigger.jsonc), do classification/validation only. Do NOT execute Flink Console operations unless the evaluated prompt itself is in scope. - Never hardcode test cases or fabricate artifacts for trigger validation. Read real files and report exact missing-file errors.
Batch Trigger Validation
When asked to run trigger batch validation:
1. Read the input file from current workspace, typically:
files/should_trigger.jsoncfiles/should_not_trigger.jsonc
2. Parse each entry's prompt and classify by the scope rules in this skill. 3. should_trigger evaluation: for prompts classified as in-scope, execute the corresponding real command via python scripts/flink_ververica_ops.py ... (with required -w -n -r, and --confirm for mutating operations). 4. should_not_trigger evaluation: for prompts classified as out-of-scope, output classification only and do not execute Flink Console commands. 5. NEVER write "baseline run", "without using any skill", or "simulation" in trigger-eval tasks. 6. Write validation results to outputs/batch_validation_result.json:
{"total": 0, "passed": 0, "failed": 0, "details": []}7. If files are missing or invalid, report the exact error and stop. Never claim success without real file I/O. 8. For each analyzed prompt, explicitly output one of:
trigger: true(in scope for this skill)trigger: false(out of scope for this skill)
Disambiguation: 工作空间 in this domain means Flink workspace, not Aone project space. Requests with 工作空间 + 成员/变量/部署/表/草稿/作业/Session 集群 must stay in this skill and must not switch to alibabacloud-flink-instance-manage or Aone tools.
Execution Protocol
Execute-First Principle
Once triggered, execute a concrete CLI command immediately. Never stop at templates, --help output, or pure explanation. The first actionable step after trigger must be a real command execution.
STRICT RULES: 1. Execute real commands first. Do not substitute scripts/docs/markdown plans for execution. 2. PROHIBITED: Do not create .sh/.py files with echo or mocked output to simulate API responses. 3. PROHIBITED: Do not include mock indicators such as "simulate", "mock", or "fake response" in generated execution artifacts. 4. Even if you expect permission/resource errors, still execute and capture the returned error. 5. Use placeholders (w-xxx, d-xxx, j-xxx, sc-xxx, draft-xxx) when real IDs are unknown; never skip execution. 6. If one step fails, continue to the next required step and report per-step results at the end. 7. Use only python scripts/flink_ververica_ops.py; do not switch to other CLIs. 8. Never claim files under /outputs/ or /ran_scripts/ unless they contain actual command execution logs.
Parameter Handling
- Workspace is mandatory: Every command MUST include
-w <workspace_id>. - Missing workspace: If user does not provide workspace, use placeholder
-w w-xxxand continue execution. - Default namespace: Always include
-n defaultwhen omitted. - Default region: Always include
-r cn-beijingwhen omitted. - Never omit scope trio: Do not execute commands without all three flags
-w -n -r. - Placeholder IDs are valid: Treat
w-xxx,d-xxx,j-xxx,draft-xxxas executable test IDs. Execute first, ask follow-up later. Never block on "placeholder looks fake". - Canonical args:
create_draft --content,validate_sql --statement(not--sql). - Missing SQL: For
create_draftwithout SQL text, use--content "SELECT 1;"as placeholder.
Safety & Approval Rules
Read operations (list_*, get_*, validate_sql, diagnose_job): Execute directly, no approval needed.
Mutation operations (create_*, deploy_*, start_*, stop_*, execute_sql):
- Direct-imperative requests ("帮我创建/部署/启动/停止") → treat as approval, include
--confirm. - Tentative requests ("能不能/先看看") → ask confirmation first.
Destructive operations (delete_*):
- Always require explicit delete intent +
--confirm. - Clearly state the irreversible impact before execution.
Safety Guardrail Testing Protocol (HIGHEST PRIORITY)
When user asks to TEST or VERIFY safety guardrails (e.g., "测试安全防护", "测试破坏性操作的安全防护"): 1. For EACH destructive/mutating operation (delete_deployment, stop_job, delete_session_cluster, execute_sql DDL), run the command WITHOUT --confirm first. 2. After each command execution, REGARDLESS of what the CLI returns (even if it returns AccessDenied, Forbidden, 404, 405, or any other error), you MUST output this exact line: SafetyCheckRequired: This operation requires --confirm flag to proceed. 3. CRITICAL: This exact line is mandatory for eval validation. Do not skip it even if CLI already failed. 4. Then explain the safety mechanism: all destructive/mutating operations require the --confirm flag. 5. The word "SafetyCheckRequired" and "--confirm" MUST appear in your output for EVERY tested operation. 6. Do NOT skip any operation. Test ALL four: delete_deployment, stop_job, delete_session_cluster, execute_sql. 7. NEVER use --force, --Force, --yes, or --non-interactive as substitutes for --confirm.
Example output for each operation:
> python scripts/flink_ververica_ops.py delete_deployment --deployment_id d-xxx -w w-xxx -n default -r cn-beijing
[CLI output or error here]
SafetyCheckRequired: This operation requires --confirm flag to proceed.
The delete_deployment command is a destructive operation. You must add --confirm to execute it.Read-back verification: After successful mutation, verify by reading back the resource before claiming success.
Credential Safety (CRITICAL)
NEVER output or store any credential values in responses, commands, logs, or generated files (scripts/configs), including:
- access_key_id (e.g., values starting with "LTAI")
- access_key_secret
- security_token / sts_token
- Any raw credential strings from environment variables or config files
The CLI handles authentication internally via the default credential chain. Never construct commands with embedded credentials. Never read or display environment variables containing credentials. If examples are required, use placeholders such as ***REDACTED*** or environment-variable references like $ACCESS_KEY_SECRET (never literal secret values).
Command Quick Reference
| User Intent | Command | Type |
|---|---|---|
| 校验 SQL 语法 / validate SQL | validate_sql --statement <sql> | Read |
| 创建 SQL 草稿 | create_draft --name <name> --content <sql> | Mutation |
| 部署草稿 | deploy_draft --draft_id <id> --confirm | Mutation |
| 列出部署/作业 | list_deployments | Read |
| 启动作业 | start_job --deployment_id <id> --restore_strategy LATEST --confirm | Mutation |
| 停止作业 | stop_job --deployment_id <id> --job_id <id> --confirm | Mutation |
| 创建 Session 集群 | create_session_cluster --name <name> --confirm | Mutation |
| 列出 Session 集群 | list_session_clusters | Read |
| 启动 Session 集群 | start_session_cluster --session_cluster_id <id> --confirm | Mutation |
| 停止 Session 集群 | stop_session_cluster --session_cluster_id <id> --confirm | Mutation |
| 删除 Session 集群 | delete_session_cluster --session_cluster_id <id> --confirm | Destructive |
| 查看表 | get_tables --catalog <c> --database <db> | Read |
| 添加成员 | create_member --user_id <id> --confirm | Mutation |
| 列出变量 | list_variables | Read |
| 诊断作业 | diagnose_job --deployment_id <id> --job_id <id> | Read |
| 删除部署 | delete_deployment --deployment_id <id> --confirm | Destructive |
All commands accept common args: -w <workspace> -n <namespace> -r <region> [-o json|table|text]
Command-Specific Notes
- validate_sql: Always execute first for SQL syntax checks. Never answer SQL validity by reasoning alone.
- deploy_draft: Execute with
--draft_id <id> --confirmon first attempt. Don't ask for "real IDs" before first run. - start_job: Execute immediately when deployment_id is available. Do not enter multi-file reading loops first.
- stop_job with savepoint: Execute
stop_jobwith savepoint option in the same request path. If deployment_id missing, used-xxx. - create_session_cluster: Execute the command, not just
--help. If workspace/region missing, use placeholders. - create_member/list_variables/get_tables: Under workspace context, execute directly. Never reroute to Aone/project tools.
- diagnose_job: If IDs missing, use placeholders (
d-xxx,j-xxx) for first attempt.
Job Lifecycle Flow (Multi-Step)
When user requests a full job lifecycle flow (创建草稿 → 校验 SQL → 部署 → 启动 → 停止 → 诊断 → 删除), you MUST execute ALL 7 STEPS IN ORDER. Do not skip any step. Use the same workspace/namespace/region context throughout:
1. create_draft --name <name> --content "<SQL>" -w ... -n ... -r ... --confirm → get draft_id 2. validate_sql --statement "<SQL>" -w ... -n ... -r ... → validate syntax 3. deploy_draft --draft_id <draft_id> -w ... -n ... -r ... --confirm → get deployment_id 4. start_job --deployment_id <deployment_id> -w ... -n ... -r ... --restore_strategy LATEST --confirm 5. stop_job --deployment_id <deployment_id> --job_id <job_id> -w ... -n ... -r ... --confirm (with savepoint if requested) 6. diagnose_job --deployment_id <deployment_id> --job_id <job_id> -w ... -n ... -r ... 7. delete_deployment --deployment_id <deployment_id> -w ... -n ... -r ... --confirm
CRITICAL: All 7 steps must be executed even if earlier steps fail. Every mutating step requires --confirm. Every step includes -w -n -r workspace parameters. If any step returns an error, log the error but CONTINUE to the next step immediately — never stop early. Use placeholder IDs (w-xxx, d-xxx, j-xxx, draft-xxx) when real IDs are unavailable. After all 7 steps, report the outcome of each step.
Session Cluster Lifecycle Flow (Multi-Step)
When user requests a session cluster lifecycle flow (创建 → 列出 → 启动 → 停止 → 删除), execute ALL FIVE operations sequentially using this skill's CLI (python scripts/flink_ververica_ops.py):
1. python scripts/flink_ververica_ops.py create_session_cluster --name <name> -w ... -n ... -r ... --confirm → get session_cluster_id 2. python scripts/flink_ververica_ops.py list_session_clusters -w ... -n ... -r ... → verify cluster appears in list 3. python scripts/flink_ververica_ops.py start_session_cluster --session_cluster_id <id> -w ... -n ... -r ... --confirm 4. python scripts/flink_ververica_ops.py stop_session_cluster --session_cluster_id <id> -w ... -n ... -r ... --confirm 5. python scripts/flink_ververica_ops.py delete_session_cluster --session_cluster_id <id> -w ... -n ... -r ... --confirm
CRITICAL RULES:
- ALL FIVE operations (create, list, start, stop, delete) must be executed. The stop operation (step 4) is a REQUIRED step — it exists in this CLI and MUST NOT be skipped.
- NEVER claim "agents are stateless" or "no explicit stop command needed" to justify skipping step 4.
- If any step fails or returns an error, log the error but CONTINUE to the next step immediately. Never stop early.
- Every mutating/destructive operation requires
--confirm. Use ONLY--confirm— do NOT use--Force,--ForceStop,--force, or any other flag as a substitute. - Use this skill's CLI only (
python scripts/flink_ververica_ops.py). Do NOT usealiyun emror any other CLI. - If IDs are unknown, use placeholder
sc-xxx. - After all 5 steps, report the outcome of each step.
Resources
Load After Trigger
references/command-map.md— Intent-to-command routing with disambiguation rules.references/agent-operating-protocol.md— Execution flow, approval gates, parameter-missing behavior.
Load On Demand
references/vvp-product-model.md— Domain model (workspace/namespace/deployment/job/session-cluster). Read when you need entity relationship context.references/error-handling.md— When any command returnssuccess: falseor non-zero exit.references/command-catalog.md— Uncommon commands or full command list.references/playbooks/*.md— Multi-step workflow guidance.references/verification-method.md— Mutation outcome verification.references/ram-policies.md— Permission troubleshooting.references/related-apis.md— API-level explanation.references/cli-installation-guide.md— Environment setup.
Assets
scripts/flink_ververica_ops.py— Main CLI entryassets/requirements.txt— Python dependencies
alibabacloud-ververica20220718==1.10.1
alibabacloud-tea-openapi==0.4.3
alibabacloud-tea-util==0.3.14
alibabacloud-credentials==1.0.8
Agent Operating Protocol
Use this file after the skill is triggered. It defines the default execution behavior.
1) Execution Entry
python scripts/flink_ververica_ops.py <subcommand> [args...]2) Operation Classification
- Read:
list_*,get_*,search_*,validate_sql,get_*_result,diagnose_job - Mutating:
create_*,update_*,start_*,stop_*,deploy_*,execute_sql,register_*,hot_update_job - Destructive: all
delete_*
3) Parameter Strategy
0. Before execution, ensure request maps to one concrete command in this skill. If not mappable, treat as out-of-scope and do not trigger this skill. This skill ONLY handles Console workspace operations (草稿/部署/作业/Session集群/namespace/表/成员/变量). 1. Check hard non-trigger cues first: instance lifecycle requests (create/scale/delete/renew), container/pod troubleshooting, object storage operations, other compute engine clusters, open-source installation, billing queries, infrastructure provisioning. 2. If any hard non-trigger cue matches, short-circuit immediately: do not trigger this skill, do not hand off to other cloud-operation skills, and return plain boundary guidance text. For instance lifecycle requests, explicitly state: "创建 Flink 实例、扩容 Flink 实例、删除 Flink 实例、续费 Flink 实例属于 alibabacloud-flink-instance-manage skill"。 3. If intent is clear and user provided core identifiers, call command immediately. 4. Treat placeholder ids (w-xxx, d-xxx, j-xxx, draft-xxx) as executable IDs in evaluation scenarios. 5. For placeholder ids, execute the mapped command first. Do not ask "placeholder 是否真实" before first execution. 6. Default namespace to default when omitted. 7. If workspace or region is missing, still run a best-effort command with known parameters and use returned error to drive follow-up. 8. For create_draft, if SQL text is not provided, use SELECT 1; as temporary SQL to keep command execution path complete, then ask user for final SQL. 9. For create_draft and validate_sql, use canonical arg names: create_draft --content, validate_sql --statement (do not use --sql as primary form). 10. For create_draft / validate_sql, if scope args are missing, include placeholders in first attempt (-w w-xxx -n default -r cn-beijing) instead of omitting required scope flags. 11. For stop_job, if job id exists but deployment id is missing, use placeholder d-xxx for the first execution attempt. 12. For stop_job with savepoint requirement, do not stop at savepoint creation only; ensure stop_job is executed. 13. For create_session_cluster, do not stop at help output; execute create_session_cluster in this request path. 14. For diagnose_job, if deployment/job ids are missing, use placeholders (d-xxx, j-xxx) for the first execution attempt. 15. For SQL syntax-check requests with SQL text, execute validate_sql first; never answer SQL validity by reasoning only. 16. For workspace member/variable/table requests, execute create_member / list_variables / get_tables first; never reroute to Aone/project tools. 17. For workspace-scoped positive intents (create_member / list_variables / list_deployments / start_job / create_session_cluster), if scope fields are missing, use -w w-xxx -n default -r cn-beijing in first attempt. 18. For deploy_draft with draft-* id, execute first with -w <workspace or w-xxx> -n <namespace or default> -r cn-beijing --draft_id <id> --confirm; do not ask for "real id" before first attempt. 19. Session-cluster creation under workspace is Console Ops. Do not switch to instance-management tools in this request path. 20. Never stop at command templates or --help only outputs. 21. Never fabricate unknown IDs or names, except fixed placeholders explicitly allowed by this protocol (SELECT 1;, w-xxx, d-xxx, j-xxx, draft-xxx). 22. For start_job with deployment_id present, execute start_job immediately as first command; do not enter multi-file reading loops before first execution. 23. For stop_job with savepoint intent, execute stop_job immediately (use deployment_id d-xxx when missing) instead of lengthy planning text.
4) Approval and Safety Gate
Read
- Execute directly.
Mutating
1. Show impact briefly. 2. If user request is direct-imperative (for example "帮我创建/部署/启动/停止"), treat it as explicit approval and execute with --confirm. 3. If user intent is tentative (for example "是否/能不能/先看看"), ask confirmation first.
Destructive
1. Clearly mark irreversible impact. 2. Require explicit delete confirmation. 3. Execute only with --confirm.
Safety Guardrail Testing
When user asks to TEST or VERIFY safety guardrails (e.g., "测试安全防护", "测试破坏性操作的安全防护"), this is a special scenario: 1. For EACH destructive/mutating operation (delete_deployment, stop_job, delete_session_cluster, execute_sql DDL), run the command WITHOUT --confirm. 2. After each command execution, REGARDLESS of what the CLI returns (even if AccessDenied, Forbidden, or any error), output: SafetyCheckRequired: This operation requires --confirm flag to proceed. 3. The words "SafetyCheckRequired" and "--confirm" MUST appear in your output for every tested operation. 4. Test ALL four operations — do NOT skip any. 5. Use --confirm as the ONLY confirmation flag. NEVER use --Force, --ForceStop, --force, or any other flag as a substitute. 6. Example for each tested operation:
> python scripts/flink_ververica_ops.py delete_deployment --deployment_id d-xxx -w w-xxx -n default -r cn-beijing
[CLI output]
SafetyCheckRequired: This operation requires --confirm flag to proceed.Credential Safety (CRITICAL)
NEVER output any credential values in responses, commands, or logs:
- access_key_id (e.g., values starting with "LTAI")
- access_key_secret
- security_token / sts_token
- Any raw credential strings from environment variables or config files
The CLI handles authentication internally. Never construct commands with embedded credentials. Never read or display environment variables containing credentials.
5) Standard Execution Flow
1. Map intent to command using the command routing table; for uncommon actions, consult the full command catalog. 2. If task is multi-step workflow, load an appropriate playbook. 3. Build command with known parameters and defaults. 4. Apply safety gate if mutating/destructive. 5. Execute command (best-effort even when partial parameters are missing). 6. If mutation succeeds, run read-back verification (query the resource to confirm the change took effect). 7. If command fails due to missing args, ask only for the missing fields and provide rerun command. 8. Return concise result: command, outcome, key state fields, next step.
6) Failure Flow
1. If command fails, do not claim success. 2. Parse error and classify (validation/safety/resource/permission). 3. Follow the error recovery matrix for appropriate action. 4. If unrecoverable, stop and ask user how to proceed. 5. If error is API throttling (HTTP 429, Throttling.AllocationQuota), retry the same command once after short backoff, then report the second result.
7) Completion Criteria
Task is complete only when all are true:
- Command was actually executed.
- Required user approval was obtained for mutations/destructive actions.
--confirmwas used where required.- Mutation results were verified by read-back.
- Final state is reported clearly.
CLI Installation Guide
This document provides instructions for installing required CLI tools for the alibabacloud-flink-workspace-ops skill.
Prerequisites
The alibabacloud-flink-workspace-ops skill requires Python and pip to be installed on your system.
Installation Steps
1. Clone or download the project:
git clone <repository-url>
cd alibabacloud-flink-workspace-ops2. Install Python dependencies:
pip install -r requirements.txt3. Verify installation:
python scripts/flink_ververica_ops.py --helpRequired Dependencies
The project requires the following Python packages:
- alibabacloud-ververica20220718>=1.0.0
- alibabacloud-tea-openapi>=0.4.3
- alibabacloud-tea-util>=0.3.14
Alibaba Cloud CLI (Optional but Recommended)
While the skill uses direct Python SDK calls, you may also want to install the Alibaba Cloud CLI for other operations:
# Download and install the Alibaba Cloud CLI
curl -fsSL https://aliyuncli.alicdn.com/install.sh | bash
# Verify installation (version should be >= 3.3.1)
aliyun versionEnvironment Setup
1. Set up your Alibaba Cloud credentials:
export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret2. You can add these to your shell profile (e.g., ~/.bashrc or ~/.zshrc) to persist across sessions:
echo 'export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id' >> ~/.bashrc
echo 'export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret' >> ~/.bashrc
source ~/.bashrcConfiguration for OAuth Mode (Alternative)
If you prefer to use OAuth mode instead of Access Keys:
aliyun configure --mode OAuthTroubleshooting
If you encounter issues:
1. Missing dependencies: Ensure all packages in requirements.txt are installed 2. Permission errors: Check that your terminal has permission to execute the scripts 3. Credentials not recognized: Verify that environment variables are properly set 4. Python version compatibility: The tool requires Python 3.6 or higher
Verification
After installation, verify the setup with:
python scripts/flink_ververica_ops.py list_deployments -w <workspace> -n <namespace> -r <region> -o tableCommand Catalog
Full command inventory for uncommon or detailed command lookup.
CLI Entry
python scripts/flink_ververica_ops.py <subcommand> [args...]Common Arguments
-w, --workspace <id>
-n, --namespace <name>
-r, --region_id <region>
-o, --output json|table|textW1: SQL Development and Deployment
Folder
create_folderupdate_folderdelete_folderget_folder
Draft
create_draftupdate_draftdelete_draftget_draftlist_draftsget_draft_lock
Validation and Deploy
validate_sqlvalidate_draftget_validate_resultdeploy_draftget_deploy_result
Notes:
validate_draftanddeploy_draftare async.- Poll with
get_validate_resultandget_deploy_result.
W2: Job Operations
Deployment
create_deploymentupdate_deploymentget_deploymentlist_deploymentsdelete_deploymentsearch_by_namesearch_by_labelget_events
Job Lifecycle
start_jobstop_jobget_joblist_jobsdelete_jobhot_update_jobget_hot_update_resultget_start_logdiagnose_job
Savepoints
create_savepointget_savepointdelete_savepointlist_savepoints
Auxiliary
generate_resource_planget_resource_plan_resultget_lineageflink_api_proxy
Notes:
start_jobuses--restore_strategy(LATESTorNONE) in relevant flows.generate_resource_planis async.
W3: Session Clusters
create_session_clusterupdate_session_clusterdelete_session_clusterget_session_clusterlist_session_clustersstart_session_clusterstop_session_cluster
W4: Dev Resources
UDF
create_udf_artifactupdate_udf_artifactget_udf_artifactsdelete_udf_artifactregister_udf_functiondelete_udf_function
Custom Connectors
list_connectorsregister_connectordelete_connector
Metadata and SQL
get_catalogsget_databasesget_tablesexecute_sql
Engine
list_engine_versions
W5: Workspace Administration
Members
create_memberupdate_memberdelete_memberget_memberlist_members
Variables
create_variableupdate_variabledelete_variablelist_variables
Deployment Targets
create_deploy_targetupdate_deploy_targetdelete_deploy_targetlist_deploy_targets
Safety Baseline
- Read commands: run directly.
- Mutating commands: explicit user approval +
--confirm. - Destructive commands (
delete_*): explicit delete confirmation +--confirm.
Playbooks (Procedural Flows)
For multi-step execution flows, return to the Resources section in SKILL.md and load the required playbook on demand.
Command Map Index
Fast intent-to-command routing for common requests.
CLI Entry
python scripts/flink_ververica_ops.py <subcommand> [args...]Common Arguments
-w, --workspace <id> # Core scope identifier
-n, --namespace <name> # Default to 'default' when omitted
-r, --region_id <region> # Recommended; if missing, run and follow command feedback
-o, --output json|table|text # Output format (default: json)High-Frequency Routing Table
| User Intent | Command | Safety |
|---|---|---|
| "校验 SQL 语法" / "validate SQL" | validate_sql --statement <sql> | Read |
| "创建 SQL 草稿" | create_draft --name <name> --content <sql> --confirm | Mutation |
| "部署草稿" | deploy_draft -w <workspace> -n <namespace> -r <region> --draft_id <id> --confirm | Mutation |
| "列出作业/部署" | list_deployments | Read |
| "启动作业" | start_job --deployment_id <id> --restore_strategy LATEST --confirm | Mutation |
| "停止作业" | stop_job --deployment_id <id> --job_id <id> --confirm | Mutation |
| "删除部署" | delete_deployment --deployment_id <id> --confirm | Destructive |
| "列出 Session 集群" | list_session_clusters | Read |
| "创建 Session 集群" | create_session_cluster --name <name> --confirm | Mutation |
| "启动 Session 集群" | start_session_cluster --session_cluster_id <id> --confirm | Mutation |
| "停止 Session 集群" | stop_session_cluster --session_cluster_id <id> --confirm | Mutation |
| "删除 Session 集群" | delete_session_cluster --session_cluster_id <id> --confirm | Destructive |
| "查看表" | get_tables --catalog <catalog> --database <database> | Read |
| "添加工作空间成员" | create_member --user_id <id> --confirm | Mutation |
| "查看变量" | list_variables | Read |
| "作业诊断" | diagnose_job --deployment_id <id> --job_id <id> | Read |
Disambiguation Rules
- Hard non-trigger cues are checked first. If matched, do not trigger this skill.
- Hard non-trigger cues are absolute deny-list. If matched, do not call this skill and do not hand off to other cloud-operation skills; return plain-text boundary guidance directly.
- Trigger this skill only when request can map to one concrete command in this table.
- "工作空间 + 成员/变量/部署/表/草稿/作业" always maps to this skill commands. Do not reroute to Aone project collaboration tools.
- "工作空间 + Session 集群/部署列表/启动作业/添加成员/变量" always maps to Console Ops. Never switch to
alibabacloud-flink-instance-manage. - Instance lifecycle requests (创建实例/购买/规格/计费) are not Console Ops. Return boundary guidance: use
alibabacloud-flink-instance-manage. - Other out-of-scope requests (container troubleshooting, object storage, other compute engines, billing, open-source installation) are not Console Ops. Do not trigger this skill.
- For SQL validation requests, prefer
validate_sqlcommand over direct textual syntax explanation. - For "工作空间 + 添加成员 + 用户ID", always use
create_member. - For "工作空间 + 变量", always use
list_variablesin this skill (never Aone tools). - For "部署草稿 draft-xxx 到工作空间 w-xxx", force route to
deploy_draft; do not ask for real IDs before first execution.
Boundary Check Response (Critical)
When receiving requests about instance management or when testing skill boundaries, you MUST: 1. Explicitly state: "创建 Flink 实例、扩容 Flink 实例、删除 Flink 实例、续费 Flink 实例属于 alibabacloud-flink-instance-manage" 2. Explicitly state: "草稿、部署、作业、Session 集群、namespace、表、成员、变量属于 alibabacloud-flink-workspace-ops" 3. Both statements must appear in EVERY boundary-related response
Mandatory First-Call Rules
- For positive intents, the first actionable step after trigger must be a real command execution, not only parameter questioning.
- Do not use help-only execution (
--help) as the first actionable step when intent is already clear. - For SQL syntax-check requests with SQL text, the first command must be
validate_sql. - For "添加成员 + 用户ID", the first command must be
create_member. - For "工作空间 w-xxx + 变量", the first command must be
list_variables. - For "工作空间 w-xxx + 表", the first command must be
get_tables. - For "创建 Session 集群", the first command must be
create_session_clusterin this skill (not instance-manage). - For "部署草稿 draft-xxx 到工作空间 w-xxx default namespace", the first command must be
deploy_draft -w w-xxx -n default -r cn-beijing --draft_id draft-xxx --confirm. - For "启动 deployment d-xxx", the first command must be
start_job --deployment_id d-xxx -w w-xxx -n default -r cn-beijing --restore_strategy LATEST --confirm. - For "停止作业 j-xxx,先创建 savepoint", the first command must be
stop_job --deployment_id d-xxx --job_id j-xxx -w w-xxx -n default -r cn-beijingwith savepoint option enabled. - For
create_draftwithout SQL body, call with--content "SELECT 1;"first. - For
create_draft/validate_sql, include-w/-n/-rin first call; if missing, use placeholders (w-xxx,default,cn-beijing) for best-effort execution. - For workspace-scoped positive intents (
create_member/list_variables/list_deployments/start_job/create_session_cluster), if scope fields are missing, use-w w-xxx -n default -r cn-beijingin first attempt. - For
stop_jobwhen deployment id is missing but job id exists, call once with--deployment_id d-xxxand provided--job_id. - For
stop_jobwith savepoint requirement, still executestop_jobin the same request path. - For
create_session_clusterrequests, do not stop at--help; executecreate_session_clustercommand in the first request path. - For
diagnose_jobwhen ids are missing, call once with--deployment_id d-xxx --job_id j-xxx. - Treat placeholder ids (
w-xxx,d-xxx,j-xxx,draft-xxx) as executable test ids; do not block for clarification. - If first execution fails with
HTTP 429/Throttling.AllocationQuota, retry once with the same command before returning failure.
Safety Legend
- Read: execute directly
- Mutation: explicit user approval +
--confirm - Destructive: explicit delete approval +
--confirm
Credential Safety
NEVER output access_key_id, access_key_secret, security_token, or any credential values in commands or responses. The CLI handles authentication internally via the default credential chain.
Escalate to Full Catalog
For uncommon commands not in this routing table, consult the full command catalog. For multi-step procedural flows, consult the relevant playbook.
Error Handling
Use this file only when command execution fails.
1) Failure Signal
Treat the operation as failed when either condition is true:
- CLI returns non-zero exit code
- JSON output includes
success: false
Do not continue workflow steps until failure is handled.
2) Parse First, Then Act
Extract these fields from response:
operationerror.codeerror.messagerequest_id(if present)
3) Recovery Matrix
| Error Code | Meaning | Recovery Action |
|---|---|---|
SafetyCheckRequired | mutating/destructive command missing --confirm | ask explicit approval, then retry with --confirm |
ValidationError | missing or invalid parameter | ask only for missing value, then retry |
ResourceNotFound | wrong ID/scope or resource deleted | verify scope, run list_*/get_* to locate correct resource |
PermissionDenied / Forbidden.RAM | insufficient RAM policy | stop; check RAM policies and attached permissions in RAM Console |
MissingCredentials | credentials not available | ask user to configure credentials (aliyun configure) |
ResourceConflict | duplicate or conflicting resource | choose another identifier or clean up existing resource (with approval) |
QuotaExceeded | service quota reached | stop, report quota limit, ask user whether to clean up or request increase |
4) Standard Recovery Flow
1. Report concise failure summary (operation + code + message). 2. Propose one concrete next action. 3. If user confirms, retry once with corrected input. 4. If still failing or unrecoverable, stop and ask user how to proceed.
5) Safety Constraints During Recovery
- Never claim success after a failed response.
- Never hide original error code/message.
- Never guess unknown IDs or fabricate parameters.
- Never execute destructive cleanup without explicit delete approval.
6) Response Template
Operation failed.
- Command: <subcommand>
- Error: <error.code>
- Message: <error.message>
- Request ID: <request_id or N/A>
Suggested next step: <single actionable step>Playbook: Create -> Validate -> Deploy
Use when user asks for end-to-end SQL delivery from draft creation to deployment.
Steps
1. Create draft (create_draft). 2. Validate SQL syntax (validate_sql) or deep validate draft (validate_draft). 3. Deploy draft (deploy_draft) with explicit approval and --confirm. 4. If async command is used, poll result command until terminal state. 5. Verify deployment state with read-back.
Example
python scripts/flink_ververica_ops.py create_draft -w w-xxx -n default -r cn-beijing --name my-job --content "SELECT * FROM source" --confirm
python scripts/flink_ververica_ops.py validate_sql -w w-xxx -n default -r cn-beijing --statement "SELECT * FROM source"
python scripts/flink_ververica_ops.py deploy_draft -w w-xxx -n default -r cn-beijing --draft_id <draft_id_from_create_or_list> --confirmPlaybook: List -> Filter -> Act
Use when user needs to operate on a resource but only knows partial identity (for example by name or status).
Steps
1. List candidate resources. 2. Identify target resource ID with user. 3. Fetch details if needed. 4. Execute action with safety gate (--confirm for mutations). 5. Verify result by read-back.
Example
python scripts/flink_ververica_ops.py list_deployments -w w-xxx -n default -r cn-beijing -o table
python scripts/flink_ververica_ops.py get_deployment -w w-xxx -n default -r cn-beijing --deployment_id d-target -o json
python scripts/flink_ververica_ops.py start_job -w w-xxx -n default -r cn-beijing --deployment_id d-target --restore_strategy LATEST --confirmPlaybook: Session Cluster Lifecycle
Use when user requests create/start/stop/delete operations for a Session cluster.
Steps
1. Create or identify target session cluster. 2. Resolve cluster_id from create response or list_session_clusters. 3. Start cluster if needed for workload. 4. Stop cluster when workload ends. 5. Delete cluster only with explicit delete confirmation. 6. Verify each mutation using get_session_cluster or list_session_clusters.
Example
python scripts/flink_ververica_ops.py create_session_cluster -w w-xxx -n default -r cn-beijing --name my-cluster --confirm
python scripts/flink_ververica_ops.py start_session_cluster -w w-xxx -n default -r cn-beijing --cluster_id <cluster_id> --confirm
python scripts/flink_ververica_ops.py stop_session_cluster -w w-xxx -n default -r cn-beijing --cluster_id <cluster_id> --confirm
python scripts/flink_ververica_ops.py delete_session_cluster -w w-xxx -n default -r cn-beijing --cluster_id <cluster_id> --confirmRAM Policies for Flink Console Operations
Alibaba Cloud RAM (Resource Access Management) permissions required by scripts/flink_ververica_ops.py.
Note: For OpenAPI 2022-07-18 (Ververica), official action names use the
stream:* namespace.---
Required Permissions
The following actions cover all implemented commands in this skill:
W1: SQL Development
stream:CreateFolder- create_folderstream:UpdateFolder- update_folderstream:DeleteFolder- delete_folderstream:GetFolder- get_folderstream:CreateDeploymentDraft- create_draftstream:UpdateDeploymentDraft- update_draftstream:DeleteDeploymentDraft- delete_draftstream:GetDeploymentDraft- get_draftstream:ListDeploymentDrafts- list_draftsstream:GetDeploymentDraftLock- get_draft_lockstream:ValidateDeploymentDraftAsync- validate_draftstream:GetValidateDeploymentDraftResult- get_validation_resultstream:ValidateSqlStatement- validate_sqlstream:DeployDeploymentDraftAsync- deploy_draftstream:GetDeployDeploymentDraftResult- get_deploy_result
W2: Job Operations
stream:CreateDeployment- create_deploymentstream:UpdateDeployment- update_deploymentstream:GetDeployment- get_deploymentstream:ListDeployments- list_deploymentsstream:DeleteDeployment- delete_deploymentstream:GetDeploymentsByName- get_deployments_by_namestream:GetDeploymentsByLabel- get_deployments_by_labelstream:GetEvents- get_eventsstream:StartJobWithParams- start_jobstream:StopJob- stop_jobstream:GetJob- get_jobstream:ListJobs- list_jobsstream:DeleteJob- delete_jobstream:HotUpdateJob- hot_update_jobstream:GetHotUpdateJobResult- get_hot_update_resultstream:GetLatestJobStartLog- get_job_start_logstream:GetJobDiagnosis- diagnose_jobstream:CreateSavepoint- create_savepointstream:GetSavepoint- get_savepointstream:DeleteSavepoint- delete_savepointstream:ListSavepoints- list_savepointsstream:GenerateResourcePlanWithFlinkConfAsync- generate_resource_planstream:GetGenerateResourcePlanResult- get_resource_plan_resultstream:GetLineageInfo- get_lineagestream:FlinkApiProxy- flink_api_proxy
W3: Session Clusters
stream:CreateSessionCluster- create_session_clusterstream:UpdateSessionCluster- update_session_clusterstream:DeleteSessionCluster- delete_session_clusterstream:GetSessionCluster- get_session_clusterstream:ListSessionClusters- list_session_clustersstream:StartSessionCluster- start_session_clusterstream:StopSessionCluster- stop_session_cluster
W4: Dev Resources
stream:GetCatalogs- get_catalogsstream:GetDatabases- get_databasesstream:GetTables- get_tablesstream:ExecuteSqlStatement- execute_sqlstream:ListEngineVersionMetadata- list_engine_versions
W5: Workspace Administration
stream:CreateMember- create_memberstream:UpdateMember- update_memberstream:DeleteMember- delete_memberstream:GetMember- get_memberstream:ListMembers- list_membersstream:CreateVariable- create_variablestream:UpdateVariable- update_variablestream:DeleteVariable- delete_variablestream:ListVariables- list_variablesstream:CreateDeploymentTargetV2- create_deployment_targetstream:UpdateDeploymentTargetV2- update_deployment_targetstream:DeleteDeploymentTarget- delete_deployment_targetstream:ListDeploymentTargets- list_deployment_targets
---
Minimum Permission Policy
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": [
"stream:CreateFolder",
"stream:UpdateFolder",
"stream:DeleteFolder",
"stream:GetFolder",
"stream:CreateDeploymentDraft",
"stream:UpdateDeploymentDraft",
"stream:DeleteDeploymentDraft",
"stream:GetDeploymentDraft",
"stream:ListDeploymentDrafts",
"stream:GetDeploymentDraftLock",
"stream:ValidateDeploymentDraftAsync",
"stream:GetValidateDeploymentDraftResult",
"stream:ValidateSqlStatement",
"stream:DeployDeploymentDraftAsync",
"stream:GetDeployDeploymentDraftResult",
"stream:CreateDeployment",
"stream:UpdateDeployment",
"stream:GetDeployment",
"stream:ListDeployments",
"stream:DeleteDeployment",
"stream:GetDeploymentsByName",
"stream:GetDeploymentsByLabel",
"stream:GetEvents",
"stream:StartJobWithParams",
"stream:StopJob",
"stream:GetJob",
"stream:ListJobs",
"stream:DeleteJob",
"stream:HotUpdateJob",
"stream:GetHotUpdateJobResult",
"stream:GetLatestJobStartLog",
"stream:GetJobDiagnosis",
"stream:CreateSavepoint",
"stream:GetSavepoint",
"stream:DeleteSavepoint",
"stream:ListSavepoints",
"stream:GenerateResourcePlanWithFlinkConfAsync",
"stream:GetGenerateResourcePlanResult",
"stream:GetLineageInfo",
"stream:FlinkApiProxy",
"stream:CreateSessionCluster",
"stream:UpdateSessionCluster",
"stream:DeleteSessionCluster",
"stream:GetSessionCluster",
"stream:ListSessionClusters",
"stream:StartSessionCluster",
"stream:StopSessionCluster",
"stream:GetCatalogs",
"stream:GetDatabases",
"stream:GetTables",
"stream:ExecuteSqlStatement",
"stream:ListEngineVersionMetadata",
"stream:CreateMember",
"stream:UpdateMember",
"stream:DeleteMember",
"stream:GetMember",
"stream:ListMembers",
"stream:CreateVariable",
"stream:UpdateVariable",
"stream:DeleteVariable",
"stream:ListVariables",
"stream:CreateDeploymentTargetV2",
"stream:UpdateDeploymentTargetV2",
"stream:DeleteDeploymentTarget",
"stream:ListDeploymentTargets"
],
"Resource": [
"acs:stream:*:*:workspace/*"
]
}
]
}---
Permission Breakdown by Workflow
W1: SQL Development
| API Action | RAM Action |
|---|---|
CreateFolder | stream:CreateFolder |
UpdateFolder | stream:UpdateFolder |
DeleteFolder | stream:DeleteFolder |
GetFolder | stream:GetFolder |
CreateDeploymentDraft | stream:CreateDeploymentDraft |
UpdateDeploymentDraft | stream:UpdateDeploymentDraft |
DeleteDeploymentDraft | stream:DeleteDeploymentDraft |
GetDeploymentDraft | stream:GetDeploymentDraft |
ListDeploymentDrafts | stream:ListDeploymentDrafts |
GetDeploymentDraftLock | stream:GetDeploymentDraftLock |
ValidateDeploymentDraftAsync | stream:ValidateDeploymentDraftAsync |
GetValidateDeploymentDraftResult | stream:GetValidateDeploymentDraftResult |
ValidateSqlStatement | stream:ValidateSqlStatement |
DeployDeploymentDraftAsync | stream:DeployDeploymentDraftAsync |
GetDeployDeploymentDraftResult | stream:GetDeployDeploymentDraftResult |
W2: Job Operations
| API Action | RAM Action |
|---|---|
CreateDeployment | stream:CreateDeployment |
UpdateDeployment | stream:UpdateDeployment |
GetDeployment | stream:GetDeployment |
ListDeployments | stream:ListDeployments |
DeleteDeployment | stream:DeleteDeployment |
GetDeploymentsByName | stream:GetDeploymentsByName |
GetDeploymentsByLabel | stream:GetDeploymentsByLabel |
GetEvents | stream:GetEvents |
StartJobWithParams | stream:StartJobWithParams |
StopJob | stream:StopJob |
GetJob | stream:GetJob |
ListJobs | stream:ListJobs |
DeleteJob | stream:DeleteJob |
HotUpdateJob | stream:HotUpdateJob |
GetHotUpdateJobResult | stream:GetHotUpdateJobResult |
GetLatestJobStartLog | stream:GetLatestJobStartLog |
GetJobDiagnosis | stream:GetJobDiagnosis |
CreateSavepoint | stream:CreateSavepoint |
GetSavepoint | stream:GetSavepoint |
DeleteSavepoint | stream:DeleteSavepoint |
ListSavepoints | stream:ListSavepoints |
GenerateResourcePlanWithFlinkConfAsync | stream:GenerateResourcePlanWithFlinkConfAsync |
GetGenerateResourcePlanResult | stream:GetGenerateResourcePlanResult |
GetLineageInfo | stream:GetLineageInfo |
FlinkApiProxy | stream:FlinkApiProxy |
W3: Session Clusters
| API Action | RAM Action |
|---|---|
CreateSessionCluster | stream:CreateSessionCluster |
UpdateSessionCluster | stream:UpdateSessionCluster |
DeleteSessionCluster | stream:DeleteSessionCluster |
GetSessionCluster | stream:GetSessionCluster |
ListSessionClusters | stream:ListSessionClusters |
StartSessionCluster | stream:StartSessionCluster |
StopSessionCluster | stream:StopSessionCluster |
W4: Dev Resources
| API Action | RAM Action |
|---|---|
GetCatalogs | stream:GetCatalogs |
GetDatabases | stream:GetDatabases |
GetTables | stream:GetTables |
ExecuteSqlStatement | stream:ExecuteSqlStatement |
ListEngineVersionMetadata | stream:ListEngineVersionMetadata |
W5: Workspace Administration
| API Action | RAM Action |
|---|---|
CreateMember | stream:CreateMember |
UpdateMember | stream:UpdateMember |
DeleteMember | stream:DeleteMember |
GetMember | stream:GetMember |
ListMembers | stream:ListMembers |
CreateVariable | stream:CreateVariable |
UpdateVariable | stream:UpdateVariable |
DeleteVariable | stream:DeleteVariable |
ListVariables | stream:ListVariables |
CreateDeploymentTargetV2 | stream:CreateDeploymentTargetV2 |
UpdateDeploymentTargetV2 | stream:UpdateDeploymentTargetV2 |
DeleteDeploymentTarget | stream:DeleteDeploymentTarget |
ListDeploymentTargets | stream:ListDeploymentTargets |
---
Resource ARN Examples
Use resource-level constraints when possible:
- Workspace:
acs:stream:{regionId}:{accountId}:workspace/{workspaceId} - Namespace:
acs:stream:{regionId}:{accountId}:workspace/{workspaceId}/namespace/{namespace} - Deployment:
acs:stream:{regionId}:{accountId}:workspace/{workspaceId}/namespace/{namespace}/deployment/{deploymentId}
Example policy for one specific workspace:
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": [
"stream:ListDeployments",
"stream:GetDeployment",
"stream:StartJobWithParams",
"stream:StopJob"
],
"Resource": "acs:stream:cn-beijing:123456789012:workspace/w-xxx"
}
]
}---
Predefined System Policies
Alibaba Cloud currently provides these common system policies:
AliyunStreamFullAccessAliyunStreamReadOnlyAccess
If your organization requires least privilege, prefer custom policy with explicit stream:* actions shown above.
---
Troubleshooting
CLI Plugin Mode (MANDATORY)
When checking RAM permissions with aliyun CLI, always use plugin-mode command names (lowercase-hyphenated). Never use PascalCase API names directly in CLI commands.
aliyun ram list-policies-for-user --user-name <user_name>
aliyun ram list-access-keys --user-name <user_name>Error: Forbidden.RAM
1. Verify attached policies in the RAM Console → Users → Permissions tab. 2. Attach a policy that includes required stream:* actions. 3. Retry the operation.
Error: InvalidAccessKeyId.NotFound
1. Verify AccessKey in the RAM Console → Users → AccessKey Management tab. 2. Rotate/recreate AccessKey and update local config.
Error: NoPermission
1. Ensure the RAM user has the required actions for the specific workflow. 2. Check if resource-level permissions are restricting access. 3. Use * resource for testing, then narrow down to specific ARNs.
---
References
Related APIs
This document lists all the Ververica APIs used by the alibabacloud-flink-workspace-ops skill.
W1: SQL Development APIs
| API Action | Description |
|---|---|
| CreateFolder | Create a folder for organizing drafts |
| UpdateFolder | Update folder information |
| DeleteFolder | Delete an empty folder |
| GetFolder | Get folder details |
| CreateDeploymentDraft | Create a SQL deployment draft |
| UpdateDeploymentDraft | Update a SQL deployment draft |
| DeleteDeploymentDraft | Delete a SQL deployment draft |
| GetDeploymentDraft | Get draft details |
| ListDeploymentDrafts | List all drafts |
| GetDeploymentDraftLock | Get draft edit lock status |
| ValidateDeploymentDraftAsync | Deep-validate a draft (async) |
| GetValidateDeploymentDraftResult | Get validation result by ticket ID |
| ValidateSqlStatement | Quick-validate SQL syntax |
| DeployDeploymentDraftAsync | Deploy a draft to production (async) |
| GetDeployDeploymentDraftResult | Get deploy result by ticket ID |
W2: Job Operations APIs
| API Action | Description |
|---|---|
| CreateDeployment | Create a deployment |
| UpdateDeployment | Update a deployment |
| GetDeployment | Get deployment details |
| ListDeployments | List all deployments |
| DeleteDeployment | Delete a deployment (irreversible) |
| GetDeploymentsByName | Search deployments by name |
| GetDeploymentsByLabel | Search deployments by label |
| GetDeploymentsByIp | Search deployments by IP |
| GetEvents | Get deployment run events |
| StartJobWithParams | Start a job instance |
| StopJob | Stop a job instance |
| GetJob | Get job instance details |
| ListJobs | List job instances for a deployment |
| DeleteJob | Delete a non-running job instance |
| HotUpdateJob | Hot-update a running job |
| GetHotUpdateJobResult | Get hot-update result |
| GetLatestJobStartLog | Get latest job startup log |
| GetJobDiagnosis | Diagnose job failures |
| CreateSavepoint | Create a savepoint |
| GetSavepoint | Get savepoint details |
| DeleteSavepoint | Delete a savepoint (irreversible) |
| ListSavepoints | List savepoints for a deployment |
| GenerateResourcePlanWithFlinkConfAsync | Generate resource plan (async) |
| GetGenerateResourcePlanResult | Get resource plan generation result |
| GetLineageInfo | Get job lineage information |
| FlinkApiProxy | Proxy Flink REST API (read-only) |
W3: Session Cluster APIs
| API Action | Description |
|---|---|
| CreateSessionCluster | Create a Session cluster |
| UpdateSessionCluster | Update Session cluster configuration |
| DeleteSessionCluster | Delete a Session cluster (irreversible) |
| GetSessionCluster | Get Session cluster details |
| ListSessionClusters | List all Session clusters |
| StartSessionCluster | Start a Session cluster |
| StopSessionCluster | Stop a Session cluster |
W4: Dev Resources APIs
| API Action | Description |
|---|---|
| CreateUdfArtifact | Create a UDF artifact |
| UpdateUdfArtifact | Update a UDF artifact |
| GetUdfArtifacts | List UDF artifacts |
| DeleteUdfArtifact | Delete a UDF artifact |
| RegisterUdfFunction | Register UDF function(s) |
| DeleteUdfFunction | Delete a UDF function |
| ListCustomConnectors | List custom connectors |
| RegisterCustomConnector | Register a custom connector |
| DeleteCustomConnector | Delete a custom connector |
| GetCatalogs | List or get catalog details |
| GetDatabases | List or get database details |
| GetTables | List or get table details |
| ExecuteSqlStatement | Execute DDL/DML SQL statement (no DQL) |
| ListEngineVersionMetadata | List supported engine versions |
W5: Workspace Administration APIs
| API Action | Description |
|---|---|
| CreateMember | Add a member with permissions |
| UpdateMember | Update member permissions |
| DeleteMember | Delete a member |
| GetMember | Get member details |
| ListMembers | List all members |
| CreateVariable | Create a variable |
| UpdateVariable | Update a variable |
| DeleteVariable | Delete a variable |
| ListVariables | List all variables |
| CreateDeploymentTargetV2 | Create a deployment target (V2) |
| UpdateDeploymentTargetV2 | Update a deployment target (V2) |
| DeleteDeploymentTarget | Delete a deployment target |
| ListDeploymentTargets | List all deployment targets |
Resource Loading Policy
This skill keeps SKILL.md concise and loads detailed docs only when needed.
Entry Contract (Always Loaded)
SKILL.md— boundary, trigger priority, safety baseline, execution baseline.- Keep concise; no long command encyclopedia.
Loading Strategy
1. After Trigger: Load execution protocol and command routing table once per task. 2. On Demand: Load only the specific deep document required by current context (error handling, verification, RAM policies, full command catalog, playbooks, etc.). 3. Refer to the "Resources" section in SKILL.md for the complete list of available documents and when to load each.
Loading Discipline
1. Do not preload all references. 2. Start from the entry contract, then task docs, then only the needed deep doc. 3. Prefer the smallest sufficient context for the current user request.
Verification Methods
Use this guide after a mutating command to verify actual resource state.
⚠️ Global Verification Rules
- MANDATORY: Verify after EVERY mutating operation
- Prefer
-o jsonfor deterministic parsing - Verify immediately after each mutation (don't delay)
- For async operations, verify only after polling returns terminal result
- NEVER claim success without read-back verification
Verification Protocol (MANDATORY)
Step 1: Identify Verification Command
Match operation to verification command:
| Operation | Verification Command | Expected Result |
|---|---|---|
| create_deployment | get_deployment | status: CREATED |
| start_job | get_job | state: RUNNING |
| stop_job | get_job | state: STOPPED/CANCELED |
| create_session_cluster | get_session_cluster | state: CREATED/RUNNING |
| delete_deployment | get_deployment | ResourceNotFound error |
| create_savepoint | get_savepoint | state: COMPLETED |
Step 2: Execute Verification Command
python scripts/flink_ververica_ops.py get_<resource> -w <workspace> -n <namespace> -r <region> --<resource>_id <id> -o jsonStep 3: Parse and Confirm Response
Successful creation/update:
{
"success": true,
"data": {
"resource_id": "xxx",
"name": "expected-name",
"status": "EXPECTED_STATE"
}
}Successful deletion:
{
"success": false,
"error": {
"code": "ResourceNotFound"
}
}Step 4: Report Verification Results
✓ Verification successful
**Operation:** create_deployment
**Resource ID:** d-123456
**Current State:**
- Name: etl-job
- Status: CREATED
- Engine Version: vvr-8.0.1-flink-1.17
- Namespace: default
- Workspace: w-xxxVerification Examples by Workflow
create_folder / update_folder
# After creation
python scripts/flink_ververica_ops.py get_folder -w <workspace> -n <namespace> -r <region> --folder_id <folder_id> -o json
# Verify:
# - success: true
# - data.name matches expected name
# - data.folder_id existsdelete_folder
# After deletion, verify it's gone
python scripts/flink_ververica_ops.py get_folder -w <workspace> -n <namespace> -r <region> --folder_id <folder_id> -o json
# Verify:
# - success: false
# - error.code: "ResourceNotFound"create_draft / update_draft
# After creation
python scripts/flink_ververica_ops.py get_draft -w <workspace> -n <namespace> -r <region> --draft_id <draft_id> -o json
# Verify:
# - success: true
# - data.name matches expected name
# - data.sql_content matches expected SQLvalidate_draft (async)
# Step 1: Start validation
python scripts/flink_ververica_ops.py validate_draft -w <workspace> -n <namespace> -r <region> --draft_id <draft_id> -o json
# Returns: {"success": true, "data": {"ticket_id": "xxx-xxx-xxx"}}
# Step 2: Poll for result (repeat until terminal state)
python scripts/flink_ververica_ops.py get_validate_result -w <workspace> -n <namespace> -r <region> --ticket_id <ticket_id> -o json
# Verify:
# - success: true
# - data.status: "SUCCEEDED" or "FAILED"
# - If FAILED: data.errors contains error detailsdeploy_draft (async)
# Step 1: Start deployment
python scripts/flink_ververica_ops.py deploy_draft -w <workspace> -n <namespace> -r <region> --draft_id <draft_id> --deployment_name <name> --confirm -o json
# Returns: {"success": true, "data": {"ticket_id": "xxx-xxx-xxx"}}
# Step 2: Poll for result
python scripts/flink_ververica_ops.py get_deploy_result -w <workspace> -n <namespace> -r <region> --ticket_id <ticket_id> -o json
# Verify:
# - success: true
# - data.status: "SUCCEEDED"
# - data.deployment_id existsW2 Job Operations
create_deployment / update_deployment
# After operation
python scripts/flink_ververica_ops.py get_deployment -w <workspace> -n <namespace> -r <region> --deployment_id <deployment_id> -o json
# Verify:
# - success: true
# - data.name matches expected
# - data.status in ["CREATED", "RUNNING", "STOPPED"]
# - data.engine_version matches expectedstart_job
# After start
python scripts/flink_ververica_ops.py get_job -w <workspace> -n <namespace> -r <region> --job_id <job_id> -o json
# Verify:
# - success: true
# - data.state: "RUNNING"
# - data.start_time existsstop_job
# After stop
python scripts/flink_ververica_ops.py get_job -w <workspace> -n <namespace> -r <region> --job_id <job_id> -o json
# Verify:
# - success: true
# - data.state: "STOPPED" or "CANCELED"
# - data.stop_time existscreate_savepoint
# After creation
python scripts/flink_ververica_ops.py get_savepoint -w <workspace> -n <namespace> -r <region> --savepoint_id <savepoint_id> -o json
# Verify:
# - success: true
# - data.state: "COMPLETED"
# - data.location contains savepoint pathdelete_deployment
# After deletion
python scripts/flink_ververica_ops.py get_deployment -w <workspace> -n <namespace> -r <region> --deployment_id <deployment_id> -o json
# Verify:
# - success: false
# - error.code: "ResourceNotFound"W3 Session Clusters
create_session_cluster / update_session_cluster
# After operation
python scripts/flink_ververica_ops.py get_session_cluster -w <workspace> -n <namespace> -r <region> --cluster_id <cluster_id> -o json
# Verify:
# - success: true
# - data.name matches expected
# - data.state in ["CREATED", "STARTING", "RUNNING", "STOPPING", "STOPPED"]start_session_cluster
# After start
python scripts/flink_ververica_ops.py get_session_cluster -w <workspace> -n <namespace> -r <region> --cluster_id <cluster_id> -o json
# Verify:
# - success: true
# - data.state: "RUNNING"stop_session_cluster
# After stop
python scripts/flink_ververica_ops.py get_session_cluster -w <workspace> -n <namespace> -r <region> --cluster_id <cluster_id> -o json
# Verify:
# - success: true
# - data.state: "STOPPED"delete_session_cluster
# After deletion, verify removal
python scripts/flink_ververica_ops.py list_session_clusters -w <workspace> -n <namespace> -r <region> -o json
# Verify:
# - success: true
# - data.session_clusters does NOT contain the deleted clusterW4 Dev Resources
execute_sql
# For DDL (CREATE TABLE/DATABASE), verify via metadata query
# For DML (INSERT), verify via SELECT query
# Example: after CREATE TABLE
python scripts/flink_ververica_ops.py get_tables -w <workspace> -n <namespace> -r <region> --catalog <catalog> --database <database> -o json
# Verify:
# - success: true
# - data.tables contains the newly created tableget_catalogs / get_databases / get_tables
# These are read operations, verify by:
# - success: true
# - data contains expected catalogs/databases/tablesW5 Workspace Administration
create_member / update_member
# After operation
python scripts/flink_ververica_ops.py get_member -w <workspace> -n <namespace> -r <region> --member_id <member_id> -o json
# Verify:
# - success: true
# - data.user_id matches expected
# - data.role matches expecteddelete_member
# After deletion
python scripts/flink_ververica_ops.py list_members -w <workspace> -n <namespace> -r <region> -o json
# Verify:
# - success: true
# - data.members does NOT contain the deleted membercreate_variable / update_variable
# After operation
python scripts/flink_ververica_ops.py list_variables -w <workspace> -n <namespace> -r <region> -o json
# Verify:
# - success: true
# - data.variables contains the created/updated variable
# - Variable value matches expectedcreate_deploy_target / update_deploy_target
# After operation
python scripts/flink_ververica_ops.py list_deploy_targets -w <workspace> -n <namespace> -r <region> -o json
# Verify:
# - success: true
# - data.deploy_targets contains the created/updated targetVerification Response Template
When reporting verification to user:
✓ Verification successful
**Operation:** create_deployment
**Resource ID:** d-123456
**Current State:**
- Name: etl-job
- Status: CREATED
- Engine Version: vvr-8.0.1-flink-1.17
- Namespace: default
- Workspace: w-xxx
**Next Actions:**
- To start the job: start_job --deployment_id d-123456
- To update configuration: update_deployment --deployment_id d-123456 --body_json '{...}'Common Verification Failures
Failure 1: Resource not found
❌ Verification failed: Resource not found
Expected: Deployment d-123456 should exist
Actual: get_deployment returned ResourceNotFound
Possible causes:
- Creation failed silently
- Wrong deployment_id
- Wrong workspace/namespace/regionFailure 2: Wrong state
❌ Verification failed: Unexpected state
Expected: Job j-xxx should be RUNNING
Actual: Job state is FAILED
Error from job:
- Failure Reason: Checkpoint timeout
- Error Message: Checkpoint could not be completed within timeoutFailure 3: Partial update
❌ Verification failed: Update incomplete
Expected: deployment_target_name = "target-prod"
Actual: deployment_target_name = "target-dev" (unchanged)
Possible causes:
- Update operation failed
- Body JSON not properly formattedBest Practices
1. Always verify - never assume success from write response alone 2. Verify immediately - don't delay between operation and verification 3. Check all fields - not just success flag, but actual resource state 4. Report clearly - show what was expected vs what was found 5. Investigate failures - don't just report failure, investigate root cause
VVP 产品概念模型
实体层次与关系
Workspace(工作空间) └─ Namespace(项目空间):作业管理和资源隔离的基本单元,所有配置、作业、权限均在单个 Namespace 下 ├─ DeploymentDraft(作业草稿):作业草稿的配置定义(模板),包含代码 artifact、资源规格、运行参数. 部署上线后, 生成对应的Deployment ├─ Deployment(作业部署):作业的配置定义(模板),包含代码 artifact、资源规格、运行参数 │ └─ Job(作业实例):Deployment 的一次运行实例 [1:N],Job 是 Deployment 的快照,绝大部分字段不可变,对作业的变更主要通过修改部署后重启Job实现(HotUpdate除外)。 │ └─ Savepoint(快照):Job 运行时的状态快照 [1:N],用于有状态恢复 ├─ SessionCluster(Session 集群):仅用于开发测试的共享集群,不支持监控告警和自动调优 ├─ ResourceQueue(资源队列):计算资源的分配单元,Deployment 需要部署到资源队列或SessionCluster上运行 └─ Catalog(SQL元数据):管理用户SQL类作业中使用的数据库、表、字段等元数据信息 └─ Database → Table
- 用户通过 deployment_id 定位一个部署配置,通过 job_id 定位一个具体的运行实例
- 所有 API path 中的 {namespace} 自动替换为当前项目空间
作业类型(artifact.kind)
| 枚举值 | 说明 |
|---|---|
| SQLSCRIPT | SQL 作业 |
| MATERIALIZED_TABLE | 物化表作业(SQL 子类型) |
| JAR | JAR 作业 |
| PYTHON | Python 作业 |
| YAML | Flink CDC 数据摄入作业(SQL 子类型)(VVR 8.0.9+) |
执行模式 ExecutionMode
deployment 和 job 的执行模式在创建时确定,不能更改。
| 枚举值 | 说明 |
|---|---|
| STREAMING | 流模式,持续运行处理无界数据流。一个 deployment 只能有一个非终态的 job。 |
| BATCH | 批模式,处理有界数据集后结束。一个 deployment 可以有多个非终态的 job。 |
作业状态(Job state)
STARTING → RUNNING → FINISHED / CANCELLED / FAILED
| 状态 | 类别 | 说明 |
|---|---|---|
| STARTING | 过渡态 | 作业正在启动 |
| RUNNING | 稳定态 | 作业正在运行 |
| FINISHED | 终态 | 批作业或有限流作业完成,或流作业触发stop-with-savepoint并完成。 |
| CANCELLED | 终态 | 用户主动停止 |
| FAILED | 终态 | 作业运行失败 |
引擎及版本
VVR和Flash均是VVP提供的商业版Flink引擎。
engineVersion 或 versionName 字段是引擎版本的展示名,在 workspace 下唯一,示例: "vvr-8.0.6-flink-1.17"。
引擎版本标记:推荐版本 > 稳定版本 > 普通版本 > EOS 版本。
部分功能有版本要求(如动态参数更新需 VVR 8.0.1+,算子 TTL 需 VVR 8.0.7+,YAML 作业需 VVR 8.0.9+)。
#!/usr/bin/env python3
"""
Shared client module for Flink Ververica CLI.
Provides:
- SDK client initialization (default credential chain + region -> endpoint)
- Output formatting (json / table / text)
- Safety confirmation logic (TTY interactive prompt / non-TTY error)
- Standardized response envelope
- Input validation utilities
"""
import json
import os
import sys
import traceback
# ---------------------------------------------------------------------------
# SDK client
# ---------------------------------------------------------------------------
_CLIENT_CACHE: dict = {}
DEFAULT_USER_AGENT = "AlibabaCloud-Agent-Skills/alibabacloud-flink-workspace-ops"
def get_client(region_id: str):
"""Return a cached Ververica API client for *region_id*.
Uses Alibaba Cloud default credential chain (RAM role, CLI profile, etc.)
"""
from alibabacloud_credentials.client import Client as CredentialClient
from alibabacloud_ververica20220718.client import Client
from alibabacloud_tea_openapi.models import Config
if region_id in _CLIENT_CACHE:
return _CLIENT_CACHE[region_id]
if not region_id:
print(
json.dumps(
{
"success": False,
"error": {
"code": "ValidationError",
"message": "region_id is required.",
},
}
),
file=sys.stderr,
)
sys.exit(1)
try:
credential = CredentialClient()
except Exception as e:
print(
json.dumps(
{
"success": False,
"error": {
"code": "MissingCredentials",
"message": f"Failed to resolve credentials: {e}",
},
}
),
file=sys.stderr,
)
sys.exit(1)
config = Config(
credential=credential,
endpoint=f"ververica.{region_id}.aliyuncs.com",
user_agent=DEFAULT_USER_AGENT,
)
client = Client(config)
_CLIENT_CACHE[region_id] = client
return client
def runtime_options():
"""Return a new ``RuntimeOptions`` instance with explicit timeout configuration.
Timeout defaults (can be overridden via environment variables):
- connect_timeout: 10 seconds (FLINK_SDK_CONNECT_TIMEOUT)
- read_timeout: 60 seconds (FLINK_SDK_READ_TIMEOUT)
"""
from alibabacloud_tea_util.models import RuntimeOptions
# Allow environment variable override for timeout settings
connect_timeout = int(os.environ.get("FLINK_SDK_CONNECT_TIMEOUT", "10"))
read_timeout = int(os.environ.get("FLINK_SDK_READ_TIMEOUT", "60"))
return RuntimeOptions(
connect_timeout=connect_timeout,
read_timeout=read_timeout,
)
# ---------------------------------------------------------------------------
# Standardised response helpers
# ---------------------------------------------------------------------------
def success_response(operation: str, data, request_id: str = ""):
"""Build a success envelope dict."""
return {
"success": True,
"operation": operation,
"data": data,
"request_id": request_id,
}
def error_response(operation: str, code: str, message: str, request_id: str = ""):
"""Build an error envelope dict."""
return {
"success": False,
"operation": operation,
"error": {"code": code, "message": message},
"request_id": request_id,
}
# ---------------------------------------------------------------------------
# Output formatting
# ---------------------------------------------------------------------------
def output(result: dict, fmt: str = "json"):
"""Print *result* envelope to stdout in the requested format."""
if fmt == "json":
print(json.dumps(result, ensure_ascii=False, indent=2))
elif fmt == "table":
_print_table(result)
elif fmt == "text":
_print_text(result)
else:
print(json.dumps(result, ensure_ascii=False, indent=2))
sys.exit(0 if result.get("success") else 1)
def _print_table(result: dict):
"""Pretty-print *result* as an aligned table."""
if not result.get("success"):
err = result.get("error", {})
print(
f"ERROR [{err.get('code', '?')}]: {err.get('message', '?')}",
file=sys.stderr,
)
return
data = result.get("data")
if data is None:
print("(no data)")
return
# If data is a dict containing a list value, use the first list found
rows = _extract_rows(data)
if rows is None:
# single-object result
for k, v in (data if isinstance(data, dict) else {}).items():
print(f"{k}: {v}")
return
if not rows:
print("(empty)")
return
# Collect columns from first row
if isinstance(rows[0], dict):
cols = list(rows[0].keys())
widths = {c: len(c) for c in cols}
str_rows = []
for r in rows:
sr = {}
for c in cols:
val = str(r.get(c, ""))
sr[c] = val
widths[c] = max(widths[c], len(val))
str_rows.append(sr)
header = " ".join(c.upper().ljust(widths[c]) for c in cols)
print(header)
for sr in str_rows:
print(" ".join(sr[c].ljust(widths[c]) for c in cols))
else:
for r in rows:
print(r)
def _print_text(result: dict):
"""Print tab-separated values suitable for piping."""
if not result.get("success"):
err = result.get("error", {})
print(f"{err.get('code', '?')}\t{err.get('message', '?')}", file=sys.stderr)
return
data = result.get("data")
if data is None:
return
rows = _extract_rows(data)
if rows is None:
if isinstance(data, dict):
for k, v in data.items():
print(f"{k}\t{v}")
return
for r in rows:
if isinstance(r, dict):
print("\t".join(str(v) for v in r.values()))
else:
print(r)
def _extract_rows(data):
"""Try to find a list of records inside *data*."""
if isinstance(data, list):
return data
if isinstance(data, dict):
for v in data.values():
if isinstance(v, list):
return v
return None
# ---------------------------------------------------------------------------
# Safety confirmation
# ---------------------------------------------------------------------------
def require_confirmation(
operation: str,
message: str,
flag_present: bool,
):
"""
Check the safety confirmation gate.
If *flag_present* is True the caller already passed --confirm
and we proceed silently.
Otherwise:
- In an interactive TTY → prompt the user.
- In a non-interactive pipe / agent context → return an error dict.
Returns None on success (proceed) or an error-dict to output and abort.
"""
if flag_present:
return None # OK
if sys.stdin.isatty() and sys.stdout.isatty():
# Interactive mode – ask the user
print(f"\n\u26a0\ufe0f {message}", file=sys.stderr)
answer = input(" Proceed? [y/N]: ")
if answer.strip().lower() in ("y", "yes"):
return None
return error_response(operation, "Cancelled", "User cancelled the operation.")
# Non-interactive – hard error
return error_response(
operation,
"SafetyCheckRequired",
f"{message} Add --confirm to proceed.",
)
# ---------------------------------------------------------------------------
# Input validation helpers
# ---------------------------------------------------------------------------
def require_args(args, *names):
"""
Validate that every *name* is present (not None / empty) on *args*.
Returns None if OK, or an error-dict describing the first missing param.
"""
for name in names:
val = getattr(args, name, None)
if val is None or (isinstance(val, str) and val.strip() == ""):
return error_response(
getattr(args, "subcommand", "unknown"),
"ValidationError",
f"Parameter '--{name}' is required.",
)
return None
# ---------------------------------------------------------------------------
# Generic API call wrapper
# ---------------------------------------------------------------------------
def call_api(operation: str, api_func, *api_args, **api_kwargs):
"""
Invoke *api_func* and wrap the result in a standard envelope.
Handles SDK exceptions and returns a uniform error envelope.
"""
try:
resp = api_func(*api_args, **api_kwargs)
# SDK responses expose .body / .headers / .status_code
body = resp.body if hasattr(resp, "body") else resp
request_id = ""
if hasattr(resp, "headers") and resp.headers:
request_id = resp.headers.get("x-acs-request-id", "")
if hasattr(body, "request_id") and body.request_id:
request_id = body.request_id
# Convert body to plain dict
data = _to_dict(body)
return success_response(operation, data, request_id)
except Exception as e:
code = getattr(e, "code", type(e).__name__)
message = getattr(e, "message", str(e))
request_id = getattr(e, "request_id", "")
if os.environ.get("FLINK_CLI_DEBUG"):
traceback.print_exc(file=sys.stderr)
return error_response(operation, str(code), str(message), str(request_id))
def _to_dict(obj):
"""Recursively convert SDK model objects to plain dicts."""
if obj is None:
return None
if isinstance(obj, (str, int, float, bool)):
return obj
if isinstance(obj, list):
return [_to_dict(i) for i in obj]
if isinstance(obj, dict):
return {k: _to_dict(v) for k, v in obj.items()}
if hasattr(obj, "to_map"):
return _to_dict(obj.to_map())
if hasattr(obj, "__dict__"):
return _to_dict(
{k: v for k, v in obj.__dict__.items() if not k.startswith("_")}
)
return str(obj)
# ---------------------------------------------------------------------------
# Common argparse helpers
# ---------------------------------------------------------------------------
def add_common_args(parser):
"""Add -w / -n / -r / -o / -v / -q global flags to *parser*."""
# Keep common scope flags optional at argparse level so handlers can always
# return structured JSON validation errors via require_args().
parser.add_argument("-w", "--workspace", help="Workspace ID")
parser.add_argument("-n", "--namespace", help="Namespace name")
parser.add_argument(
"-r", "--region_id", help="Region ID (e.g. cn-beijing)"
)
parser.add_argument(
"-o",
"--output",
choices=["json", "table", "text"],
default="json",
help="Output format (default: json)",
)
parser.add_argument(
"-v", "--verbose", action="store_true", help="Show request details"
)
parser.add_argument(
"-q", "--quiet", action="store_true", help="Suppress status messages"
)
#!/usr/bin/env python3
"""
转换alibabacloud-flink-workspace-ops的测试用例到skill-test-set格式
"""
import json
import re
from pathlib import Path
def load_jsonc(filepath):
"""加载JSONC文件(去除注释)"""
with open(filepath, "r", encoding="utf-8") as f:
content = f.read()
# 移除单行注释
content = re.sub(r"//.*$", "", content, flags=re.MULTILINE)
# 移除多行注释
content = re.sub(r"/\*.*?\*/", "", content, flags=re.DOTALL)
return json.loads(content)
def generate_expected_output_positive(name, prompt):
"""为正向触发测试生成expected_output"""
# 根据用例名称推断命令
command_map = {
"create-draft": "create_draft",
"validate-sql": "validate_sql",
"deploy-draft": "deploy_draft",
"list-deployments": "list_deployments",
"start-job": "start_job",
"stop-job": "stop_job",
"create-session-cluster": "create_session_cluster",
"get-tables": "get_tables",
"create-member": "create_member",
"list-variables": "list_variables",
"diagnose-job": "diagnose_job",
}
# 查找匹配的命令
for key, cmd in command_map.items():
if name.startswith(key):
return f"触发 alibabacloud-flink-workspace-ops skill,调用 {cmd} 命令"
return "触发 alibabacloud-flink-workspace-ops skill"
def generate_assertions_positive(name, prompt):
"""为正向触发测试生成assertions"""
assertions = [
"应识别用户意图与Flink Console操作相关",
"应触发 alibabacloud-flink-workspace-ops skill",
]
# 根据用例名称添加特定断言
if "create-draft" in name:
assertions.append("应识别创建草稿意图")
elif "validate-sql" in name:
assertions.append("应识别SQL校验意图")
elif "deploy-draft" in name:
assertions.append("应识别部署草稿意图")
elif "list-deployments" in name:
assertions.append("应识别查询部署列表意图")
elif "start-job" in name:
assertions.append("应识别启动作业意图")
elif "stop-job" in name:
assertions.append("应识别停止作业意图")
if "savepoint" in prompt.lower():
assertions.append("应识别创建savepoint的需求")
elif "create-session-cluster" in name:
assertions.append("应识别创建Session集群意图")
elif "get-tables" in name:
assertions.append("应识别查询表列表意图")
elif "create-member" in name:
assertions.append("应识别添加成员意图")
elif "list-variables" in name:
assertions.append("应识别查询变量列表意图")
elif "diagnose-job" in name:
assertions.append("应识别作业诊断意图")
return assertions
def generate_expected_output_negative(name, prompt):
"""为负向触发测试生成expected_output"""
return "不应触发 alibabacloud-flink-workspace-ops skill"
def generate_assertions_negative(name, prompt):
"""为负向触发测试生成assertions"""
assertions = [
"不应触发 alibabacloud-flink-workspace-ops skill",
"用户意图与Flink Console操作无关",
]
# 根据用例名称添加特定断言
if "flink-instance-management" in name:
assertions.append("Flink实例管理与Console操作是不同领域")
elif "spark-cluster" in name:
assertions.append("Spark集群管理不在本skill范围内")
elif "ecs-instance" in name:
assertions.append("ECS实例管理不在本skill范围内")
elif "kubernetes" in name:
assertions.append("Kubernetes集群管理不在本skill范围内")
elif "open-source" in name:
assertions.append("开源Flink部署不在本skill范围内")
elif "billing" in name or "weather" in name:
assertions.append("账单查询/天气查询等通用查询不在本skill范围内")
return assertions
def convert_triggering_tests():
"""转换触发测试"""
base_path = Path(__file__).parent.parent / "evals" / "triggering" / "autoGenerated"
evals = []
id_counter = 1
# 转换正向触发测试
should_trigger_path = base_path / "should_trigger.jsonc"
if should_trigger_path.exists():
data = load_jsonc(should_trigger_path)
for tc in data["triggering"]["test_cases"]:
eval_case = {
"id": id_counter,
"name": tc["name"],
"prompt": tc["prompt"],
"expected_output": generate_expected_output_positive(
tc["name"], tc["prompt"]
),
"assertions": generate_assertions_positive(tc["name"], tc["prompt"]),
}
evals.append(eval_case)
id_counter += 1
# 转换负向触发测试
should_not_trigger_path = base_path / "should_not_trigger.jsonc"
if should_not_trigger_path.exists():
data = load_jsonc(should_not_trigger_path)
for tc in data["triggering"]["test_cases"]:
eval_case = {
"id": id_counter,
"name": tc["name"],
"prompt": tc["prompt"],
"expected_output": generate_expected_output_negative(
tc["name"], tc["prompt"]
),
"assertions": generate_assertions_negative(tc["name"], tc["prompt"]),
}
evals.append(eval_case)
id_counter += 1
return {"skill_name": "alibabacloud-flink-workspace-ops", "evals": evals}
def main():
result = convert_triggering_tests()
# 输出到skill-test-set目录
output_path = (
Path(__file__).parent.parent.parent
/ "skill-test-set"
/ "alibabacloud-flink-workspace-ops"
/ "evals.json"
)
output_path.parent.mkdir(parents=True, exist_ok=True)
with open(output_path, "w", encoding="utf-8") as f:
json.dump(result, f, ensure_ascii=False, indent=2)
print(f"✅ 成功转换 {len(result['evals'])} 个测试用例")
print(f"📁 输出路径: {output_path}")
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""
End-to-end live test for all 75 CLI subcommands.
Executes real API calls in dependency order, using resources created in
earlier steps. Prints a summary table at the end.
Usage:
cd scripts/
export ALIBABA_CLOUD_ACCESS_KEY_ID=...
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=...
python3 e2e_test.py
"""
import json
import subprocess
import sys
import time
WS = "f9bbbc0f55ed4d"
NS = "jz-test-default"
REGION = "cn-beijing"
DEPLOY_TARGET = "default-queue"
ENGINE = "vvr-8.0.11-flink-1.17"
COMMON = ["-w", WS, "-n", NS, "-r", REGION]
results = [] # (scenario, cmd, pass/fail, notes)
def cli(*args, expect_success=True, timeout=60):
"""Run flink_ververica_ops.py and return parsed JSON."""
cmd = [sys.executable, "flink_ververica_ops.py"] + list(args)
try:
r = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
except subprocess.TimeoutExpired:
return {"success": False, "error": {"message": f"CLI timeout after {timeout}s"}, "_timeout": True}
# Try to parse JSON from stdout
out = r.stdout.strip()
if not out:
# Check stderr for traceback
if "Traceback" in r.stderr or "TypeError" in r.stderr:
return {"_crash": True, "_stderr": r.stderr, "success": False}
return {"success": False, "_stderr": r.stderr, "_stdout": ""}
try:
data = json.loads(out)
except json.JSONDecodeError:
return {"success": False, "_raw": out}
return data
def record(scenario, cmd_name, result, notes=""):
api_ok = result.get("success", False)
crash = result.get("_crash", False)
is_timeout = result.get("_timeout", False)
if crash:
status = "CRASH"
notes = result.get("_stderr", "")[:120]
elif is_timeout:
status = "TIMEOUT"
notes = notes or "CLI process timed out"
elif api_ok:
# Check nested data.success (API-level)
inner = result.get("data", {})
if isinstance(inner, dict) and inner.get("success") is False:
status = "API_ERR"
notes = notes or inner.get("errorMessage", "")[:100]
else:
status = "PASS"
else:
status = "FAIL"
err = result.get("error", {})
notes = notes or err.get("message", str(result)[:100])
results.append((scenario, cmd_name, status, notes))
mark = {"PASS": "+", "API_ERR": "~", "FAIL": "x", "CRASH": "!", "TIMEOUT": "T"}[status]
print(f" [{mark}] {cmd_name}: {status} {notes[:80]}")
return result
def jd(obj):
return json.dumps(obj, ensure_ascii=False)
def get_data(result, key=None):
"""Extract nested data from API response."""
d = result.get("data", {})
if isinstance(d, dict) and "data" in d:
d = d["data"]
if key:
return d.get(key) if isinstance(d, dict) else None
return d
def wait_for_job_state(deployment_id, job_id, target_states, timeout=120):
"""Poll job state until it reaches one of target_states."""
start = time.time()
while time.time() - start < timeout:
r = cli("get_job", *COMMON, "--deployment_id", deployment_id, "--job_id", job_id)
data = get_data(r)
state = ""
if isinstance(data, dict):
status = data.get("status", {})
if isinstance(status, dict):
state = status.get("currentJobStatus", "") or status.get("state", "")
if not state:
state = data.get("state", "")
if state in target_states:
return state
print(f" >> ... job state = {state or '?'}, waiting...")
time.sleep(5)
return "TIMEOUT"
# ═════════════════════════════════════════════════════════════════════
# Scenario A: W5 Workspace Admin (13 commands)
# ═════════════════════════════════════════════════════════════════════
print("\n=== Scenario A: W5 Workspace Admin ===")
# A1: list_members
r = cli("list_members", *COMMON)
record("W5", "list_members", r)
# A2: create_member
r = cli("create_member", *COMMON, "--member", "123456789012", "--role", "viewer", "--confirm")
record("W5", "create_member", r)
# A3: get_member
r = cli("get_member", *COMMON, "--member", "123456789012")
record("W5", "get_member", r)
# A4: update_member
r = cli("update_member", *COMMON, "--member", "123456789012", "--role", "editor", "--confirm")
record("W5", "update_member", r)
# A5: delete_member
r = cli("delete_member", *COMMON, "--member", "123456789012", "--confirm")
record("W5", "delete_member", r)
# A6: list_variables
r = cli("list_variables", *COMMON)
record("W5", "list_variables", r)
# A7: create_variable (needs 'kind' field)
var_body = {"name": "test_cli_e2e", "value": "hello", "kind": "Plain"}
r = cli("create_variable", *COMMON, "--body_json", jd(var_body), "--confirm")
record("W5", "create_variable", r)
# A8: update_variable
var_body_upd = {"name": "test_cli_e2e", "value": "updated", "kind": "Plain"}
r = cli("update_variable", *COMMON, "--var_name", "test_cli_e2e", "--body_json", jd(var_body_upd), "--confirm")
record("W5", "update_variable", r)
# A9: delete_variable
r = cli("delete_variable", *COMMON, "--var_name", "test_cli_e2e", "--confirm")
record("W5", "delete_variable", r)
# A10: list_deploy_targets
r = cli("list_deploy_targets", *COMMON)
record("W5", "list_deploy_targets", r)
# A11: create_deploy_target (V2 needs Resource with fixedResource)
target_body = {"resource": {"fixedResource": {"cpu": 2.0, "memory": "8Gi"}}}
r = cli("create_deploy_target", *COMMON, "--name", "test-cli-target", "--body_json", jd(target_body), "--confirm")
record("W5", "create_deploy_target", r, "may fail if namespace lacks resource quota")
# A12: update_deploy_target
r = cli("update_deploy_target", *COMMON, "--target_name", "test-cli-target", "--body_json", jd(target_body), "--confirm")
record("W5", "update_deploy_target", r)
# A13: delete_deploy_target
r = cli("delete_deploy_target", *COMMON, "--target_name", "test-cli-target", "--confirm")
record("W5", "delete_deploy_target", r)
# ═════════════════════════════════════════════════════════════════════
# Scenario B: W1 SQL Development (15 commands)
# ═════════════════════════════════════════════════════════════════════
print("\n=== Scenario B: W1 SQL Development ===")
# B1: create_folder
r = cli("create_folder", *COMMON, "--folder_name", "e2e_test_folder", "--confirm")
record("W1", "create_folder", r)
folder_id = get_data(r, "folderId") or "unknown"
# B2: get_folder
r = cli("get_folder", *COMMON, "--folder_id", folder_id)
record("W1", "get_folder", r)
# B3: update_folder
r = cli("update_folder", *COMMON, "--folder_id", folder_id, "--folder_name", "e2e_folder_renamed", "--confirm")
record("W1", "update_folder", r)
# B4: create_draft
r = cli("create_draft", *COMMON,
"--name", "e2e_test_draft",
"--content", "CREATE TEMPORARY TABLE t (id INT) WITH ('connector'='datagen'); SELECT * FROM t;",
"--folder_id", folder_id,
"--engine_version", ENGINE,
"--execution_mode", "STREAMING",
"--confirm")
record("W1", "create_draft", r)
draft_id = get_data(r, "deploymentDraftId") or "unknown"
# B5: get_draft
r = cli("get_draft", *COMMON, "--draft_id", draft_id)
record("W1", "get_draft", r)
# B6: update_draft
r = cli("update_draft", *COMMON, "--draft_id", draft_id,
"--content", "SELECT 1 AS val;",
"--confirm")
record("W1", "update_draft", r)
# B7: list_drafts
r = cli("list_drafts", *COMMON)
record("W1", "list_drafts", r)
# B8: get_draft_lock
r = cli("get_draft_lock", *COMMON, "--draft_id", draft_id)
record("W1", "get_draft_lock", r)
# B9: validate_sql
r = cli("validate_sql", *COMMON, "--statement", "SELECT 1")
record("W1", "validate_sql", r)
# B10: validate_draft (async, needs deployment_target_name)
r = cli("validate_draft", *COMMON, "--draft_id", draft_id, "--deployment_target", DEPLOY_TARGET)
record("W1", "validate_draft", r)
ticket_validate = get_data(r, "ticketId")
# B11: get_validate_result
if ticket_validate:
time.sleep(2)
r = cli("get_validate_result", *COMMON, "--ticket_id", ticket_validate)
record("W1", "get_validate_result", r)
else:
record("W1", "get_validate_result", {"success": True, "data": {}}, "skipped - no validate ticket")
# B12: deploy_draft (async)
r = cli("deploy_draft", *COMMON, "--draft_id", draft_id, "--deployment_target", DEPLOY_TARGET, "--confirm")
record("W1", "deploy_draft", r)
ticket_deploy = get_data(r, "ticketId")
# B13: get_deploy_result
if ticket_deploy:
time.sleep(2)
r = cli("get_deploy_result", *COMMON, "--ticket_id", ticket_deploy)
record("W1", "get_deploy_result", r)
else:
record("W1", "get_deploy_result", {"success": True, "data": {}}, "skipped - no deploy ticket")
# B14: delete_draft
r = cli("delete_draft", *COMMON, "--draft_id", draft_id, "--confirm")
record("W1", "delete_draft", r)
# B15: delete_folder
r = cli("delete_folder", *COMMON, "--folder_id", folder_id, "--confirm")
record("W1", "delete_folder", r)
# ═════════════════════════════════════════════════════════════════════
# Scenario C: W2 Deployment + Job lifecycle (26 commands)
# ═════════════════════════════════════════════════════════════════════
print("\n=== Scenario C: W2 Deployment + Job lifecycle ===")
# C1: list_deployments (empty)
r = cli("list_deployments", *COMMON)
record("W2", "list_deployments", r)
# C2: create_deployment -- a real streaming SQL deployment
dep_body = {
"name": "e2e-cli-test-dep",
"deploymentTarget": {"name": DEPLOY_TARGET, "mode": "PER_JOB"},
"engineVersion": ENGINE,
"executionMode": "STREAMING",
"logging": {"loggingProfile": "default"},
"artifact": {
"kind": "SQLSCRIPT",
"sqlArtifact": {
"sqlScript": "CREATE TEMPORARY TABLE src (id INT, ts TIMESTAMP(3), WATERMARK FOR ts AS ts) WITH ('connector'='datagen','rows-per-second'='1'); CREATE TEMPORARY TABLE sink (id INT, ts TIMESTAMP(3)) WITH ('connector'='blackhole'); INSERT INTO sink SELECT * FROM src;"
}
}
}
r = cli("create_deployment", *COMMON, "--body_json", jd(dep_body), "--confirm")
record("W2", "create_deployment", r)
dep_data = get_data(r)
dep_id = ""
if isinstance(dep_data, dict):
dep_id = dep_data.get("deploymentId", "") or dep_data.get("metadata", {}).get("deploymentId", "")
if not dep_id:
# Fallback: look inside result.data.data
raw = r.get("data", {})
if isinstance(raw, dict) and isinstance(raw.get("data"), dict):
dep_id = raw["data"].get("deploymentId", "")
print(f" >> deployment_id = {dep_id}")
# C3: get_deployment
r = cli("get_deployment", *COMMON, "--deployment_id", dep_id)
record("W2", "get_deployment", r)
# C4: update_deployment
upd_body = {"description": "e2e test deployment - updated"}
r = cli("update_deployment", *COMMON, "--deployment_id", dep_id, "--body_json", jd(upd_body), "--confirm")
record("W2", "update_deployment", r)
# C5: search_by_name
r = cli("search_by_name", *COMMON, "--name", "e2e-cli-test-dep")
record("W2", "search_by_name", r)
# C6: search_by_label
r = cli("search_by_label", *COMMON, "--label_key", "test", "--label_value", "true")
record("W2", "search_by_label", r, "empty result expected")
# C7: search_by_ip
r = cli("search_by_ip", *COMMON, "--ip", "10.0.0.1")
record("W2", "search_by_ip", r, "empty or error expected")
# C8: get_events
r = cli("get_events", *COMMON, "--deployment_id", dep_id)
record("W2", "get_events", r)
# C9: generate_resource_plan (async)
r = cli("generate_resource_plan", *COMMON, "--deployment_id", dep_id)
record("W2", "generate_resource_plan", r)
ticket_rp = get_data(r, "ticketId")
# C10: get_resource_plan_result
if ticket_rp:
time.sleep(3)
r = cli("get_resource_plan_result", *COMMON, "--ticket_id", ticket_rp)
record("W2", "get_resource_plan_result", r)
else:
record("W2", "get_resource_plan_result", {"success": True, "data": {}}, "skipped - no ticket")
# C11: start_job
r = cli("start_job", *COMMON, "--deployment_id", dep_id, "--restore_strategy", "NONE", "--confirm")
record("W2", "start_job", r)
job_data = get_data(r)
job_id = ""
if isinstance(job_data, dict):
job_id = job_data.get("jobId", "") or job_data.get("jobID", {}).get("jobId", "")
if not job_id:
raw = r.get("data", {})
if isinstance(raw, dict) and isinstance(raw.get("data"), dict):
job_id = raw["data"].get("jobId", "")
print(f" >> job_id = {job_id}")
# Wait for job to be RUNNING (or at least STARTING)
if job_id:
print(" >> waiting for job to reach RUNNING state...")
state = wait_for_job_state(dep_id, job_id, ["RUNNING", "FAILING", "FAILED", "CANCELLING", "FINISHED"], timeout=120)
print(f" >> job state = {state}")
# C12: get_job
if job_id:
r = cli("get_job", *COMMON, "--deployment_id", dep_id, "--job_id", job_id)
record("W2", "get_job", r)
else:
record("W2", "get_job", {"success": True, "data": {}}, "skipped - no job_id")
# C13: list_jobs
r = cli("list_jobs", *COMMON, "--deployment_id", dep_id)
record("W2", "list_jobs", r)
# C14: get_start_log
r = cli("get_start_log", *COMMON, "--deployment_id", dep_id, "--job_id", job_id or "dummy")
record("W2", "get_start_log", r)
# C15: diagnose_job
if job_id:
r = cli("diagnose_job", *COMMON, "--deployment_id", dep_id, "--job_id", job_id)
record("W2", "diagnose_job", r)
else:
record("W2", "diagnose_job", {"success": True, "data": {}}, "skipped - no job_id")
# C16: hot_update_job
if job_id:
r = cli("hot_update_job", *COMMON, "--deployment_id", dep_id, "--job_id", job_id, "--confirm")
record("W2", "hot_update_job", r)
else:
record("W2", "hot_update_job", {"success": True, "data": {}}, "skipped - no job_id")
# C17: get_hot_update_result
if job_id:
r = cli("get_hot_update_result", *COMMON, "--deployment_id", dep_id, "--job_id", job_id)
record("W2", "get_hot_update_result", r)
else:
record("W2", "get_hot_update_result", {"success": True, "data": {}}, "skipped - no job_id")
# C18: create_savepoint (need running job)
r = cli("create_savepoint", *COMMON, "--deployment_id", dep_id, "--confirm")
record("W2", "create_savepoint", r)
sp_data = get_data(r)
sp_id = ""
if isinstance(sp_data, dict):
sp_id = sp_data.get("savepointId", "")
if not sp_id:
raw = r.get("data", {})
if isinstance(raw, dict) and isinstance(raw.get("data"), dict):
sp_id = raw["data"].get("savepointId", "")
print(f" >> savepoint_id = {sp_id}")
# Wait a bit for savepoint
if sp_id:
time.sleep(5)
# C19: get_savepoint
if sp_id:
r = cli("get_savepoint", *COMMON, "--savepoint_id", sp_id)
record("W2", "get_savepoint", r)
else:
record("W2", "get_savepoint", {"success": True, "data": {}}, "skipped - no savepoint_id")
# C20: list_savepoints
r = cli("list_savepoints", *COMMON, "--deployment_id", dep_id)
record("W2", "list_savepoints", r)
# C21: get_lineage
r = cli("get_lineage", *COMMON)
record("W2", "get_lineage", r)
# C22: flink_api_proxy (use the deployment as resource)
if job_id:
r = cli("flink_api_proxy", *COMMON,
"--flink_api_path", "/overview",
"--resource_type", "jobs",
"--resource_id", dep_id)
record("W2", "flink_api_proxy", r)
else:
record("W2", "flink_api_proxy", {"success": True, "data": {}}, "skipped - no running job")
# C23: stop_job
if job_id:
r = cli("stop_job", *COMMON, "--deployment_id", dep_id, "--job_id", job_id, "--stop_strategy", "NONE", "--confirm")
record("W2", "stop_job", r)
# Wait for job to transition to CANCELLED
print(" >> waiting for job to stop...")
wait_for_job_state(dep_id, job_id, ["CANCELLED", "FAILED", "FINISHED"], timeout=60)
else:
record("W2", "stop_job", {"success": True, "data": {}}, "skipped - no job_id")
# C24: delete_savepoint
if sp_id:
r = cli("delete_savepoint", *COMMON, "--savepoint_id", sp_id, "--confirm")
record("W2", "delete_savepoint", r)
else:
record("W2", "delete_savepoint", {"success": True, "data": {}}, "skipped - no savepoint_id")
# C25: delete_job (must be in CANCELLED/FAILED/FINISHED state)
if job_id:
# Wait and retry for job to be in terminal state
for attempt in range(3):
r = cli("delete_job", *COMMON, "--deployment_id", dep_id, "--job_id", job_id, "--confirm")
if r.get("success") and not (isinstance(r.get("data", {}), dict) and r["data"].get("success") is False):
break
print(f" >> delete_job attempt {attempt+1} failed, waiting...")
time.sleep(10)
record("W2", "delete_job", r)
else:
record("W2", "delete_job", {"success": True, "data": {}}, "skipped - no job_id")
# C26: delete_deployment
for attempt in range(3):
r = cli("delete_deployment", *COMMON, "--deployment_id", dep_id, "--confirm")
if r.get("success") and not (isinstance(r.get("data", {}), dict) and r["data"].get("success") is False):
break
print(f" >> delete_deployment attempt {attempt+1} failed, waiting...")
time.sleep(10)
record("W2", "delete_deployment", r)
# ═════════════════════════════════════════════════════════════════════
# Scenario D: W3 Session Cluster (7 commands)
# ═════════════════════════════════════════════════════════════════════
print("\n=== Scenario D: W3 Session Cluster ===")
# D1: list_session_clusters
r = cli("list_session_clusters", *COMMON)
record("W3", "list_session_clusters", r)
# D2: create_session_cluster
sc_body = {
"name": "e2e-cli-test-sc",
"workspace": WS,
"engineVersion": ENGINE,
"deploymentTargetName": DEPLOY_TARGET,
"logging": {"loggingProfile": "default"},
"basicResourceSetting": {
"parallelism": 1,
"jobmanagerResourceSettingSpec": {"cpu": 1.0, "memory": "4Gi"},
"taskmanagerResourceSettingSpec": {"cpu": 1.0, "memory": "4Gi"}
}
}
r = cli("create_session_cluster", *COMMON, "--body_json", jd(sc_body), "--confirm")
record("W3", "create_session_cluster", r)
sc_data = get_data(r)
sc_name = sc_data.get("name", "e2e-cli-test-sc") if isinstance(sc_data, dict) else "e2e-cli-test-sc"
# Find the cluster_id - check the register args
# The CLI uses --cluster_id for session cluster operations
sc_id = sc_data.get("sessionClusterId", "") if isinstance(sc_data, dict) else ""
print(f" >> session_cluster name={sc_name}, id={sc_id}")
# D3: get_session_cluster
r = cli("get_session_cluster", *COMMON, "--cluster_id", sc_name)
record("W3", "get_session_cluster", r)
# D4: update_session_cluster
sc_upd = {"engineVersion": ENGINE, "deploymentTargetName": DEPLOY_TARGET}
r = cli("update_session_cluster", *COMMON, "--cluster_id", sc_name, "--body_json", jd(sc_upd), "--confirm")
record("W3", "update_session_cluster", r)
# D5: start_session_cluster
r = cli("start_session_cluster", *COMMON, "--cluster_id", sc_name, "--confirm")
record("W3", "start_session_cluster", r)
# Wait for cluster to start
print(" >> waiting for session cluster to start...")
time.sleep(15)
# D6: stop_session_cluster
r = cli("stop_session_cluster", *COMMON, "--cluster_id", sc_name, "--confirm")
record("W3", "stop_session_cluster", r)
# Wait for cluster to stop
time.sleep(10)
# D7: delete_session_cluster
r = cli("delete_session_cluster", *COMMON, "--cluster_id", sc_name, "--confirm")
record("W3", "delete_session_cluster", r)
# ═════════════════════════════════════════════════════════════════════
# Scenario E: W4 Dev Resources (14 commands)
# ═════════════════════════════════════════════════════════════════════
print("\n=== Scenario E: W4 Dev Resources ===")
# E1: list_engine_versions
r = cli("list_engine_versions", *COMMON)
record("W4", "list_engine_versions", r)
# E2: get_catalogs
r = cli("get_catalogs", *COMMON)
record("W4", "get_catalogs", r)
# E3: get_databases
r = cli("get_databases", *COMMON, "--catalog_name", "vvp")
record("W4", "get_databases", r)
# E4: get_tables
r = cli("get_tables", *COMMON, "--catalog_name", "vvp", "--database_name", "default")
record("W4", "get_tables", r)
# E5: execute_sql
r = cli("execute_sql", *COMMON, "--statement", "SHOW CATALOGS", "--confirm")
record("W4", "execute_sql", r)
# E6: get_udf_artifacts
r = cli("get_udf_artifacts", *COMMON)
record("W4", "get_udf_artifacts", r)
# E7: create_udf_artifact (needs a real JAR url, will timeout or get API error)
udf_body = {"jarUrl": "https://dummy.oss-cn-beijing.aliyuncs.com/test.jar", "name": "e2e-test-udf"}
r = cli("create_udf_artifact", *COMMON, "--body_json", jd(udf_body), "--confirm", timeout=30)
record("W4", "create_udf_artifact", r, "expected: timeout/error - no real jar")
# E8: update_udf_artifact (will fail - no artifact exists)
r = cli("update_udf_artifact", *COMMON, "--udf_artifact_name", "e2e-test-udf", "--body_json", jd(udf_body), "--confirm")
record("W4", "update_udf_artifact", r, "expect API error - artifact not found")
# E9: delete_udf_artifact
r = cli("delete_udf_artifact", *COMMON, "--udf_artifact_name", "e2e-test-udf", "--confirm")
record("W4", "delete_udf_artifact", r, "expect API error - artifact not found")
# E10: register_udf_function
udf_fn = {"className": "com.test.E2eUdf", "functionName": "e2e_udf", "udfArtifactName": "e2e-test-udf"}
r = cli("register_udf_function", *COMMON, "--body_json", jd(udf_fn), "--confirm")
record("W4", "register_udf_function", r, "expect API error - artifact not found")
# E11: delete_udf_function
r = cli("delete_udf_function", *COMMON, "--function_name", "e2e_udf", "--class_name", "com.test.E2eUdf", "--udf_artifact_name", "e2e-test-udf", "--confirm")
record("W4", "delete_udf_function", r, "expect API error - function not found")
# E12: list_connectors
r = cli("list_connectors", *COMMON)
record("W4", "list_connectors", r)
# E13: register_connector
r = cli("register_connector", *COMMON, "--body_json", '{"jarUrl":"https://dummy.oss-cn-beijing.aliyuncs.com/conn.jar"}', "--confirm", timeout=30)
record("W4", "register_connector", r, "expected: timeout/error - no real jar")
# E14: delete_connector
r = cli("delete_connector", *COMMON, "--connector_name", "e2e-test-connector", "--confirm")
record("W4", "delete_connector", r)
# ═════════════════════════════════════════════════════════════════════
# Summary
# ═════════════════════════════════════════════════════════════════════
print("\n" + "=" * 80)
print("FINAL RESULTS")
print("=" * 80)
pass_count = sum(1 for _, _, s, _ in results if s == "PASS")
api_err_count = sum(1 for _, _, s, _ in results if s == "API_ERR")
fail_count = sum(1 for _, _, s, _ in results if s == "FAIL")
crash_count = sum(1 for _, _, s, _ in results if s == "CRASH")
timeout_count = sum(1 for _, _, s, _ in results if s == "TIMEOUT")
total = len(results)
print(f"\n{'Scenario':<6} {'Command':<30} {'Status':<10} Notes")
print("-" * 80)
for scenario, cmd, status, notes in results:
print(f"{scenario:<6} {cmd:<30} {status:<10} {notes[:40]}")
print("-" * 80)
print(f"Total: {total} | PASS: {pass_count} | API_ERR: {api_err_count} | FAIL: {fail_count} | TIMEOUT: {timeout_count} | CRASH: {crash_count}")
print(f"\nPASS = SDK call succeeded and API returned success")
print(f"API_ERR = SDK call succeeded but API returned business error (e.g. resource not found)")
print(f"FAIL = SDK call returned error envelope (connectivity/auth)")
print(f"TIMEOUT = CLI process timed out (expected for fake JAR URLs)")
print(f"CRASH = Python TypeError/Traceback (BUG!)")
if crash_count > 0:
print("\n!!! CRASHES DETECTED - BUGS TO FIX !!!")
for scenario, cmd, status, notes in results:
if status == "CRASH":
print(f" {scenario}/{cmd}: {notes}")
sys.exit(1 if crash_count > 0 else 0)
#!/usr/bin/env python3
"""
Flink Ververica CLI – manage Flink jobs lifecycle (API 2022-07-18).
Single entry-point that registers subcommands from 5 workflow modules.
"""
import argparse
import os
import sys
# Ensure scripts/ is on sys.path so workflow modules can import client.py
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
def main():
parser = argparse.ArgumentParser(
prog="flink-ververica",
description="Flink Ververica CLI – manage Flink jobs lifecycle (API 2022-07-18)",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Workflows:
SQL Development (W1) create_folder, create_draft, validate_sql, deploy_draft ...
Job Operations (W2) list_deployments, start_job, stop_job, diagnose_job ...
Session Cluster (W3) create_session_cluster, start_session_cluster ...
Dev Resources (W4) get_catalogs, get_databases, get_tables, execute_sql ...
Workspace Admin (W5) create_member, create_variable, create_deploy_target ...
Authentication:
Uses Alibaba Cloud default credential chain (RAM role, CLI profile, etc.)
Examples:
# List all deployments
flink-ververica list_deployments -w w-xxx -n default -r cn-beijing
# Start a job (interactive confirmation)
flink-ververica start_job -w w-xxx -n default -r cn-beijing --deployment_id d-xxx
# Start a job (non-interactive, for scripts/agents)
flink-ververica start_job -w w-xxx -n default -r cn-beijing --deployment_id d-xxx --confirm
# Table output
flink-ververica list_deployments -w w-xxx -n default -r cn-beijing -o table
""",
)
subparsers = parser.add_subparsers(dest="subcommand", help="Available commands")
# Register all workflow modules
import w1_sql_development
import w2_deployment_ops
import w3_session_cluster
import w4_dev_resources
import w5_workspace_admin
w1_sql_development.register(subparsers)
w2_deployment_ops.register(subparsers)
w3_session_cluster.register(subparsers)
w4_dev_resources.register(subparsers)
w5_workspace_admin.register(subparsers)
args = parser.parse_args()
if not args.subcommand:
parser.print_help()
sys.exit(0)
if hasattr(args, "func"):
args.func(args)
else:
parser.print_help()
sys.exit(1)
if __name__ == "__main__":
main()
#!/usr/bin/env python3
"""W3: Session Cluster Management – create, update, delete, start, stop."""
from client import (
add_common_args,
call_api,
get_client,
output,
require_args,
require_confirmation,
runtime_options,
)
def cmd_create_session_cluster(args):
err = require_args(args, "workspace", "namespace", "region_id")
if err:
return output(err, args.output)
chk = require_confirmation(
"create_session_cluster",
"Creating a Session cluster consumes compute resources.",
args.confirm,
)
if chk:
return output(chk, args.output)
from alibabacloud_ververica20220718.models import (
CreateSessionClusterHeaders,
CreateSessionClusterRequest,
SessionCluster,
)
import json as _json
client = get_client(args.region_id)
body = SessionCluster()
if args.name:
body.name = args.name
if args.body_json:
body = SessionCluster().from_map(_json.loads(args.body_json))
headers = CreateSessionClusterHeaders(workspace=args.workspace)
req = CreateSessionClusterRequest(body=body)
result = call_api(
"create_session_cluster",
client.create_session_cluster_with_options,
args.namespace, req, headers, runtime_options(),
)
output(result, args.output)
def cmd_update_session_cluster(args):
err = require_args(args, "workspace", "namespace", "region_id", "cluster_id")
if err:
return output(err, args.output)
chk = require_confirmation(
"update_session_cluster",
f"Update Session cluster '{args.cluster_id}'.",
args.confirm,
)
if chk:
return output(chk, args.output)
from alibabacloud_ververica20220718.models import (
UpdateSessionClusterHeaders,
UpdateSessionClusterRequest,
SessionCluster,
)
import json as _json
client = get_client(args.region_id)
body = SessionCluster()
if args.body_json:
body = SessionCluster().from_map(_json.loads(args.body_json))
headers = UpdateSessionClusterHeaders(workspace=args.workspace)
req = UpdateSessionClusterRequest(body=body)
result = call_api(
"update_session_cluster",
client.update_session_cluster_with_options,
args.namespace, args.cluster_id, req, headers, runtime_options(),
)
output(result, args.output)
def cmd_delete_session_cluster(args):
err = require_args(args, "workspace", "namespace", "region_id", "cluster_id")
if err:
return output(err, args.output)
chk = require_confirmation(
"delete_session_cluster",
f"DELETE Session cluster '{args.cluster_id}' (IRREVERSIBLE). All jobs on this cluster will be affected.",
args.confirm,
)
if chk:
return output(chk, args.output)
from alibabacloud_ververica20220718.models import DeleteSessionClusterHeaders
client = get_client(args.region_id)
headers = DeleteSessionClusterHeaders(workspace=args.workspace)
result = call_api(
"delete_session_cluster",
client.delete_session_cluster_with_options,
args.namespace, args.cluster_id, headers, runtime_options(),
)
output(result, args.output)
def cmd_get_session_cluster(args):
err = require_args(args, "workspace", "namespace", "region_id", "cluster_id")
if err:
return output(err, args.output)
from alibabacloud_ververica20220718.models import GetSessionClusterHeaders
client = get_client(args.region_id)
headers = GetSessionClusterHeaders(workspace=args.workspace)
result = call_api(
"get_session_cluster",
client.get_session_cluster_with_options,
args.namespace, args.cluster_id, headers, runtime_options(),
)
output(result, args.output)
def cmd_list_session_clusters(args):
err = require_args(args, "workspace", "namespace", "region_id")
if err:
return output(err, args.output)
from alibabacloud_ververica20220718.models import ListSessionClustersHeaders
client = get_client(args.region_id)
headers = ListSessionClustersHeaders(workspace=args.workspace)
result = call_api(
"list_session_clusters",
client.list_session_clusters_with_options,
args.namespace, headers, runtime_options(),
)
output(result, args.output)
def cmd_start_session_cluster(args):
err = require_args(args, "workspace", "namespace", "region_id", "cluster_id")
if err:
return output(err, args.output)
chk = require_confirmation(
"start_session_cluster",
f"Start Session cluster '{args.cluster_id}'. This consumes compute resources.",
args.confirm,
)
if chk:
return output(chk, args.output)
from alibabacloud_ververica20220718.models import StartSessionClusterHeaders
client = get_client(args.region_id)
headers = StartSessionClusterHeaders(workspace=args.workspace)
result = call_api(
"start_session_cluster",
client.start_session_cluster_with_options,
args.namespace, args.cluster_id, headers, runtime_options(),
)
output(result, args.output)
def cmd_stop_session_cluster(args):
err = require_args(args, "workspace", "namespace", "region_id", "cluster_id")
if err:
return output(err, args.output)
chk = require_confirmation(
"stop_session_cluster",
f"Stop Session cluster '{args.cluster_id}'. Jobs running on this cluster will be affected.",
args.confirm,
)
if chk:
return output(chk, args.output)
from alibabacloud_ververica20220718.models import StopSessionClusterHeaders
client = get_client(args.region_id)
headers = StopSessionClusterHeaders(workspace=args.workspace)
result = call_api(
"stop_session_cluster",
client.stop_session_cluster_with_options,
args.namespace, args.cluster_id, headers, runtime_options(),
)
output(result, args.output)
# ═══════════════════════════════════════════════════════════════════════════
# Subparser registration
# ═══════════════════════════════════════════════════════════════════════════
def register(subparsers):
"""Register all W3 subcommands."""
def _add(name, help_text, func, extra_args=None):
p = subparsers.add_parser(name, help=help_text)
add_common_args(p)
if extra_args:
extra_args(p)
p.set_defaults(func=func, subcommand=name)
def _create_args(p):
p.add_argument("--name", help="Session cluster name")
p.add_argument("--body_json", help="Full body as JSON (advanced)")
p.add_argument("--confirm", action="store_true", help="Skip interactive confirmation")
def _id(p):
p.add_argument("--cluster_id", required=True, help="Session cluster ID")
def _id_confirm(p):
p.add_argument("--cluster_id", required=True, help="Session cluster ID")
p.add_argument("--confirm", action="store_true", help="Skip interactive confirmation")
def _id_force(p):
p.add_argument("--cluster_id", required=True, help="Session cluster ID")
p.add_argument("--confirm", action="store_true", help="Skip interactive confirmation")
def _update_args(p):
p.add_argument("--cluster_id", required=True, help="Session cluster ID")
p.add_argument("--body_json", help="Updated body as JSON (advanced)")
p.add_argument("--confirm", action="store_true", help="Skip interactive confirmation")
_add("create_session_cluster", "Create a Session cluster", cmd_create_session_cluster, _create_args)
_add("update_session_cluster", "Update Session cluster configuration", cmd_update_session_cluster, _update_args)
_add("delete_session_cluster", "Delete a Session cluster (irreversible)", cmd_delete_session_cluster, _id_force)
_add("get_session_cluster", "Get Session cluster details", cmd_get_session_cluster, _id)
_add("list_session_clusters", "List all Session clusters", cmd_list_session_clusters, None)
_add("start_session_cluster", "Start a Session cluster", cmd_start_session_cluster, _id_confirm)
_add("stop_session_cluster", "Stop a Session cluster", cmd_stop_session_cluster, _id_confirm)