Alibabacloud Dataworks Infra Manage

DataWorks Infrastructure Management

Unified management of Data Sources, Compute Resources, and Resource Groups in Alibaba Cloud DataWorks workspaces, supporting create and query operations.

Architecture

DataWorks
├── Workspaces ─── Query and search workspaces
│   ├── Data Sources ─── 50 types: MySQL, Hologres, MaxCompute, ...
│   └── Compute Resources ─── Hologres, MaxCompute, Flink, Spark
└── Resource Groups ─── Serverless resource group management (cross-workspace)

Dependencies:
  Workspace ◀── Data Sources, Compute Resources (must belong to a workspace)
  Workspace ◀── Resource Groups (associated via binding; one resource group can bind to multiple workspaces)
  Connectivity Test ──depends on──▶ Resource Group (must be bound to the workspace of the data source)
  Standard Mode ──requires──▶ Dev (Development) + Prod (Production) dual data sources and compute resources

---

Global Rules

Prerequisites

1. Aliyun CLI >= 3.3.1: aliyun version (Installation guide: references/cli-installation-guide.md) 2. First-time use: aliyun configure set --auto-plugin-install true 3. jq (required for resource group operations): which jq 4. Credential status: aliyun configure list, verify valid credentials exist 5. DataWorks edition: Basic edition or above required

Security Rules: DO NOT read/print/echo AK/SK values, DO NOT let users input AK/SK directly, ONLY use aliyun configure list to check credential status.

Command Formatting

• User-Agent (mandatory): All aliyun CLI commands must include the --user-agent AlibabaCloud-Agent-Skills parameter to identify the source.
• Single-line commands: When executing Bash commands, must construct as a single-line string; do not use \ for line breaks.
• jq step-by-step execution: First execute the aliyun command to get JSON, then format with jq (to avoid multi-line security prompts).
• Endpoint mandatory: When specifying the --region parameter, you must also add --endpoint dataworks.<REGION_ID>.aliyuncs.com. Not needed when --region is not specified.

Parameter Confirmation

Before executing any command, all user-customizable parameters must be confirmed by the user. Do not assume or use default values.

Exception: When the user has explicitly specified parameter values in the conversation, use them directly without re-confirmation.

Resource group related parameters (mandatory user selection): VPC, VSwitch, Resource Group ID (for binding/connectivity testing) — involve networking and billing, DO NOT auto-select; must display a list for the user to explicitly choose. Confirm even if there is only one option.

⚠️ Write API Execution Gate — MUST Check Before Every Write Operation

MANDATORY: Before calling any Write API (Create / Update / Delete / Bind / Unbind / Associate / Dissociate / Test), you MUST perform the following checks in order:

>

1. Scan the entire SKILL.md for a Security Restriction or Disabled Operations notice that mentions the target API or module.

2. If a restriction exists: BLOCK the operation immediately. Do NOT call the API. Respond to the user with:

- What operation is blocked and why

- The recommended alternative (e.g., use the DataWorks console, contact administrator)

3. If no restriction exists: Proceed normally with parameter confirmation and execution.

>

This check is NOT optional. It applies to every single write operation without exception. Never skip this step.

>

Quick Reference — Blocked APIs in this skill:

| Module | Blocked APIs | Reason |

|--------|-------------|--------|

| Data Sources (Module 1) | UpdateDataSource, DeleteDataSource | Prevent accidental data loss, credential exposure, disruption of running tasks |

| Compute Resources (Module 2) | UpdateComputeResource, DeleteComputeResource | Prevent disruption of running development and scheduling tasks |

>

Allowed Write APIs: CreateDataSource, CreateComputeResource, CreateResourceGroup, AssociateProjectToResourceGroup, DissociateProjectFromResourceGroup, TestDataSourceConnectivity

RAM Permissions

All operations require dataworks:<APIAction> permissions. Creating resource groups additionally requires AliyunBSSOrderAccess and vpc:DescribeVpcs, vpc:DescribeVSwitches.

Full permission matrix: references/ram-policies.md

---

Quick Start: New Workspace Infrastructure Initialization

When the user is unsure about specific operations or has vague requirements, guide them through the following process:

1. Environment check — Check CLI and credentials per Prerequisites 2. Confirm workspace — Use ListProjects to locate the workspace, GetProject to confirm the mode (Simple/Standard) 3. Create compute resources — Guide engine type selection; the system will automatically create corresponding data sources. Standard Mode requires Dev+Prod pairs. Only pure storage-type data sources (MySQL, Kafka, etc.) need separate data source creation 4. Create/bind resource groups — Query existing resource groups → let user select → bind. Guide creation when no resource groups are available 5. Test connectivity — Test with bound resource groups; when all pass, inform "Infrastructure configuration complete"

After each step, proactively suggest the next action.

---

Next Step Guidance

After each write operation is completed and verified, proactively suggest follow-up actions:

---

Trigger Rules

Trigger scenarios: Data source create/query, compute resource create/query, resource group management, infrastructure initialization, colloquial aliases (DW database connection failure, configure holo/mc resources, create rg)

Not triggered: Data development tasks, scheduling configuration, MaxCompute table management, data integration tasks, ECS/RDS/OSS, workspace member management, data quality/lineage/preview. Standalone workspace queries are handled by the alibabacloud-dataworks-workspace-manage skill.

Interaction Flow

All operations follow: Identify module → Environment check → Collect parameters → Execute command → Verify result → Guide next step

Common aliases: DW=DataWorks, holo=Hologres, mc/MC/odps=MaxCompute, pg=PostgreSQL, rg=Resource Group, ds=Data Source, RDS=InstanceMode MySQL/PG/SQLServer, ADB=AnalyticDB

Naming suggestions: Data source {type}_{business}_{purpose}, Compute resource {type}_{business}, Resource group dw_{purpose}_rg_{env}

---

Module 0: Workspace Query

If the alibabacloud-dataworks-workspace-manage skill is available, prefer using it for workspace queries. The following is only a fallback.

aliyun dataworks-public ListProjects --user-agent AlibabaCloud-Agent-Skills --Status Available --PageSize 100

When searching by name, first get the full list then filter .PagingInfo.Projects[] by Name/DisplayName using jq.

---

Module 1: Data Source Management

Supports 50 data source types. See references/data-sources/README.md for details.

When do you need to create a data source separately? Creating a compute resource (Module 2) will automatically create the corresponding data source. Only pure storage-type databases (MySQL, PostgreSQL, Kafka, MongoDB, etc.) need separate creation.

Note: The following types do not currently support OpenAPI: hdfs

Connection modes: UrlMode (self-hosted databases, requires host/port) or InstanceMode (Alibaba Cloud managed instances, requires instanceId). When unsure, proactively ask the user. InstanceMode is preferred.

Instance query APIs: references/data-sources/instance-apis.md

⚠️ Security Restriction — See Write API Execution Gate (Global Rules) for mandatory pre-check

IMPORTANT: The DeleteDataSource and UpdateDataSource APIs are supported by the DataWorks service, but this skill has disabled modifying or deleting data sources for security reasons. Before attempting any write operation, the agent MUST check the Write API Execution Gate section.

If you need to modify or delete a data source, please use the DataWorks console directly or contact your administrator.

Connection Mode Quick Reference

ConnectionPropertiesMode selection determines required fields. InstanceMode is preferred when both are available.

hdfs — not supported via OpenAPI.

Full details: references/data-sources/README.md

Workspace Mode

Environment note: Prod (Production) is for production data processing; Dev (Development) is for development and debugging, physically isolated from production.

aliyun dataworks-public GetProject --user-agent AlibabaCloud-Agent-Skills --Id <PROJECT_ID> — check DevEnvironmentEnabled:

• false → Simple Mode (1 data source, envType=Prod)
• true → Standard Mode (2 data sources, Dev + Prod, physically isolated)

Full mode comparison: references/data-sources/README.md

Task 1.1: Create Data Source (CreateDataSource)

aliyun dataworks-public CreateDataSource --user-agent AlibabaCloud-Agent-Skills [--region <REGION_ID> --endpoint dataworks.<REGION_ID>.aliyuncs.com] --ProjectId <PROJECT_ID> --Name <NAME> --Type <TYPE> --ConnectionPropertiesMode <UrlMode|InstanceMode> --ConnectionProperties '<JSON>' --Description "<DESC>"

ConnectionProperties common structure:

• UrlMode: {"envType":"Prod","address":[{"host":"<IP>","port":<PORT>}],"database":"<DB>","username":"<USER>","password":"<PWD>"}
• InstanceMode: {"envType":"Prod","instanceId":"<ID>","regionId":"<REGION>","database":"<DB>","username":"<USER>","password":"<PWD>"}

Special type structures (Oracle, MaxCompute, HBase, etc.): see references/data-sources/ per-type docs

Cross-account data source configuration: references/cross-account-datasources.md

Task 1.2: Get Data Source (GetDataSource)

aliyun dataworks-public GetDataSource --user-agent AlibabaCloud-Agent-Skills --Id <DATASOURCE_ID> [--region <REGION_ID> --endpoint dataworks.<REGION_ID>.aliyuncs.com]

Task 1.3: List Data Sources (ListDataSources)

aliyun dataworks-public ListDataSources --user-agent AlibabaCloud-Agent-Skills --ProjectId <PROJECT_ID> [--Types '["mysql"]'] [--EnvType <Dev|Prod>] [--PageNumber 1] [--PageSize 20]

Returns nested structure DataSources[].DataSource[]; Name/Type are in the outer layer, Id/Description in the inner layer.

Task 1.4: Test Connectivity (TestDataSourceConnectivity)

Process: Query resource group list → Let user select a resource group → Execute test.

# Step 1: Query project resource groups
aliyun dataworks-public ListResourceGroups --user-agent AlibabaCloud-Agent-Skills --ProjectId <PROJECT_ID>

# Step 2: Execute test after user selects a resource group
aliyun dataworks-public TestDataSourceConnectivity --user-agent AlibabaCloud-Agent-Skills --DataSourceId <ID> --ProjectId <PROJECT_ID> --ResourceGroupId "<RG_ID>"

If error "resourceGroupId is not in the project", the resource group needs to be bound first (confirm with user, then execute AssociateProjectToResourceGroup).

---

Module 2: Compute Resource Management

Supports Hologres, MaxCompute, Flink, Spark, and other types. The system will automatically create corresponding data sources upon creation.

⚠️ Security Restriction — See Write API Execution Gate (Global Rules) for mandatory pre-check

IMPORTANT: For security reasons, this skill does NOT support modifying or deleting compute resources. Before attempting any write operation, the agent MUST check the Write API Execution Gate section. These operations are disabled to prevent:

- Accidental data loss or service interruption

- Disruption of running data development and scheduling tasks

- Unintended changes to production compute resource configurations

>

If you need to modify or delete a compute resource, please use the DataWorks console directly or contact your administrator.

authType Rules

• Dev environment: authType is fixed as Executor
• Prod environment: Options are PrimaryAccount (recommended), TaskOwner, SubAccount, RamRole. Default recommendation is PrimaryAccount unless user has special requirements

authType details and guidance: references/compute-resources/README.md

Type-Specific Notes

• Hologres: Only supports InstanceMode, requires instanceId, securityProtocol
• MaxCompute: Only supports UrlMode, requires project, endpointMode

Full ConnectionProperties examples: references/compute-resources/README.md

Task 2.1: Create Compute Resource (CreateComputeResource)

aliyun dataworks-public CreateComputeResource --user-agent AlibabaCloud-Agent-Skills [--region <REGION_ID> --endpoint dataworks.<REGION_ID>.aliyuncs.com] --ProjectId <PROJECT_ID> --Name <NAME> --Type <TYPE> --ConnectionPropertiesMode <InstanceMode|UrlMode> --ConnectionProperties '<JSON>' [--Description "<DESC>"]

After creation, use ListDataSources to verify the corresponding data source was auto-generated.

Task 2.2: Get Compute Resource (GetComputeResource)

aliyun dataworks-public GetComputeResource --user-agent AlibabaCloud-Agent-Skills --Id <ID> --ProjectId <PROJECT_ID>

Task 2.3: List Compute Resources (ListComputeResources)

aliyun dataworks-public ListComputeResources --user-agent AlibabaCloud-Agent-Skills --ProjectId <PROJECT_ID> [--Name <FILTER>] [--EnvType <Dev|Prod>] [--PageSize 20] [--SortBy CreateTime] [--Order Desc]

Returns nested structure ComputeResources[].ComputeResource[]; Name/Type are in the outer layer, Id in the inner layer.

---

Module 3: Resource Group Management

Manages the full lifecycle of Serverless resource groups.

Task 3.1: Create Resource Group (CreateResourceGroup)

Requires AliyunBSSOrderAccess permission.

Interaction flow (let user choose at each step, DO NOT auto-select):

1. Query and select VPC:

aliyun vpc DescribeVpcs --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION_ID>" --PageSize 50

If the list is empty, guide the user to create a VPC; DO NOT auto-create.

2. Query and select VSwitch:

aliyun vpc DescribeVSwitches --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION_ID>" --VpcId "<VPC_ID>" --PageSize 50

3. Confirm name and specification → Execute creation:

aliyun dataworks-public CreateResourceGroup --user-agent AlibabaCloud-Agent-Skills [--region <REGION_ID> --endpoint dataworks.<REGION_ID>.aliyuncs.com] --Name "<NAME>" --PaymentType PostPaid --VpcId "<VPC_ID>" --VswitchId "<VSWITCH_ID>" --ClientToken "$(uuidgen 2>/dev/null || echo "token-$(date +%s)")" --Remark "Created by Agent"

After creation, poll GetResourceGroup until status becomes Normal (every 10 seconds, up to 10 minutes).

Task 3.2: Get Resource Group (GetResourceGroup)

aliyun dataworks-public GetResourceGroup --user-agent AlibabaCloud-Agent-Skills --Id "<ID>"

Task 3.3: List Resource Groups (ListResourceGroups)

aliyun dataworks-public ListResourceGroups --user-agent AlibabaCloud-Agent-Skills [--ProjectId <PROJECT_ID>] [--Statuses '["Normal"]'] --PageSize 100

Task 3.4: Bind Resource Group (AssociateProjectToResourceGroup)

Process: Query available resource groups → Display list for user to select → Bind after user confirms.

aliyun dataworks-public AssociateProjectToResourceGroup --user-agent AlibabaCloud-Agent-Skills --ResourceGroupId "<RG_ID>" --ProjectId "<PROJECT_ID>"

Task 3.5: Query Binding Relationships

aliyun dataworks-public ListResourceGroupAssociateProjects --user-agent AlibabaCloud-Agent-Skills --ResourceGroupId "<RG_ID>"

Task 3.6: Unbind Resource Group (DissociateProjectFromResourceGroup)

aliyun dataworks-public DissociateProjectFromResourceGroup --user-agent AlibabaCloud-Agent-Skills --ResourceGroupId "<RG_ID>" --ProjectId "<PROJECT_ID>"

---

Success Verification

After all write operations, use the corresponding Get/List command to verify the result.

Common Errors

Region

Common: cn-hangzhou, cn-shanghai, cn-beijing, cn-shenzhen. Endpoint: dataworks.<region-id>.aliyuncs.com

Full list: references/related-apis.md

Best Practices

1. Query before action — Confirm current state before create operations 2. Manage by environment — Manage Dev and Prod resources separately 3. Verify operations — Use Get/List to verify after each write operation 4. Proactive guidance — Suggest the next step after each step completes 5. Protect data sources and compute resources — Never modify or delete data sources or compute resources via this skill; use the DataWorks console for such operations

Reference Links

Aliyun CLI Installation & Configuration Guide

Aliyun CLI 3.3.1+ required for full plugin ecosystem support.

Installation

macOS (Homebrew)

brew install aliyun-cli
brew upgrade aliyun-cli
aliyun version  # verify >= 3.3.1

macOS (Binary)

wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz
tar -xzf aliyun-cli-macosx-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
aliyun version

Linux

wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
aliyun version

Windows (PowerShell)

Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip"
Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli
$env:Path += ";C:\aliyun-cli"
aliyun version

Configuration

# Interactive configuration
aliyun configure

# Or set via environment variables (see official docs for details)
# Note: AK/SK should be configured through `aliyun configure` or credential files,
# not echoed/printed in chat. See SKILL.md security rules.
export ALIBABA_CLOUD_REGION_ID=cn-hangzhou

Enable Auto Plugin Install

aliyun configure set --auto-plugin-install true

Verification

aliyun ecs describe-regions --user-agent AlibabaCloud-Agent-Skills   # test authentication
aliyun configure list         # show current configuration

References

• Official Documentation: https://help.aliyun.com/zh/cli/
• RAM Console: https://ram.console.aliyun.com/
• Access Key Management: https://ram.console.aliyun.com/manage/ak

Compute Resource ConnectionProperties Reference

Workspace Mode and Compute Resource Configuration

Important: Workspace mode (Simple Mode / Standard Mode) affects the quantity and authType configuration of compute resources.

Query Workspace Mode

aliyun dataworks-public GetProject --user-agent AlibabaCloud-Agent-Skills --Id <PROJECT_ID> 2>/dev/null | jq '.Project | {Id, Name, DevEnvironmentEnabled}'

• DevEnvironmentEnabled: false → Simple Mode
• DevEnvironmentEnabled: true → Standard Mode

Simple Mode vs Standard Mode

Compute Resource Creation Strategy

Simple Mode Workspace

Create 1 compute resource with envType: Prod:

{
  "envType": "Prod",
  "authType": "PrimaryAccount",
  ...
}

Standard Mode Workspace

Create 2 compute resources. Different environments use different authType:

Production environment:

{
  "envType": "Prod",
  "authType": "PrimaryAccount",
  ...
}

Development environment:

{
  "envType": "Dev",
  "authType": "Executor",
  ...
}

---

authType Rules (by environment)

• Dev environment (envType: "Dev"): authType is fixed as `Executor`
• Prod environment (envType: "Prod"): authType supports the following values:

---

Hologres (InstanceMode only)

Prod environment example:

{
  "envType": "Prod",
  "regionId": "cn-hangzhou",
  "instanceId": "hgprecn-cn-xxxxx",
  "database": "mydb",
  "securityProtocol": "authTypeNone",
  "authType": "PrimaryAccount"
}

Dev environment example:

{
  "envType": "Dev",
  "regionId": "cn-hangzhou",
  "instanceId": "hgprecn-cn-xxxxx",
  "database": "mydb",
  "securityProtocol": "authTypeNone",
  "authType": "Executor"
}

MaxCompute (UrlMode only)

Prod environment example:

{
  "envType": "Prod",
  "regionId": "cn-hangzhou",
  "project": "my_maxcompute_project",
  "authType": "PrimaryAccount",
  "endpointMode": "SelfAdaption"
}

Dev environment example:

{
  "envType": "Dev",
  "regionId": "cn-hangzhou",
  "project": "my_maxcompute_project_dev",
  "authType": "Executor",
  "endpointMode": "SelfAdaption"
}

Flink (InstanceMode)

{
  "envType": "Prod",
  "regionId": "cn-hangzhou",
  "instanceId": "flink-xxxxx",
  "workspaceName": "my_flink_workspace",
  "authType": "PrimaryAccount"
}

Spark (InstanceMode)

{
  "envType": "Prod",
  "regionId": "cn-hangzhou",
  "clusterId": "spark-xxxxx",
  "authType": "PrimaryAccount"
}

---

Important Notes

• All compute resource types use camelCase for ConnectionProperties fields
• Dev environment authType is fixed as Executor; Prod environment supports PrimaryAccount/TaskOwner/SubAccount/RamRole
• Hologres compute resources support InstanceMode only
• MaxCompute compute resources support UrlMode only

Cross-Account Data Source Configuration Guide

Use cross-account mode when you need to access resources under other Alibaba Cloud accounts.

---

Cross-Account Configuration Process

Step 1: Resource Account (Target Account) Preparation

In the account where the resource resides (e.g., <TARGET_ACCOUNT_ID>):

1. Create a RAM role (e.g., <CROSS_ACCOUNT_ROLE_NAME>) 2. Configure trust policy to allow the source account (e.g., <SOURCE_ACCOUNT_ID>) to assume 3. Grant the role permissions to access target resources (e.g., MaxCompute project access)

Trust Policy Example:

{
  "Statement": [{
    "Action": "sts:AssumeRole",
    "Effect": "Allow",
    "Principal": {
      "RAM": ["acs:ram::<SOURCE_ACCOUNT_ID>:root"],
      "Service": ["<SOURCE_ACCOUNT_ID>@engine.dataworks.aliyuncs.com"]
    }
  }],
  "Version": "1"
}

Step 2: Create Data Source in the DataWorks Account (Source Account)

Create a data source in the DataWorks account and configure cross-account parameters:

• crossAccountOwnerId: Alibaba Cloud UID of the resource account
• crossAccountRoleName: RAM role name in the resource account
• authType: RamRole (required for MaxCompute/Hologres cross-account scenarios)

---

Cross-Account Data Source Type Quick Reference

Note: Only MaxCompute and Hologres require explicitly setting authType: RamRole for cross-account. Other types do not need to set authType.

---

Cross-Account Configuration Examples

MaxCompute Cross-Account

{
  "project": "target_mc_project",
  "regionId": "cn-zhangjiakou",
  "endpointMode": "SelfAdaption",
  "authType": "RamRole",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

Hologres Cross-Account

{
  "instanceId": "hgpostcn-cn-xxxxx",
  "regionId": "cn-zhangjiakou",
  "database": "mydb",
  "authType": "RamRole",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

MySQL Cross-Account

{
  "instanceId": "rm-xxxxx",
  "regionId": "cn-zhangjiakou",
  "database": "mydb",
  "username": "myuser",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

PolarDB Cross-Account

{
  "clusterId": "pc-xxxxx",
  "regionId": "cn-zhangjiakou",
  "database": "mydb",
  "dbType": "mysql",
  "username": "myuser",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

PostgreSQL Cross-Account

{
  "instanceId": "pgm-xxxxx",
  "regionId": "cn-zhangjiakou",
  "database": "postgres",
  "username": "myuser",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

SQLServer Cross-Account

{
  "instanceId": "rm-xxxxx",
  "regionId": "cn-zhangjiakou",
  "database": "mydb",
  "username": "myuser",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

AnalyticDB MySQL Cross-Account

{
  "instanceId": "am-xxxxx",
  "regionId": "cn-zhangjiakou",
  "database": "mydb",
  "username": "myuser",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

AnalyticDB PostgreSQL Cross-Account

{
  "instanceId": "gp-xxxxx",
  "regionId": "cn-zhangjiakou",
  "database": "postgres",
  "username": "myuser",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

StarRocks Cross-Account

{
  "instanceId": "c-xxxxx",
  "instanceType": "serverless",
  "regionId": "cn-zhangjiakou",
  "database": "mydb",
  "username": "sr_user",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<TARGET_ACCOUNT_ID>",
  "crossAccountRoleName": "<CROSS_ACCOUNT_ROLE_NAME>",
  "envType": "Prod"
}

---

CLI Command Examples

Complete command for creating a cross-account data source:

aliyun dataworks-public CreateDataSource --user-agent AlibabaCloud-Agent-Skills \
  --region cn-zhangjiakou \
  --ProjectId 12345 \
  --Name cross_account_mysql \
  --Type mysql \
  --ConnectionPropertiesMode InstanceMode \
  --ConnectionProperties '{"envType":"Prod","instanceId":"rm-xxxxx","regionId":"cn-zhangjiakou","database":"mydb","username":"myuser","password":"<PASSWORD>","crossAccountOwnerId":"<TARGET_ACCOUNT_ID>","crossAccountRoleName":"<CROSS_ACCOUNT_ROLE_NAME>"}' \
  --Description "Cross-account MySQL datasource"

---

Detailed Configuration Documentation

For detailed cross-account configuration of each data source type, please refer to:

• data-sources/maxcompute.md
• data-sources/hologres.md
• data-sources/mysql.md
• data-sources/postgresql.md
• data-sources/polardb.md
• data-sources/sqlserver.md
• data-sources/analyticdb_for_mysql.md
• data-sources/analyticdb_for_postgresql.md
• data-sources/starrocks.md

AnalyticDB MySQL Datasource Documentation

Overview

Last Updated: October 17, 2024, 10:22:06

Property Definition

• Datasource Type: analyticdb_for_mysql
• Supported Configuration Modes (ConnectionPropertiesMode):
• UrlMode (Connection String Mode)
• InstanceMode (Instance Mode)

---

Same Account Instance Mode

Configuration Example - Same Account Instance Mode

{
    "database": "database",
    "password": "***",
    "instanceId": "am-xxxxx",
    "regionId": "cn-shanghai",
    "envType": "Dev",
    "username": "username"
}

---

Cross-Account Instance Mode

Configuration Example - Cross-Account Instance Mode

{
    "database": "database",
    "password": "***",
    "instanceId": "am-xxxxx",
    "regionId": "cn-shanghai",
    "envType": "Dev",
    "username": "username",
    "crossAccountOwnerId": "<ACCOUNT_ID>",
    "crossAccountRoleName": "dw-ds2.0-role"
}

---

Connection String Mode (UrlMode)

Configuration Example - Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "aliyun",
    "password": "xxx",
    "envType": "Dev"
}

---

Summary

AnalyticDB PostgreSQL DataSource Documentation

Overview

• DataSource Type: analyticdb_for_postgresql
• Supported Configuration Modes:
• UrlMode (Connection String Mode)
• InstanceMode (Instance Mode)

---

Configuration Modes

1. Same-Account Instance Mode

Example:

{
    "database": "dw",
    "password": "***",
    "instanceId": "gp-xxxxx",
    "regionId": "cn-hangzhou",
    "envType": "Prod",
    "username": "dw"
}

---

2. Cross-Account Instance Mode

Example:

{
    "crossAccountOwnerId": "<ACCOUNT_ID>",
    "crossAccountRoleName": "dw-role",
    "database": "dw",
    "password": "***",
    "instanceId": "gp-xxxxx",
    "regionId": "cn-shanghai",
    "envType": "Prod",
    "username": "dw"
}

---

3. Connection String Mode

Example:

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "aliyun",
    "password": "xxx",
    "envType": "Dev"
}

---

Summary Table

OceanBase ConnectionProperties Documentation

Property Definition

• Datasource Type: apsaradb_for_oceanbase
• Supported Configuration Modes (ConnectionPropertiesMode):
• UrlMode (Connection String Mode)
• InstanceMode (Instance Mode)

---

Same-Account Instance Mode

---

Cross-Account Instance Mode

---

Connection String Mode

---

Datasource Configuration Examples

Same-Account Instance Mode

{
    "envType": "Prod",
    "instanceId": "obxxxxxxxxxx",
    "tenant": "txxxxxxxxxx",
    "regionId": "cn-shanghai",
    "database": "db",
    "username": "aliyun",
    "password": "xxx"
}

Cross-Account Instance Mode

{
    "envType": "Prod",
    "instanceId": "obxxxxxxxxxx",
    "tenant": "txxxxxxxxxx",
    "regionId": "cn-shanghai",
    "database": "db",
    "username": "aliyun",
    "password": "xxx",
    "crossAccountOwnerId": "1234567890",
    "crossAccountRoleName": "my_ram_role"
}

Connection String Mode

{
    "envType": "Prod",
    "address": [
        {
            "host": "127.0.0.1",
            "port": "5432"
        }
    ],
    "dbMode": "mysql",
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "aliyun",
    "password": "xxx"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/oceanbase

BigQuery Datasource Documentation

Overview

• Datasource Type: bigquery
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

ConnectionProperties Parameters

Connection String Mode Parameters

---

Configuration Example

Connection String Mode

{
  "bigQueryProjectId": "bigquery_id",
  "bigQueryAuth": "123",
  "envType": "Dev"
}

---

API Reference Summary

---

Source: Aliyun DataWorks Developer Reference - BigQuery Last Updated: 2024-10-15

ClickHouse Datasource Documentation

Property Definition

• Datasource Type: clickhouse
• Supported Configuration Mode (ConnectionPropertiesMode): UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Datasource Configuration Example

Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 8123
        }
    ],
    "securityProtocol": "authTypeNone",
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "aliyun",
    "password": "xxx",
    "envType": "Dev"
}

DataHub ConnectionProperties Documentation

Overview

• Data Source Type: datahub
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Example

Connection String Mode

{
    "envType": "Prod",
    "endpoint": "http://dh-cn-shanghai.aliyuncs.com",
    "project": "jiangcheng-test1",
    "accessId": "xxx",
    "accessKey": "xxx"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/datahub

DB2 ConnectionProperties Documentation

Overview

• Datasource Type: db2
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Example

Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "database": "db",
    "properties": {
        "currentSchema": "abc"
    },
    "jdbcDriver": "db2_1",
    "username": "xxxxx",
    "password": "xxxxx",
    "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/db2

Data Lake Formation Datasource Documentation

Property Definition

• Datasource type: dlf
• Supported configuration mode (ConnectionPropertiesMode):
• InstanceMode (Instance Mode)

---

Property Parameters

---

Datasource Configuration Example

{
  "envType": "Prod",
  "authType": "SubAccount",
  "database": "testdb01",
  "catalogId": "clg-paimon-xx",
  "catalogName": "xx",
  "catalogType": "Paimon",
  "endpoint": "http://cn-hangzhou-vpc.dlf.aliyuncs.com",
  "authIdentity": "xxx"
}

DM (Dameng) Datasource ConnectionProperties Documentation

Overview

• Datasource Type: dm
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Example

Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "properties": {
        "schema": "db1"
    },
    "username": "xxxxx",
    "password": "xxxxx",
    "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/dm

Doris Datasource Documentation

Overview

• Datasource Type: doris
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

Connection Properties (UrlMode)

---

Field Details

address

• Type: Array
• Required: Yes
• Example:

  [
    {
      "host": "127.0.0.1",
      "port": 3306
    }
  ]

• Note: Only a single address is allowed.

loadAddress

• Type: String
• Required: Yes
• Example:

  [
    {
      "host": "127.0.0.1",
      "port": 3306
    }
  ]

• Note: FE Endpoint; multiple addresses are allowed.

database

• Type: String
• Required: Yes
• Example: dbName
• Description: The database name to connect to.

username

• Type: String
• Required: Yes
• Example: xxxxx
• Description: Username for authentication.

password

• Type: String
• Required: Yes
• Example: xxxxx
• Description: Password for authentication.

properties

• Type: JSON Object
• Required: No
• Example:

  {
    "useSSL": "false"
  }

• Description: Additional driver properties.

envType

• Type: String
• Required: Yes
• Example: Dev
• Description: Represents the datasource environment information.
• Dev: Development environment
• Prod: Production environment

---

Configuration Example (UrlMode)

{
    "envType": "Prod",
    "address": [
        {
            "host": "127.0.0.1",
            "port": "3306"
        }
    ],
    "loadAddress": [
        {
            "host": "127.0.0.2",
            "port": "8031"
        }
    ],
    "database": "my_database",
    "username": "my_username",
    "password": "<PASSWORD>",
    "properties": {
        "socketTimeout": "2000"
    }
}

---

API Reference Summary

---

Source: Aliyun DataWorks Developer Reference - Doris Last Updated: 2024-10-15

DRDS Datasource Documentation

Property Definition

• Datasource Type: drds
• Supported Configuration Mode (ConnectionPropertiesMode): InstanceMode, UrlMode

---

InstanceMode Parameters

---

UrlMode Parameters

---

Configuration Examples

InstanceMode

{
  "envType": "Prod",
  "regionId": "cn-shanghai",
  "instanceId": "drds-xxxxx",
  "database": "mydb",
  "username": "root",
  "password": "<PASSWORD>"
}

InstanceMode (Cross-Account)

{
  "envType": "Prod",
  "regionId": "cn-shanghai",
  "instanceId": "drds-xxxxx",
  "database": "mydb",
  "username": "root",
  "password": "<PASSWORD>",
  "crossAccountOwnerId": "<ACCOUNT_ID>",
  "crossAccountRoleName": "cross-account-role"
}

UrlMode

{
  "envType": "Prod",
  "address": [{"host": "192.168.1.100", "port": "3306"}],
  "database": "mydb",
  "username": "root",
  "password": "<PASSWORD>"
}

Elasticsearch Datasource Documentation

Property Definition

• Datasource Type: elasticsearch
• Supported Configuration Mode (ConnectionPropertiesMode): InstanceMode, UrlMode

---

InstanceMode Parameters

---

UrlMode Parameters

---

Configuration Examples

InstanceMode

{
  "envType": "Prod",
  "regionId": "cn-shanghai",
  "instanceId": "es-xxxxx",
  "instanceType": "cloudNative",
  "username": "elastic",
  "password": "<PASSWORD>"
}

UrlMode (With Authentication)

{
  "envType": "Prod",
  "endpoint": "http://esxxx.elasticsearch.aliyuncs.com:9200",
  "authEnable": "enable",
  "username": "elastic",
  "password": "<PASSWORD>"
}

UrlMode (Anonymous)

{
  "envType": "Prod",
  "endpoint": "http://192.168.1.100:9200",
  "authEnable": "disable"
}

FTP Data Source Connection Properties Documentation

Overview

• Data Source Type: ftp
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Example

Connection String Mode

{
  "host": "127.0.0.1",
  "port": "5432",
  "protocol": "sftp",
  "securityProtocol": "passwordAuth",
  "username": "xxxxx",
  "password": "xxxxx",
  "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/ftp

Last Updated: 2024-10-17

GBASE 8A ConnectionProperties Documentation

Data Source Type

• Type: gbase8a

Supported Configuration Modes

• ConnectionPropertiesMode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Example

Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "xxxxx",
    "password": "xxxxx",
    "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/gbase-8a

Graph Database Connection Properties Documentation

Overview

• Data Source Type: graph_database
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Example

Connection String Mode

{
  "host": "127.0.0.1",
  "port": "5432",
  "username": "xxxxx",
  "password": "xxxxx",
  "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/graph-database

Last Updated: 2024-10-15 09:31:48

HBase Datasource Documentation

Overview

Datasource Type: hbase

Supported Configuration Mode: UrlMode (Connection String Mode)

---

ConnectionProperties Parameters (UrlMode)

---

Configuration Example (UrlMode)

{
  "hbaseConfig": {
    "hbase.zookeeper.quorum": "localhost:2181",
    "hbaseVersion": "0.9.4"
  },
  "securityProtocol": "authTypeClientPassword",
  "username": "xxxxx",
  "password": "xxxxx",
  "envType": "Dev"
}

---

Authentication Types Summary

---

Last Updated: 2024-11-06

Hive Datasource Documentation

Property Definition

• Datasource Type: hive
• Supported Configuration Modes (ConnectionPropertiesMode):
• UrlMode (Connection String Mode)
• InstanceMode (Instance Mode)
• CdhMode (CDH Cluster Mode)

---

1. Same-Account Instance Mode

---

2. Cross-Account Instance Mode

---

3. Connection String Mode (UrlMode)

---

4. CDH Mode

---

Configuration Examples

Same-Account Instance Mode

{
    "clusterId": "c-xxxxxxxxx",
    "regionId": "cn-shanghai",
    "database": "db",
    "loginMode": "Anonymous",
    "version": "2.3.9",
    "authType": "Executor",
    "securityProtocol": "authTypeNone",
    "envType": "Dev"
}

Cross-Account Instance Mode

{
    "clusterId": "c-xxxxxxxxx",
    "regionId": "cn-shanghai",
    "database": "db",
    "loginMode": "LDAP",
    "version": "2.3.9",
    "authType": "Executor",
    "securityProtocol": "authTypeNone",
    "envType": "Dev"
}

Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 10000
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "aliyun",
    "password": "xxx",
    "metastoreUris": "thrift://127.0.0.1:9083",
    "metaType": "HiveMetastore",
    "version": "2.3.9",
    "loginMode": "Anonymous",
    "securityProtocol": "authTypeNone",
    "envType": "Dev"
}

CDH Mode

{
    "clusterIdentifier": "c-xxxxxxxxx",
    "database": "db",
    "loginMode": "Anonymous",
    "authType": "Executor",
    "securityProtocol": "authTypeNone",
    "envType": "Dev"
}

---

Important Notes

⚠️ loginMode Parameter Note: The loginMode parameter for Hive data sources only supports the following two values:

- Anonymous: Anonymous login

- LDAP: LDAP authentication login

>

Other values such as simple and disable are not supported and will cause an error: Unsupported loginMode=simple for Hive data source, only Anonymous and LDAP are supported

Hologres Datasource Documentation

Property Definition

• Datasource Type: hologres
• Supported Configuration Mode (ConnectionPropertiesMode): InstanceMode (Instance Mode)

---

Same Account Access Identity Mode

---

Cross-Account Mode

---

Datasource Configuration Examples

Executor Access Identity Mode

{
  "database": "database",
  "instanceId": "hgpostcn-cn-xxxxx",
  "securityProtocol": "authTypeNone",
  "regionId": "cn-beijing",
  "envType": "Dev",
  "authType": "Executor"
}

Cross-Account Mode

{
  "crossAccountOwnerId": "<ACCOUNT_ID>",
  "crossAccountRoleName": "holo-accross-role-name",
  "database": "database",
  "instanceId": "hgpostcn-cn-xxxxx",
  "securityProtocol": "authTypeNone",
  "regionId": "cn-beijing",
  "envType": "Dev",
  "authType": "RamRole"
}

HttpFile Datasource Documentation

Property Definition

• Datasource Type: httpfile
• Supported Configuration Mode (ConnectionPropertiesMode): UrlMode (Connection String Mode)

---

UrlMode Parameters

---

Configuration Examples

Basic Configuration

{
  "envType": "Prod",
  "urlPrefix": "http://example.com",
  "defaultHeaders": "{}"
}

With Custom Headers

{
  "envType": "Prod",
  "urlPrefix": "http://example.com",
  "defaultHeaders": "{\"Authorization\": \"Bearer <TOKEN>\", \"Content-Type\": \"application/json\"}"
}

InstanceMode Data Source Instance Query API Reference

When creating InstanceMode data sources, you can call the corresponding cloud product's OpenAPI to query the instance list for user selection.

Important: The user may not have permissions to call these APIs. If the instance list query fails, do not block the subsequent process; prompt the user to manually input the instance ID.

---

Instance Query API Summary

---

Detailed API Description

RDS MySQL (mysql)

API: DescribeDBInstances - Query Instance List Documentation: https://help.aliyun.com/zh/rds/apsaradb-rds-for-mysql/api-rds-2014-08-15-describedbinstances-mysql

aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" --Engine MySQL --PageSize 50 2>/dev/null | jq -r '
  .Items.DBInstance[] |
  "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)"
'

---

RDS PostgreSQL (postgresql)

API: DescribeDBInstances - Query Instance List Documentation: https://help.aliyun.com/zh/rds/apsaradb-rds-for-postgresql/api-rds-2014-08-15-describedbinstances-postgresql

aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" --Engine PostgreSQL --PageSize 50 2>/dev/null | jq -r '
  .Items.DBInstance[] |
  "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)"
'

---

RDS SQL Server (sqlserver)

API: DescribeDBInstances - Query Instance List Documentation: https://help.aliyun.com/zh/rds/apsaradb-rds-for-sql-server/api-rds-2014-08-15-describedbinstances-sqlserver

aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" --Engine SQLServer --PageSize 50 2>/dev/null | jq -r '
  .Items.DBInstance[] |
  "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)"
'

---

PolarDB MySQL (polardb)

API: DescribeDBClusters - Query Cluster List Documentation: https://help.aliyun.com/zh/polardb/api-polardb-2017-08-01-describedbclusters

aliyun polardb DescribeDBClusters --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" --DBType MySQL 2>/dev/null | jq -r '
  .Items.DBCluster[] |
  "ID: \(.DBClusterId) | Name: \(.DBClusterDescription // .DBClusterId) | Status: \(.DBClusterStatus)"
'

Note: When creating a data source, use the clusterId parameter (not instanceId), and specify the dbType parameter (mysql or postgresql).

---

DRDS (drds)

API: DescribeDrdsInstances - Query Instance List Documentation: https://help.aliyun.com/zh/drds/api-drds-2019-01-23-describedrdsinstances

aliyun drds DescribeDrdsInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" 2>/dev/null | jq -r '
  .Instances.Instance[] |
  "ID: \(.DrdsInstanceId) | Name: \(.Description // .DrdsInstanceId) | Status: \(.Status)"
'

Note: The returned JSON path is .Instances.Instance[], not .Data.Instances.Instance[].

---

Hologres (hologres)

API: ListInstances - Get Instance List Documentation: https://help.aliyun.com/zh/hologres/developer-reference/api-hologram-2022-06-01-listinstances

aliyun hologram POST /api/v1/instances --user-agent AlibabaCloud-Agent-Skills --region "<REGION>" --body '{}' 2>/dev/null | jq -r '
  .InstanceList[] |
  "ID: \(.InstanceId) | Name: \(.InstanceName) | Status: \(.InstanceStatus)"
'

Note: The Hologres API requires a POST request with an empty body --body '{}'. When creating a data source, username/password are not required; the authType parameter is needed.

---

AnalyticDB MySQL (analyticdb_for_mysql)

API: DescribeDBClusters - Query Instance List Documentation: https://help.aliyun.com/zh/analyticdb/analyticdb-for-mysql/developer-reference/api-adb-2019-03-15-describedbclusters

aliyun adb DescribeDBClusters --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" 2>/dev/null | jq -r '
  .Items.DBCluster[] |
  "ID: \(.DBClusterId) | Name: \(.DBClusterDescription // .DBClusterId) | Status: \(.DBClusterStatus)"
'

Note: This API does not support the --PageSize parameter; it returns 30 records by default.

---

AnalyticDB PostgreSQL (analyticdb_for_postgresql)

API: DescribeDBInstances - Query Database Instance List Documentation: https://help.aliyun.com/zh/analyticdb/analyticdb-for-postgresql/developer-reference/api-gpdb-2016-05-03-describedbinstances

aliyun gpdb DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" --PageSize 50 2>/dev/null | jq -r '
  .Items.DBInstance[] |
  "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)"
'

---

StarRocks (starrocks)

API: DescribeInstances - Query Instance List Documentation: https://help.aliyun.com/zh/emr-serverless-starrocks/developer-reference/api-starrocks-2022-10-19-describeinstances

aliyun starrocks DescribeInstances --user-agent AlibabaCloud-Agent-Skills --region "<REGION>" 2>/dev/null | jq -r '
  .Data[] |
  "ID: \(.InstanceId) | Name: \(.InstanceName) | Status: \(.InstanceStatus)"
'

Note: The returned JSON path is .Data[], and field names are lowercase. When creating a data source, specify the instanceType parameter (serverless or emr-olap).

---

Milvus (milvus)

API: list-instances - Get Instance List Documentation: https://help.aliyun.com/zh/milvus/developer-reference/api-milvus-2023-10-12-listinstances

Prerequisites: You need to install the Milvus CLI plugin first:

```bash

aliyun plugin install --names aliyun-cli-milvus

```

aliyun milvus list-instances --user-agent AlibabaCloud-Agent-Skills --region "<REGION>" 2>/dev/null | jq -r '
  .Data[] |
  "ID: \(.InstanceId) | Name: \(.ClusterName) | Status: \(.InstanceStatus)"
'

Note: The command and parameters are lowercase (list-instances, --region). The returned JSON path is .Data[].

---

MongoDB (mongodb)

API: DescribeDBInstances - Query MongoDB Instance List Documentation: https://help.aliyun.com/zh/mongodb/developer-reference/api-dds-2015-12-01-describedbinstances

aliyun dds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" 2>/dev/null | jq -r '
  .DBInstances.DBInstance[] |
  "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)"
'

Note: This API returns 30 records by default; the --PageSize parameter is not needed.

---

Kafka (kafka)

API: GetInstanceList - Query Instance Information for a Specific Region Documentation: https://help.aliyun.com/zh/apsaramq-for-kafka/cloud-message-queue-for-kafka/developer-reference/api-alikafka-2019-09-16-getinstancelist

aliyun alikafka GetInstanceList --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" 2>/dev/null | jq -r '
  .InstanceList.InstanceVO[] |
  "ID: \(.InstanceId) | Name: \(.InstanceName) | Status: \(.Status)"
'

---

OpenSearch (opensearch)

API: ListInstances - Get Instance List Documentation: https://help.aliyun.com/zh/open-search/developer-reference/api-searchengine-2021-10-25-listinstances

aliyun searchengine ListInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" 2>/dev/null | jq -r '
  .result[] |
  "ID: \(.instanceId) | Name: \(.description // .instanceId) | Type: \(.edition)"
'

Note: The returned JSON fields are lowercase: result, instanceId, etc.

---

Recommended Process

When creating an InstanceMode data source:

1. Try to query the instance list

   aliyun <product> <api> --RegionId "<REGION>" ...

2. Query successful → Display list for user to select instance ID

3. Query failed → Prompt user: "Unable to retrieve the instance list, please enter the instance ID directly"

---

Error Handling

Common Errors and Handling:

Key Principle: When instance list query fails, do not block the data source creation process; fall back to letting the user manually input the instance ID.

Kafka Datasource Documentation

Property Definition

• Datasource Type: kafka
• Supported Configuration Mode (ConnectionPropertiesMode): InstanceMode, UrlMode

---

UrlMode Parameters

---

InstanceMode Parameters

---

Configuration Examples

UrlMode (No Authentication)

{
  "envType": "Prod",
  "address": [{"host": "192.168.1.100", "port": "9092"}],
  "version": "2.0",
  "securityProtocol": "authTypeNone"
}

UrlMode (SASL Plaintext)

{
  "envType": "Prod",
  "address": [{"host": "192.168.1.100", "port": "9092"}],
  "version": "2.0",
  "securityProtocol": "authTypeSaslPlaintext",
  "saslMechanism": "plain",
  "saslUsername": "user",
  "saslPassword": "<PASSWORD>"
}

InstanceMode

{
  "envType": "Prod",
  "regionId": "cn-shanghai",
  "instanceId": "alikafka-xxxxx",
  "version": "2.0",
  "securityProtocol": "authTypeNone"
}

KingbaseES Datasource ConnectionProperties Documentation

Overview

• Datasource Type: kingbasees
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Parameter Details

address

Type: JSON Array

Example:

[
    {
        "host": "127.0.0.1",
        "port": 3306
    }
]

Note: Although it's an array format, only one set of host and port is allowed.

---

database

Type: String

Example: mysql_database

Description: The name of the database.

---

username

Type: String

Example: xxxxx

Description: The username for authentication.

---

password

Type: String

Example: xxxxx

Description: The password for authentication.

---

properties

Type: JSON Object

Example:

{
    "useSSL": "false"
}

Description: Driver properties (optional).

---

envType

Type: String

Example: Dev

Description: Indicates the datasource environment information.

Valid Values:

• Dev: Development environment
• Prod: Production environment

---

Configuration Example

Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "xxxxx",
    "password": "xxxxx",
    "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/kingbasees

Lindorm Datasource Documentation

Last Updated: 2024-10-15 09:31:57

Property Definition

Note:

- Wide Table Engine: Only supports UrlMode (Connection String Mode).

- Compute Engine: Supports both UrlMode and InstanceMode. InstanceMode supports cross-account access.

---

Connection String Mode Parameters (UrlMode)

---

Instance Mode Parameters (InstanceMode)

Applicable to: Compute Engine only.

---

Configuration Examples

Connection String Mode (UrlMode)

{
  "seedserver": "ld-xxxxxxxxxxxxx.rds.aliyuncs.com:30020",
  "username": "user",
  "password": "pass",
  "namespace": "default",
  "envType": "Dev"
}

Instance Mode (InstanceMode) - Compute Engine

{
  "envType": "Prod",
  "engineType": "compute",
  "regionId": "cn-xxxxx",
  "instanceId": "ld-xxxxxxxxxxxxx",
  "namespace": "db",
  "username": "user",
  "password": "pass"
}

Instance Mode (InstanceMode) - Cross-Account Compute Engine

{
  "envType": "Prod",
  "engineType": "compute",
  "regionId": "cn-xxxxx",
  "instanceId": "ld-xxxxxxxxxxxxx",
  "namespace": "db",
  "username": "user",
  "password": "pass",
  "crossAccountOwnerId": "111111111",
  "crossAccountRoleName": "xx-role"
}

---

API Reference Summary

• Source: DataWorks Developer Reference > Appendix > Data Source Connection Information (ConnectionProperties)
• Platform: Alibaba Cloud DataWorks
• Navigation: Previous: KingbaseES | Next: LogHub

Related Lindorm OpenAPIs

LogHub Data Source ConnectionProperties

Property Definition

• Data source type: loghub
• Supported configuration mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Examples

Connection String Mode

{
    "envType": "Prod",
    "endpoint": "http://cn-beijing.log.aliyuncs.com",
    "project": "jiangcheng-test1",
    "accessId": "xxx",
    "accessKey": "xxx"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/loghub

MariaDB DataSource ConnectionProperties Documentation

Attribute Definition

• Datasource type: mariadb
• Supported configuration mode: UrlMode (Connection String Mode)

---

Connection String Mode

---

Datasource Configuration Example

Connection String Mode

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "xxxxx",
    "password": "xxxxx",
    "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/mariadb

MaxCompute Datasource Documentation

Overview

---

Query MaxCompute Projects

Before creating a MaxCompute data source, you can query the list of available MaxCompute projects.

Reference: MaxCompute ListProjects API

aliyun maxcompute ListProjects --user-agent AlibabaCloud-Agent-Skills --RegionId "<REGION>" --maxItem 50 2>/dev/null | jq -r '
  .data.projects[] |
  "Project Name: \(.name) | Status: \(.status)"
'

---

ConnectionProperties Parameters

1. Same-Account Access Identity Mode

---

2. Cross-Account Mode

---

Configuration Examples

PrimaryAccount Access Identity Mode (Recommended)

Note: MaxCompute data source does not support Executor identity. Please use PrimaryAccount.

{
    "envType": "Prod",
    "project": "skyfire_20221228",
    "regionId": "cn-shanghai",
    "endpointMode": "SelfAdaption",
    "authType": "PrimaryAccount"
}

Custom Endpoint Mode

{
    "endpoint": "http://service.cn-shanghai.maxcompute.aliyun-inc.com/api",
    "envType": "Prod",
    "project": "skyfire_20221228",
    "authType": "PrimaryAccount",
    "endpointMode": "Custom",
    "regionId": "cn-shanghai"
}

Cross-Account Mode

{
    "project": "skyfire_20221228",
    "crossAccountOwnerId": "<ACCOUNT_ID>",
    "crossAccountRoleName": "mc-accross-role-name",
    "endpointMode": "SelfAdaption",
    "envType": "Prod",
    "authType": "RamRole",
    "regionId": "cn-shanghai"
}

AuthType Enum Values

Note: MaxCompute data source does not support Executor identity. PrimaryAccount is recommended.

Endpoint Format Notes

• Service Endpoint Formats:
• Classic: service.${regionId}.maxcompute.aliyun-inc.com/api
• New format: service.${regionId}-intranet.maxcompute.aliyun-inc.com/api

• Tunnel Endpoint Formats:
• Classic: dt.${regionId}.maxcompute.aliyun-inc.com
• New format: dt.${regionId}-intranet.maxcompute.aliyun-inc.com

Memcache Data Source ConnectionProperties

Basic Information

• Data source type: memcache
• Supported configuration mode: UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

Configuration Examples

Connection String Mode

{
  "proxy": "127.0.0.1",
  "port": "5432",
  "username": "xxxxx",
  "password": "xxxxx",
  "envType": "Dev"
}

---

Last updated: 2024-10-15 09:32:01

Source: https://help.aliyun.com/zh/dataworks/developer-reference/memcache

Milvus Data Source ConnectionProperties

Data source type: milvus

Supported configuration modes:

• InstanceMode (Instance Mode)
• UrlMode (Connection String Mode)

---

Instance Mode

Instance Mode Configuration Example

{
    "envType": "Prod",
    "regionId": "cn-beijing",
    "instanceId": "c-dd8f71372xxxx",
    "database": "default",
    "username": "root",
    "password": "xxxxxxx"
}

---

Connection String Mode

Connection String Mode Configuration Example

{
    "envType": "Prod",
    "endpoint": "http://c-dd8xxxxx.milvus.aliyuncs.com:19530",
    "database": "default",
    "username": "root",
    "password": "xxxx",
    "authType": "USERNAME_PASSWORD"
}

---

Last updated: 2025-03-27 14:05:17

Source: https://help.aliyun.com/zh/dataworks/developer-reference/milvus

MongoDB Datasource Documentation

Property Definition

• Datasource Type: mongodb
• Supported Configuration Mode (ConnectionPropertiesMode): InstanceMode, UrlMode

---

InstanceMode Parameters

---

UrlMode Parameters

---

Configuration Examples

InstanceMode

{
  "envType": "Prod",
  "instanceId": "dds-xxxxx",
  "regionId": "cn-shanghai",
  "database": "my_db",
  "username": "root",
  "password": "<PASSWORD>",
  "authDb": "admin",
  "engineVersion": "5.x"
}

UrlMode

{
  "envType": "Prod",
  "address": [{"host": "192.168.1.100", "port": "27017"}],
  "database": "my_db",
  "username": "root",
  "password": "<PASSWORD>",
  "authDb": "admin",
  "engineVersion": "5.x"
}

UrlMode with SSL

{
  "envType": "Prod",
  "address": [{"host": "192.168.1.100", "port": "27017"}],
  "database": "my_db",
  "username": "root",
  "password": "<PASSWORD>",
  "authDb": "admin",
  "engineVersion": "5.x",
  "authType": "authTypeSsl",
  "truststoreFile": "<FILE_ID>"
}

MySQL DataSource ConnectionProperties Documentation

Overview

• Data Source Type: mysql
• Supported Configuration Modes:
• UrlMode (Connection String Mode)
• InstanceMode (Instance Mode)

---

1. Same-Account Instance Mode

Example

{
    "instanceId": "rm-xxxxxxxxx",
    "regionId": "cn-shanghai",
    "database": "db",
    "username": "aliyun",
    "password": "xxx",
    "securityProtocol": "authTypeNone",
    "envType": "Dev"
}

---

2. Cross-Account Instance Mode

Example

{
    "instanceId": "rm-xxxxxxxxx",
    "regionId": "cn-shanghai",
    "database": "db",
    "username": "aliyun",
    "password": "xxx",
    "crossAccountOwnerId": "1234567890",
    "crossAccountRoleName": "my_ram_role",
    "securityProtocol": "authTypeNone",
    "envType": "Dev"
}

---

3. Connection String Mode

Example

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 3306
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "aliyun",
    "password": "xxx",
    "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/mysql

OpenSearch ConnectionProperties Documentation

Overview

Data Source Type: opensearch

Supported Configuration Mode (ConnectionPropertiesMode):

• InstanceMode (Instance Mode)

---

Instance Mode Parameters

Important Notes

For Vector Search Version and Recall Engine Version, both username/password and AccessKey are required:

• AccessKey: Used by plugins to retrieve table schema and pull table structure information in wizard mode.
• Username/Password: Used for actual runtime data writing.

---

Configuration Example

Instance Mode

{
    "envType": "Prod",
    "regionId": "cn-beijing",
    "instanceType": "vectorSearchVersion",
    "instanceId": "ha-xxxx",
    "username": "admin",
    "password": "xxxx"
}

---

Document Information

• Last Updated: 2024-12-30 18:28:58
• Source: https://help.aliyun.com/zh/dataworks/developer-reference/opensearch

Oracle DataSource ConnectionProperties Documentation

Property Definition

• Data source type: oracle
• Supported configuration mode (ConnectionPropertiesMode): UrlMode (Connection String Mode)

---

Connection String Mode Parameters

---

DataSource Configuration Example

Connection String Mode

{
  "jdbcUrl": "jdbc:oracle:thin:@host:port:SID",
  "username": "xxxxx",
  "password": "xxxxx",
  "securityProtocol": "authTypeNone",
  "envType": "Dev"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/oracle

OSS Data Source Documentation

Property Definition

• Data source type: oss
• Supported configuration mode (ConnectionPropertiesMode): UrlMode (Connection string mode)

Connection String Mode Parameters

Data Source Configuration Examples

Connection String Mode (RamRole Recommended)

Note: OSS data source supports both RamRole and Ak authentication methods. RamRole is recommended.

Create via RAM Role Mode (Recommended):

{
    "envType": "Prod",
    "endpoint": "http://oss-beijing.aliyuncs.com",
    "bucket": "test-oss-sh",
    "authType": "RamRole",
    "authIdentity": "1123455"
}

Create via AK Mode (Alternative):

{
    "envType": "Prod",
    "endpoint": "http://oss-beijing.aliyuncs.com",
    "bucket": "test-oss-sh",
    "authType": "Ak",
    "accessId": "xxx",
    "accessKey": "xxx"
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/oss

PolarDB-X 2.0 ConnectionProperties Documentation

Attribute Definition

• Datasource type: polardbx20
• Supported configuration modes (ConnectionPropertiesMode):
• UrlMode (Connection String Mode)
• InstanceMode (Instance Mode)

---

Same-Account Instance Mode

---

Cross-Account Instance Mode

---

Connection String Mode

---

Datasource Configuration Examples

Same-Account Instance Mode

{
    "envType": "Prod",
    "ownerId": "<ACCOUNT_ID>",
    "regionId": "cn-beijing",
    "instanceId": "pxc-bjrikym49rir1s",
    "database": "my_database",
    "username": "my_username",
    "password": "<PASSWORD>",
    "properties": {
        "socketTimeout": "2000"
    }
}

Cross-Account Instance Mode

{
    "envType": "Prod",
    "regionId": "cn-beijing",
    "instanceId": "pxc-bjrikym49rir1s",
    "database": "my_database",
    "username": "my_username",
    "password": "<PASSWORD>",
    "properties": {
        "socketTimeout": "2000"
    },
    "crossAccountOwnerId": "123123123",
    "crossAccountRoleName": "cross-role"
}

Connection String Mode

{
    "envType": "Prod",
    "address": [
        {
            "host": "127.0.0.1",
            "port": "3306"
        }
    ],
    "database": "my_database",
    "username": "my_username",
    "password": "<PASSWORD>",
    "properties": {
        "socketTimeout": "2000"
    }
}

---

Source: https://help.aliyun.com/zh/dataworks/developer-reference/polardb-x-2-0

Redshift Datasource Documentation

Overview

• Datasource Type: redshift
• Supported Configuration Mode: UrlMode (Connection String Mode)

---

ConnectionProperties Parameters (UrlMode)

---

Configuration Example (UrlMode)

{
    "address": [
        {
            "host": "127.0.0.1",
            "port": 5432
        }
    ],
    "database": "db",
    "properties": {
        "connectTimeout": "2000"
    },
    "username": "xxxxx",
    "password": "xxxxx",
    "envType": "Dev"
}

---

Field Details

address

• Type: JSON Array
• Required: Yes
• Notes: Although formatted as an array, it only accepts a single host/port configuration pair.

database

• Type: String
• Required: Yes
• Description: The name of the database to connect to.

username

• Type: String
• Required: Yes
• Description: Authentication username.

password

• Type: String
• Required: Yes
• Description: Authentication password.

properties

• Type: JSON Object
• Required: No
• Description: Additional JDBC driver properties (e.g., useSSL, connectTimeout).

envType

• Type: String
• Required: Yes
• Valid Values:
• Dev - Development environment
• Prod - Production environment