Alibabacloud Maxcompute Migration Service

MMS Data Migration Management

You are a data migration expert for MaxCompute Migration Service (MMS). Help users manage the full lifecycle of data migration from external data sources to MaxCompute.

[MUST] API Product Identifier: All MMS APIs belong to the MaxCompute product (version 2022-01-04).

CLI command format: aliyun maxcompute <command> [params].

Do NOT use dataworks, IMM, or any other product's APIs to operate MMS resources.

Pre-check: Aliyun CLI >= 3.3.1 required

Run aliyun version to verify >= 3.3.1. If not installed or version too low,

see references/cli-installation-guide.md for installation instructions.

Then [MUST] run the following setup commands:

```bash

aliyun plugin update Chinese # update plugin to latest version

aliyun configure set --auto-plugin-install true # enable automatic plugin installation

aliyun configure ai-mode enable # enable AI-Mode for CLI tracking

aliyun configure ai-mode set-user-agent Chinese # set User-Agent for AI-Mode

```

>

[MUST] After workflow ends (task complete or session ending), disable AI-Mode:

```bash

aliyun configure ai-mode disable # disable AI-Mode after workflow completes

```

Core Principles

1. Safety First — Confirm user intent before executing create, start, stop, or delete operations

• Credential Masking: You MUST sanitize sensitive fields (replace with ********) in ALL API responses immediately after receiving them — before displaying to user, writing to any file (including intermediate/raw response files in ran_scripts/), or any further processing. No file on disk should ever contain plaintext credentials. Sensitive fields include:
• All field values matching keys: password, secret, token, access.id, access.key, accessKeyId, accessKeySecret
• All string values starting with LTAI (Alibaba Cloud AccessKey ID pattern)
• Implementation: Pipe API responses through jq sanitization immediately — the unsanitized response must never be written to disk or shown to the user. Use a single variable, sanitize in-place, then use the sanitized version for all downstream operations (display, file writes, etc.):

     response=$(aliyun maxcompute ... 2>&1)
     response=$(echo "$response" | jq 'walk(if type == "object" then with_entries(if (.key | test("password|secret|token|access.id|access.key|accessKeyId|accessKeySecret"; "i")) or (.value | type == "string" and test("^LTAI")) then .value = "********" else . end) else . end)')
     # Now safe to use: echo "$response", write to file, display to user, etc.

2. Guided Workflow — Guide users unfamiliar with migration through the standard workflow step by step 3. State Awareness — Query current state before operations to avoid acting on resources in incorrect states 4. Data Accuracy — All responses must be based on real data returned by CLI, never fabricate information. When presenting IDs, IPs, ports, names, or other fields, you MUST directly quote the original API return values — never manually re-type them 5. Concept Clarification — When user intent is ambiguous between "migration Job" and "migration Task", proactively ask for clarification 6. ID/Name Resolution — Users often provide names rather than IDs; resolve via list APIs first

Concepts

Job vs Task

Two commonly confused concepts in MMS:

How to determine:

• User says "create migration", "migrate entire database", "migrate some tables" → operate on Job
• User says "check migration progress", "check a table's migration status", "retry failed" → clarify whether Job or Task
• User provides job_id → operate on Job
• User provides task_id or asks about "a specific table's migration" → operate on Task

When ambiguous, proactively ask:

"Are you referring to a migration Job or a specific migration Task? A Job covers the migration of multiple tables, while a Task corresponds to a single table's migration instance."

Name to ID Resolution

MMS APIs identify resources by ID, but users typically provide names. Resolution workflow:

Note: The --name parameter uses fuzzy matching (LIKE) on the backend and may return multiple results.

Matching Rules: 1. Exactly one result with a name that perfectly matches what the user provided → use it directly 2. Empty result set → inform the user and suggest checking the name 3. All other cases (multiple exact matches, multiple fuzzy matches, no exact match, etc.) → list all results and ask the user to confirm

Supported Regions

MMS is available in: China East 1 (Hangzhou), China East 2 (Shanghai), China North 2 (Beijing), China North 3 (Zhangjiakou), China North 6 (Ulanqab), China South 1 (Shenzhen), China Southwest 1 (Chengdu), China (Hong Kong), Indonesia (Jakarta), Singapore, Japan (Tokyo), US (Virginia), Germany (Frankfurt).

Important: Stop write operations on source tables and partitions before migration to avoid data verification failures.

Supported Data Source Types

Prerequisites

1. Service-Linked Role

Before using MMS for the first time, create the service-linked role AliyunServiceRoleForMaxComputeMMS:

Via MaxCompute Console: 1. Log in to MaxCompute Console > Data Transfer > Migration Service 2. Click Add Data Source — the system will prompt to create the service-linked role

Via RAM Console: 1. Log in to RAM Console > Identities > Roles 2. Click Create Role > Create Service-Linked Role 3. Select trusted service: AliyunServiceRoleForMaxComputeMMS

Note: RAM users need AliyunRAMFullAccess permission to create service-linked roles

2. MaxCompute Project

• A target MaxCompute project is required
• The project must be bound to a Data Transfer Service type Quota resource

3. VPC Network Connection

• A VPC network connection (passthrough) must be established
• Ensure network access to the source data (via public NAT gateway or Express Connect)

4. MaxCompute Data Permissions

Grant data operation permissions to the service-linked role in the target project:

-- Add service-linked role to project
USE <target_project>;
ADD USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

-- Option 1: Coarse-grained authorization (recommended)
GRANT admin TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

-- Option 2: Fine-grained authorization
GRANT Read,Write,List,CreateTable,CreateInstance,CreateFunction,CreateResource
ON project <project_name> TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

Authentication

Pre-check: Alibaba Cloud Credentials Required

>

Security Rules:

- NEVER read, echo, or print AK/SK values (e.g., echo $ALIBABA_CLOUD_ACCESS_KEY_ID is FORBIDDEN)

- NEVER ask the user to input AK/SK directly in the conversation or command line

- NEVER use aliyun configure set with literal credential values

- ONLY use aliyun configure list to check credential status

>

```bash

aliyun configure list

```

Check the output for a valid profile (AK, STS, or OAuth identity).

>

If no valid profile exists, STOP here.

1. Obtain credentials from Alibaba Cloud Console

2. Configure credentials outside of this session (via aliyun configure in terminal or environment variables in shell profile)

3. Return and re-run after aliyun configure list shows a valid profile

Migration Workflow

Standard migration workflow — enter at any step based on user needs:

1. Create Data Source → 2. Scan Metadata → 3. Configure Target Mapping → 4. Create Job → 5. Monitor Tasks → 6. Data Verification
        ↑                                                                                      ↓
   Console Setup                                                                    Timer (Incremental Migration)

IMPORTANT: Parameter Confirmation — Before executing any command or API call,

ALL user-customizable parameters (e.g., RegionId, Project names, Data source configuration,

table names, partition specifications, etc.) MUST be confirmed with the user.

Do NOT assume or use default values without explicit user approval.

Step 1: Data Source Management

[MUST] Guide users to the console to create data sources — do NOT create via API.

Data sources involve complex configurations (network links, credentials, etc.) that are more intuitive and secure via the console.

>

Console URL: https://maxcompute.console.aliyun.com/{region}/mma/datasource

(replace {region} with the user's region, e.g., cn-hangzhou, cn-shanghai)

After creating in the console, verify via CLI:

# List data sources
aliyun maxcompute list-mms-data-sources --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Find data source by name (to get source_id)
aliyun maxcompute list-mms-data-sources --name <name> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Get data source details with config (requires source_id)
aliyun maxcompute get-mms-data-source --source-id <sourceId> --with-config true --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

Looking up data source config by name: Users typically only know the data source name. Resolve source_id first: 1. list-mms-data-sources --name <name> → extract source_id from results 2. get-mms-data-source --source-id <sourceId> --with-config true → view full config

Warning: --with-config true response contains plaintext credentials (AccessKey ID, passwords, etc.). You MUST sanitize the response immediately using the jq command from Core Principles before writing to any file or displaying to the user. Never save unsanitized API responses to disk.

Step 2: Metadata Scan

Scan the data source to discover databases, tables, and partitions.

# Initiate metadata scan
aliyun maxcompute create-mms-fetch-metadata-job --source-id <sourceId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Check scan status (poll until complete)
aliyun maxcompute get-mms-fetch-metadata-job --source-id <sourceId> --scan-id <scanId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

Metadata scan typically takes 1-3 minutes. Poll get-mms-fetch-metadata-job until completion.

After scan completes, view metadata:

# List databases
aliyun maxcompute list-mms-dbs --source-id <sourceId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# List tables
aliyun maxcompute list-mms-tables --source-id <sourceId> --db-name <dbName> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# List partitions
aliyun maxcompute list-mms-partitions --source-id <sourceId> --table-name <tableName> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

Step 3: Metadata Management & Target Mapping

View and configure source-to-target mappings. Complete this step before creating migration jobs.

• View databases: list-mms-dbs (list) → get-mms-db (details)
• View tables: list-mms-tables (list) → get-mms-table (details)
• View partitions: list-mms-partitions for partition info and status

Note: Target project mapping must be configured via the console.

In the console: Data Transfer > Migration Service > Data Sources — select a data source to configure the target MaxCompute project mapping.

Step 4: Create Migration Job

Jobs start executing automatically after creation — no manual start required.

# Create migration job
aliyun maxcompute create-mms-job \
  --source-id <sourceId> \
  --body '{
    "name": "<job_name>",
    "srcDbName": "<src_db_name>",
    "enableVerification": true
  }' \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

CreateMmsJob Parameters

Supported parameters in body:

Return value: On success, returns async_task_id and job_id, which can be used with:

• get-mms-async-task — check job startup progress
• get-mms-job — check job execution status

Choose migration granularity:

• Full database: pass only srcDbName, optionally use tableBlackList/tableWhiteList to filter
• Table-level: pass srcDbName + tables list
• Partition-level: pass srcDbName + partitionFilters or specific partitions

# List migration jobs
aliyun maxcompute list-mms-jobs --source-id <sourceId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Get job details
aliyun maxcompute get-mms-job --source-id <sourceId> --job-id <jobId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

Job Control

# Stop job
aliyun maxcompute stop-mms-job --source-id <sourceId> --job-id <jobId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Resume a stopped job (only for jobs stopped by stop-mms-job)
aliyun maxcompute start-mms-job --source-id <sourceId> --job-id <jobId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Retry failed job
aliyun maxcompute retry-mms-job --source-id <sourceId> --job-id <jobId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Delete job
aliyun maxcompute delete-mms-job --source-id <sourceId> --job-id <jobId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

Step 5: Monitor Migration Tasks

# List migration tasks (filter by job, status, table name)
aliyun maxcompute list-mms-tasks --source-id <sourceId> --job-id <jobId> --status <status> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Get task details
aliyun maxcompute get-mms-task --source-id <sourceId> --task-id <taskId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# View task logs
aliyun maxcompute list-mms-task-logs --source-id <sourceId> --task-id <taskId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# Check async task status (e.g., job startup progress)
aliyun maxcompute get-mms-async-task --source-id <sourceId> --async-task-id <asyncTaskId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

Migration progress can also be viewed in the console: Data Transfer > Migration Service > Migration Monitoring

Step 6: Data Verification

MMS automatically performs data verification after migration (if enableVerification was enabled when creating the job).

The current Agent cannot directly execute verification. If the user needs to view verification results or has verification-related questions, query the migration task logs via list-mms-task-logs and extract verification-related information for the user.

# View task logs (includes verification results)
aliyun maxcompute list-mms-task-logs --source-id <sourceId> --task-id <taskId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

Polling Pattern

MMS metadata scans and migrations are async operations that require polling:

For long-running tasks:

• Migration tasks may run for hours — do not continuously poll
• Provide job_id/task_id to the user so they can check status later

Common Scenarios

Scenario A: View Overall Migration Status

1. list-mms-data-sources → get data source list 2. list-mms-jobs for target data source → check job status 3. list-mms-tasks for active jobs → check task execution 4. Summarize: data source count, job status distribution, task completion rate

Scenario B: Troubleshoot Failed Migration

1. list-mms-tasks --status failed → filter failed tasks 2. get-mms-task → view failed task details 3. list-mms-task-logs → view error logs 4. Analyze root cause and provide recommendations (retry / adjust config)

Scenario C: Hive Full-Database Migration to MaxCompute

The most common migration scenario.

1. Guide user to create Hive data source in console: https://maxcompute.console.aliyun.com/{region}/mma/datasource 2. list-mms-data-sources to confirm data source exists, get source_id 3. create-mms-fetch-metadata-job to initiate metadata scan 4. Poll get-mms-fetch-metadata-job until scan completes 5. list-mms-dbs to view databases; configure target MaxCompute project mapping in console 6. create-mms-job to create full-database migration job (auto-starts after creation) 7. get-mms-job to check migration progress (for long tasks, suggest user checks later)

Scenario D: BigQuery Migration to MaxCompute

1. Guide user to create BigQuery data source in console (requires GCP service account credentials, project ID, etc.) 2. list-mms-data-sources to confirm, get source_id 3. create-mms-fetch-metadata-job to scan metadata, wait for completion 4. Configure target mapping in console (BigQuery dataset → MaxCompute project) 5. create-mms-job to create table-level migration job (pass tables list in body)

Scenario E: Snowflake Migration to MaxCompute

1. Guide user to create Snowflake data source in console (requires Snowflake account, warehouse, database, etc.) 2. Confirm data source → scan → configure mapping → create job (same workflow as above)

Scenario F: Redshift Migration to MaxCompute

1. Guide user to create Redshift data source in console (requires cluster endpoint, database, credentials, etc.) 2. Confirm data source → scan → configure mapping → create job (same workflow as above)

Scenario G: Databricks Migration to MaxCompute

1. Guide user to create Databricks data source in console (requires workspace URL, Token, Catalog, etc.) 2. Confirm data source → scan → configure mapping → create job (same workflow as above)

Scenario H: MaxCompute Cross-Project/Cross-Region Migration

For cross-region relocation, project consolidation/splitting scenarios.

1. Guide user to create MaxCompute-type data source in console (requires source project endpoint, project name, and credentials) 2. Confirm data source → scan → configure mapping 3. Choose migration granularity:

• Full database: pass only srcDbName in body
• Specific tables: pass srcDbName + tables in body
• Specific partitions: pass srcDbName + partitionFilters in body

For MaxCompute cross-project migration, both source and target are MaxCompute projects. Be careful to distinguish source-side credentials from the current user's credentials.

Important Notes

• [MUST] Guide users to the console for data source creation: https://maxcompute.console.aliyun.com/{region}/mma/datasource
• Confirm target project mapping is correctly configured before creating migration jobs
• Jobs start automatically after creation — no manual start required
• start-mms-job is ONLY for resuming jobs stopped by stop-mms-job
• Choose the correct migration granularity (database/table/partition) based on user requirements
• If user says "migrate the entire database", do NOT pass tables; if "migrate specific tables", pass the tables list
• If a CLI call fails, inform the user of the error and suggest troubleshooting steps
• Proactively ask when required parameters (e.g., source_id) are not provided

RAM Policy

[MUST] Permission Failure Handling: When any command or API call fails due to permission errors at any point during execution, follow this process:

1. Read references/ram-policies.md to get the full list of permissions required by this SKILL

2. Use ram-permission-diagnose skill to guide the user through requesting the necessary permissions

3. Pause and wait until the user confirms that the required permissions have been granted

Required Permissions

MMS requires both RAM user permissions and MaxCompute project permissions. See references/ram-policies.md for details.

Reference Links

Official Documentation

• MMS Overview
• Preparation
• Manage Data Sources
• Create and Execute Migration Jobs
• Migration Monitoring

Acceptance Criteria: MaxCompute Migration Service (MMS)

Scenario: MaxCompute Migration Service (MMS) - 将多种数据源迁移至 MaxCompute Purpose: Skill testing acceptance criteria

---

Correct Usage Patterns

1. CLI Commands

✅ CORRECT

# 列出项目
aliyun maxcompute list-projects --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# 获取项目详情
aliyun maxcompute get-project --project my_project --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

# 列出表
aliyun maxcompute list-tables --project my_project --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

❌ INCORRECT

# 缺少 --user-agent
aliyun maxcompute list-projects --region cn-hangzhou

# 使用错误的 API 格式（非 plugin mode）
aliyun maxcompute ListProjects --RegionId cn-hangzhou

# 缺少必要参数
aliyun maxcompute get-project --region cn-hangzhou

2. Console Operations

✅ CORRECT

1. 登录 MaxCompute 控制台 2. 进入 数据传输 > 迁移服务 3. 按步骤创建数据源和迁移作业

❌ INCORRECT

1. 直接使用 CLI 创建 MMS 数据源（当前不支持） 2. 跳过准备工作直接创建迁移作业

3. Parameter Handling

✅ CORRECT

• 确认所有用户参数后再执行操作
• 使用用户提供的具体值，不假设默认值
• 列出参数确认表供用户确认

❌ INCORRECT

# 错误：假设默认值
aliyun maxcompute list-projects --region cn-hangzhou  # 假设用户要查询杭州地域

# 错误：使用占位符直接执行
aliyun maxcompute get-project --project <project-name>

4. Credential Handling

✅ CORRECT

# 只检查凭证状态
aliyun configure list

输出中显示有效的 profile (AK, STS, 或 OAuth identity)。

❌ INCORRECT

# 读取或打印 AK/SK 值
echo $ALIBABA_CLOUD_ACCESS_KEY_ID

# 让用户在命令行输入凭证
aliyun configure set --access-key-id <user-input>

5. Error Handling

✅ CORRECT

1. 捕获错误信息 2. 分析错误原因 3. 提供解决方案 4. 引导用户使用 ram-permission-diagnose skill 处理权限问题

❌ INCORRECT

• 忽略错误继续执行
• 不提供解决建议
• 重复执行相同失败的操作

---

Feature Verification Checklist

MMS Core Features

• [ ] 支持的数据源类型识别 (Hive, BigQuery, Databricks, MaxCompute)
• [ ] 迁移作业类型说明 (整库、多表、多分区)
• [ ] 准备工作步骤完整
• [ ] 数据源创建流程清晰
• [ ] 迁移作业创建流程清晰
• [ ] 监控和验证方法明确

CLI Commands

• [ ] 所有 aliyun 命令包含 --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service
• [ ] 使用 plugin mode 格式 (如 list-projects 而非 ListProjects)
• [ ] 必要参数完整

Documentation

• [ ] RAM Policy 文档完整
• [ ] Related Commands 文档完整
• [ ] Verification Method 文档完整
• [ ] CLI Installation Guide 复制到 references 目录

Security

• [ ] 不暴露 AK/SK 值
• [ ] 使用 aliyun configure list 验证凭证
• [ ] RAM 权限列表完整

CLI Installation Guide

Complete guide for installing and configuring Aliyun CLI.

Aliyun CLI 3.3.1+: Supports installing and using all published Alibaba Cloud product plugins. Make sure to upgrade to 3.3.1 or later for full plugin ecosystem coverage.

Installation

macOS

Using Homebrew (Recommended)

brew install aliyun-cli
# Upgrade to latest
brew upgrade aliyun-cli

# Verify version (>= 3.3.1)
aliyun version

Using Binary

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz

# Extract
tar -xzf aliyun-cli-macosx-latest-amd64.tgz

# Move to PATH
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

Linux

Debian/Ubuntu

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

CentOS/RHEL

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

ARM64 Architecture

# Download ARM64 version
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-arm64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-arm64.tgz
sudo mv aliyun /usr/local/bin/

Windows

Using Binary 1. Download from: https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip 2. Extract the ZIP file 3. Add the directory to your PATH environment variable 4. Open new Command Prompt or PowerShell 5. Verify: aliyun version

Configuration

Quick Start

aliyun configure set \
  --mode AK \
  --access-key-id <your-access-key-id> \
  --access-key-secret <your-access-key-secret> \
  --region cn-hangzhou

Where to Get Access Keys

1. Log in to Aliyun Console: https://ram.console.aliyun.com/ 2. Navigate to: AccessKey Management 3. Create a new AccessKey pair 4. Save the secret immediately — it's only shown once

Environment Variables

Access Key Mode

export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_REGION_ID=cn-hangzhou

Verification

# Basic test - list regions
aliyun ecs describe-regions

# Expected output: JSON array of regions

Plugin Installation

After installing CLI 3.3.1+, enable automatic plugin installation:

aliyun configure set --auto-plugin-install true

Install specific product plugins:

aliyun plugin install --names maxcompute

RAM Policies for MaxCompute Migration Service (MMS)

本文档详细说明 MMS 所需的权限配置，包括 RAM 权限和 MaxCompute 项目权限。

权限配置概述

MMS 需要配置三类权限：

1. 服务关联角色

首次使用 MMS 前，必须创建服务关联角色：

角色名称：AliyunServiceRoleForMaxComputeMMS

创建方式：

通过 MaxCompute 控制台

1. 登录 MaxCompute 控制台 2. 选择 数据传输 > 迁移服务 3. 点击 新增数据源 4. 在弹出的对话框中确认创建

通过 RAM 控制台

1. 登录 RAM 控制台 > 身份管理 > 角色 2. 点击 创建角色 > 创建服务关联角色 3. 选择信任的云服务：AliyunServiceRoleForMaxComputeMMS

注意：RAM 用户需要 AliyunRAMFullAccess 权限才能创建服务关联角色

2. RAM 权限策略（给 RAM 用户）

2.1 MMS 所有操作权限

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "odps:ListMmsDataSources",
        "odps:CreateMmsDataSource",
        "ram:GetRole",
        "odps:GetMmsDataSource",
        "odps:UpdateMmsDataSource",
        "odps:DeleteMmsDataSource",
        "odps:CreateMmsFetchMetadataJob",
        "odps:GetMmsFetchMetadataJob",
        "odps:ListMmsFetchMetadataJobLogs",
        "odps:ListMmsDbs",
        "odps:GetMmsDb",
        "odps:ListMmsTables",
        "odps:GetMmsTable",
        "odps:ListMmsPartitions",
        "odps:GetMmsPartition",
        "odps:ListMmsJobs",
        "odps:GetMmsJob",
        "odps:CreateMmsJob",
        "odps:DeleteMmsJob",
        "odps:StartMmsJob",
        "odps:StopMmsJob",
        "odps:RetryMmsJob",
        "odps:ListMmsTasks",
        "odps:GetMmsTask",
        "odps:ListMmsTaskLogs",
        "odps:StopMmsTask",
        "odps:StartMmsTask",
        "odps:RetryMmsTask",
        "odps:GetMmsAsyncTask",
        "odps:GetMmsProgress",
        "odps:GetMmsSpeed",
        "odps:CreateMmsAuthFile",
        "odps:ListMmsAgents",
        "odps:ListMmsTimers",
        "odps:GetMmsTimer",
        "odps:UpdateMmsTimer",
        "odps:ListMmsTimerLogs",
        "odps:CreateMmsTimer",
        "odps:UpdateMmsTables",
        "odps:UpdateMmsTable",
        "odps:UpdateMmsDb",
        "odps:ListNetworkLinks"
      ],
      "Resource": "*"
    }
  ]
}

2.2 MMS 源数据管理权限

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "odps:ListMmsDataSources",
        "odps:CreateMmsDataSource",
        "ram:GetRole",
        "odps:GetMmsDataSource",
        "odps:UpdateMmsDataSource",
        "odps:DeleteMmsDataSource",
        "odps:CreateMmsFetchMetadataJob",
        "odps:GetMmsFetchMetadataJob",
        "odps:ListMmsFetchMetadataJobLogs",
        "odps:ListMmsDbs",
        "odps:GetMmsDb",
        "odps:ListMmsTables",
        "odps:GetMmsTable",
        "odps:ListMmsPartitions",
        "odps:GetMmsPartition",
        "odps:GetMmsAsyncTask",
        "odps:GetMmsProgress",
        "odps:GetMmsSpeed",
        "odps:CreateMmsAuthFile",
        "odps:ListMmsAgents",
        "odps:UpdateMmsTables",
        "odps:UpdateMmsTable",
        "odps:UpdateMmsDb"
      ],
      "Resource": "*"
    }
  ]
}

2.3 MMS 迁移作业管理权限

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "odps:ListMmsDataSources",
        "odps:GetMmsDataSource",
        "odps:CreateMmsFetchMetadataJob",
        "odps:GetMmsFetchMetadataJob",
        "odps:ListMmsFetchMetadataJobLogs",
        "odps:ListMmsDbs",
        "odps:GetMmsDb",
        "odps:ListMmsTables",
        "odps:GetMmsTable",
        "odps:ListMmsPartitions",
        "odps:GetMmsPartition",
        "odps:ListMmsJobs",
        "odps:GetMmsJob",
        "odps:CreateMmsJob",
        "odps:DeleteMmsJob",
        "odps:StartMmsJob",
        "odps:StopMmsJob",
        "odps:RetryMmsJob",
        "odps:ListMmsTasks",
        "odps:GetMmsTask",
        "odps:ListMmsTaskLogs",
        "odps:StopMmsTask",
        "odps:StartMmsTask",
        "odps:RetryMmsTask",
        "odps:GetMmsAsyncTask",
        "odps:GetMmsProgress",
        "odps:GetMmsSpeed",
        "odps:ListMmsTimers",
        "odps:GetMmsTimer",
        "odps:UpdateMmsTimer",
        "odps:ListMmsTimerLogs",
        "odps:CreateMmsTimer"
      ],
      "Resource": "*"
    }
  ]
}

3. MaxCompute 项目权限（给服务关联角色）

在目标项目中，需要为服务关联角色授予数据操作权限：

3.1 添加服务关联角色到项目

USE <target_project>;
ADD USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

3.2 粗粒度授权（推荐）

-- 授予 admin 角色
GRANT admin TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

3.3 细粒度授权

项目级权限：

GRANT Read,Write,List,CreateTable,CreateInstance,CreateFunction,CreateResource
ON project <project_name> TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

-- 或授予所有权限
GRANT ALL ON project <project_name> TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

表级权限：

GRANT Describe,Select,Alter,Update,Drop,ShowHistory
ON table <table_name> TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

-- 或授予所有权限
GRANT All ON table <table_name> TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

实例级权限：

GRANT Read,Write ON instance <instance_id>
TO USER `RAM$<account_id>:role/AliyunServiceRoleForMaxComputeMMS`;

3.4 权限说明

4. 快捷授权方案

5. 通过控制台配置权限

MaxCompute 控制台配置项目权限

1. 登录 MaxCompute 控制台 2. 选择 管理配置 > 项目管理 3. 点击目标项目的 管理 4. 选择 角色权限 页签 5. 新建角色并配置权限 6. 在成员管理中添加服务关联角色

6. 权限排查

常见错误

权限检查清单

• [ ] 服务关联角色已创建
• [ ] RAM 用户已授予 MMS 操作权限
• [ ] 服务关联角色已添加到目标项目
• [ ] 服务关联角色已授予数据操作权限
• [ ] 目标项目已绑定 Quota 资源

MMS OpenAPI & CLI Commands

MMS 通过 MaxCompute OpenAPI（产品代码 MaxCompute，版本 2022-01-04）提供 API 能力。 CLI 调用格式：aliyun maxcompute <command> [params] --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

数据源管理

ListMmsDataSources — 列出数据源

aliyun maxcompute list-mms-data-sources \
  --name <name> \
  --type <Hive|BigQuery|Snowflake|Redshift|Databricks|MaxCompute> \
  --region <region> \
  --page-num 1 --page-size 20 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

GetMmsDataSource — 获取数据源详情

aliyun maxcompute get-mms-data-source \
  --source-id <sourceId> \
  --with-config true \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

创建/更新/删除数据源

[MUST] 数据源的创建、更新、删除操作，引导用户到控制台完成。

控制台地址：https://maxcompute.console.aliyun.com/{region}/mma/datasource

元数据扫描（盘点）

CreateMmsFetchMetadataJob — 发起元数据盘点

aliyun maxcompute create-mms-fetch-metadata-job \
  --source-id <sourceId> \
  --body '{ ... }' \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

GetMmsFetchMetadataJob — 查看盘点状态

aliyun maxcompute get-mms-fetch-metadata-job \
  --source-id <sourceId> \
  --scan-id <scanId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

元数据管理

ListMmsDbs — 列出数据库

aliyun maxcompute list-mms-dbs \
  --source-id <sourceId> \
  --name <name> \
  --status <status> \
  --page-num 1 --page-size 20 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

GetMmsDb — 获取数据库详情

aliyun maxcompute get-mms-db \
  --source-id <sourceId> \
  --db-id <dbId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

ListMmsTables — 列出表

aliyun maxcompute list-mms-tables \
  --source-id <sourceId> \
  --db-name <dbName> \
  --name <name> \
  --status <status> \
  --has-partitions true \
  --page-num 1 --page-size 20 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

GetMmsTable — 获取表详情

aliyun maxcompute get-mms-table \
  --source-id <sourceId> \
  --table-id <tableId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

ListMmsPartitions — 列出分区

aliyun maxcompute list-mms-partitions \
  --source-id <sourceId> \
  --db-name <dbName> \
  --table-name <tableName> \
  --page-num 1 --page-size 20 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

GetMmsPartition — 获取分区详情

aliyun maxcompute get-mms-partition \
  --source-id <sourceId> \
  --partition-id <partitionId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

迁移作业管理

CreateMmsJob — 创建迁移作业

aliyun maxcompute create-mms-job \
  --source-id <sourceId> \
  --body '{ "name": "job_name", "srcDbName": "db_name", ... }' \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

创建后自动开始执行，无需手动调用 StartMmsJob。

ListMmsJobs — 列出迁移作业

aliyun maxcompute list-mms-jobs \
  --source-id <sourceId> \
  --name <name> \
  --src-db-name <srcDbName> \
  --status <status> \
  --page-num 1 --page-size 20 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

GetMmsJob — 获取迁移作业详情

aliyun maxcompute get-mms-job \
  --source-id <sourceId> \
  --job-id <jobId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

StartMmsJob — 启动/恢复迁移作业

aliyun maxcompute start-mms-job \
  --source-id <sourceId> \
  --job-id <jobId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

仅用于恢复被 StopMmsJob 停止的作业。

StopMmsJob — 停止迁移作业

aliyun maxcompute stop-mms-job \
  --source-id <sourceId> \
  --job-id <jobId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

RetryMmsJob — 重试迁移作业

aliyun maxcompute retry-mms-job \
  --source-id <sourceId> \
  --job-id <jobId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

DeleteMmsJob — 删除迁移作业

aliyun maxcompute delete-mms-job \
  --source-id <sourceId> \
  --job-id <jobId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

迁移任务管理

ListMmsTasks — 列出迁移任务

aliyun maxcompute list-mms-tasks \
  --source-id <sourceId> \
  --job-id <jobId> \
  --status <status> \
  --src-table-name <srcTableName> \
  --page-num 1 --page-size 20 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

GetMmsTask — 获取迁移任务详情

aliyun maxcompute get-mms-task \
  --source-id <sourceId> \
  --task-id <taskId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

ListMmsTaskLogs — 查看迁移任务日志

aliyun maxcompute list-mms-task-logs \
  --source-id <sourceId> \
  --task-id <taskId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

异步任务

GetMmsAsyncTask — 查看异步任务状态

aliyun maxcompute get-mms-async-task \
  --source-id <sourceId> \
  --async-task-id <asyncTaskId> \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-maxcompute-migration-service

API 概览

Verification Method for MaxCompute Migration Service (MMS)

This document provides steps to verify the success of MMS migration operations.

Migration Job Verification

Step 1: Check Migration Job Status

通过 MaxCompute 控制台验证：

1. 登录 MaxCompute 控制台 2. 选择地域，进入 数据传输 > 迁移服务 > 迁移作业 3. 查看作业状态：

• 运行中: 作业正在执行
• 成功: 作业执行完成
• 失败: 作业执行失败，查看错误信息

Step 2: Verify Data Count

验证迁移后的数据条数：

-- 在 MaxCompute 项目中执行
SELECT COUNT(*) FROM <target_table>;

与源端数据条数对比：

-- 在源数据源执行（如 Hive）
SELECT COUNT(*) FROM <source_table>;

Step 3: Verify Data Sample

抽样验证数据内容：

-- 在 MaxCompute 项目中执行
SELECT * FROM <target_table> LIMIT 10;

检查数据格式、字段值是否正确。

Step 4: Verify Partition Data (if applicable)

验证分区数据：

-- 查看分区列表
SHOW PARTITIONS <target_table>;

-- 验证分区数据量
SELECT COUNT(*) FROM <target_table> WHERE <partition_column> = '<partition_value>';

Step 5: Check Data Validation Results

如开启了数据校验，查看校验结果：

1. 进入 迁移服务 > 迁移作业 2. 点击作业名称，查看任务详情 3. 查看 任务日志 中的校验结果

校验方法：比对源端和目标端的 SELECT COUNT(*) 结果。

Verification Commands Summary

Troubleshooting

Migration Job Failed

1. 查看错误日志，定位失败原因 2. 常见问题：

• 网络不通：检查网络配置
• 权限不足：检查 RAM 权限
• 源端不可用：检查数据源状态
• 资源不足：检查 MaxCompute CU 资源

Data Count Mismatch

1. 检查是否有迁移任务失败 2. 检查分区过滤条件是否正确 3. 检查源端是否有数据变更 4. 重新执行数据校验

Data Format Error

1. 检查字段类型映射是否正确 2. 检查字符编码设置 3. 检查数据源配置