Alibabacloud Pts Ops

Alibaba Cloud PTS Stress Testing Scenario Management

This skill enables you to create and manage stress testing scenarios using Alibaba Cloud PTS (Performance Testing Service). It supports both PTS native HTTP/HTTPS stress testing and JMeter-based stress testing.

Scenario Description

PTS (Performance Testing Service) is Alibaba Cloud's fully managed performance testing platform that helps you validate the performance, capacity, and stability of your applications. This skill covers:

1. PTS Native Stress Testing - Create HTTP/HTTPS stress testing scenarios with configurable APIs, serial links, and load models 2. JMeter Stress Testing - Upload and run JMeter scripts with distributed load generation

Architecture

User → Aliyun CLI → PTS Service → Target Application
                         ↓
                   Stress Testing Report

Pre-check

Pre-check: Aliyun CLI >= 3.3.1 required

Run aliyun version to verify >= 3.3.1. If not installed or version too low,

see references/cli-installation-guide.md for installation instructions.

Then [MUST] run aliyun configure set --auto-plugin-install true to enable automatic plugin installation.

# Verify CLI version
aliyun version

# Enable auto plugin installation
aliyun configure set --auto-plugin-install true

Timeout Settings

All CLI commands should include timeout parameters to avoid hanging:

# Recommended timeout settings for PTS operations
--read-timeout 60 --connect-timeout 10

• read-timeout: 60 seconds (stress testing operations may take longer)
• connect-timeout: 10 seconds

Environment Variables

No additional environment variables required beyond CLI authentication.

Parameter Confirmation

IMPORTANT: Parameter Confirmation — Before executing any command or API call,

ALL user-customizable parameters (e.g., RegionId, scene names, target URLs,

concurrency, duration, JMX files, etc.) MUST be confirmed with the user.

Do NOT assume or use default values without explicit user approval.

User-Customizable Parameters

Authentication

This skill relies on the Aliyun CLI's default credential chain. Ensure your CLI is already authenticated before use.

Verify current authentication:

aliyun configure get

If CLI is not yet configured, see references/cli-installation-guide.md for setup instructions.

RAM Policy

Users must have appropriate PTS permissions. See references/ram-policies.md for detailed policies.

Idempotency

PTS APIs do not support ClientToken-based idempotency. Scene names are not unique — multiple PTS or JMeter scenarios may share the same SceneName. Never treat “same name” as one resource; always use `SceneId` (returned by the API) as the stable identifier.

To prevent duplicate resources or unintended side-effects when retrying after timeouts or errors, always use the check-then-act pattern before every write operation:

Core Workflow

IMPORTANT: Parameter Confirmation — Before executing any command or API call,

ALL user-customizable parameters (e.g., RegionId, scene names, target URLs,

concurrency, duration, etc.) MUST be confirmed with the user.

Do NOT assume or use default values without explicit user approval.

Workflow 1: Create and Run PTS Native Stress Testing

Task 1.1: Create PTS Scenario

Note: Use save-pts-scene instead of create-pts-scene. The --scene parameter accepts a JSON object directly (not wrapped in a Scene field).

Idempotency: SceneName may duplicate across scenarios. Do not skip creation or pick a

scene based on name alone. After save-pts-scene succeeds, record the returned `SceneId` for

all later steps. If the command fails or times out with unknown outcome, use list-pts-scene

together with the user to identify the intended SceneId before retrying — avoid blind retries

that create extra scenes.

aliyun pts save-pts-scene \
  --scene '{
    "SceneName": "<SCENE_NAME>",
    "RelationList": [
      {
        "RelationName": "serial-link-1",
        "ApiList": [
          {
            "ApiName": "api-1",
            "Url": "<TARGET_URL>",
            "Method": "<HTTP_METHOD>",
            "TimeoutInSecond": 10,
            "RedirectCountLimit": 10,
            "HeaderList": [
              {
                "HeaderName": "User-Agent",
                "HeaderValue": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
              }
            ],
            "CheckPointList": [
              {
                "CheckPoint": "",
                "CheckType": "STATUS_CODE",
                "Operator": "eq",
                "ExpectValue": "200"
              }
            ]
          }
        ]
      }
    ],
    "LoadConfig": {
      "TestMode": "concurrency_mode",
      "MaxRunningTime": <DURATION_MINUTES>,
      "AutoStep": false,
      "Configuration": {
        "AllConcurrencyBegin": <CONCURRENCY>,
        "AllConcurrencyLimit": <CONCURRENCY>
      }
    },
    "AdvanceSetting": {
      "LogRate": 1,
      "ConnectionTimeoutInSecond": 5
    }
  }' \
  --user-agent AlibabaCloud-Agent-Skills

Parameter Notes:

• MaxRunningTime: Duration in minutes (not seconds), range [1-1440]
• TestMode: Use concurrency_mode for concurrent user testing or tps_mode for RPS testing
• TimeoutInSecond: Request timeout in seconds (recommended: 10)
• RedirectCountLimit: Maximum redirects allowed (use 10 for normal, 0 to disable)
• HeaderList: HTTP headers, User-Agent is recommended for better compatibility
• CheckPointList: Assertions for response validation (STATUS_CODE, BODY_JSON, etc.)
• AdvanceSetting.LogRate: Log sampling rate (1-100)
• AdvanceSetting.ConnectionTimeoutInSecond: Connection timeout (recommended: 5)

For complete JSON structure with all fields (POST requests, file parameters, global variables, etc.), see references/pts-scene-json-reference.md

Task 1.2: Start Stress Testing

[MUST] Pre-flight Safety Checks — Starting a stress test sends significant traffic to the

target system. ALL of the following checks MUST pass before executing start-pts-scene:

>

1. Idempotency guard — Run get-pts-scene-running-status --scene-id <SCENE_ID>.

If the status is RUNNING or SYNCING, the test is already in progress — skip the start

command and proceed to monitoring. Do NOT start a duplicate test.

2. Retrieve and verify scene configuration — Run get-pts-scene --scene-id <SCENE_ID> and

confirm the response contains a valid SceneName, at least one RelationList entry with a

non-empty Url, and a valid LoadConfig (non-zero MaxRunningTime and concurrency).

If any field is missing or empty, abort and notify the user.

3. Display test summary and require explicit user confirmation — Present the following to

the user and wait for explicit approval (e.g., "yes" / "确认"):

- Target URL(s)

- Concurrency level

- Test duration

- Test mode (concurrency / TPS)

>

Do NOT proceed without the user's explicit "go-ahead" confirmation.

# Idempotency guard: Skip if test is already running
aliyun pts get-pts-scene-running-status \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills
# ↑ If status is RUNNING or SYNCING, skip start-pts-scene and go to monitoring.

# Pre-flight check: Verify scene configuration is complete
aliyun pts get-pts-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

# Start stress testing (only after all checks pass and user confirms)
aliyun pts start-pts-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Task 1.3: Monitor Testing Status

aliyun pts get-pts-scene-running-status \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Task 1.4: Get Testing Report

aliyun pts get-pts-report-details \
  --scene-id <SCENE_ID> \
  --plan-id <PLAN_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Workflow 2: Create and Run JMeter Stress Testing

Task 2.1: Create JMeter Scenario

Idempotency: SceneName may duplicate across JMeter scenarios. Do not dedupe by name.

After save-open-jmeter-scene succeeds, record the returned `SceneId`. On uncertain

failure, use list-open-jmeter-scenes with the user to disambiguate before retrying.

aliyun pts save-open-jmeter-scene \
  --open-jmeter-scene '{
    "SceneName": "<SCENE_NAME>",
    "TestFile": "<JMX_FILENAME>",
    "Duration": <DURATION>,
    "Concurrency": <CONCURRENCY>,
    "Mode": "CONCURRENCY"
  }' \
  --user-agent AlibabaCloud-Agent-Skills

Task 2.2: Start JMeter Testing

[MUST] Pre-flight Safety Checks — Starting a JMeter stress test sends significant traffic

to the target system. ALL of the following checks MUST pass before executing

start-testing-jmeter-scene:

>

1. Idempotency guard — Run get-open-jmeter-scene --scene-id <SCENE_ID> and check the

scene status. If the test is already running, skip the start command and proceed to

monitoring. Do NOT start a duplicate test.

2. Verify scene configuration — From the same response, confirm it contains a valid

SceneName, a non-empty TestFile, and non-zero Duration and Concurrency.

If any field is missing or empty, abort and notify the user.

3. Display test summary and require explicit user confirmation — Present the following to

the user and wait for explicit approval (e.g., "yes" / "确认"):

- Scene name and JMX file

- Concurrency level

- Test duration

>

Do NOT proceed without the user's explicit "go-ahead" confirmation.

# Idempotency guard + pre-flight check: Verify scene config and check if already running
aliyun pts get-open-jmeter-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills
# ↑ If already running, skip start command. If config is incomplete, abort.

# Start JMeter testing (only after all checks pass and user confirms)
aliyun pts start-testing-jmeter-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Task 2.3: Get JMeter Report

aliyun pts get-jmeter-report-details \
  --report-id <REPORT_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Workflow 3: Manage Scenarios

Task 3.1: List All PTS Scenarios

aliyun pts list-pts-scene \
  --page-number 1 \
  --page-size 10 \
  --user-agent AlibabaCloud-Agent-Skills

Task 3.2: List All JMeter Scenarios

aliyun pts list-open-jmeter-scenes \
  --page-number 1 \
  --page-size 10 \
  --user-agent AlibabaCloud-Agent-Skills

Task 3.3: Get Scenario Details

# PTS scenario
aliyun pts get-pts-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

# JMeter scenario
aliyun pts get-open-jmeter-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Task 3.4: Debug Scenario (PTS only)

aliyun pts start-debug-pts-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Task 3.5: Stop Running Test

# Stop PTS test
aliyun pts stop-pts-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

# Stop JMeter test
aliyun pts stop-testing-jmeter-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Success Verification Method

IMPORTANT: start-pts-scene may return Success: true even when the stress test fails to actually launch (e.g., due to target site protection or missing configuration). Always verify actual execution status.

After each operation, verify success using the verification commands in references/verification-method.md.

Verify scenario creation:

# Use list-pts-scene instead of get-pts-scene (more reliable)
aliyun pts list-pts-scene \
  --page-number 1 \
  --page-size 10 \
  --user-agent AlibabaCloud-Agent-Skills

Verify stress test is actually running:

# Check running status first
aliyun pts get-pts-scene-running-status \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

# Then verify with running data (requires plan-id from start-pts-scene)
aliyun pts get-pts-scene-running-data \
  --scene-id <SCENE_ID> \
  --plan-id <PLAN_ID> \
  --user-agent AlibabaCloud-Agent-Skills

Key indicators of successful execution:

• Status: Should be "RUNNING" or "SYNCING" (not "STOPPED" immediately)
• AliveAgents: Should be > 0
• Concurrency: Should match configured value
• TotalRequestCount: Should be increasing

Cleanup

Delete scenarios when no longer needed.

[MUST] Pre-delete Safety Checks — Before deleting any scenario, ALL of the following

checks MUST pass:

>

1. Idempotency guard — Using the target `SceneId` (not name), verify it still exists

(e.g. list-pts-scene / list-open-jmeter-scenes or get-*). If that SceneId is absent,

treat deletion as already done and skip the delete command.

2. Check if the scenario is currently running — Run

get-pts-scene-running-status --scene-id <SCENE_ID> (PTS) or check JMeter scene status.

If the scenario status is RUNNING or SYNCING, you MUST stop it first using

stop-pts-scene / stop-testing-jmeter-scene and wait for it to fully stop before deleting.

Do NOT delete a running scenario.

3. Require explicit user confirmation — Display the scene name and ID to the user and

ask for explicit deletion confirmation (e.g., "yes" / "确认删除"). Do NOT proceed without

the user's explicit approval.

# Pre-delete check: Verify scenario is not running
aliyun pts get-pts-scene-running-status \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

# Delete PTS scenario (only after confirming it is not running and user approves)
aliyun pts delete-pts-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

# Delete JMeter scenario (only after confirming it is not running and user approves)
aliyun pts remove-open-jmeter-scene \
  --scene-id <SCENE_ID> \
  --user-agent AlibabaCloud-Agent-Skills

API and Command Tables

See references/related-apis.md for complete API and CLI command reference.

Best Practices

1. Use complete scene configuration - Always include TimeoutInSecond, HeaderList (with User-Agent), CheckPointList, and AdvanceSetting for reliable test execution 2. Always confirm parameters - Verify target URLs, concurrency settings, and duration with the user before execution 3. Start with low concurrency - Begin with low concurrency and gradually increase to identify performance thresholds 4. Verify actual execution - Don't trust Success: true from start-pts-scene; always check get-pts-scene-running-data with --plan-id 5. Use debug mode first - For PTS scenarios, use start-debug-pts-scene to validate configuration before full tests 6. Monitor during tests - Regularly check running status during stress tests 7. Review reports thoroughly - Analyze response times, error rates, and throughput in reports 8. Clean up after testing - Delete test scenarios to avoid unnecessary costs 9. Use appropriate test duration - Longer tests provide more accurate results but consume more resources 10. Include warmup period - Allow time for systems to warm up before measuring peak performance

Reference Links

Acceptance Criteria: PTS Stress Testing Scenario Skill

Scenario: PTS Performance Testing Service - Create and Manage Stress Testing Scenarios Purpose: Skill testing acceptance criteria

---

Correct CLI Command Patterns

1. Product — verify product name exists

✅ CORRECT

aliyun pts <action>

The product name pts is correct for PTS (Performance Testing Service).

❌ INCORRECT

aliyun PTS <action>     # Wrong: uppercase
aliyun performance <action>  # Wrong: not the product code

2. Command — verify action exists under the product

✅ CORRECT PTS Native Commands

aliyun pts create-pts-scene
aliyun pts get-pts-scene
aliyun pts list-pts-scene
aliyun pts start-pts-scene
aliyun pts stop-pts-scene
aliyun pts delete-pts-scene
aliyun pts start-debug-pts-scene
aliyun pts get-pts-report-details

✅ CORRECT JMeter Commands

aliyun pts save-open-jmeter-scene
aliyun pts get-open-jmeter-scene
aliyun pts list-open-jmeter-scenes
aliyun pts start-testing-jmeter-scene
aliyun pts stop-testing-jmeter-scene
aliyun pts remove-open-jmeter-scene
aliyun pts get-jmeter-report-details

❌ INCORRECT

aliyun pts CreatePtsScene        # Wrong: PascalCase API style
aliyun pts create-scene          # Wrong: missing 'pts' prefix
aliyun pts createPtsScene        # Wrong: camelCase

3. Parameters — verify each parameter name exists for the command

✅ CORRECT Parameter Names

# For create-pts-scene
aliyun pts create-pts-scene --scene '...'

# For get-pts-scene
aliyun pts get-pts-scene --scene-id <id>

# For list-pts-scene
aliyun pts list-pts-scene --page-number 1 --page-size 10

# For start-pts-scene
aliyun pts start-pts-scene --scene-id <id>

# For save-open-jmeter-scene
aliyun pts save-open-jmeter-scene --open-jmeter-scene '...'

# For get-jmeter-report-details
aliyun pts get-jmeter-report-details --report-id <id>

❌ INCORRECT Parameter Names

aliyun pts get-pts-scene --sceneId <id>     # Wrong: camelCase
aliyun pts get-pts-scene --SceneId <id>     # Wrong: PascalCase
aliyun pts list-pts-scene --pageNumber 1    # Wrong: camelCase

4. User-Agent Flag — must be present in every command

✅ CORRECT

aliyun pts list-pts-scene \
  --page-number 1 \
  --page-size 10 \
  --user-agent AlibabaCloud-Agent-Skills

❌ INCORRECT

# Missing --user-agent flag
aliyun pts list-pts-scene --page-number 1 --page-size 10

5. JSON Parameter Format — verify complex parameters use proper JSON

✅ CORRECT JSON Format for PTS Scene

aliyun pts create-pts-scene \
  --scene '{"name":"test-scene","type":"HTTP","requests":[{"url":"https://example.com","method":"GET"}]}' \
  --user-agent AlibabaCloud-Agent-Skills

✅ CORRECT JSON Format for JMeter Scene

aliyun pts save-open-jmeter-scene \
  --open-jmeter-scene '{"scene_name":"MyJMeterTest","test_file":"example.jmx","duration":300,"concurrency":100,"mode":"CONCURRENCY"}' \
  --user-agent AlibabaCloud-Agent-Skills

❌ INCORRECT JSON Format

# Wrong: unquoted strings
aliyun pts create-pts-scene --scene {name:test-scene}

# Wrong: double quotes not properly escaped
aliyun pts create-pts-scene --scene "{"name":"test"}"

---

Parameter Confirmation Requirements

Required User Confirmation Parameters

The following parameters MUST be confirmed with the user before execution:

✅ CORRECT: Parameter Confirmation Flow

1. List all required parameters 2. Ask user to confirm or provide values 3. Show preview of command before execution 4. Execute only after user approval

❌ INCORRECT: Hardcoding Values

# Wrong: hardcoded URL without user confirmation
aliyun pts create-pts-scene \
  --scene '{"name":"test","requests":[{"url":"https://example.com"}]}' \
  --user-agent AlibabaCloud-Agent-Skills

---

Success Verification Patterns

✅ CORRECT Verification Pattern

After each operation, verify success by checking the result:

# 1. Create scenario
SCENE_RESULT=$(aliyun pts create-pts-scene --scene '...' --user-agent AlibabaCloud-Agent-Skills)

# 2. Extract scene ID
SCENE_ID=$(echo $SCENE_RESULT | jq -r '.SceneId')

# 3. Verify creation
aliyun pts get-pts-scene --scene-id $SCENE_ID --user-agent AlibabaCloud-Agent-Skills

❌ INCORRECT: No Verification

# Wrong: no verification after creation
aliyun pts create-pts-scene --scene '...' --user-agent AlibabaCloud-Agent-Skills
# Immediately proceeding without checking if creation succeeded

---

Error Handling Patterns

✅ CORRECT Error Handling

Check command exit status and handle errors:

if ! aliyun pts get-pts-scene --scene-id $SCENE_ID --user-agent AlibabaCloud-Agent-Skills; then
  echo "Error: Failed to get scene details"
  exit 1
fi

❌ INCORRECT: Ignoring Errors

# Wrong: ignoring potential errors
aliyun pts start-pts-scene --scene-id $SCENE_ID --user-agent AlibabaCloud-Agent-Skills
aliyun pts get-pts-report-details --scene-id $SCENE_ID --plan-id $PLAN_ID --user-agent AlibabaCloud-Agent-Skills

---

Cleanup Patterns

✅ CORRECT Cleanup

Always provide cleanup commands after examples:

# Cleanup: Delete the test scenario
aliyun pts delete-pts-scene \
  --scene-id $SCENE_ID \
  --user-agent AlibabaCloud-Agent-Skills

# For JMeter scenarios
aliyun pts remove-open-jmeter-scene \
  --scene-id $JMETER_SCENE_ID \
  --user-agent AlibabaCloud-Agent-Skills

❌ INCORRECT: No Cleanup

Examples without cleanup commands may leave orphaned resources.

---

Version Requirements

✅ CORRECT Version Check

# Must check CLI version >= 3.3.1
CLI_VERSION=$(aliyun version | head -1)
if [[ "$CLI_VERSION" < "3.3.1" ]]; then
  echo "Please upgrade aliyun CLI to version 3.3.1 or later"
  exit 1
fi

❌ INCORRECT: No Version Check

Running commands without verifying CLI version may lead to compatibility issues.

Aliyun CLI Installation & Configuration Guide

Complete guide for installing and configuring Aliyun CLI.

Aliyun CLI 3.3.1+: Supports installing and using all published Alibaba Cloud product plugins. Make sure to upgrade to 3.3.1 or later for full plugin ecosystem coverage.

Installation

macOS

Using Homebrew (Recommended)

brew install aliyun-cli
# Upgrade to latest
brew upgrade aliyun-cli

# Verify version (>= 3.3.1)
aliyun version

Using Binary

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz

# Extract
tar -xzf aliyun-cli-macosx-latest-amd64.tgz

# Move to PATH
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

Linux

Debian/Ubuntu

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

CentOS/RHEL

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

ARM64 Architecture

# Download ARM64 version
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-arm64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-arm64.tgz
sudo mv aliyun /usr/local/bin/

Windows

Using Binary 1. Download from: https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip 2. Extract the ZIP file 3. Add the directory to your PATH environment variable 4. Open new Command Prompt or PowerShell 5. Verify: aliyun version

Using PowerShell

# Download
Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip"

# Extract
Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli

# Add to PATH (requires admin privileges)
$env:Path += ";C:\aliyun-cli"
[Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::Machine)

# Verify
aliyun version

Configuration

Default Credential Chain

The Aliyun CLI uses a default credential chain to resolve authentication automatically. This skill relies on the default credential chain — do not configure credentials explicitly.

Verify your CLI is authenticated:

aliyun configure get

All aliyun configure commands support non-interactive flags, which is the recommended approach — it works in scripts, CI/CD pipelines, and agent-driven automation without hanging on stdin prompts.

Supported Authentication Modes

Aliyun CLI supports 6 authentication modes. Use aliyun configure --mode <MODE> to set up:

Refer to the official Aliyun CLI documentation for mode-specific setup instructions.

Recommended: EcsRamRole Mode

When running on ECS instances, use EcsRamRole mode for credential-free authentication:

aliyun configure set \
  --mode EcsRamRole \
  --ram-role-name <ROLE_NAME> \
  --region cn-hangzhou

Requirements: must be running on an ECS instance with a RAM role attached.

Managing Multiple Profiles

# List all profiles
aliyun configure list

# Switch default profile
aliyun configure set --current <PROFILE_NAME>

# Use specific profile for a command
aliyun ecs describe-instances --profile <PROFILE_NAME>

Credential Priority

Credentials are loaded in this order (first found wins):

1. Command-line flag: --profile <name> 2. Environment variable: ALIBABA_CLOUD_PROFILE 3. Credentials file: ~/.aliyun/config.json (current profile) 4. ECS Instance RAM Role: If running on ECS with attached role

Verification

Test Authentication

# Basic test - list regions
aliyun ecs describe-regions

# Expected output: JSON array of regions

If successful, you'll see:

{
  "Regions": {
    "Region": [
      {
        "RegionId": "cn-hangzhou",
        "RegionEndpoint": "ecs.cn-hangzhou.aliyuncs.com",
        "LocalName": "华东 1（杭州）"
      },
      ...
    ]
  },
  "RequestId": "..."
}

If failed, you'll see error messages:

• InvalidAccessKeyId.NotFound - Wrong Access Key ID
• SignatureDoesNotMatch - Wrong Access Key Secret
• InvalidSecurityToken.Expired - STS token expired (for StsToken mode)
• Forbidden.RAM - Insufficient permissions

Debug Configuration

# Show current configuration
aliyun configure get

# Test with debug logging
aliyun ecs describe-regions --log-level=debug

# Check credential provider
aliyun configure get mode

Security Best Practices

1. Use RAM Users (Not Root Account)

❌ Don't: Use Aliyun root account credentials ✅ Do: Create RAM users with specific permissions in the RAM console

2. Principle of Least Privilege

Grant only the minimum permissions needed. Attach managed policies when possible (e.g., AliyunECSReadOnlyAccess).

3. Use ECS RAM Roles When Possible

Prefer credential-free authentication via ECS instance RAM roles:

aliyun configure set --mode EcsRamRole --ram-role-name <ROLE_NAME> --region cn-hangzhou

4. Use Temporary Credentials for Short-Lived Access

For CI/CD and automation, prefer STS temporary credentials or RAM role assumption over long-lived keys.

5. Never Commit Credentials

# Add to .gitignore
echo "~/.aliyun/config.json" >> .gitignore

# Use environment variables in CI/CD instead

6. Secure Config File

# Restrict permissions
chmod 600 ~/.aliyun/config.json

Troubleshooting

Issue: Command Not Found

# Check installation
which aliyun

# Check PATH
echo $PATH

# Reinstall or add to PATH

Issue: Authentication Failed

# Verify configuration
aliyun configure get

# Test with debug
aliyun ecs describe-regions --log-level=debug

Issue: Permission Denied

# Error: Forbidden.RAM

# Check RAM user permissions
# Attach necessary policies in RAM console
# Example: AliyunECSFullAccess for ECS operations

Issue: STS Token Expired

# Error: InvalidSecurityToken.Expired
# Reconfigure with a new STS token using: aliyun configure --mode StsToken

Issue: Wrong Region

# Some resources may not exist in the specified region

# Check available regions
aliyun ecs describe-regions

# Update default region
aliyun configure set region cn-shanghai

Advanced Configuration

Custom Endpoint

# Use custom or private endpoint
export ALIBABA_CLOUD_ECS_ENDPOINT=ecs-vpc.cn-hangzhou.aliyuncs.com

Proxy Settings

# HTTP proxy
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080

# No proxy for specific domains
export NO_PROXY=localhost,127.0.0.1,.aliyuncs.com

Timeout Settings

# Connection timeout (default: 10s)
export ALIBABA_CLOUD_CONNECT_TIMEOUT=30

# Read timeout (default: 10s)
export ALIBABA_CLOUD_READ_TIMEOUT=30

Next Steps

After installation and configuration:

1. Install plugins for services you need (v3.3.1+ supports all published product plugins):

   aliyun plugin install --names ecs vpc rds

   # List all available plugins
   aliyun plugin list-remote

2. Explore commands:

   aliyun ecs --help
   aliyun fc --help

3. Read documentation:

• Command Syntax Guide
• Global Flags Reference
• Common Scenarios

References

• Official Documentation: https://help.aliyun.com/zh/cli/
• RAM Console: https://ram.console.aliyun.com/
• Access Key Management: https://ram.console.aliyun.com/manage/ak
• Plugin Repository: https://github.com/aliyun/aliyun-cli

PTS Scene JSON Reference

This document provides complete JSON structure reference for creating PTS stress testing scenarios.

Basic Scene Structure (GET Request)

Minimum required fields for a working PTS scene:

{
  "SceneName": "<SCENE_NAME>",
  "RelationList": [
    {
      "RelationName": "serial-link-1",
      "ApiList": [
        {
          "ApiName": "api-1",
          "Url": "<TARGET_URL>",
          "Method": "GET",
          "TimeoutInSecond": 10,
          "RedirectCountLimit": 10,
          "HeaderList": [
            {
              "HeaderName": "User-Agent",
              "HeaderValue": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
            }
          ],
          "CheckPointList": [
            {
              "CheckPoint": "",
              "CheckType": "STATUS_CODE",
              "Operator": "eq",
              "ExpectValue": "200"
            }
          ]
        }
      ]
    }
  ],
  "LoadConfig": {
    "TestMode": "concurrency_mode",
    "MaxRunningTime": 1,
    "AutoStep": false,
    "Configuration": {
      "AllConcurrencyBegin": 10,
      "AllConcurrencyLimit": 10
    }
  },
  "AdvanceSetting": {
    "LogRate": 1,
    "ConnectionTimeoutInSecond": 5
  }
}

Complete Scene Structure (All Fields)

For advanced scenarios with POST requests, file parameters, and global variables:

{
  "SceneName": "my-stress-test",
  "RelationList": [
    {
      "RelationName": "user-flow",
      "ApiList": [
        {
          "ApiName": "login-api",
          "Url": "https://api.example.com/login",
          "Method": "POST",
          "Body": {
            "ContentType": "application/json",
            "BodyValue": "{\"username\":\"${name}\",\"token\":\"${global}\"}"
          },
          "TimeoutInSecond": 10,
          "RedirectCountLimit": 0,
          "HeaderList": [
            {
              "HeaderName": "Content-Type",
              "HeaderValue": "application/json"
            }
          ],
          "ExportList": [
            {
              "ExportName": "userId",
              "ExportType": "BODY_JSON",
              "ExportValue": "$.data.userId"
            }
          ],
          "CheckPointList": [
            {
              "CheckPoint": "",
              "CheckType": "STATUS_CODE",
              "Operator": "eq",
              "ExpectValue": "200"
            }
          ]
        }
      ],
      "FileParameterExplainList": [
        {
          "FileName": "users.csv",
          "FileParamName": "name,uid",
          "CycleOnce": false,
          "BaseFile": true
        }
      ]
    }
  ],
  "LoadConfig": {
    "TestMode": "concurrency_mode",
    "MaxRunningTime": 10,
    "AutoStep": true,
    "Increment": 30,
    "KeepTime": 3,
    "Configuration": {
      "AllConcurrencyBegin": 10,
      "AllConcurrencyLimit": 100
    }
  },
  "AdvanceSetting": {
    "LogRate": 10,
    "ConnectionTimeoutInSecond": 10,
    "SuccessCode": "429",
    "DomainBindingList": [
      {
        "Domain": "api.example.com",
        "Ips": ["1.1.1.1", "2.2.2.2"]
      }
    ]
  },
  "FileParameterList": [
    {
      "FileName": "users.csv",
      "FileOssAddress": "https://bucket.oss.aliyuncs.com/users.csv"
    }
  ],
  "GlobalParameterList": [
    {
      "ParamName": "global",
      "ParamValue": "test-token-123"
    }
  ]
}

Field Reference

Root Level Fields

API Configuration Fields

Body Configuration

HeaderList Item

CheckPointList Item

ExportList Item

LoadConfig Fields

Configuration Fields

AdvanceSetting Fields

FileParameterList Item

FileParameterExplainList Item

GlobalParameterList Item

Common Patterns

Simple GET Request

{
  "ApiName": "homepage",
  "Url": "https://example.com",
  "Method": "GET",
  "TimeoutInSecond": 10,
  "RedirectCountLimit": 10,
  "HeaderList": [
    {"HeaderName": "User-Agent", "HeaderValue": "Mozilla/5.0"}
  ],
  "CheckPointList": [
    {"CheckPoint": "", "CheckType": "STATUS_CODE", "Operator": "eq", "ExpectValue": "200"}
  ]
}

POST with JSON Body

{
  "ApiName": "login",
  "Url": "https://api.example.com/login",
  "Method": "POST",
  "Body": {
    "ContentType": "application/json",
    "BodyValue": "{\"username\":\"test\",\"password\":\"123456\"}"
  },
  "TimeoutInSecond": 10,
  "RedirectCountLimit": 0,
  "HeaderList": [
    {"HeaderName": "Content-Type", "HeaderValue": "application/json"}
  ],
  "CheckPointList": [
    {"CheckPoint": "", "CheckType": "STATUS_CODE", "Operator": "eq", "ExpectValue": "200"}
  ]
}

Gradual Load Increase

{
  "LoadConfig": {
    "TestMode": "concurrency_mode",
    "MaxRunningTime": 10,
    "AutoStep": true,
    "Increment": 60,
    "KeepTime": 2,
    "Configuration": {
      "AllConcurrencyBegin": 10,
      "AllConcurrencyLimit": 100
    }
  }
}

This configuration starts with 10 concurrent users and increases every 60 seconds, holding each level for 2 minutes, until reaching 100 concurrent users.

PTS RAM Policies

This document lists the RAM (Resource Access Management) permissions required for PTS operations.

Required Permissions

The following permissions are required for this skill to function. Each permission is listed in the format {Product}:{Action} — Description.

PTS Native Stress Testing Permissions

• pts:CreatePtsScene — Create PTS stress testing scenario
• pts:GetPtsScene — Get PTS scenario details
• pts:ListPtsScene — List PTS scenarios
• pts:StartPtsScene — Start PTS stress testing
• pts:StopPtsScene — Stop PTS stress testing
• pts:DeletePtsScene — Delete PTS scenario
• pts:StartDebugPtsScene — Debug PTS scenario
• pts:GetPtsReportDetails — Get PTS report details
• pts:GetPtsSceneBaseLine — Get PTS scenario baseline data
• pts:GetPtsSceneRunningData — Get PTS scenario running data
• pts:GetPtsSceneRunningStatus — Get PTS scenario running status

JMeter Stress Testing Permissions

• pts:SaveOpenJMeterScene — Create or update JMeter scenario
• pts:GetOpenJMeterScene — Get JMeter scenario details
• pts:ListOpenJMeterScenes — List JMeter scenarios
• pts:StartTestingJMeterScene — Start JMeter stress testing
• pts:StopTestingJMeterScene — Stop JMeter stress testing
• pts:RemoveOpenJMeterScene — Delete JMeter scenario
• pts:GetJMeterReportDetails — Get JMeter report details

Policy Templates

Full Access Policy

For users who need complete PTS management capabilities:

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "pts:CreatePtsScene",
        "pts:GetPtsScene",
        "pts:ListPtsScene",
        "pts:StartPtsScene",
        "pts:StopPtsScene",
        "pts:DeletePtsScene",
        "pts:StartDebugPtsScene",
        "pts:GetPtsReportDetails",
        "pts:GetPtsSceneBaseLine",
        "pts:GetPtsSceneRunningData",
        "pts:GetPtsSceneRunningStatus",
        "pts:SaveOpenJMeterScene",
        "pts:GetOpenJMeterScene",
        "pts:ListOpenJMeterScenes",
        "pts:StartTestingJMeterScene",
        "pts:StopTestingJMeterScene",
        "pts:RemoveOpenJMeterScene",
        "pts:GetJMeterReportDetails"
      ],
      "Resource": "*"
    }
  ]
}

Read-Only Policy

For users who only need to view PTS scenarios and reports:

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "pts:GetPtsScene",
        "pts:ListPtsScene",
        "pts:GetPtsReportDetails",
        "pts:GetPtsSceneBaseLine",
        "pts:GetPtsSceneRunningData",
        "pts:GetPtsSceneRunningStatus",
        "pts:GetOpenJMeterScene",
        "pts:ListOpenJMeterScenes",
        "pts:GetJMeterReportDetails"
      ],
      "Resource": "*"
    }
  ]
}

Scenario Management Policy

For users who need to create and manage scenarios but not run stress tests:

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "pts:CreatePtsScene",
        "pts:GetPtsScene",
        "pts:ListPtsScene",
        "pts:DeletePtsScene",
        "pts:SaveOpenJMeterScene",
        "pts:GetOpenJMeterScene",
        "pts:ListOpenJMeterScenes",
        "pts:RemoveOpenJMeterScene"
      ],
      "Resource": "*"
    }
  ]
}

Stress Testing Execution Policy

For users who need to execute and monitor stress tests:

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "pts:StartPtsScene",
        "pts:StopPtsScene",
        "pts:StartDebugPtsScene",
        "pts:GetPtsSceneRunningStatus",
        "pts:GetPtsSceneRunningData",
        "pts:GetPtsReportDetails",
        "pts:StartTestingJMeterScene",
        "pts:StopTestingJMeterScene",
        "pts:GetJMeterReportDetails"
      ],
      "Resource": "*"
    }
  ]
}

System Policies

Alibaba Cloud provides the following system policies for PTS:

How to Attach Policies

Using Console

1. Log in to RAM Console: https://ram.console.aliyun.com/ 2. Navigate to: Identities > Users 3. Select the target user 4. Click "Add Permissions" 5. Select or create the appropriate policy

Using CLI

# Attach system policy
aliyun ram attach-policy-to-user \
  --user-name <UserName> \
  --policy-name AliyunPTSFullAccess \
  --policy-type System \
  --user-agent AlibabaCloud-Agent-Skills

# Attach custom policy
aliyun ram attach-policy-to-user \
  --user-name <UserName> \
  --policy-name MyPTSPolicy \
  --policy-type Custom \
  --user-agent AlibabaCloud-Agent-Skills

Best Practices

1. Use Least Privilege: Grant only the minimum permissions required for the task 2. Separate Duties: Use different policies for different roles (viewers, operators, administrators) 3. Use System Policies: Prefer Alibaba Cloud managed policies when they meet your needs 4. Regular Audit: Periodically review and audit permissions

References

• RAM Policy Overview
• PTS Authorization

PTS Related APIs and CLI Commands

This document lists all APIs and CLI commands related to Alibaba Cloud Performance Testing Service (PTS).

Product Information

PTS Native Stress Testing APIs

JMeter Stress Testing APIs

File Management APIs

Common Parameters

All CLI commands support the following common parameters:

Example CLI Commands

Create PTS Scenario

aliyun pts create-pts-scene \
  --scene '{"name":"test-scene","type":"HTTP","requests":[{"url":"https://example.com","method":"GET"}]}' \
  --user-agent AlibabaCloud-Agent-Skills

Start PTS Stress Testing

aliyun pts start-pts-scene \
  --scene-id <SceneId> \
  --user-agent AlibabaCloud-Agent-Skills

Create JMeter Scenario

aliyun pts save-open-jmeter-scene \
  --open-jmeter-scene '{"scene_name":"MyJMeterTest","test_file":"example.jmx","duration":300,"concurrency":100,"mode":"CONCURRENCY"}' \
  --user-agent AlibabaCloud-Agent-Skills

Start JMeter Stress Testing

aliyun pts start-testing-jmeter-scene \
  --scene-id <SceneId> \
  --user-agent AlibabaCloud-Agent-Skills

References

• PTS API Documentation
• Create PTS Scenario
• Create JMeter Scenario

PTS Verification Methods

This document provides verification steps to confirm successful execution of PTS operations.

1. Verify CLI Authentication

Before executing any PTS commands, verify that CLI authentication is properly configured:

# Check CLI version (must be >= 3.3.1)
aliyun version

# Test authentication by listing regions
aliyun ecs describe-regions --user-agent AlibabaCloud-Agent-Skills

Expected Result: Returns a list of regions without authentication errors.

2. Verify PTS Scenario Creation

2.1 Verify PTS Native Scenario Created

After creating a PTS scenario, verify it exists:

# List all PTS scenarios
aliyun pts list-pts-scene \
  --page-number 1 \
  --page-size 10 \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: The newly created scenario should appear in the PtsSceneViewList array.

2.2 Verify Scenario Details

Get detailed information about the created scenario:

# Get scenario details
aliyun pts get-pts-scene \
  --scene-id <SceneId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: Returns complete scenario configuration including:

• Scene name
• API configurations (URLs, methods, headers)
• Load configuration
• Duration settings

3. Verify JMeter Scenario Creation

3.1 Verify JMeter Scenario Created

After creating a JMeter scenario, verify it exists:

# List all JMeter scenarios
aliyun pts list-open-jmeter-scenes \
  --page-number 1 \
  --page-size 10 \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: The newly created JMeter scenario should appear in the response.

3.2 Verify JMeter Scenario Details

Get detailed information about the created JMeter scenario:

# Get JMeter scenario details
aliyun pts get-open-jmeter-scene \
  --scene-id <SceneId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: Returns complete JMeter scenario configuration including:

• Scene name
• JMX file information
• Concurrency settings
• Duration settings

4. Verify Stress Testing Execution

CRITICAL WARNING: start-pts-scene may return Success: true even when the stress test fails to actually launch. This "false success" can occur due to:

- Missing configuration fields (e.g., TimeoutInSecond, AdvanceSetting)

- Target site protection blocking traffic

- Account quota limits

>

Always verify actual execution status using the methods below.

4.1 Verify PTS Task Started

After starting a PTS stress testing task:

Step 1: Check running status

aliyun pts get-pts-scene-running-status \
  --scene-id <SceneId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Status Values:

• SYNCING - Data uploading, preparing agents
• RUNNING - Test is actively running
• STOPPED - Test has stopped (check if it ran successfully or failed immediately)

Step 2: Verify with running data (REQUIRED)

# The --plan-id is REQUIRED and comes from start-pts-scene response
aliyun pts get-pts-scene-running-data \
  --scene-id <SceneId> \
  --plan-id <PlanId> \
  --user-agent AlibabaCloud-Agent-Skills

Key Indicators of Successful Execution:

Indicators of Failed Execution:

4.2 Verify JMeter Task Started

After starting a JMeter stress testing task:

# The response from start-testing-jmeter-scene includes a report ID
# Use it to check status via get-jmeter-report-details
aliyun pts get-jmeter-report-details \
  --report-id <ReportId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: Returns report details showing test is in progress or completed.

5. Verify Stress Testing Results

5.1 Verify PTS Report

After the stress test completes:

# Get PTS report details
aliyun pts get-pts-report-details \
  --scene-id <SceneId> \
  --plan-id <PlanId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: Returns complete report including:

• Total requests
• Average response time
• Success rate
• TPS (Transactions Per Second)
• Error details (if any)

5.2 Verify JMeter Report

After the JMeter test completes:

# Get JMeter report details
aliyun pts get-jmeter-report-details \
  --report-id <ReportId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: Returns complete JMeter report including:

• Test duration
• Throughput
• Response times
• Error rates

6. Verify Scenario Deletion

6.1 Verify PTS Scenario Deleted

After deleting a PTS scenario:

# Try to get the deleted scenario
aliyun pts get-pts-scene \
  --scene-id <DeletedSceneId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: Should return an error indicating the scenario does not exist.

6.2 Verify JMeter Scenario Deleted

After deleting a JMeter scenario:

# Try to get the deleted JMeter scenario
aliyun pts get-open-jmeter-scene \
  --scene-id <DeletedSceneId> \
  --user-agent AlibabaCloud-Agent-Skills

Expected Result: Should return an error indicating the scenario does not exist.

7. Common Error Handling

Authentication Errors

API Errors

8. Debug Commands

Enable debug logging to troubleshoot issues:

# Run command with debug logging
aliyun pts list-pts-scene \
  --page-number 1 \
  --page-size 10 \
  --log-level debug \
  --user-agent AlibabaCloud-Agent-Skills

Output includes: Request/response headers, body content, timestamps.