Alibabacloud Pai Rec Diagnosis

PAI-Rec Engine Diagnosis and Configuration Validation

This skill provides comprehensive diagnostic and validation capabilities for Alibaba Cloud PAI-Rec (Programmable Recommendation System) engines, including interface troubleshooting and configuration analysis.

Scenario Description

PAI-Rec is Alibaba Cloud's programmable recommendation system that provides intelligent recommendation capabilities. This skill helps users:

1. Diagnose PAI-Rec Engine Interface Issues: When engine API returns errors or unexpected results, trace the request through EAS service logs and engine configurations to identify root causes.

2. Validate Engine Configurations: Analyze engine configuration files for potential issues, inconsistencies, or misconfigurations before deployment.

Architecture: PAI-EAS Service + PAI-Rec Engine + Engine Configuration Management

Key Components

• PAI-EAS Service: Elastic Algorithm Service hosting the recommendation engine
• PAI-Rec Engine: The recommendation engine processing requests
• Engine Configuration: Configuration files defining engine behavior
• Service Logs: EAS service logs containing request traces

---

Installation

Pre-check: Aliyun CLI >= 3.3.3 required

Run aliyun version to verify >= 3.3.3. If not installed or version too low,

run curl -fsSL https://aliyuncli.alicdn.com/setup.sh | bash to install/update,

or see references/cli-installation-guide.md for installation instructions.

Pre-check: Aliyun CLI plugin update required

[MUST] run aliyun configure set --auto-plugin-install true to enable automatic plugin installation.

[MUST] run aliyun plugin update to ensure that any existing plugins are always up-to-date.

---

Authentication

Pre-check: Alibaba Cloud Credentials Required

>

Security Rules:

- NEVER read, echo, or print AK/SK values (e.g., echo $ALIBABA_CLOUD_ACCESS_KEY_ID is FORBIDDEN)

- NEVER ask the user to input AK/SK directly in the conversation or command line

- NEVER use aliyun configure set with literal credential values

- ONLY use aliyun configure list to check credential status

>

```bash

aliyun configure list

```

Check the output for a valid profile (AK, STS, or OAuth identity).

>

If no valid profile exists, STOP here.

1. Obtain credentials from Alibaba Cloud Console

2. Configure credentials outside of this session (via aliyun configure in terminal or environment variables in shell profile)

3. Return and re-run after aliyun configure list shows a valid profile

---

RAM Policy

[MUST] Permission Failure Handling: When any command or API call fails due to permission errors at any point during execution, follow this process:

1. Read references/ram-policies.md to get the full list of permissions required by this SKILL

2. Use ram-permission-diagnose skill to guide the user through requesting the necessary permissions

3. Pause and wait until the user confirms that the required permissions have been granted

For detailed RAM policies required by this skill, see references/ram-policies.md.

---

Parameter Confirmation

IMPORTANT: Parameter Confirmation — Before executing any command or API call,

ALL user-customizable parameters (e.g., RegionId, service names, instance IDs,

environment names, config names, etc.) MUST be confirmed with the user.

Do NOT assume or use default values without explicit user approval.

Required Parameters

---

Observability (MUST follow for every aliyun command)

Upon loading this skill, generate a random session ID (32-char lowercase hex string) once for the entire session. Use it as {session-id} below.

Rule: Every `aliyun` CLI command that calls a cloud API MUST include the `--user-agent` flag. Local utility commands (e.g. configure, plugin, version) do not support this flag and should be excluded.

--user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/{session-id}

Example (assuming session-id is a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6):

aliyun eas describe-service --cluster-id cn-hangzhou --service-name my_service --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

Do not skip, alter the format, or omit --user-agent on any aliyun API command invocation.

---

Core Workflow

[MUST] Allocate a per-session work directory — All transient artifacts

(raw config dumps, log dumps, sanitized output) MUST be written under a unique

per-session directory to avoid concurrent overwrite between parallel skill

invocations. Run the following at the start of the workflow, before any

artifact-producing CLI invocation, and reuse $WORKDIR for the whole session.

Pass the full template as a positional arg (works on both BSD/macOS and

GNU/Linux mktemp); do NOT use -t prefix, which falls back to $TMPDIR

(e.g. /var/folders on macOS) and may even fail under sandboxed shells.

```bash

export WORKDIR=$(mktemp -d /tmp/pairec-diag-XXXXXX)

```

All file paths shown below ($WORKDIR/engine_configs_list.json, etc.) live

inside this directory and MUST NOT be replaced with hard-coded /tmp/... paths.

[MUST] Use ONLY the workflows defined below. Do NOT invent additional

steps (e.g. instance resource checks, network probes) or substitute manual

analysis for a defined workflow step.

Workflow 1: PAI-Rec Engine Interface Diagnosis

This workflow helps diagnose issues when a PAI-Rec engine API returns errors or unexpected results.

Input Example:

Service Name: embedding_recall
API Response:
{
    "code": 299,
    "msg": "items size not enough",
    "request_id": "941b4e14-d1c5-489f-a184-b2b17f8b4fdb",
    "size": 0,
    "experiment_id": "",
    "items": []
}

Step 1: Retrieve EAS Service Information

Get the service details to find the EAS service ID and configuration:

aliyun eas describe-service \
  --cluster-id <cluster-id> \
  --service-name <service-name>

What to extract:

• Resource: EAS service resource ID (e.g., eas-r-1v4qb1yan3qmnjwxqe)
• ServiceConfig.envs: Environment variables containing:
• REGION: The region
• INSTANCE_ID: PAI-Rec instance ID
• CONFIG_NAME: Engine configuration name
• PAIREC_ENVIRONMENT: Environment (product/prepub)

Step 2: Extract Request ID from API Response

Parse the API response JSON to get the request_id field. This will be used to search service logs.

Step 3: Query EAS Service Logs

Use the request ID as the sole filter to search service logs. Do NOT pass --start-time / --end-time when searching PAI-Rec business logs:

aliyun eas describe-service-log \
  --cluster-id <cluster-id> \
  --service-name <service-name> \
  --keyword <request-id> \
  --page-size 500

[CRITICAL] `--keyword <request-id>` is MANDATORY — local post-processing is FORBIDDEN:

• You MUST pass --keyword <request-id> (server-side filter, exact case-sensitive match on the full request_id). The API returns only log lines matching the keyword.
• You MUST NOT omit --keyword and then filter locally (e.g., piping through head, grep, python3, jq, or any script).
• You MUST NOT make multiple describe-service-log calls without --keyword hoping to find relevant lines by scanning the full log stream.
• If a call with --keyword returns empty results, report that no matching logs were found — do NOT fall back to fetching unfiltered logs.
• --page-size 500 captures the entire trace in a single page; matched entries for one request are usually < 30.

❌ WRONG (fetches ALL logs, filters locally — FORBIDDEN):

```bash

aliyun eas describe-service-log --cluster-id cn-beijing --service-name embedding_recall | head -300

aliyun eas describe-service-log --cluster-id cn-beijing --service-name embedding_recall | grep "request_id"

```

>

✅ RIGHT (server-side keyword filter — REQUIRED):

```bash

aliyun eas describe-service-log --cluster-id cn-beijing --service-name embedding_recall --keyword 0c6cbd91-5618-4705-8e08-9126bf4600f7 --page-size 500

```

[CRITICAL] Time range silently drops business logs:

• With --keyword only (no time range), the CLI returns the full PAI-Rec application trace (controller.go / feed.go / recall.go / rank_service.go etc.) matching the request_id.
• Adding --start-time / --end-time — even when the window covers the real timestamp — silently drops business logs and returns only infrastructure noise (/bin/sh heartbeats, 502 Bad Gateway retries, postgres.go dbstat).
• Use time ranges only for broad scans WITHOUT --keyword, in yyyy-MM-dd HH:mm:ss UTC format (no T / no Z); ISO-8601 forms like 2025-04-28T00:00:00Z are rejected with InvalidParameter.

Step 4: List Engine Configurations

Map the environment and list matching configurations:

Environment Mapping:

• product → Prod
• prepub → Pre

aliyun pairecservice list-engine-configs \
  --instance-id <instance-id> \
  --environment <Prod|Pre> \
  --status Released \
  --name <config-name> > "$WORKDIR/engine_configs_list.json" 2>&1

[MUST] Always pass `--name <config-name>` for server-side filtering: <config-name> is already known from Step 1 (ServiceConfig.envs.CONFIG_NAME); forward it as --name. Omitting it returns the entire instance's config inventory (often hundreds of unrelated entries), forces client-side filtering, wastes tokens, and risks hitting CLI default pagination so the target row is silently dropped. --name is an exact-match filter on the server; do NOT substitute with grep / jq select post-processing. The same rule applies to every list-engine-configs invocation in this skill (Workflow 2 Step 1 included).

What to extract:

• Find the configuration with Status: Released
• Get EngineConfigId and Version

Step 5: Get Engine Configuration Details

aliyun pairecservice get-engine-config \
  --instance-id <instance-id> \
  --engine-config-id <engine-config-id> > "$WORKDIR/raw_engine_config.json" 2>&1

[MUST] Sanitize before display — Config may contain plaintext passwords or access keys. Always pipe through the sanitizer before printing to terminal; only sanitized output (with credentials replaced by ***REDACTED***) should appear there. The raw file at $WORKDIR/raw_engine_config.json can be passed directly to scripts/validate.py (which does not print credential values).

python3 scripts/sanitize_config.py "$WORKDIR/raw_engine_config.json"

What to extract:

• ConfigValue: The actual engine configuration (JSON/YAML)

Step 5.5 (Optional): Static Config Sanity Check

Run scripts/validate.py against the retrieved ConfigValue to rule out structural / reference errors. See references/config-validation.md.

printf '%s' "$CONFIG_VALUE" | python3 scripts/validate.py --stdin

When to run: when logs point at a config element or when diagnosing the config for the first time. When to skip: when logs show a non-config root cause (missing scene_id, upstream 5xx). [MUST NOT] Do not replace or duplicate validate.py (same restriction as Workflow 2 § Step 3). [MUST] Scoping rule: findings enter the final diagnosis ONLY when tied to log evidence for the current request_id.

Step 5a (Conditional): Retrieve Experiment Configuration

Condition: experiment_id in the API response is non-empty (e.g., "ER14_L21_L26#EG21_L38#EG38#E44_GL36_GL37").

Parse: Extract EG{id} (experiment group) and E{id} (experiment) numeric IDs from the string. Ignore ER, L, GL prefixes — they carry no config.

# For each EG{id}:
aliyun pairecservice get-experiment-group \
  --instance-id <instance-id> \
  --experiment-group-id <id> > "$WORKDIR/experiment_group_<id>.json" 2>&1
# For each E{id}:
aliyun pairecservice get-experiment \
  --instance-id <instance-id> \
  --experiment-id <id> > "$WORKDIR/experiment_<id>.json" 2>&1

What to extract: The Config field — contains override parameters (e.g., default.RecallNames, rankconf, filterNames, default.SortNames) that supersede the base engine config.

Override priority (low → high): Base Engine Config < ExperimentGroup.Config < Experiment.Config. Apply in Step 6 to understand actual runtime behavior.

Validate experiment configs against the base config (reference existence check):

python3 scripts/validate.py "$WORKDIR/raw_engine_config.json" \
  --experiment-config "$WORKDIR/experiment_group_<id>.json" \
  --experiment-config "$WORKDIR/experiment_<id>.json"

Step 6: Comprehensive Analysis

Analyze the following components together: 1. API Response: Error code, message, and returned data 2. Service Logs: Trace logs for the request_id showing processing flow 3. Engine Configuration: Settings that may affect the behavior 4. Experiment Overrides (if experiment_id non-empty): Effective config = base config with experiment parameters applied on top

Common Issues to Check:

• Configuration mismatches (e.g., recall settings, filtering rules)
• Experiment overrides (e.g., experiment changed RecallNames / rankconf from base config)
• Resource limitations (e.g., insufficient items, timeout settings)
• Data source issues (e.g., table access, feature availability)
• Environment inconsistencies (e.g., prod config in prepub environment)

[MUST] Evidence-only reporting rule:

The final diagnosis delivered to the user MUST be grounded strictly in what the EAS service logs and the engine configuration directly show. Apply the following constraints:

• Report only what is observed. Quote the exact log line (file:line, level, message) and the exact config fragment that proves each claim.
• State the direct causal chain from log evidence to the API response, and stop there.
• Do NOT add any of the following unless the user explicitly asks:
• Speculative root causes not visible in logs/config (e.g., "client probably sent wrong X")
• Fix recommendations or remediation steps
• Conditional "if X then Y" scenarios
• Tangential best-practice advice (security, fallback design, naming, etc.)
• Guesses about upstream systems, client code, or data sources not covered by the logs/config
• If the evidence is insufficient to reach a conclusion, state explicitly what additional data (specific log lines, other config versions, other environments) is needed, instead of guessing.
• Recommendations are opt-in only. Provide fixes/suggestions only when the user explicitly requests them in a follow-up.

---

Workflow 2: PAI-Rec Engine Configuration Validation

This workflow validates engine configurations for potential issues.

Input: Configuration name and environment (Prod/Pre)

Step 1: List Configuration Versions

If user doesn't provide engine-config-id, list available versions:

aliyun pairecservice list-engine-configs \
  --instance-id <instance-id> \
  --environment <Prod|Pre> \
  --name <config-name>

Display to user:

• Version: Version number
• Status: Configuration status (Released/Draft/Archived)
• GmtCreateTime: Creation timestamp
• EngineConfigId: Version ID

Ask user to select a version or provide the engine-config-id.

Step 2: Retrieve Configuration Details

aliyun pairecservice get-engine-config \
  --instance-id <instance-id> \
  --engine-config-id <engine-config-id> > "$WORKDIR/raw_engine_config.json" 2>&1

[MUST] Sanitize before display — Always sanitize before printing to terminal:

python3 scripts/sanitize_config.py "$WORKDIR/raw_engine_config.json"

Step 3: Run Schema + Rule Validation

[MUST] Feed the extracted ConfigValue JSON into scripts/validate.py. The script enforces JSON Schema (references/schema.json) + reference-consistency rules and exits with status 0 on pass, 1 on failure.

# From a saved JSON file (recommended)
python3 scripts/validate.py "$WORKDIR/raw_engine_config.json"
# Or pipe ConfigValue directly via stdin
printf '%s' "$CONFIG_VALUE" | python3 scripts/validate.py --stdin

Requires jsonschema (pip install jsonschema); if missing the script falls back to rule-only validation without Schema checks.

[MUST NOT] Do not replace or duplicate `validate.py`:

• Do NOT skip it; do NOT hand-write equivalent checks in Python / jq / grep / any other tool — the script is the authoritative validator.
• Do NOT re-implement, re-check, or "double-confirm" any rule after the script has run; trust its output verbatim, including a clean 0 error(s), 0 warning(s) run.
• If the script cannot run (missing Python, dependency issue, etc.), fix the environment and re-run — do NOT fall back to manual checking.
• Inspections OUTSIDE the script's scope are still allowed (see Step 4).

What the script checks (summary):

1. Structure — JSON well-formedness, required fields, types (RecallConfs, FilterConfs, SortConfs, AlgoConfs, SceneConfs, RankConf, FeatureConfs, UserFeatureConfs, DebugConfs, FeatureLogConfs, CallBackConfs, PipelineConfs, etc.) 2. Enum values — RecallType / FilterType / SortType / DebugConfs.OutputType / GeneralRankConfs.ActionConfs[].ActionType 3. Reference consistency — SceneConfs.RecallNames → RecallConfs; FilterNames → FilterConfs; SortNames → SortConfs; RankConf.RankAlgoList → AlgoConfs; any DaoConf.AdapterType + *Name → the corresponding *Confs (Hologres / Redis / MySQL / TableStore / FeatureStore / …) 4. Business rules

• User2ItemExposureFilter with WriteLog=true + FeatureStore adapter: must set

TimeInterval > 0

• PriorityAdjustCountFilter in accumulator mode: Count must be strictly

increasing (use Type="fix" for independent per-recall caps)

• PipelineConfs.*.Name must be globally unique
• DebugConfs.Rate must be an integer in [0, 100]

5. Duplicate name detection within RecallConfs, FilterConfs, SortConfs, AlgoConfs

Detailed usage, exit codes, example outputs and the full rule list live in references/config-validation.md.

Step 4: Evidence-Grounded Report

[MUST] Required first line of the report: Quote validate.py's stdout verbatim — either Validation passed: configuration is well-formed or Validation finished: N error(s), M warning(s). A report missing this exact line is INVALID; restart from Step 3.

Manual inspection is allowed only for concerns out of scope of validate.py: env / region / model-signature mismatches, cross-version diffs, naming collisions between RankScore variables and model output fields, and root-cause reading of any [WARNING] the script itself asks a human to judge. Do NOT add findings the script did not report unless you can tie them to one of these out-of-scope concerns.

Report structure:

• ✅ Checks passed — quote validate.py's 0 error(s), 0 warning(s) line
• ⚠️ Warnings — copy each [WARNING] <path>: <message> from the script,

plus any out-of-scope inconsistency from manual inspection

• ❌ Errors — copy each [ERROR] <path>: <message> from the script
• Missing-evidence notes — ONLY when ≥1 ⚠️ warning is listed: state what extra data would upgrade that warning into a confirmed error. With 0 warnings, OMIT this section; do NOT fill it with generic out-of-scope disclaimers (cross-version diff, remote connectivity, region/endpoint consistency) — those are volunteered best-practice advice forbidden by the evidence-only rule.

Do not add speculative fixes or best-practice tangents; suggestions are provided only when the user explicitly asks for them.

---

Success Verification Method

For detailed verification steps, see references/verification-method.md.

Quick Verification:

1. For Diagnosis Workflow:

• Service information retrieved successfully
• Logs found containing the request_id
• Configuration loaded correctly
• Root cause identified

2. For Validation Workflow:

• Configuration retrieved successfully
• All validation checks executed
• Issues clearly reported
• Recommendations provided (if applicable)

---

Cleanup

This skill performs read-only Alibaba Cloud API calls (no remote resources are created). Transient artifacts go to a per-session local $WORKDIR under /tmp (see Core Workflow preamble). The skill does NOT delete $WORKDIR automatically — the OS-level temp policy reclaims it (macOS reaps /tmp periodically; most Linux distros reap on reboot or via systemd-tmpfiles). To free disk space sooner, manually run rm -rf /tmp/pairec-diag-* outside the workflow.

---

Best Practices

1. Log queries — keyword only, no time range, no local filtering: For request-level diagnosis, pass --keyword <request_id> to aliyun eas describe-service-log and leave --start-time / --end-time unset. NEVER omit --keyword and post-process locally (e.g., | head, | grep, | python3) — this defeats server-side filtering, wastes tokens, and may miss logs beyond the first page. Combining keyword with a time range filters out business logs due to a CLI quirk (see Workflow 1, Step 3). Only use time ranges for broad non-request scans, and only with the yyyy-MM-dd HH:mm:ss UTC format (no T / no Z). 2. Trust `validate.py`: For Workflow 2, treat scripts/validate.py as the single source of truth for the rules in its catalogue. Do NOT skip it and hand-write checks, and do NOT re-validate its rules manually after a clean run. Manual inspection is reserved for concerns out of its scope (env / region / model signature, cross-version diffs, RankScore vs model output naming). 3. Environment awareness: Always verify that configurations match the target environment (Prod vs Pre); compare against a known-good version when issues persist. 4. Log retention: EAS service logs are retained for limited periods; diagnose issues promptly after occurrence. 5. Evidence-only conclusions: Ground every statement on a specific log line or config fragment. Follow the systematic workflow rather than jumping to conclusions from error messages alone. Do not speculate, do not propose fixes, and do not volunteer best-practice advice unless the user explicitly asks. If the evidence is insufficient, say what is missing rather than inferring.

---

Reference Links

Acceptance Criteria: alibabacloud-pai-rec-diagnosis

Scenario: PAI-Rec Engine Diagnosis and Configuration Validation Purpose: Skill testing acceptance criteria

This document defines correct and incorrect patterns for the PAI-Rec diagnosis skill to ensure proper implementation and usage.

---

Correct CLI Command Patterns

1. Product Names — Verify product name exists

✅ CORRECT

# EAS product commands
aliyun eas describe-service --cluster-id cn-hangzhou --service-name test
aliyun eas describe-service-log --cluster-id cn-hangzhou --service-name test

# PAI-RecService product commands
aliyun pairecservice list-engine-configs --instance-id pairec-cn-xxxxx
aliyun pairecservice get-engine-config --instance-id pairec-cn-xxxxx --engine-config-id config-123

Why correct: Uses valid product names eas and pairecservice that exist in Aliyun CLI.

❌ INCORRECT

# Wrong product names
aliyun pai-eas describe-service ...        # Product is 'eas', not 'pai-eas'
aliyun rec describe-service ...            # Product is 'pairecservice', not 'rec'
aliyun pairec list-engine-configs ...      # Product is 'pairecservice', not 'pairec'
aliyun pai list-engine-configs ...         # Product is 'pairecservice', not 'pai'

Why incorrect: These product names don't exist in the Aliyun CLI.

---

2. Command Names — Verify action exists under the product

✅ CORRECT

# EAS commands
aliyun eas describe-service
aliyun eas describe-service-log

# PAI-RecService commands
aliyun pairecservice list-engine-configs
aliyun pairecservice get-engine-config
aliyun pairecservice apply-engine-config
aliyun pairecservice create-engine-config
aliyun pairecservice get-experiment-group
aliyun pairecservice get-experiment

Why correct: All commands use lowercase words connected with hyphens (plugin mode format).

❌ INCORRECT

# Traditional API format (not plugin mode)
aliyun eas DescribeService                  # Use describe-service
aliyun eas DescribeServiceLog               # Use describe-service-log
aliyun pairecservice ListEngineConfigs      # Use list-engine-configs
aliyun pairecservice GetEngineConfig        # Use get-engine-config

# Wrong command names
aliyun eas get-service                      # Command is describe-service
aliyun eas list-logs                        # Command is describe-service-log
aliyun pairecservice get-configs            # Command is list-engine-configs
aliyun pairecservice describe-config        # Command is get-engine-config

Why incorrect: Must use plugin mode format (lowercase with hyphens), and exact command names.

---

3. Parameter Names — Verify each parameter exists for the command

✅ CORRECT - EAS describe-service

aliyun eas describe-service \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --region cn-hangzhou

Parameters:

• --cluster-id: Required, the cluster ID
• --service-name: Required, the service name
• --region: Optional, region override

✅ CORRECT - EAS describe-service-log

aliyun eas describe-service-log \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --keyword "941b4e14-d1c5-489f-a184-b2b17f8b4fdb" \
  --start-time "2025-04-28T08:00:00Z" \
  --end-time "2025-04-28T09:00:00Z" \
  --page-num 1 \
  --page-size 500

Parameters:

• --cluster-id: Required
• --service-name: Required
• --keyword: Optional, filter keyword
• --start-time: Optional, UTC format
• --end-time: Optional, UTC format
• --page-num: Optional, default 1
• --page-size: Optional, default 500

✅ CORRECT - PAI-RecService list-engine-configs

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod \
  --name my_config \
  --status Released \
  --version v1.2.3 \
  --page-number 1 \
  --page-size 20

Parameters:

• --instance-id: Required
• --environment: Optional, valid values: Prod or Pre
• --name: Optional, config name filter
• --status: Optional, e.g., Released, Draft, Archived
• --version: Optional
• --page-number: Optional
• --page-size: Optional

✅ CORRECT - PAI-RecService get-engine-config

aliyun pairecservice get-engine-config \
  --instance-id pairec-cn-xxxxx \
  --engine-config-id config-12345

Parameters:

• --instance-id: Required
• --engine-config-id: Required

✅ CORRECT - PAI-RecService get-experiment-group

aliyun pairecservice get-experiment-group \
  --instance-id pairec-cn-xxxxx \
  --experiment-group-id 21

Parameters:

• --instance-id: Required
• --experiment-group-id: Required, numeric ID extracted from experiment_id string (EG{id} pattern)

✅ CORRECT - PAI-RecService get-experiment

aliyun pairecservice get-experiment \
  --instance-id pairec-cn-xxxxx \
  --experiment-id 44

Parameters:

• --instance-id: Required
• --experiment-id: Required, numeric ID extracted from experiment_id string (E{id} pattern)

❌ INCORRECT - Wrong parameter names

# EAS describe-service - wrong parameters
aliyun eas describe-service \
  --cluster cn-hangzhou \              # Use --cluster-id
  --name embedding_recall              # Use --service-name

# EAS describe-service-log - wrong parameters
aliyun eas describe-service-log \
  --cluster-id cn-hangzhou \
  --service-name test \
  --search-keyword "test" \            # Use --keyword
  --from "2025-04-28T08:00:00Z" \      # Use --start-time
  --to "2025-04-28T09:00:00Z" \        # Use --end-time
  --page 1 \                           # Use --page-num
  --limit 500                          # Use --page-size

# PAI-RecService list-engine-configs - wrong parameters
aliyun pairecservice list-engine-configs \
  --instance pairec-cn-xxxxx \         # Use --instance-id
  --env Prod \                         # Use --environment
  --config-name my_config \            # Use --name
  --page 1                             # Use --page-number

# PAI-RecService get-engine-config - wrong parameters
aliyun pairecservice get-engine-config \
  --instance pairec-cn-xxxxx \         # Use --instance-id
  --config-id config-12345             # Use --engine-config-id

# PAI-RecService get-experiment-group - wrong parameters
aliyun pairecservice get-experiment-group \
  --instance pairec-cn-xxxxx \         # Use --instance-id
  --group-id 21                        # Use --experiment-group-id

# PAI-RecService get-experiment - wrong parameters
aliyun pairecservice get-experiment \
  --instance pairec-cn-xxxxx \         # Use --instance-id
  --exp-id 44                          # Use --experiment-id

Why incorrect: Parameter names must match exactly as defined in --help output.

---

4. Parameter Values — Verify format and enum values

✅ CORRECT - Environment parameter values

# Environment must be "Prod" or "Pre" (case-sensitive)
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Pre

Valid values: Prod, Pre (capitalized)

❌ INCORRECT - Wrong environment values

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment product            # Use Prod

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment prepub             # Use Pre

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment prod               # Use Prod (capital P)

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment pre                # Use Pre (capital P)

✅ CORRECT - Time format (UTC)

aliyun eas describe-service-log \
  --start-time "2025-04-28T08:00:00Z" \
  --end-time "2025-04-28T09:00:00Z"

Format: ISO 8601 UTC format with 'Z' suffix

❌ INCORRECT - Wrong time formats

aliyun eas describe-service-log \
  --start-time "2025-04-28 08:00:00" \        # Missing 'T' and 'Z'
  --end-time "04/28/2025 09:00:00"            # Wrong format

aliyun eas describe-service-log \
  --start-time "2025-04-28T16:00:00+08:00" \  # Use UTC, not local timezone
  --end-time "2025-04-28T17:00:00+08:00"

---

5. Per-command --user-agent (Observability)

✅ CORRECT

# Session-id is a 32-char lowercase hex string generated once per session
# Every aliyun API command includes --user-agent with session-id
aliyun eas describe-service \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

❌ INCORRECT

# Missing --user-agent flag
aliyun eas describe-service --cluster-id cn-hangzhou --service-name test

# Wrong user-agent format (missing session-id)
aliyun eas describe-service --cluster-id cn-hangzhou --service-name test \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis

# Using quoted user-agent value (should not be quoted)
aliyun eas describe-service --cluster-id cn-hangzhou --service-name test \
  --user-agent "AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4"

---

6. Plugin Management Commands

✅ CORRECT

# Enable auto plugin install
aliyun configure set --auto-plugin-install true

# Update plugins
aliyun plugin update

❌ INCORRECT

# Wrong command
aliyun plugin install --auto           # Use configure set
aliyun update-plugins                   # Use plugin update

---

Correct Workflow Patterns

1. Diagnosis Workflow

✅ CORRECT - Complete workflow

# 1. Session-id is a 32-char lowercase hex string generated once per session
# Example: a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 2. Get service information
aliyun eas describe-service \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 3. Query service logs by request_id
aliyun eas describe-service-log \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --keyword "941b4e14-d1c5-489f-a184-b2b17f8b4fdb" \
  --page-size 500 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 4. List engine configs
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod \
  --status Released \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 5. Get specific config
aliyun pairecservice get-engine-config \
  --instance-id pairec-cn-xxxxx \
  --engine-config-id config-12345 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

Why correct:

• Session-id is a 32-char lowercase hex string generated once per session
• Every API command includes --user-agent (without quotes) with session-id
• Local utility commands (configure, plugin, version) excluded
• Logical sequence of operations
• Proper parameter usage

❌ INCORRECT - Missing steps or wrong order

# Missing --user-agent (no observability)
aliyun eas describe-service --cluster-id cn-hangzhou --service-name test

# Wrong environment mapping (using "product" instead of "Prod")
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment product

---

2. Configuration Validation Workflow

✅ CORRECT

# 1. Session-id is a 32-char lowercase hex string generated once per session
# Example: a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 2. List config versions (if user didn't provide ID)
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod \
  --name my_config \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 3. Get specific config for validation
aliyun pairecservice get-engine-config \
  --instance-id pairec-cn-xxxxx \
  --engine-config-id config-12345 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

❌ INCORRECT

# Skipping version listing when user didn't provide ID
# (Should list versions for user to choose)

# Using wrong status filter
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --status Active                    # Use Released, Draft, or Archived

---

Correct Environment Mapping

✅ CORRECT

# From service config: PAIREC_ENVIRONMENT=product
# Map to CLI parameter:
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod

# From service config: PAIREC_ENVIRONMENT=prepub
# Map to CLI parameter:
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Pre

❌ INCORRECT

# Using service env value directly (wrong)
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment product            # Should be Prod

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment prepub             # Should be Pre

---

Correct Parameter Confirmation Patterns

✅ CORRECT - Confirm before execution

Before executing, please confirm the following parameters:
- Cluster ID: cn-hangzhou
- Service Name: embedding_recall
- Instance ID: pairec-cn-xxxxx
- Environment: Prod (mapped from "product")
- Time Range: 2025-04-28 08:00:00 to 09:00:00 UTC

Proceed? (yes/no)

❌ INCORRECT - Using hardcoded or assumed values

# Hardcoded region without confirmation
aliyun eas describe-service \
  --cluster-id cn-hangzhou \           # Never assume
  --service-name embedding_recall

# Assumed time range without asking user
--start-time "2025-04-28T00:00:00Z" \  # Should confirm with user
--end-time "2025-04-28T23:59:59Z"

---

Correct Error Handling Patterns

✅ CORRECT - Handle errors gracefully

# Check if command succeeded
if ! SERVICE_INFO=$(aliyun eas describe-service --cluster-id cn-hangzhou --service-name test --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6 2>&1); then
  echo "Error: Failed to retrieve service information"
  echo "Details: $SERVICE_INFO"
  exit 1
fi

Why correct: Includes --user-agent for observability and handles errors gracefully.

❌ INCORRECT - No error handling

# No error checking
SERVICE_INFO=$(aliyun eas describe-service --cluster-id cn-hangzhou --service-name test)

# Process results without checking if command succeeded
# (May fail silently or with confusing errors)

---

Correct Time Range Patterns

✅ CORRECT - Appropriate time windows

# For specific issue diagnosis (±30 min to ±1 hour)
--start-time "2025-04-28T08:00:00Z" \
--end-time "2025-04-28T09:00:00Z"

# For pattern analysis (last 1-24 hours)
--start-time "2025-04-27T08:00:00Z" \
--end-time "2025-04-28T08:00:00Z"

# For recent issues (last hour)
START_TIME=$(date -u -d '1 hour ago' '+%Y-%m-%dT%H:%M:%SZ')
END_TIME=$(date -u '+%Y-%m-%dT%H:%M:%SZ')

❌ INCORRECT - Inappropriate time windows

# Too broad (entire year - too many logs)
--start-time "2025-01-01T00:00:00Z" \
--end-time "2025-12-31T23:59:59Z"

# Too narrow (1 minute - may miss logs)
--start-time "2025-04-28T08:15:00Z" \
--end-time "2025-04-28T08:16:00Z"

# Wrong timezone (not UTC)
--start-time "2025-04-28T16:00:00+08:00"

---

Verification Patterns

✅ CORRECT - Verify results

# Check if service info contains required fields
if echo "$SERVICE_INFO" | jq -e '.Resource' > /dev/null; then
  echo "✅ Service info retrieved successfully"
else
  echo "❌ Failed to get service resource ID"
fi

# Check if logs found
LOG_COUNT=$(echo "$LOGS" | jq -r '.TotalCount // 0')
if [ "$LOG_COUNT" -gt 0 ]; then
  echo "✅ Found $LOG_COUNT log entries"
else
  echo "❌ No logs found for request_id"
fi

# Check if configs exist
CONFIG_COUNT=$(echo "$CONFIGS" | jq -r '.EngineConfigs | length')
if [ "$CONFIG_COUNT" -gt 0 ]; then
  echo "✅ Found $CONFIG_COUNT configurations"
else
  echo "❌ No configurations found"
fi

❌ INCORRECT - No verification

# Assume commands always succeed
SERVICE_INFO=$(aliyun eas describe-service ...)
# Process without checking if result is valid

# No check if logs/configs found
# May process empty results

---

Summary Checklist

CLI Commands

• [ ] Use correct product names: eas, pairecservice
• [ ] Use plugin mode format: lowercase with hyphens
• [ ] Use exact parameter names from --help
• [ ] Use correct enum values (e.g., Prod/Pre, not product/prepub)
• [ ] Use UTC time format: YYYY-MM-DDTHH:MM:SSZ

Workflow

• [ ] Generate session-id (32-char lowercase hex) at start
• [ ] Include --user-agent on every aliyun API command
• [ ] Exclude --user-agent from local utility commands (configure, plugin, version)
• [ ] Confirm parameters with user before execution
• [ ] Execute commands in logical order

Environment Mapping

• [ ] Map product → Prod
• [ ] Map prepub → Pre

Error Handling

• [ ] Check command success
• [ ] Handle errors gracefully

Verification

• [ ] Verify command results
• [ ] Check required fields present
• [ ] Validate data formats
• [ ] Confirm counts when listing

---

Related Documentation

• Related Commands - Full CLI command reference
• Verification Method - Detailed verification steps
• Troubleshooting Guide - Common issues and solutions

Aliyun CLI Installation & Configuration Guide

Complete guide for installing and configuring Aliyun CLI.

Aliyun CLI 3.3.3+: Supports installing and using all published Alibaba Cloud product plugins. Make sure to upgrade to 3.3.3 or later for full plugin ecosystem coverage.

Installation

macOS

Using Homebrew (Recommended)

brew install aliyun-cli
# Upgrade to latest
brew upgrade aliyun-cli

# Verify version (>= 3.3.3)
aliyun version

Using Binary

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz

# Extract
tar -xzf aliyun-cli-macosx-latest-amd64.tgz

# Move to PATH
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

Linux

Debian/Ubuntu

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

CentOS/RHEL

# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/

# Verify
aliyun version

ARM64 Architecture

# Download ARM64 version
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-arm64.tgz

# Extract and install
tar -xzf aliyun-cli-linux-latest-arm64.tgz
sudo mv aliyun /usr/local/bin/

Windows

Using Binary 1. Download from: https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip 2. Extract the ZIP file 3. Add the directory to your PATH environment variable 4. Open new Command Prompt or PowerShell 5. Verify: aliyun version

Using PowerShell

# Download
Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip"

# Extract
Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli

# Add to PATH (requires admin privileges)
$env:Path += ";C:\aliyun-cli"
[Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::Machine)

# Verify
aliyun version

Configuration

Quick Start

aliyun configure set \
  --mode AK \
  --access-key-id <your-access-key-id> \
  --access-key-secret <your-access-key-secret> \
  --region cn-hangzhou

All aliyun configure commands support non-interactive flags, which is the recommended approach — it works in scripts, CI/CD pipelines, and agent-driven automation without hanging on stdin prompts.

Where to Get Access Keys

1. Log in to Aliyun Console: https://ram.console.aliyun.com/ 2. Navigate to: AccessKey Management 3. Create a new AccessKey pair 4. Save the secret immediately — it's only shown once

Configuration Modes

Aliyun CLI supports 6 authentication modes. All examples below use non-interactive flags.

1. AK Mode (Access Key)

Most common mode for personal accounts and scripts.

aliyun configure set \
  --mode AK \
  --access-key-id LTAI5tXXXXXXXX \
  --access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
  --region cn-hangzhou

Configuration is stored in ~/.aliyun/config.json:

{
  "current": "default",
  "profiles": [
    {
      "name": "default",
      "mode": "AK",
      "access_key_id": "LTAI5tXXXXXXXX",
      "access_key_secret": "8dXXXXXXXXXXXXXXXXXXXXXXXX",
      "region_id": "cn-hangzhou",
      "output_format": "json",
      "language": "en"
    }
  ]
}

2. StsToken Mode (Temporary Credentials)

For short-lived access (tokens expire in 1-12 hours).

aliyun configure set \
  --mode StsToken \
  --access-key-id LTAI5tXXXXXXXX \
  --access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
  --sts-token v1.0:XXXXXXXXXXXXXXXX \
  --region cn-hangzhou

Use cases: CI/CD pipelines, temporary access for external contractors, cross-account access.

3. RamRoleArn Mode (Assume RAM Role)

Assume a RAM role for elevated or cross-account access.

aliyun configure set \
  --mode RamRoleArn \
  --access-key-id LTAI5tXXXXXXXX \
  --access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
  --ram-role-arn acs:ram::123456789012:role/AdminRole \
  --role-session-name my-session \
  --region cn-hangzhou

Use cases: cross-account resource access, temporary elevated privileges, role-based access control.

4. EcsRamRole Mode (ECS Instance RAM Role)

Use the RAM role attached to an ECS instance — no credentials needed.

aliyun configure set \
  --mode EcsRamRole \
  --ram-role-name MyEcsRole \
  --region cn-hangzhou

Requirements: must be running on an ECS instance with a RAM role attached.

Use cases: scripts and automation running on ECS instances.

5. RsaKeyPair Mode (RSA Key Pair)

Use RSA key pair for authentication (generate key pair in Aliyun Console first).

aliyun configure set \
  --mode RsaKeyPair \
  --private-key /path/to/private-key.pem \
  --key-pair-name my-key-pair \
  --region cn-hangzhou

6. RamRoleArnWithEcs Mode (ECS + RAM Role)

Combine ECS instance role with RAM role assumption for cross-account access from ECS.

aliyun configure set \
  --mode RamRoleArnWithEcs \
  --ram-role-name MyEcsRole \
  --ram-role-arn acs:ram::123456789012:role/TargetRole \
  --role-session-name my-session \
  --region cn-hangzhou

Environment Variables

Highest priority - overrides config file

Access Key Mode

export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_REGION_ID=cn-hangzhou

STS Token Mode

export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_SECURITY_TOKEN=your_sts_token
export ALIBABA_CLOUD_REGION_ID=cn-hangzhou

ECS RAM Role Mode

export ALIBABA_CLOUD_ECS_METADATA=role_name

Use Case:

• CI/CD pipelines
• Docker containers
• Temporary credential override

Managing Multiple Profiles

Create Named Profiles

aliyun configure set --profile projectA \
  --mode AK \
  --access-key-id LTAI5tAAAAAAAA \
  --access-key-secret 8dAAAAAAAAAAAAAAAAAAAAAAAA \
  --region cn-hangzhou

aliyun configure set --profile projectB \
  --mode AK \
  --access-key-id LTAI5tBBBBBBBB \
  --access-key-secret 8dBBBBBBBBBBBBBBBBBBBBBBBB \
  --region cn-shanghai

Use Specific Profile

aliyun ecs describe-instances --profile projectA

export ALIBABA_CLOUD_PROFILE=projectA
aliyun ecs describe-instances   # Uses projectA

List and Switch Profiles

aliyun configure list                      # List all profiles
aliyun configure set --current projectA    # Switch default profile

Credential Priority

Credentials are loaded in this order (first found wins):

1. Command-line flag: --profile <name> 2. Environment variable: ALIBABA_CLOUD_PROFILE 3. Environment credentials: ALIBABA_CLOUD_ACCESS_KEY_ID, etc. 4. Configuration file: ~/.aliyun/config.json (current profile) 5. ECS Instance RAM Role: If running on ECS with attached role

Verification

Test Authentication

# Basic test - list regions
aliyun ecs describe-regions

# Expected output: JSON array of regions

If successful, you'll see:

{
  "Regions": {
    "Region": [
      {
        "RegionId": "cn-hangzhou",
        "RegionEndpoint": "ecs.cn-hangzhou.aliyuncs.com",
        "LocalName": "East China 1 (Hangzhou)"
      },
      ...
    ]
  },
  "RequestId": "..."
}

If failed, you'll see error messages:

• InvalidAccessKeyId.NotFound - Wrong Access Key ID
• SignatureDoesNotMatch - Wrong Access Key Secret
• InvalidSecurityToken.Expired - STS token expired (for StsToken mode)
• Forbidden.RAM - Insufficient permissions

Debug Configuration

# Show current configuration
aliyun configure get

# Test with debug logging
aliyun ecs describe-regions --log-level=debug

# Check credential provider
aliyun configure get mode

Security Best Practices

1. Use RAM Users (Not Root Account)

❌ Don't: Use Aliyun root account credentials ✅ Do: Create RAM users with specific permissions

# Create RAM user in console
# Attach only necessary policies
# Use RAM user's access keys

2. Principle of Least Privilege

Grant only the minimum permissions needed:

# Example: Read-only ECS access
# Attach policy: AliyunECSReadOnlyAccess

3. Rotate Access Keys Regularly

# Create new access key in RAM Console, then update configuration
aliyun configure set --access-key-id NEW_KEY --access-key-secret NEW_SECRET
# Delete old access key from console

4. Use STS Tokens for Temporary Access

aliyun configure set --mode StsToken \
  --access-key-id XXXX --access-key-secret XXXX \
  --sts-token XXXX --region cn-hangzhou

5. Use ECS RAM Roles When Possible

aliyun configure set --mode EcsRamRole --ram-role-name MyRole --region cn-hangzhou

6. Never Commit Credentials

# Add to .gitignore
echo "~/.aliyun/config.json" >> .gitignore

# Use environment variables in CI/CD instead

7. Secure Config File

# Restrict permissions
chmod 600 ~/.aliyun/config.json

Troubleshooting

Issue: Command Not Found

# Check installation
which aliyun

# Check PATH
echo $PATH

# Reinstall or add to PATH

Issue: Authentication Failed

# Verify configuration
aliyun configure get

# Test with debug
aliyun ecs describe-regions --log-level=debug

# Check credentials in console
# Verify access key is active

Issue: Permission Denied

# Error: Forbidden.RAM

# Check RAM user permissions
# Attach necessary policies in RAM console
# Example: AliyunECSFullAccess for ECS operations

Issue: STS Token Expired

# Error: InvalidSecurityToken.Expired

# Reconfigure with new token
aliyun configure set --mode StsToken \
  --access-key-id XXXX --access-key-secret XXXX \
  --sts-token NEW_TOKEN --region cn-hangzhou

Issue: Wrong Region

# Some resources may not exist in the specified region

# Check available regions
aliyun ecs describe-regions

# Update default region
aliyun configure set region cn-shanghai

Advanced Configuration

Custom Endpoint

# Use custom or private endpoint
export ALIBABA_CLOUD_ECS_ENDPOINT=ecs-vpc.cn-hangzhou.aliyuncs.com

Proxy Settings

# HTTP proxy
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080

# No proxy for specific domains
export NO_PROXY=localhost,127.0.0.1,.aliyuncs.com

Timeout Settings

# Connection timeout (default: 10s)
export ALIBABA_CLOUD_CONNECT_TIMEOUT=30

# Read timeout (default: 10s)
export ALIBABA_CLOUD_READ_TIMEOUT=30

Next Steps

After installation and configuration:

1. Install plugins for services you need (v3.3.3+ supports all published product plugins):

   aliyun plugin install --names ecs vpc rds

   # List all available plugins
   aliyun plugin list-remote

2. Explore commands:

   aliyun ecs --help
   aliyun fc --help

3. Read documentation:

• Command Syntax Guide
• Global Flags Reference
• Common Scenarios

References

• Official Documentation: https://help.aliyun.com/zh/cli/
• RAM Console: https://ram.console.aliyun.com/
• Access Key Management: https://ram.console.aliyun.com/manage/ak
• Plugin Repository: https://github.com/aliyun/aliyun-cli

PAI-Rec Engine Configuration Validation

Reference for scripts/validate.py — the JSON Schema + rule-based validator used by Workflow 2 (Engine Configuration Validation) in SKILL.md.

---

1. Files

All files live inside this skill — no external skill dependency.

---

2. Dependencies

• Python 3.8+
• jsonschema >=4.0,<5.0 (optional but recommended):

  pip install "jsonschema>=4.0,<5.0"

If jsonschema is not installed the script still runs but skips Schema checks and relies on the built-in rule validator only. A single warning-equivalent note is NOT emitted in this case, so install jsonschema for full coverage.

---

3. CLI Usage

# From a saved JSON file
python3 scripts/validate.py /tmp/engine-config.json

# Inline JSON string
python3 scripts/validate.py '{"RunMode":"product","RecallConfs":[...]}'

# From stdin (recommended when ConfigValue is already in a shell variable)
printf '%s' "$CONFIG_VALUE" | python3 scripts/validate.py --stdin

# Validate with experiment configs (reference-existence check)
python3 scripts/validate.py /tmp/raw_engine_config.json \
  --experiment-config /tmp/experiment_group_21.json \
  --experiment-config /tmp/experiment_44.json

Exit codes

Output format

Validation finished: <E> error(s), <W> warning(s)

  [ERROR]   <json.path>: <message>
  [WARNING] <json.path>: <message>
  ...

<json.path> uses dotted / bracketed notation, e.g. RecallConfs[3], SceneConfs.default.default.RecallNames[1], FilterConfs[0].DaoConf.

Fragment auto-wrapping

If the input is a single fragment (object containing RecallType, FilterType, or SortType at the top level) and the corresponding *Confs array is missing, the script automatically wraps it into { "<Conf>s": [ <fragment> ] } before validation. This lets you validate one recall / filter / sort entry in isolation.

---

4. Library Usage

from scripts.validate import validate_config, ValidationError

errors: list[ValidationError] = validate_config(config_dict, use_schema=True)
for e in errors:
    print(e.severity, e.path, e.message)

ValidationError fields: path, message, severity ("error" | "warning").

---

5. Rule Catalogue

5.1 Structural / Type checks (JSON Schema)

Enforced by the bundled JSON Schema (Draft-7) via jsonschema.Draft7Validator. Covers:

• Top-level keys: RunMode, ListenConf, RecallConfs, FilterConfs,

SortConfs, AlgoConfs, SceneConfs, FilterNames, SortNames, RankConf, GeneralRankConfs, FeatureConfs, UserFeatureConfs, DebugConfs, FeatureLogConfs, CallBackConfs, PipelineConfs, HologresConfs, RedisConfs, MysqlConfs, TableStoreConfs, FeatureStoreConfs, ClickHouseConfs, GraphConfs, LindormConfs, RecallEngineConfs, …

• Required fields and primitive types for every nested object.
• Enumerations for AdapterType, operation modes, etc. where representable in

JSON Schema.

5.2 Enum validation (rule-based)

5.2a Built-in recall instance names

SceneConfs.<scene>.<category>.RecallNames[] may reference the following built-in recall instances that are implemented inside the engine and do NOT need to appear in RecallConfs:

5.2b Built-in sort (rerank) instance names

SceneConfs.<scene>.<category>.SortNames[] (and the top-level SortNames[]) may reference the following built-in sort instances that are implemented inside the engine and do NOT need to appear in SortConfs. The validator treats these names as pre-defined and will not emit an "undefined sort name" error when they are referenced:

Note: if a SortConfs[*] entry is defined with SortType: ItemRankScore, the enum check in §5.2 still applies; the built-in name above covers the case where ItemRankScore is referenced directly in SortNames[] without any SortConfs definition.

5.3 Required-field checks

• RecallConfs[*]: Name, RecallType, RecallCount (positive integer).
• FilterConfs[*]: Name, FilterType.
• SortConfs[*]: Name, SortType.
• AlgoConfs[*]: Name, Type.
• Data-source adapter blocks (HologresConfs[*], RedisConfs[*], …): connection

identifiers required by Schema.

5.4 Reference-consistency (cross-section)

The validator resolves every name reference and reports errors when a referenced entity is not defined:

5.5 Data-source adapter mapping

DaoConf.AdapterType (case-insensitive) is mapped to the top-level confs key holding the connection definition; the *Name field inside DaoConf must match an entry there:

Any other AdapterType is flagged as invalid.

5.6 Business rules

• `User2ItemExposureFilter` + FeatureStore pseudo-exposure

When WriteLog == true and DaoConf.AdapterType == "featurestore", TimeInterval is required and must be a positive integer (seconds). Missing or non-positive values are flagged as errors.

• `PriorityAdjustCountFilter` accumulator mode

Inside AdjustCountConfs[*], for entries with Type == "accumulator" (the default), the sequence of Count values must be strictly increasing. A non-increasing pair emits a warning with a hint that Type = "fix" should be used for independent per-recall caps. Reported once per filter to avoid duplicates.

• *`PipelineConfs[].Name` uniqueness**

All pipeline names must be globally unique across PipelineConfs.

• `DebugConfs.Rate`

Must be an integer in [0, 100] (inclusive). Values outside this range or of non-integer types are errors.

5.7 Duplicate-name detection

Within each of RecallConfs, FilterConfs, SortConfs, AlgoConfs the Name field must be unique; duplicates yield errors at the later occurrence.

---

6. How the Skill Uses the Validator

Workflow 2 (SKILL.md §"Workflow 2") runs the following sequence after fetching the configuration from pairecservice get-engine-config:

1. Extract the ConfigValue string from the API response. 2. Pipe it to python3 scripts/validate.py --stdin. 3. Capture stdout + exit code. 4. Translate the output into the evidence-grounded ✅ / ⚠️ / ❌ report defined in SKILL.md §"Workflow 2 / Step 4". Every ⚠️ / ❌ line MUST quote the exact [SEVERITY] path: message produced by the script (or the exact JSON fragment of ConfigValue for findings that come from manual inspection outside the script's rule set).

---

7. Experiment Config Validation

When --experiment-config <file> is passed, the script validates experiment override parameters against the base engine config.

Recognized predefined keys

Keys not matching any pattern above are treated as user-defined custom parameters and skipped without error.

Validation rules

1. Type check — predefined keys must have the expected value type. 2. Reference existence — names in list-type keys must exist in the corresponding base config section:

• {cat}.RecallNames values → RecallConfs[].Name (exception: ContextItemRecall)
• filterNames values → FilterConfs[].Name (exception: UniqueFilter)
• {cat}.SortNames values → SortConfs[].Name (exception: ItemRankScore)
• rankconf.RankAlgoList values → AlgoConfs[].Name
• sort.{X} → X must exist in SortConfs[].Name
• recall.{X} → X must exist in RecallConfs[].Name

Auto-detection

The script auto-detects the API response wrapper:

• Base config file: if it contains a ConfigValue string field, that field is

parsed as the actual engine config.

• Experiment file: the Config string field is parsed as the experiment

parameters JSON.

---

8. Extending the Validator

When adding new rules:

1. Prefer expressing structural constraints in the bundled JSON Schema (Draft-7). 2. Put cross-section / business-logic rules in PairecConfigValidator._validate_* methods of scripts/validate.py. 3. Keep ValidationError.severity aligned with real impact:

• error — config will break at runtime or is definitely invalid.
• warning — likely misconfiguration that has a legitimate alternative.

4. Add/extend the enum sets (VALID_RECALL_TYPES, VALID_FILTER_TYPES, VALID_SORT_TYPES, VALID_RESPONSE_FUNC_NAMES, BUILTIN_RECALL_NAMES, DATASOURCE_ADAPTER_MAP) at the top of validate.py when new types are supported. For VALID_RESPONSE_FUNC_NAMES, keep in sync with upstream pairec/algorithm/eas/client.go SetResponseFunc().

PAI-Rec Engine Configuration Examples

This document provides sample PAI-Rec engine configurations and common patterns to help with validation and diagnosis.

Configuration Structure Overview

A typical PAI-Rec engine configuration contains these main sections:

{
  "scene_id": "default",
  "recall": {
    "modules": []
  },
  "rank": {
    "modules": []
  },
  "filter": {
    "rules": []
  },
  "experiment": {
    "layers": []
  }
}

---

Basic Configuration Example

Minimal Working Configuration

{
  "scene_id": "embedding_recall_scene",
  "recall": {
    "modules": [
      {
        "name": "embedding_recall",
        "type": "vector",
        "params": {
          "table": "user_item_embedding",
          "index_type": "hnsw",
          "topk": 100,
          "timeout_ms": 200
        }
      }
    ]
  },
  "rank": {
    "modules": [
      {
        "name": "simple_scorer",
        "type": "feature_based",
        "params": {
          "features": ["user_age", "item_category", "click_rate"],
          "model_endpoint": "http://eas-service.cn-hangzhou.aliyuncs.com/scoring"
        }
      }
    ]
  },
  "filter": {
    "rules": [
      {
        "name": "basic_filter",
        "type": "threshold",
        "params": {
          "min_score": 0.1,
          "max_results": 50
        }
      }
    ]
  }
}

---

Recall Module Examples

1. Vector Similarity Recall

{
  "recall": {
    "modules": [
      {
        "name": "user_embedding_recall",
        "type": "vector",
        "params": {
          "table": "user_item_embeddings_v2",
          "index_type": "hnsw",
          "metric": "cosine",
          "topk": 500,
          "timeout_ms": 300,
          "filters": {
            "category": "electronics",
            "status": "active"
          }
        }
      }
    ]
  }
}

Common Parameters:

• table: Source embedding table name
• index_type: hnsw, flat, ivf
• metric: cosine, euclidean, inner_product
• topk: Number of results to retrieve
• timeout_ms: Query timeout
• filters: Pre-filter conditions

Validation Checks:

• ✅ Table exists and is accessible
• ✅ Index type is supported
• ✅ Metric matches index configuration
• ✅ Timeout is reasonable (50-500ms)
• ⚠️ topk too large (>1000) may impact performance

---

2. Collaborative Filtering Recall

{
  "recall": {
    "modules": [
      {
        "name": "cf_recall",
        "type": "collaborative_filtering",
        "params": {
          "table": "user_behavior_matrix",
          "algorithm": "item_cf",
          "topk": 200,
          "min_similarity": 0.3,
          "decay_days": 30
        }
      }
    ]
  }
}

Common Parameters:

• algorithm: user_cf, item_cf, swing
• min_similarity: Minimum similarity threshold
• decay_days: Time decay for historical data

---

3. Hot Items Recall

{
  "recall": {
    "modules": [
      {
        "name": "hot_items",
        "type": "popular",
        "params": {
          "table": "item_popularity_stats",
          "time_window_hours": 24,
          "category_aware": true,
          "topk": 100
        }
      }
    ]
  }
}

---

4. Multiple Recall Modules

{
  "recall": {
    "modules": [
      {
        "name": "embedding_recall",
        "type": "vector",
        "weight": 0.5,
        "params": {
          "table": "embeddings",
          "topk": 300
        }
      },
      {
        "name": "cf_recall",
        "type": "collaborative_filtering",
        "weight": 0.3,
        "params": {
          "table": "user_behavior",
          "topk": 200
        }
      },
      {
        "name": "hot_items",
        "type": "popular",
        "weight": 0.2,
        "params": {
          "topk": 100
        }
      }
    ],
    "merge_strategy": "weighted",
    "final_topk": 500
  }
}

Validation Checks:

• ✅ Weights sum to 1.0 (if using weighted strategy)
• ✅ No duplicate module names
• ✅ final_topk >= max individual topk
• ⚠️ Too many modules may slow processing

---

Rank Module Examples

1. Feature-Based Ranking

{
  "rank": {
    "modules": [
      {
        "name": "feature_ranker",
        "type": "feature_based",
        "params": {
          "features": [
            "user_age",
            "user_gender",
            "item_category",
            "item_price",
            "ctr_7d",
            "cvr_7d"
          ],
          "feature_tables": {
            "user_features": "user_profile_v1",
            "item_features": "item_attributes_v1",
            "behavior_features": "user_item_stats_v1"
          },
          "model_endpoint": "http://eas.cn-hangzhou.aliyuncs.com/rank_model_v2",
          "timeout_ms": 100,
          "batch_size": 50
        }
      }
    ]
  }
}

Validation Checks:

• ✅ All feature tables accessible
• ✅ Model endpoint reachable
• ✅ Batch size reasonable (20-100)
• ✅ Timeout appropriate for batch size
• ⚠️ Too many features (>50) may slow inference

---

2. Multi-Stage Ranking

{
  "rank": {
    "modules": [
      {
        "name": "coarse_ranker",
        "type": "simple_scorer",
        "stage": 1,
        "params": {
          "features": ["ctr_7d", "cvr_7d"],
          "topk": 200
        }
      },
      {
        "name": "fine_ranker",
        "type": "deep_model",
        "stage": 2,
        "params": {
          "model_endpoint": "http://eas.cn-hangzhou.aliyuncs.com/dnn_ranker",
          "features": ["all"],
          "topk": 50
        }
      }
    ]
  }
}

---

Filter Module Examples

1. Threshold Filters

{
  "filter": {
    "rules": [
      {
        "name": "score_threshold",
        "type": "threshold",
        "params": {
          "field": "score",
          "min_value": 0.1,
          "max_value": 1.0
        }
      },
      {
        "name": "diversity_filter",
        "type": "diversity",
        "params": {
          "field": "category",
          "max_same_category": 5
        }
      }
    ]
  }
}

---

2. Bloom Filter (Dedupe)

{
  "filter": {
    "rules": [
      {
        "name": "dedupe_filter",
        "type": "bloom",
        "params": {
          "table": "user_exposure_history",
          "window_days": 7,
          "field": "item_id"
        }
      }
    ]
  }
}

---

3. Business Rules Filter

{
  "filter": {
    "rules": [
      {
        "name": "business_rules",
        "type": "custom",
        "params": {
          "rules": [
            {
              "condition": "item_status == 'active'",
              "action": "keep"
            },
            {
              "condition": "item_price > user_budget",
              "action": "remove"
            },
            {
              "condition": "item_stock < 1",
              "action": "remove"
            }
          ]
        }
      }
    ]
  }
}

Validation Checks:

• ✅ Condition syntax is valid
• ✅ Referenced fields exist
• ✅ Actions are valid (keep/remove)
• ⚠️ Complex rules may impact performance

---

Experiment Configuration Examples

1. A/B Testing

{
  "experiment": {
    "enabled": true,
    "layers": [
      {
        "layer_id": "rank_model_ab_test",
        "experiments": [
          {
            "exp_id": "exp_baseline",
            "traffic": 0.5,
            "config": {
              "rank": {
                "modules": [
                  {
                    "name": "baseline_ranker",
                    "model_endpoint": "http://eas/baseline_model"
                  }
                ]
              }
            }
          },
          {
            "exp_id": "exp_new_model",
            "traffic": 0.5,
            "config": {
              "rank": {
                "modules": [
                  {
                    "name": "new_ranker",
                    "model_endpoint": "http://eas/new_model_v2"
                  }
                ]
              }
            }
          }
        ]
      }
    ]
  }
}

Validation Checks:

• ✅ Traffic percentages sum to 1.0
• ✅ Experiment IDs are unique
• ✅ Config overrides are valid
• ⚠️ Too many experiments may complicate analysis

---

Common Configuration Patterns

Pattern 1: E-commerce Recommendation

{
  "scene_id": "product_recommendation",
  "recall": {
    "modules": [
      {
        "name": "user_embedding",
        "type": "vector",
        "params": {
          "table": "user_item_embeddings",
          "topk": 300
        }
      },
      {
        "name": "browse_history_cf",
        "type": "collaborative_filtering",
        "params": {
          "table": "browse_behavior",
          "topk": 200
        }
      },
      {
        "name": "trending_products",
        "type": "popular",
        "params": {
          "time_window_hours": 24,
          "topk": 100
        }
      }
    ],
    "merge_strategy": "weighted"
  },
  "rank": {
    "modules": [
      {
        "name": "ctr_cvr_ranker",
        "type": "feature_based",
        "params": {
          "features": ["user_profile", "item_attributes", "ctr_cvr_features"],
          "model_endpoint": "http://eas/rank_model"
        }
      }
    ]
  },
  "filter": {
    "rules": [
      {
        "name": "in_stock_filter",
        "type": "custom",
        "params": {
          "condition": "item_stock > 0"
        }
      },
      {
        "name": "exposure_dedupe",
        "type": "bloom",
        "params": {
          "table": "user_exposure",
          "window_days": 3
        }
      },
      {
        "name": "diversity",
        "type": "diversity",
        "params": {
          "field": "category",
          "max_same_category": 3
        }
      }
    ]
  }
}

---

Pattern 2: Content Recommendation

{
  "scene_id": "news_feed",
  "recall": {
    "modules": [
      {
        "name": "content_embedding",
        "type": "vector",
        "params": {
          "table": "article_embeddings",
          "topk": 500,
          "filters": {
            "publish_time": "last_7_days"
          }
        }
      },
      {
        "name": "hot_news",
        "type": "popular",
        "params": {
          "time_window_hours": 6,
          "topk": 100
        }
      }
    ]
  },
  "rank": {
    "modules": [
      {
        "name": "engagement_ranker",
        "type": "feature_based",
        "params": {
          "features": ["read_time", "comment_count", "share_count", "user_interest_match"],
          "model_endpoint": "http://eas/news_ranker"
        }
      }
    ]
  },
  "filter": {
    "rules": [
      {
        "name": "freshness_filter",
        "type": "threshold",
        "params": {
          "field": "publish_time",
          "min_value": "now-7d"
        }
      },
      {
        "name": "read_history_dedupe",
        "type": "bloom",
        "params": {
          "table": "user_read_history",
          "window_days": 30
        }
      }
    ]
  }
}

---

Validation Checklists by Section

Recall Configuration Validation

• [ ] All table references exist and are accessible
• [ ] Index types match table configurations
• [ ] topk values are reasonable (typically 50-1000)
• [ ] Timeout values appropriate (50-500ms)
• [ ] Filter conditions use valid fields
• [ ] Multiple modules have consistent merge strategy
• [ ] Weights sum to 1.0 (if weighted merge)

Rank Configuration Validation

• [ ] Feature names match available features
• [ ] Feature tables accessible in target environment
• [ ] Model endpoints are reachable
• [ ] Batch sizes appropriate (20-100)
• [ ] Timeout accounts for batch processing
• [ ] Multi-stage ranking has proper topk cascade
• [ ] No circular dependencies

Filter Configuration Validation

• [ ] Threshold values are reasonable
• [ ] Referenced fields exist
• [ ] Bloom filter tables accessible
• [ ] Custom rule syntax is valid
• [ ] Diversity constraints make sense
• [ ] Filter order is optimal (cheap filters first)

Experiment Configuration Validation

• [ ] Traffic splits sum to 1.0
• [ ] Experiment IDs are unique
• [ ] Config overrides are syntactically valid
• [ ] Layer definitions don't conflict
• [ ] Experiment is enabled/disabled appropriately

---

Common Configuration Issues

Issue 1: Items Size Not Enough

Error:

{
  "code": 299,
  "msg": "items size not enough"
}

Possible Causes: 1. Recall topk too low 2. Filters too aggressive 3. No data in recall tables 4. User has insufficient history

Configuration Fix:

{
  "recall": {
    "modules": [
      {
        "topk": 500  // Increased from 100
      }
    ]
  },
  "filter": {
    "rules": [
      {
        "min_score": 0.05  // Lowered from 0.1
      }
    ]
  }
}

---

Issue 2: Timeout Errors

Error:

{
  "code": 500,
  "msg": "request timeout"
}

Possible Causes: 1. Recall timeout too low 2. Too many recall modules 3. Large batch size in ranking 4. Slow model endpoint

Configuration Fix:

{
  "recall": {
    "modules": [
      {
        "params": {
          "timeout_ms": 500  // Increased from 200
        }
      }
    ]
  },
  "rank": {
    "modules": [
      {
        "params": {
          "batch_size": 30,  // Reduced from 100
          "timeout_ms": 200
        }
      }
    ]
  }
}

---

Issue 3: Environment Mismatch

Error:

Resource 'prod_user_features' not found

Cause: Using production table reference in Pre environment

Configuration Fix:

{
  "rank": {
    "modules": [
      {
        "params": {
          "feature_tables": {
            "user_features": "pre_user_features_v1"  // Changed from prod_
          }
        }
      }
    ]
  }
}

---

Best Practices

1. Naming Conventions

• Use descriptive module names: user_embedding_recall, not recall1
• Include version in table names: user_profile_v2
• Prefix tables by environment: prod_, pre_

2. Performance Optimization

• Set appropriate topk at each stage (cascade: 1000 → 200 → 50)
• Use cheap filters before expensive ones
• Batch model inference when possible
• Set reasonable timeouts (recall: 200-500ms, rank: 100-200ms)

3. Robustness

• Always include fallback recall (e.g., hot items)
• Set min/max constraints on values
• Handle missing features gracefully
• Use default values for optional parameters

4. Experimentation

• Start with small traffic splits (5%-10%)
• Keep baseline configuration stable
• Document experiment hypotheses
• Monitor key metrics

---

Related Documentation

• Verification Method - How to validate configurations
• Troubleshooting Guide - Common issues and solutions
• Related Commands - CLI commands for config management

RAM Policies for PAI-Rec Diagnosis Skill

This document lists all RAM (Resource Access Management) permissions required by the PAI-Rec Engine Diagnosis and Configuration Validation skill.

Overview

This skill requires read-only permissions for:

• PAI-EAS (Elastic Algorithm Service) - for service information and logs
• PAI-RecService - for engine configuration management

Required Permissions

PAI-EAS Permissions

PAI-RecService Permissions

Complete RAM Policy Document

Minimal Policy (Read-Only)

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "eas:DescribeService",
        "eas:DescribeServiceLog"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "pairecservice:ListEngineConfigs",
        "pairecservice:GetEngineConfig",
        "pairecservice:GetExperimentGroup",
        "pairecservice:GetExperiment"
      ],
      "Resource": "*"
    }
  ]
}

Resource-Specific Policy (Recommended)

For better security, restrict access to specific resources:

{
  "Version": "1",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "eas:DescribeService",
        "eas:DescribeServiceLog"
      ],
      "Resource": [
        "acs:eas:*:*:service/<your-service-name>",
        "acs:eas:*:*:cluster/<your-cluster-id>"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "pairecservice:ListEngineConfigs",
        "pairecservice:GetEngineConfig",
        "pairecservice:GetExperimentGroup",
        "pairecservice:GetExperiment"
      ],
      "Resource": [
        "acs:pairecservice:*:*:instance/<your-instance-id>"
      ]
    }
  ]
}

Permission Scope

Service-Level Permissions

• EAS Service: Read access to service configurations and logs
• PAI-Rec Instance: Read access to engine configurations

Data-Level Permissions

• No write permissions required
• No resource creation/deletion permissions required
• Read-only access to configurations and logs

Applying Permissions

Using RAM Console

1. Log in to RAM Console 2. Navigate to Users or Roles 3. Select the target principal 4. Click Add Permissions 5. Choose Custom Policy and paste the policy JSON above 6. Click OK to apply

Using Aliyun CLI

Create a custom policy:

aliyun ram create-policy \
  --policy-name PAIRecDiagnosisReadOnly \
  --policy-document '{
    "Version": "1",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "eas:DescribeService",
          "eas:DescribeServiceLog",
          "pairecservice:ListEngineConfigs",
          "pairecservice:GetEngineConfig",
          "pairecservice:GetExperimentGroup",
          "pairecservice:GetExperiment"
        ],
        "Resource": "*"
      }
    ]
  }'

Attach policy to user:

aliyun ram attach-policy-to-user \
  --policy-name PAIRecDiagnosisReadOnly \
  --policy-type Custom \
  --user-name <your-user-name>

Troubleshooting Permission Issues

Common Permission Errors

1. Forbidden.RAM

• Error: User not authorized to perform operation
• Solution: Ensure all required permissions are granted
• Check: Verify policy is attached and resources match

2. InvalidAccessKeyId.NotFound

• Error: Access key does not exist
• Solution: Verify credentials are configured correctly
• Check: Run aliyun configure list

3. Forbidden.SubUser

• Error: Sub-account lacks permissions
• Solution: Contact account admin to grant permissions
• Check: Verify you're using the correct account

Verification Steps

After applying permissions, verify access:

# Test EAS service access
aliyun eas describe-service \
  --cluster-id <cluster-id> \
  --service-name <service-name>

# Test PAI-Rec config access
aliyun pairecservice list-engine-configs \
  --instance-id <instance-id>

Security Best Practices

1. Principle of Least Privilege: Grant only required permissions 2. Resource-Specific Policies: Limit access to specific instances/services 3. Regular Audits: Review and update permissions periodically 4. Separate Roles: Use different roles for different environments (Prod/Pre) 5. Credential Rotation: Rotate access keys regularly 6. Audit Logging: Enable ActionTrail to track API calls

Related Documentation

• RAM Overview
• EAS API Reference
• PAI-RecService API Reference

Related CLI Commands

This document provides a comprehensive reference of all Alibaba Cloud CLI commands used in the PAI-Rec Engine Diagnosis and Configuration Validation skill.

PAI-EAS Commands

describe-service

Queries the details about a PAI-EAS service.

Usage:

aliyun eas describe-service \
  --cluster-id <cluster-id> \
  --service-name <service-name>

Parameters:

• --cluster-id (required): The ID of the region where the service is deployed
• --service-name (required): The service name
• --region (optional): Override region ID

Response Fields:

• ServiceId: Service identifier
• ServiceName: Name of the service
• Resource: EAS service resource ID (e.g., eas-r-xxxxxxxxxxxx)
• ServiceConfig: Service configuration including environment variables
• Status: Service status (Running/Stopped/etc.)
• CurrentVersion: Current deployed version
• Metadata: Additional service metadata

Example:

aliyun eas describe-service \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall

Common Use Cases:

• Get EAS service resource ID
• Extract environment variables (INSTANCE_ID, CONFIG_NAME, REGION, PAIREC_ENVIRONMENT)
• Check service status before diagnosis
• Verify service configuration

---

describe-service-log

Queries the information about the logs of a PAI-EAS service.

Usage:

aliyun eas describe-service-log \
  --cluster-id <cluster-id> \
  --service-name <service-name> \
  [--keyword <keyword>] \
  [--start-time <utc-time>] \
  [--end-time <utc-time>] \
  [--page-num <number>] \
  [--page-size <number>]

Parameters:

• --cluster-id (required): The ID of the region where the service is deployed
• --service-name (required): The service name
• --keyword (optional, strongly recommended for request-level diagnosis): Keyword to filter logs, e.g., request_id. See pitfall note below.
• --start-time (optional): Beginning of time range. Format MUST be yyyy-MM-dd HH:mm:ss in UTC (space separator, no T / no Z). ISO-8601 like 2025-04-28T00:00:00Z will be rejected with InvalidParameter.
• --end-time (optional): End of time range (same format as --start-time).
• --page-num (optional): Page number (default: 1)
• --page-size (optional): Entries per page (default: 500)
• --container-name (optional): Container name
• --instance-name (optional): Instance name
• --ip (optional): Instance IP address
• --previous (optional): Query logs before last restart

[CRITICAL] Pitfall — keyword + time range returns no business logs: When --keyword is combined with --start-time / --end-time, PAI-Rec application logs (controller.go, feed.go, recall.go, rank_service.go, etc.) are silently dropped by the CLI, even if the time window covers the real log timestamp. Only /bin/sh wrapper heartbeats and 502 Bad Gateway noise remain. For request-level diagnosis, always use --keyword without --start-time / --end-time.

Response Fields:

• Logs: Array of log entries
• PageNum: Current page number
• TotalCount: Total log count
• TotalPageNum: Total pages

Example — recommended pattern for request-level diagnosis (keyword only):

aliyun eas describe-service-log \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --keyword "941b4e14-d1c5-489f-a184-b2b17f8b4fdb" \
  --page-size 500

Example — broad time-window scan without keyword (infrastructure logs only):

aliyun eas describe-service-log \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --start-time "2025-04-28 08:00:00" \
  --end-time   "2025-04-28 09:00:00" \
  --page-size 500

Common Use Cases:

• Trace request processing by request_id (use keyword-only pattern)
• Debug error messages
• Monitor service behavior
• Analyze performance issues

Query Strategy Recommendations:

• Request-level diagnosis: --keyword <request_id> only, no time range
• Service-level scan: omit --keyword, use time window in yyyy-MM-dd HH:mm:ss UTC
• Log retention: EAS logs have a limited retention window; diagnose issues promptly

---

PAI-RecService Commands

list-engine-configs

Lists engine configuration versions for a PAI-Rec instance.

Usage:

aliyun pairecservice list-engine-configs \
  --instance-id <instance-id> \
  [--environment <Prod|Pre>] \
  [--name <config-name>] \
  [--status <status>] \
  [--version <version>] \
  [--page-number <number>] \
  [--page-size <number>]

Parameters:

• --instance-id (required): The PAI-Rec instance ID
• --environment (optional): Environment filter (Prod or Pre)
• --name (optional but strongly recommended when the config name is known): Exact-match server-side filter on configuration name. Always pass this when you already know the target config (e.g. obtained from ServiceConfig.envs.CONFIG_NAME); omitting it returns the entire instance inventory and may exceed the default page size.
• --status (optional): Status filter (e.g., Released, Draft, Archived)
• --version (optional): Version filter
• --page-number (optional): Page number for pagination
• --page-size (optional): Number of results per page

Response Fields:

• EngineConfigs: Array of engine configurations
• EngineConfigId: Configuration version ID
• Name: Configuration name
• Version: Version number
• Status: Configuration status
• Environment: Target environment
• GmtCreateTime: Creation timestamp
• GmtModifiedTime: Last modification timestamp

Example:

aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod \
  --status Released \
  --name my_engine_config

Common Use Cases:

• Find released configurations
• List all versions of a configuration
• Filter by environment (Prod/Pre)
• Select configuration for validation

Environment Mapping:

• Service environment product → CLI parameter Prod
• Service environment prepub → CLI parameter Pre

---

get-engine-config

Retrieves detailed information about a specific engine configuration.

Usage:

aliyun pairecservice get-engine-config \
  --instance-id <instance-id> \
  --engine-config-id <config-id>

Parameters:

• --instance-id (required): The PAI-Rec instance ID
• --engine-config-id (required): The specific configuration version ID

Response Fields:

• EngineConfig: Configuration object
• EngineConfigId: Configuration ID
• Name: Configuration name
• Version: Version number
• Status: Current status
• Environment: Target environment
• ConfigValue: The actual configuration content (JSON/YAML)
• Description: Configuration description
• GmtCreateTime: Creation time
• GmtModifiedTime: Last modified time

Example:

aliyun pairecservice get-engine-config \
  --instance-id pairec-cn-xxxxx \
  --engine-config-id config-12345

Common Use Cases:

• Retrieve configuration for validation
• Export configuration for backup
• Compare different versions
• Debug configuration issues

ConfigValue Structure: The ConfigValue field contains the engine configuration in JSON or YAML format, typically including:

• Recall configurations
• Ranking configurations
• Filter rules
• Scene definitions
• Feature mappings
• Model endpoints

---

Utility Commands

aliyun configure list

Checks current credential configuration status.

Usage:

aliyun configure list

Output: Lists all configured profiles with their authentication method (AK, STS, OAuth).

Common Use Cases:

• Verify credentials before running commands
• Check which profile is active
• Troubleshoot authentication issues

---

Per-command --user-agent (Observability)

Every aliyun CLI command that calls a cloud API MUST include --user-agent for request tracing. Local utility commands (e.g. configure, plugin, version) do not support this flag and should be excluded.

Usage:

aliyun <service> <command> [options] \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/{session-id}

Common Use Cases:

• Track skill invocations end-to-end via a unique session-id (32-char lowercase hex)
• Correlate multiple CLI calls within a single skill session

---

aliyun plugin update

Updates all installed CLI plugins to latest versions.

Usage:

aliyun plugin update

Common Use Cases:

• Ensure latest plugin versions before execution
• Required as part of skill prerequisites

---

Command Chaining Examples

Full Diagnosis Flow

# 1. Session-id is a 32-char lowercase hex string generated once per session
# Example: a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 2. Get service info
aliyun eas describe-service \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 3. Query logs by request_id (keyword-only pattern — do NOT add --start-time / --end-time)
aliyun eas describe-service-log \
  --cluster-id cn-hangzhou \
  --service-name embedding_recall \
  --keyword "941b4e14-d1c5-489f-a184-b2b17f8b4fdb" \
  --page-size 500 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 4. List engine configs (always pass --name when the config name is known)
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod \
  --status Released \
  --name embedding_config \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 5. Get specific config
aliyun pairecservice get-engine-config \
  --instance-id pairec-cn-xxxxx \
  --engine-config-id config-12345 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

Configuration Validation Flow

# 1. Session-id is a 32-char lowercase hex string generated once per session
# Example: a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 2. List available versions
aliyun pairecservice list-engine-configs \
  --instance-id pairec-cn-xxxxx \
  --environment Prod \
  --name my_config \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

# 3. Get specific version for validation
aliyun pairecservice get-engine-config \
  --instance-id pairec-cn-xxxxx \
  --engine-config-id config-67890 \
  --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6

Error Handling

Common Error Codes

Debugging Commands

Check command execution details:

aliyun eas describe-service \
  --cluster-id cn-hangzhou \
  --service-name test \
  --log-level DEBUG

Dry-run mode (no actual execution):

aliyun eas describe-service \
  --cluster-id cn-hangzhou \
  --service-name test \
  --cli-dry-run

Related Documentation

• EAS API Reference
• PAI-RecService API Reference
• Aliyun CLI User Guide

Troubleshooting Guide

This guide provides solutions to common issues encountered when using the PAI-Rec Engine Diagnosis and Configuration Validation skill.

---

[MUST READ] Reporting Principle — Evidence-Only Conclusions

When delivering a diagnosis conclusion to the user, strictly follow the evidence-only rule defined in SKILL.md (Step 6 → [MUST] Evidence-only reporting rule).

Allowed in a conclusion:

• Exact log lines actually retrieved from EAS (with file:line, level, message)
• Exact config fragments actually retrieved from the engine configuration
• The direct causal chain that connects the above evidence to the observed API response
• An explicit statement of what evidence is still missing, if any

NOT allowed (unless the user explicitly asks for it in a follow-up):

• Speculative root causes that are not directly visible in the logs or config
• Guesses about client-side behavior, upstream systems, or data sources
• Fix recommendations or remediation steps
• Conditional "if X then Y" scenarios
• Tangential best-practice advice (security, fallback design, naming, etc.)

This principle applies to every troubleshooting item in this guide: the workarounds below describe how to gather evidence; they do NOT authorize adding speculation on top of that evidence when reporting to the user.

---

Table of Contents

1. CLI and Authentication Issues 2. Service Access Issues 3. Log Query Issues 4. Configuration Retrieval Issues 5. Engine API Issues 6. Configuration Validation Issues 7. Performance Issues

---

CLI and Authentication Issues

Issue: Aliyun CLI Not Found

Symptoms:

bash: aliyun: command not found

Solution: 1. Install Aliyun CLI:

   curl -fsSL https://aliyuncli.alicdn.com/setup.sh | bash

2. Reload shell configuration:

   source ~/.bashrc  # or ~/.zshrc

3. Verify installation:

   aliyun version

Related Documentation: CLI Installation Guide

---

Issue: Aliyun CLI Version Too Old

Symptoms:

aliyun version
# Output: 3.0.x (needs >= 3.3.3)

Solution: 1. Update Aliyun CLI:

   curl -fsSL https://aliyuncli.alicdn.com/setup.sh | bash

2. Verify update:

   aliyun version  # Should show >= 3.3.3

---

Issue: No Credentials Configured

Symptoms:

Error: No credential configured

Solution: 1. Check current configuration:

   aliyun configure list

2. If empty, configure credentials outside this session:

   # In a separate terminal
   aliyun configure

3. Provide credentials when prompted:

• Access Key ID
• Access Key Secret
• Region (e.g., cn-hangzhou)

Security Note: NEVER type credentials in chat or skill execution context.

---

Issue: Forbidden.RAM - Insufficient Permissions

Symptoms:

{
  "Code": "Forbidden.RAM",
  "Message": "User not authorized to perform this operation"
}

Solution: 1. Check required permissions in RAM Policies

2. Request permissions from account administrator

3. Verify permissions granted:

   aliyun ram get-user-policy \
     --user-name <your-username> \
     --policy-name <policy-name>

4. Test access after permissions granted:

   aliyun eas describe-service \
     --cluster-id <cluster-id> \
     --service-name <service-name>

Common Missing Permissions:

• eas:DescribeService
• eas:DescribeServiceLog
• pairecservice:ListEngineConfigs
• pairecservice:GetEngineConfig

---

Issue: Missing --user-agent Flag

Symptoms: CLI calls succeed but cannot be traced back to this skill session.

Solution: Ensure a session-id (32-char lowercase hex string) was generated at session start and every aliyun API command includes:

--user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-rec-diagnosis/{session-id}

Prevention: Generate the session-id once at the very start of the workflow. Local utility commands (configure, plugin, version) do not support --user-agent and should be excluded.

---

Service Access Issues

Issue: Service Not Found

Symptoms:

{
  "Code": "InvalidService.NotFound",
  "Message": "The specified service does not exist"
}

Diagnostic Steps: 1. Verify service name spelling:

   # List all services in cluster
   aliyun eas list-services --cluster-id <cluster-id>

2. Check if service exists in different cluster/region

3. Verify cluster ID is correct

Solution:

• Use exact service name from service list
• Specify correct cluster ID
• Check if service has been deleted

---

Issue: Cluster ID Unknown

Symptoms: Need cluster ID but don't know it

Solution: 1. Cluster ID is typically the region ID:

• cn-hangzhou
• cn-shanghai
• cn-beijing
• etc.

2. Or list services to find cluster:

   aliyun eas list-services --region <region-id>

3. Check EAS console for cluster information: EAS Console

---

Issue: Service Configuration Missing Environment Variables

Symptoms: Service info retrieved but environment variables not found

Diagnostic Steps: 1. Check ServiceConfig structure:

   aliyun eas describe-service \
     --cluster-id <cluster-id> \
     --service-name <service-name> \
     --cli-query 'ServiceConfig'

2. Look for environment variables in different locations:

• ServiceConfig.envs
• ServiceConfig.metadata
• ServiceConfig.configs

Solution:

• If env vars missing, service may not be PAI-Rec service
• Check service deployment configuration
• Consult service owner for configuration details

---

Log Query Issues

Issue: No Logs Found for Request ID

Symptoms:

{
  "Logs": [],
  "TotalCount": 0
}

#1 root cause — combining `--keyword` with a time range:

aliyun eas describe-service-log has a well-known quirk: when --keyword is supplied together with --start-time / --end-time, PAI-Rec application (business) logs are silently dropped even if the window covers the real log time; only /bin/sh wrapper heartbeats and 502 Bad Gateway noise remain. The correct pattern is to use --keyword without any time parameters:

# [CORRECT] Keyword-only lookup returns the full business trace
aliyun eas describe-service-log \
  --cluster-id <cluster-id> \
  --service-name <service-name> \
  --keyword <request-id> \
  --page-size 500

# [WRONG] Adding a time range drops all controller.go/feed.go/recall.go lines
aliyun eas describe-service-log \
  --cluster-id <cluster-id> \
  --service-name <service-name> \
  --keyword <request-id> \
  --start-time "2025-04-28 06:00:00" \
  --end-time   "2025-04-28 10:00:00"

Other diagnostic steps (only if keyword-only lookup is already zero):

1. Verify request ID format:

• Exact match, case-sensitive
• No truncation, full UUID
• Copy directly from the API response request_id field

2. Verify log retention:

• EAS log retention is limited; old requests may no longer be queryable
• Diagnose promptly after the issue occurs

3. Cross-check the service actually received the request:

• Confirm the service name and cluster/region match the endpoint that was called
• Consider traffic mirroring rules (ExtraData.mirror) which may redirect traffic to another service

4. Sanity check with a broader time-window scan (no keyword):

   # Dumps raw /bin/sh wrapper output only (good for confirming the service is producing logs)
   aliyun eas describe-service-log \
     --cluster-id <cluster-id> \
     --service-name <service-name> \
     --start-time "2025-04-28 01:30:00" \
     --end-time   "2025-04-28 02:10:00" \
     --page-size 500

Solutions:

• Prefer keyword-only lookup for request-level diagnosis
• Verify request ID exactly matches API response
• Diagnose promptly before log retention expires

---

Issue: Too Many Logs Returned

Symptoms: Thousands of log entries, hard to analyze

Solution: 1. Use the full `request_id` as keyword (do NOT add a time range):

   aliyun eas describe-service-log \
     --cluster-id <cluster-id> \
     --service-name <service-name> \
     --keyword "941b4e14-d1c5-489f-a184-b2b17f8b4fdb" \
     --page-size 500

A single request's full trace is usually < 30 entries.

2. For broad time-window scans without keyword, use pagination:

   aliyun eas describe-service-log \
     --cluster-id <cluster-id> \
     --service-name <service-name> \
     --start-time "2025-04-28 08:10:00" \
     --end-time   "2025-04-28 08:20:00" \
     --page-num 1 \
     --page-size 100

Time format must be yyyy-MM-dd HH:mm:ss UTC (space separator, no T / no Z).

---

Issue: Log Timestamps Don't Match

Symptoms: Log timestamps in different timezone than expected; --start-time / --end-time rejected with InvalidParameter

Solution: 1. Use the exact format `yyyy-MM-dd HH:mm:ss` in UTC:

• Correct: 2025-04-28 08:15:23
• Wrong: 2025-04-28T08:15:23Z (ISO-8601 is rejected)

2. Convert local time to UTC (example for CST / UTC+8):

   Local (Beijing): 2025-04-28 16:15:23
   UTC:             2025-04-28 08:15:23

3. Note: Log lines shown in the EAS console appear in Beijing time ([YYYY-MM-DD HH:MM:SS] prefix), but CLI parameters require UTC. Subtract 8 hours when translating a console timestamp into CLI --start-time / --end-time.

4. Prefer `--keyword <request_id>` only (no time parameters) for request-level diagnosis — this sidesteps timezone confusion entirely.

---

Configuration Retrieval Issues

Issue: No Released Configurations Found

Symptoms:

{
  "EngineConfigs": [],
  "TotalCount": 0
}

Diagnostic Steps:

1. Check all statuses:

   # Remove status filter
   aliyun pairecservice list-engine-configs \
     --instance-id <instance-id> \
     --environment <Prod|Pre>

2. Check environment mapping:

• Service env product → CLI param Prod
• Service env prepub → CLI param Pre
• Verify correct mapping used

3. List all configs:

   # Remove all filters
   aliyun pairecservice list-engine-configs \
     --instance-id <instance-id>

Solutions:

• Use correct environment parameter
• Check if configurations exist in different environment
• Verify instance ID is correct
• Check if configurations have been archived

---

Issue: Wrong Environment Configurations Returned

Symptoms: Got Pre environment config when expecting Prod

Root Cause: Environment variable PAIREC_ENVIRONMENT mapping incorrect

Solution: 1. Verify environment mapping:

• product → Prod
• prepub → Pre
• Case-sensitive

2. Double-check service environment:

   aliyun eas describe-service \
     --cluster-id <cluster-id> \
     --service-name <service-name> \
     --cli-query 'ServiceConfig.envs.PAIREC_ENVIRONMENT'

---

Issue: Multiple Versions Returned

Symptoms: List returns many configuration versions, unclear which to use

Solution: 1. Filter by Released status:

   aliyun pairecservice list-engine-configs \
     --instance-id <instance-id> \
     --environment Prod \
     --status Released

2. Select most recent:

• Sort by GmtCreateTime
• Use latest unless specified otherwise

3. Ask user to confirm:

• Display version list with metadata
• Let user select specific version

---

Issue: ConfigValue is Empty

Symptoms:

{
  "EngineConfig": {
    "ConfigValue": ""
  }
}

Diagnostic Steps: 1. Check if configuration was properly saved 2. Verify configuration is not in Draft state 3. Check permissions to view configuration

Solution:

• Use Released configuration
• Verify configuration was deployed successfully
• Contact configuration owner

---

Engine API Issues

Issue: Code 299 - Items Size Not Enough

Error Response:

{
  "code": 299,
  "msg": "items size not enough",
  "size": 0,
  "items": []
}

Common Causes:

1. Recall topk too low:

   // Fix: Increase topk
   "recall": {
     "modules": [{
       "params": {
         "topk": 500  // Increased from 100
       }
     }]
   }

2. Filters too aggressive:

   // Fix: Relax filter thresholds
   "filter": {
     "rules": [{
       "params": {
         "min_score": 0.05  // Lowered from 0.2
       }
     }]
   }

3. No data for user:

• Check if user exists in system
• Verify user has historical behavior
• Check data freshness

4. Table access issues:

• Verify recall tables have data
• Check table permissions
• Validate table names in config

Diagnosis Workflow: 1. Check service logs for detailed error 2. Review recall configuration 3. Verify data availability 4. Test with known good user

---

Issue: Request Timeout

Error Response:

{
  "code": 500,
  "msg": "request timeout"
}

Common Causes:

1. Recall timeout too low:

   "params": {
     "timeout_ms": 500  // Increased from 200
   }

2. Too many recall modules:

• Reduce number of parallel recalls
• Optimize recall order
• Use async where possible

3. Large batch in ranking:

   "params": {
     "batch_size": 30  // Reduced from 100
   }

4. Slow model endpoint:

• Check model service status
• Optimize model inference
• Increase timeout

Diagnosis: 1. Check logs for which stage timed out 2. Review timeout configurations 3. Test individual modules separately

---

Issue: Invalid Request Parameters

Error Response:

{
  "code": 400,
  "msg": "invalid parameter: user_id"
}

Solution: 1. Check API documentation for required parameters 2. Validate parameter formats 3. Review request payload in logs 4. Test with known good request

---

Configuration Validation Issues

Issue: JSON Parsing Error

Symptoms: Cannot parse ConfigValue as JSON

Solutions:

1. Check for escape characters:

   # ConfigValue may be escaped string
   echo "$CONFIG_VALUE" | jq -r . | jq .

2. Validate JSON syntax:

   echo "$CONFIG_VALUE" | jq .

3. Check encoding:

• Ensure UTF-8 encoding
• Remove BOM if present

4. Handle YAML:

• Some configs may be YAML format
• Use YAML parser if needed

---

Issue: Reference Validation Failures

Symptoms: Configuration references tables/models that don't exist

Diagnostic Steps:

1. For table references:

• Check if table exists in MaxCompute/OSS
• Verify table name spelling
• Check environment (Prod vs Pre tables)

2. For model endpoints:

• Verify endpoint URL is reachable
• Check model service status
• Test endpoint separately

3. For feature names:

• Check feature table schema
• Verify feature names match exactly
• Check if features are computed

Solution:

• Update config with correct references
• Ensure resources exist in target environment
• Use environment-specific resource names

---

Issue: Validation Warnings vs Errors

Question: Should warnings block deployment?

Guideline:

Errors (❌): Must be fixed before deployment

• Invalid syntax
• Missing required fields
• Inaccessible resources
• Type mismatches

Warnings (⚠️): Should review but may proceed

• Suboptimal settings
• Performance concerns
• Deprecated parameters
• Style violations

Best Practice:

• Fix all errors
• Review warnings with domain expert
• Document accepted warnings
• Monitor metrics post-deployment

---

Performance Issues

Issue: Slow Diagnosis Workflow

Symptoms: Diagnosis takes >5 minutes

Optimization:

1. Parallel API calls:

   # Run independent queries in parallel
   aliyun eas describe-service ... &
   aliyun pairecservice list-engine-configs ... &
   wait

2. Narrow log search:

• For request-level diagnosis, use --keyword <request_id> only (do NOT add --start-time / --end-time, which filters out business logs)
• For broad scans, use time windows in yyyy-MM-dd HH:mm:ss UTC format
• Limit page size

3. Cache service info:

• Service config rarely changes
• Can reuse for multiple diagnoses

---

Issue: Configuration Validation Takes Too Long

Symptoms: Validation >1 minute for single config

Optimization:

1. Prioritize checks:

• Syntax validation first (fast)
• Structure validation second
• Resource validation last (slow)

2. Parallel resource checks:

• Check tables in parallel
• Batch model endpoint tests

3. Cache validation results:

• Table existence checks
• Model availability

---

General Troubleshooting Tips

1. Enable Debug Logging

aliyun eas describe-service \
  --cluster-id <cluster-id> \
  --service-name <service-name> \
  --log-level DEBUG

2. Use Dry-Run Mode

aliyun eas describe-service \
  --cluster-id <cluster-id> \
  --service-name <service-name> \
  --cli-dry-run

3. Check API Quotas

Some APIs have rate limits:

• Pace requests if hitting limits
• Use pagination appropriately
• Cache results when possible

4. Verify Region Consistency

Ensure all resources in same region:

• EAS service region
• PAI-Rec instance region
• Data tables region

5. Consult Official Documentation

• PAI-Rec Documentation
• EAS Documentation
• Aliyun CLI Documentation

---

Getting Help

If issues persist:

1. Collect diagnostic information:

• Error messages
• Command outputs
• Service/instance IDs
• Timestamps

2. Check service status:

• EAS console
• PAI-Rec console
• Service health metrics

3. Contact support:

• Alibaba Cloud support ticket
• PAI-Rec support channels
• Include diagnostic information

---

Related Documentation

• RAM Policies - Permission requirements
• Related Commands - CLI command reference
• Verification Method - How to verify success
• Configuration Examples - Sample configurations