
Alibabacloud Oss Manage Metaquery
- 184 installs
- 208 repo stars
- Updated August 4, 2026
- aliyun/alibabacloud-aiops-skills
Query and manage OSS object metadata at scale for inventory audits, compliance scans, data discovery, and pipeline orchestration over large storage namespaces.
About
Operations skill for Alibaba Cloud OSS MetaQuery: enable metadata indexes, run structured queries over object tags and properties, support inventory audits and compliance checks, and automate data discovery across large OSS namespaces.
- OSS MetaQuery index enablement
- SQL-like metadata search over objects
- Inventory and compliance reporting
- Large-bucket discovery without full listing
- Automated governance over stored assets
Alibabacloud Oss Manage Metaquery by the numbers
- 184 all-time installs (skills.sh)
- Ranked #470 of 1,039 Cloud & Infrastructure skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/aliyun/alibabacloud-aiops-skills --skill alibabacloud-oss-manage-metaqueryAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 184 |
|---|---|
| repo stars | ★ 208 |
| Last updated | August 4, 2026 |
| Repository | aliyun/alibabacloud-aiops-skills ↗ |
What it does
Query and manage OSS object metadata at scale for inventory audits, compliance scans, data discovery, and pipeline orchestration over large storage namespaces.
Files
OSS Vector Search & AI Content Awareness
Leverage multimodal AI models to extract semantic descriptions and concise summaries from images, videos, audio, and documents stored in OSS Buckets. Build searchable vector indexes to enable advanced retrieval capabilities such as text-to-image and text-to-video search.
Prerequisites
1. Aliyun CLI (>= 3.3.3)
Pre-check: Aliyun CLI >= 3.3.3 required
This skill uses Aliyun CLI for all OSS operations except opening MetaQuery.
If Aliyun CLI is already installed, ossutil does not need to be installed separately.
Run the following command to verify the version: aliyun versionIf not installed or the version is below 3.3.3,
run curl -fsSL https://aliyuncli.alicdn.com/setup.sh | bash to install/update,or refer to references/cli-installation-guide.md for installation instructions.
>
Then [MUST] run aliyun configure set --auto-plugin-install true to enable automatic plugin installation.[MUST] run aliyun plugin update to ensure that any existing plugins on your local machine are always up-to-date.2. Update the aliyun ossutil plugin
Troubleshooting: `--user-agent` unknown flag error
If you encounter Error: unknown flag: --user-agent when running a command, the ossutil plugin version is too old. Run the following command to update to the latest version:```bash
aliyun ossutil update
```
After the update, re-run the original command.
# Python SDK dependencies
pip install oss2==2.19.1 alibabacloud-credentials==1.0.8Notes:
- Required: Aliyun CLI, Python dependencies oss2 and alibabacloud-credentials
- Not required: ossutil
- The only operation that requires Python: opening MetaQuery (AI Content Awareness + semantic search)
Security Rules:
- NEVER read, echo, or print AK/SK values (e.g., echo $ALIBABA_CLOUD_ACCESS_KEY_ID is FORBIDDEN)- NEVER ask the user to input AK/SK directly in the conversation or command line
At the start of the Core Workflow (before any CLI invocation): [MUST] Enable AI-Mode — AI-mode is required for Agent Skill execution. Run the following commands before any CLI invocation:
aliyun configure ai-mode enable
aliyun configure ai-mode set-user-agent --user-agent "AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery"[MUST] Disable AI-Mode at EVERY exit point — Before delivering the final response for ANY reason, always disable AI-mode first. This applies to ALL exit paths: workflow success, workflow failure, error/exception, user cancellation, session end, or any other scenario where no further CLI commands will be executed. AI-mode is only used for Agent Skill invocation scenarios and MUST NOT remain enabled after the skill stops running.
aliyun configure ai-mode disable[MUST] CLI User-Agent — Every aliyun CLI command invocation must include: --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
Architecture
User Request -> OSS Bucket -> AI Content Awareness Engine -> Semantic Feature Extraction -> Vector Index -> Semantic Search
|
Images/Videos/Audio/Docs -> Detailed Description (~100 chars) + Concise Summary (<=20 chars)Core Components: OSS Bucket + Data Index + Vector Search + AI Content Awareness
Usage Restrictions
Supported Regions
| Region Category | Region List |
|---|---|
| East China | cn-hangzhou, cn-shanghai |
| North China | cn-qingdao, cn-beijing, cn-zhangjiakou |
| South China | cn-shenzhen, cn-guangzhou |
| Southwest China | cn-chengdu |
| Other | cn-hongkong, ap-southeast-1 (Singapore), us-east-1 (Virginia) |
Note: If the user's Bucket is in a region not listed above, vector-mode MetaQuery and content awareness cannot be enabled, and an EC Code 0037-00000001 error will be returned. Guide the user to create a new Bucket in a supported region.File Types
- Supported: Images, videos, audio, documents
- Multipart uploads: Only objects that have been assembled via
CompleteMultipartUploadare shown
---
Performance Reference
OSS Internal Bandwidth and QPS
| Region | Internal Bandwidth | Default QPS |
|---|---|---|
| cn-beijing, cn-hangzhou, cn-shanghai, cn-shenzhen | 10Gbps | 1250 |
| Other regions | 1Gbps | 1250 |
This bandwidth and QPS is provided exclusively for vector search and does not consume the Bucket's QoS quota.
Existing File Index Build Time
| File Type | 10 Million Files | 100 Million Files | 1 Billion Files |
|---|---|---|---|
| Structured data & images | 2-3 hours | 1 day | ~10 days |
| Videos, documents, audio | 2-3 days | 7-9 days | - |
Incremental Updates and Search Latency
- Incremental updates: When QPS < 1250, latency is typically minutes to hours
- Search response: Sub-second, default timeout 30 seconds
---
Dangerous Operation Confirmation
Before executing any of the following dangerous operations, you MUST confirm with the user first and obtain explicit consent before proceeding:
- Delete Bucket:
aliyun ossutil rm oss://<bucket-name> -b --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery-- Deletes the entire Bucket, irreversible - Delete Object:
aliyun ossutil rm oss://<bucket-name>/<object-key> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery-- Deletes a specific file - Batch Delete Objects:
aliyun ossutil rm oss://<bucket-name>/ --recursive --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery-- Recursively deletes all files in the Bucket - Close MetaQuery:
aliyun ossutil api close-meta-query --bucket <bucket-name> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery-- Closes the metadata index; all indexed data will be cleared - Open MetaQuery:
python scripts/open_metaquery.py --region <your-region> --bucket <your-bucket-name> --endpoint <your-endpoint>-- Opens the metadata index; existing data will start being indexed. If the bucket has more than 1000 objects, confirm with the user first. - Create Bucket:
aliyun ossutil api put-bucket --bucket <bucket> --region <region-id> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery-- Creates a Bucket
When confirming, explain the following to the user: 1. The specific operation to be performed 2. The scope of impact (which files/resources will be deleted or closed) 3. Whether the operation is reversible (most delete operations are irreversible)
RAM Permissions
See references/ram-policies.md
---
Critical Rules (Must Follow)
Rule 1: Opening MetaQuery MUST use the Python script
PROHIBITED:
aliyun ossutil api open-meta-query oss://my-bucket --mode semanticREQUIRED:
python scripts/open_metaquery.py --region cn-hangzhou --bucket my-bucketReason: Only the Python script or SDK can correctly configure WorkflowParameters to enable AI Content Awareness (ImageInsightEnable and VideoInsightEnable). Without this, semantic search quality will be severely degraded.
Rule 2: Must ask the user when Bucket name conflicts
When creating a Bucket and encountering a BucketAlreadyExists error: 1. Immediately stop all subsequent operations 2. Inform the user: "The Bucket name is already taken" 3. Ask the user to choose:
- Option 1: Use the existing bucket (requires explicit user confirmation)
- Option 2: Choose a new bucket name (user provides the new name)
4. Wait for the user's response before continuing PROHIBITED:
- Automatically modifying the bucket name (e.g., appending
-2,-new, etc.) - Using an existing bucket without asking the user
Rule 3: Use Aliyun CLI by default for all operations except opening MetaQuery
The following operations should use Aliyun CLI by default:
- Create Bucket
- Query Bucket info
- Query Bucket statistics
- Upload files
- Query MetaQuery status
- Execute semantic search
- Close MetaQuery
- Delete Object / Bucket
Goal: Use the `aliyun` command uniformly, minimizing dependency on ossutil.
Rule 4: If Aliyun CLI is installed, ossutil is not needed
This skill does not require ossutil to be installed by default. As long as Aliyun CLI >= 3.3.3 is installed and the following has been executed:
aliyun configure set --auto-plugin-install trueIt can be used as the default execution tool.
Core Workflows
Task 1: Create Bucket and Upload Files
Always confirm with the user before creating a bucket. Proceed only after the user agrees.
# 1.1 Create Bucket
aliyun ossutil api put-bucket --bucket examplebucket --region <region-id> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
# 1.2 Download files
aliyun ossutil cp oss://example-bucket/test_medias/ /tmp/test_medias_download/ -r --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
# 1.3 Upload files
aliyun ossutil cp /tmp/test_medias_download/ oss://example-bucket/test_medias/ -r --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaqueryTask 2: Enable Vector Search & AI Content Awareness (Python script or SDK only)
WARNING: You MUST usepython scripts/open_metaquery.pyto open MetaQuery. Usingaliyun ossutil api open-meta-queryis STRICTLY PROHIBITED (it cannot configure WorkflowParameters, which prevents enabling AI Content Awareness features ImageInsightEnable and VideoInsightEnable, severely degrading semantic search quality).
Using the Python Script (Mandatory)
Before executing the Python script, complete the following environment setup:
1. Install Python dependencies:
pip install oss2==2.19.1 alibabacloud-credentials==1.0.82. Configure credentials: The Python script uses the alibabacloud-credentials default credential chain to automatically discover credentials (supporting environment variables, ~/.aliyun/config.json, ECS instance roles, etc.). No explicit AK/SK handling is needed in the code. Ensure credentials are configured via the aliyun configure command.
3. Verify RAM permissions: Users must have the minimum RAM permissions required for MetaQuery. See references/ram-policies.md. If the user encounters an AccessDenied error, check that RAM permissions are correctly configured.
Enablement Process: 1. Prepare the Bucket: a. If the user requests creating a new Bucket:
- Run
aliyun ossutil api put-bucket --bucket examplebucket --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquerywith the user-specified bucket name - If creation fails with a
BucketAlreadyExistserror: - Immediately stop the operation
- Inform the user: "The Bucket name
<bucket-name>is already taken (it may have been created by you or another user)" - You MUST ask the user: "Would you like to: 1) Use this existing bucket? or 2) Choose a new bucket name?"
- Wait for the user's explicit response before continuing. Do not modify the bucket name or use the existing bucket without permission.
b. If the user provides an existing bucket:
- First verify the bucket exists using
aliyun ossutil api get-bucket-info --bucket <bucket-name> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery - If it does not exist, ask the user whether to create it
2. Verify Bucket object count: After the user provides a bucket, check the object count. If it exceeds 1000, warn the user that enabling MetaQuery will incur costs. Use the following command to get the bucket's object count:
aliyun ossutil api get-bucket-stat --bucket <your-bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaqueryThe ObjectCount field in the response indicates the number of objects.
- If the object count exceeds 1000, warn the user that enabling MetaQuery will incur costs and confirm whether to proceed.
- If the object count is 0, ask the user which files to upload. Upload command:
aliyun ossutil api put-object --bucket <your-bucket-name> --key <object-key> --body file://<local-file-path> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery3. Run the Python script: After the above steps are complete, attempt to open MetaQuery using the Python script. Python script example:
python scripts/open_metaquery.py --region <your-region> --bucket <your-bucket-name> --endpoint <your-endpoint>Troubleshooting MetaQuery Enablement Issues
Use the get-meta-query-status command to check MetaQuery status:
aliyun ossutil api get-meta-query-status --bucket <your-bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaqueryBased on the returned status:
- Status is `Deleted`: MetaQuery is being closed. The user should retry later.
- Status is `Running` or `Ready`: MetaQuery has already been created. Check the following two conditions:
- Whether
MetaQueryModeissemantic - Whether
WorkflowParameterscontains the following configuration:
<WorkflowParameters>
<WorkflowParameter><Name>ImageInsightEnable</Name><Value>True</Value></WorkflowParameter>
<WorkflowParameter><Name>VideoInsightEnable</Name><Value>True</Value></WorkflowParameter>
</WorkflowParameters>If MetaQueryMode=semantic and both VideoInsightEnable and ImageInsightEnable are True, the user has successfully enabled MetaQuery in vector mode with content awareness (which greatly improves semantic search quality). No further action is needed. If these conditions are not met, recommend the user switch to a different bucket and start over.
Task 3: Execute Semantic Search
Prerequisites for MetaQuery Search
Before using MetaQuery for search, confirm the following: 1. Verify MetaQuery is enabled:
aliyun ossutil api get-meta-query-status --bucket <your-bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
2. **If MetaQuery is not enabled**: Complete the enablement process first. Refer to Task 2 to enable it using the Python script.
3. **Check index scan status**: The `Phase` field from `get-meta-query-status` indicates the current scan phase:
- `FullScanning`: Full scan in progress. **Search is not available yet**. Wait for the full scan to complete.
- `IncrementalScanning`: Incremental scan in progress. The index has been largely built and search can be performed normally.
4. **Verify MetaQuery state is `Running`**: MetaQuery is only available when `State` is `Running`. If the state is `Ready` or any non-`Running` state, you may need to wait or re-enable it.
**1. Prepare the meta-query.xml file:**
Create a `meta-query.xml` file to define query conditions. For detailed format, field descriptions, and complete examples, see [references/metaquery.md](references/metaquery.md).
Example of semantic vector search for video files containing "person" (MediaTypes can only be one of: video, image, audio, document):<MetaQuery> <MediaTypes><MediaType>video</MediaType></MediaTypes> <Query>person</Query> </MetaQuery>
Example of scalar search where file size > 30B and file modification time > 2025-06-03T09:20:47.999Z:<MetaQuery> <Query>{"SubQueries":[{"Field":"Size","Value":"30","Operation":"gt"},{"Field":"FileModifiedTime","Value":"2025-06-03T09:20:47.999Z","Operation":"gt"}],"Operation":"and"}</Query> </MetaQuery>
**2. Execute the search command:**
This example uses semantic vector search. The `meta-query.xml` file defines the query conditions, and search results return the most similar files.aliyun ossutil api do-meta-query --bucket <bucket-name> --meta-query file://meta-query.xml --meta-query-mode semantic --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
For scalar search, use `--meta-query-mode basic`
> For detailed command parameters, see the DoMetaQuery section in [references/related-apis.md](references/related-apis.md).
**3. Optimizing search result display:**
After search completes, when displaying results to the user, use the `x-oss-process` parameter to generate preview images or cover frames for image and video files, making it easier for the user to visually review search results. If the user's current channel supports multimedia files, send them directly to the user.
**Video files -- Get video cover snapshot:**aliyun ossutil presign oss://<bucket-name>/<video-object-key> --query-param x-oss-process=video/snapshot,t_0,f_png,w_0,h_0 --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
Parameters: `t_0`: Capture frame at 0ms as cover; `f_png`: Output format PNG; `w_0,h_0`: Width/height 0 means original resolution.
**Image files -- Get image preview link:**aliyun ossutil presign oss://<bucket-name>/<image-object-key> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
> **Note**: The `aliyun ossutil presign` command generates a signed temporary access URL that can be opened directly in a browser for preview during its validity period. For image files, you can also add image processing parameters via `x-oss-process` (e.g., resize, crop):
> ```bash
> aliyun ossutil presign oss://<bucket-name>/<image-object-key> --query-param x-oss-process=image/resize,w_200 --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
> ```
> This generates a thumbnail preview to reduce loading time.
### Troubleshooting MetaQuery Search Issues
#### User asks "Why wasn't a specific file found?"
When a user reports that a specific uploaded file is missing from search results, troubleshoot based on the MetaQuery configuration:
**a. Content awareness is NOT enabled:**
If the user's MetaQuery does not have content awareness enabled (i.e., `VideoInsightEnable` or `ImageInsightEnable` is not `True` in `WorkflowParameters`), possible reasons include:
- The file's metadata index has not been fully built yet. Wait for the index scan to complete (check the `Phase` field via `aliyun ossutil api get-meta-query-status --bucket <bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery`).
- Without content awareness, search is based only on basic file metadata (filename, size, type, etc.) and cannot perform semantic understanding of file contents, resulting in limited search effectiveness.
- **Recommendation**: Suggest the user enable content awareness to improve search quality. Since existing MetaQuery configurations cannot be directly modified, recommend the user switch to a new bucket and re-enable MetaQuery with content awareness following the Task 2 process.
**b. Content awareness IS enabled:**
If the user's MetaQuery has content awareness enabled but a specific file still cannot be found, possible reasons include:
- **File is still being processed**: Content awareness requires deep analysis of files (e.g., image recognition, video understanding), which takes longer, especially for video files. Check the `Phase` field via `aliyun ossutil api get-meta-query-status --bucket <bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery`:
- `FullScanning`: The overall index is still in full scan mode. Wait patiently.
- `IncrementalScanning`: Newly uploaded files are being processed incrementally. Usually wait a few minutes.
- **Unsupported file format**: Some file formats may not be supported by content awareness. In this case, search can only use basic metadata.
- **Search keywords don't match**: The user's search keywords may not semantically match the file content. Suggest the user try adjusting their search keywords to use descriptions closer to the actual file content.
### Task 4: Query Data Index Status (aliyun ossutil)aliyun ossutil api get-meta-query-status --bucket <bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
> For detailed descriptions of returned fields (State, Phase, MetaQueryMode, etc.), see the GetMetaQueryStatus section in [references/related-apis.md](references/related-apis.md).
## Verification
See [references/verification-method.md](references/verification-method.md)
## Resource CleanupClose the data index. (Dangerous operation -- confirm with the user first)
aliyun ossutil api close-meta-query --bucket <bucket-name> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
> **Warning**: After closing the data index, all indexed data will be cleared. (Dangerous operation -- confirm with the user first)
## Alternative Python Scripts for OSS Operations
When aliyun ossutil is unavailable, you can use Python scripts as alternatives. See the Python SDK Scripts section in [references/related-apis.md](references/related-apis.md).
---
Acceptance Criteria: OSS Vector Search & AI Content Awareness
Scenario: OSS Vector Search & AI Content Awareness Semantic Search Purpose: Skill test acceptance criteria
Table of Contents
- Operation Dependencies
- Correct aliyun ossutil Command Patterns
- Correct Python SDK Code Patterns
- Correct Java SDK Code Patterns
- Operation Checklist
- Acceptance Test Cases
- Common Errors and Troubleshooting
---
Operation Dependencies
+---------------------------------------------------------------------+
| Operation Dependency Flow |
+---------------------------------------------------------------------+
| |
| 1. Create Bucket -----------------------------------------------+ |
| (CLI/SDK/Console) | |
| | | |
| v | |
| 2. Upload Files ----------------------------------------+ | |
| (CLI/SDK/Console) | | |
| | | | |
| v | | |
| 3. Initial Role Authorization (Console only) | | |
| AliyunMetaQueryDefaultRole | | |
| | | | |
| v | | |
| 4. Enable Vector Search + AI Content Awareness <--------+ | |
| (SDK/Console) | |
| - mode: semantic | |
| - WorkflowParameters: VideoInsightEnable, ImageInsightEnable | |
| - Filters: File filtering rules (optional) | |
| | | |
| v | |
| 5. Wait for Index Build <---------------------------------------+ |
| Query status: GetMetaQueryStatus |
| State: Ready -> Running (FullScanning -> IncrementalScanning) |
| | |
| v |
| 6. Execute Semantic Search |
| (CLI/SDK/Console) |
| DoMetaQuery (mode=semantic) |
| |
+---------------------------------------------------------------------+Dependency Table
| Step | Operation | Dependencies | Implementation Method |
|---|---|---|---|
| 1 | Create Bucket | None | CLI/SDK/Console |
| 2 | Upload Files | Bucket created | CLI/SDK/Console |
| 3 | Role Authorization | None | Console only (first time) |
| 4 | Enable Vector Search + AI Content Awareness | Role authorized | SDK/Console |
| 5 | Query Index Status | Vector search enabled | aliyun ossutil/SDK/Console |
| 6 | Execute Semantic Search | Index build complete (State=Running) | aliyun ossutil/SDK/Console |
| 7 | Close Data Index | Vector search enabled | aliyun ossutil/SDK/Console |
---
Correct Command Patterns
For correct aliyun ossutil command patterns and core rules, refer to the "Critical Rules" and "Core Workflows" sections in SKILL.md.
---
Correct Python SDK Code Patterns
For correct and incorrect Python SDK code patterns (import, credential initialization, enabling vector search, semantic search, scalar search), refer to the code examples and script usage in the "Critical Rules" and "Core Workflows" sections of SKILL.md, as well as the verification scripts in verification-method.md.
---
Correct Java SDK Code Patterns
1. Dependency Version
CORRECT
<!-- Java SDK 3.18.2+ supports vector search -->
<dependency>
<groupId>com.aliyun.oss</groupId>
<artifactId>aliyun-sdk-oss</artifactId>
<version>3.18.2</version>
</dependency>2. Client Initialization
CORRECT
EnvironmentVariableCredentialsProvider credentialsProvider =
CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
ClientBuilderConfiguration clientConfig = new ClientBuilderConfiguration();
clientConfig.setSignatureVersion(SignVersion.V4); // Must use V4 signature
OSS ossClient = OSSClientBuilder.create()
.endpoint(endpoint)
.credentialsProvider(credentialsProvider)
.clientConfiguration(clientConfig)
.region(region)
.build();3. Semantic Search Request
CORRECT
DoMetaQueryRequest request = new DoMetaQueryRequest(
bucketName, maxResults, query, sort,
MetaQueryMode.SEMANTIC, // Semantic mode
mediaTypes, simpleQuery
);
DoMetaQueryResult result = ossClient.doMetaQuery(request);---
Operation Checklist
Prerequisite Checks
- [ ] Valid credentials configured (via
aliyun configurein~/.aliyun/config.json) - [ ] Bucket region supports vector search functionality
- [ ] Correct SDK version installed (Java >= 3.18.2, Python oss2)
Role Authorization (First time only, Console only)
- [ ]
AliyunMetaQueryDefaultRolerole authorization completed in Console
Enable Vector Search & AI Content Awareness (SDK/Console)
- [ ] Vector search enabled (mode=semantic)
- [ ] Image content awareness configured (ImageInsightEnable=True)
- [ ] Video content awareness configured (VideoInsightEnable=True)
- [ ] (Optional) File filtering rules configured
Index Status Check
- [ ] State =
Running(operational) - [ ] Phase =
FullScanning(full scan) orIncrementalScanning(incremental scan)
---
Acceptance Test Cases
For specific commands and code for the following test cases, refer to the corresponding "Core Workflows" Task sections in SKILL.md.
Test Case 1: Bucket Creation and File Upload
Corresponds to: SKILL.md Task 1
Prerequisites: Valid credentials configured (via aliyun configure in ~/.aliyun/config.json)
Expected Results:
- Bucket created successfully
- Files uploaded successfully
---
Test Case 2: Enable Vector Search & AI Content Awareness
Corresponds to: SKILL.md Task 2
Prerequisites:
- Bucket created
- Role authorization completed (Console required for first time)
- Bucket object count verified (confirm costs if exceeding 1000 objects)
Expected Results:
- Successful status code returned (200)
- Index build started
- Content awareness features enabled (VideoInsightEnable and ImageInsightEnable are True)
---
Test Case 3: Query Index Status
Corresponds to: SKILL.md Task 4
Prerequisites: Vector search enabled
Expected Results:
- State:
Ready->Running - Phase:
FullScanningorIncrementalScanning - MetaQueryMode:
semantic
---
Test Case 4: Execute Semantic Search
Corresponds to: SKILL.md Task 3
Prerequisites:
- Index state is Running
- Files have been indexed
Expected Results:
- Matching file list returned
- Each file contains AI metadata (description, summary)
---
Test Case 5: AI Content Awareness Result Verification
Prerequisites: Test Case 4 completed
Expected Results:
oss_ai_meta.description: Approximately 100 characters describing the file contentoss_ai_meta.summary: No more than 20 characters, concise summary
---
Test Case 6: Close Data Index
Corresponds to: SKILL.md Resource Cleanup
Prerequisites: Vector search enabled
Expected Results:
- Successful status code returned
- Index status changes to
Deleted
---
Common Errors and Troubleshooting
| Error | Cause | Solution |
|---|---|---|
AccessDenied | Missing permissions | Add oss:OpenMetaQuery and other required permissions |
BucketNotFound | Bucket does not exist | Check Bucket name and region |
MetaQueryNotOpened | Vector search not enabled | Call OpenMetaQuery first |
InvalidMode | Invalid mode parameter | Use semantic or basic |
| No search results | Index build incomplete | Wait for index build to complete |
| No AI metadata | Content awareness not enabled | Configure WorkflowParameters |
0037-00000001 | Bucket region does not support vector search | Create a new Bucket in a supported region; refer to the supported regions list |
MetaQueryAlreadyExist | Bucket already has MetaQuery enabled or is being closed | Use get-meta-query-status to check current status |
Aliyun CLI Installation & Configuration Guide
Complete guide for installing and configuring Aliyun CLI.
Table of Contents
- Installation
- macOS
- Linux
- Windows
- Configuration
- Quick Start
- Configuration Modes
- Environment Variables
- Managing Multiple Profiles
- Credential Priority
- Verification
- Security Best Practices
- Troubleshooting
- Advanced Configuration
- Next Steps
- References
Aliyun CLI 3.3.3+: Supports installing and using all published Alibaba Cloud product plugins. Make sure to upgrade to 3.3.3 or later for full plugin ecosystem coverage.
Installation
macOS
Using Homebrew (Recommended)
brew install aliyun-cli
# Upgrade to latest
brew upgrade aliyun-cli
# Verify version (>= 3.3.3)
aliyun versionUsing Binary
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz
# Extract
tar -xzf aliyun-cli-macosx-latest-amd64.tgz
# Move to PATH
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionLinux
Debian/Ubuntu
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionCentOS/RHEL
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionARM64 Architecture
# Download ARM64 version
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-arm64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-arm64.tgz
sudo mv aliyun /usr/local/bin/Windows
Using Binary 1. Download from: https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip 2. Extract the ZIP file 3. Add the directory to your PATH environment variable 4. Open new Command Prompt or PowerShell 5. Verify: aliyun version Using PowerShell
# Download
Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip"
# Extract
Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli
# Add to PATH (requires admin privileges)
$env:Path += ";C:\aliyun-cli"
[Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::Machine)
# Verify
aliyun versionConfiguration
Quick Start
aliyun configure set \
--mode AK \
--access-key-id <your-access-key-id> \
--access-key-secret <your-access-key-secret> \
--region cn-hangzhouAll aliyun configure commands support non-interactive flags, which is the recommended approach — it works in scripts, CI/CD pipelines, and agent-driven automation without hanging on stdin prompts. Where to Get Access Keys 1. Log in to Aliyun Console: https://ram.console.aliyun.com/ 2. Navigate to: AccessKey Management 3. Create a new AccessKey pair 4. Save the secret immediately — it's only shown once
Configuration Modes
Aliyun CLI supports 6 authentication modes. All examples below use non-interactive flags.
1. AK Mode (Access Key)
Most common mode for personal accounts and scripts.
aliyun configure set \
--mode AK \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--region cn-hangzhouConfiguration is stored in ~/.aliyun/config.json:
{
"current": "default",
"profiles": [
{
"name": "default",
"mode": "AK",
"access_key_id": "LTAI5tXXXXXXXX",
"access_key_secret": "8dXXXXXXXXXXXXXXXXXXXXXXXX",
"region_id": "cn-hangzhou",
"output_format": "json",
"language": "en"
}
]
}2. StsToken Mode (Temporary Credentials)
For short-lived access (tokens expire in 1-12 hours).
aliyun configure set \
--mode StsToken \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--sts-token v1.0:XXXXXXXXXXXXXXXX \
--region cn-hangzhouUse cases: CI/CD pipelines, temporary access for external contractors, cross-account access.
3. RamRoleArn Mode (Assume RAM Role)
Assume a RAM role for elevated or cross-account access.
aliyun configure set \
--mode RamRoleArn \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--ram-role-arn acs:ram::123456789012:role/AdminRole \
--role-session-name my-session \
--region cn-hangzhouUse cases: cross-account resource access, temporary elevated privileges, role-based access control.
4. EcsRamRole Mode (ECS Instance RAM Role)
Use the RAM role attached to an ECS instance — no credentials needed.
aliyun configure set \
--mode EcsRamRole \
--ram-role-name MyEcsRole \
--region cn-hangzhouRequirements: must be running on an ECS instance with a RAM role attached. Use cases: scripts and automation running on ECS instances.
5. RsaKeyPair Mode (RSA Key Pair)
Use RSA key pair for authentication (generate key pair in Aliyun Console first).
aliyun configure set \
--mode RsaKeyPair \
--private-key /path/to/private-key.pem \
--key-pair-name my-key-pair \
--region cn-hangzhou6. RamRoleArnWithEcs Mode (ECS + RAM Role)
Combine ECS instance role with RAM role assumption for cross-account access from ECS.
aliyun configure set \
--mode RamRoleArnWithEcs \
--ram-role-name MyEcsRole \
--ram-role-arn acs:ram::123456789012:role/TargetRole \
--role-session-name my-session \
--region cn-hangzhouEnvironment Variables
Highest priority - overrides config file Access Key Mode
export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_REGION_ID=cn-hangzhouSTS Token Mode
export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_SECURITY_TOKEN=your_sts_token
export ALIBABA_CLOUD_REGION_ID=cn-hangzhouECS RAM Role Mode
export ALIBABA_CLOUD_ECS_METADATA=role_nameUse Case:
- CI/CD pipelines
- Docker containers
- Temporary credential override
Managing Multiple Profiles
Create Named Profiles
aliyun configure set --profile projectA \
--mode AK \
--access-key-id LTAI5tAAAAAAAA \
--access-key-secret 8dAAAAAAAAAAAAAAAAAAAAAAAA \
--region cn-hangzhou
aliyun configure set --profile projectB \
--mode AK \
--access-key-id LTAI5tBBBBBBBB \
--access-key-secret 8dBBBBBBBBBBBBBBBBBBBBBBBB \
--region cn-shanghaiUse Specific Profile
aliyun ecs describe-instances --profile projectA
export ALIBABA_CLOUD_PROFILE=projectA
aliyun ecs describe-instances # Uses projectAList and Switch Profiles
aliyun configure list # List all profiles
aliyun configure set --current projectA # Switch default profileCredential Priority
Credentials are loaded in this order (first found wins): 1. Command-line flag: --profile <name> 2. Environment variable: ALIBABA_CLOUD_PROFILE 3. Environment credentials: ALIBABA_CLOUD_ACCESS_KEY_ID, etc. 4. Configuration file: ~/.aliyun/config.json (current profile) 5. ECS Instance RAM Role: If running on ECS with attached role
Verification
Test Authentication
# Basic test - list regions
aliyun ecs describe-regions
# Expected output: JSON array of regionsIf successful, you'll see:
{
"Regions": {
"Region": [
{
"RegionId": "cn-hangzhou",
"RegionEndpoint": "ecs.cn-hangzhou.aliyuncs.com",
"LocalName": "China East 1 (Hangzhou)"
},
...
]
},
"RequestId": "..."
}If failed, you'll see error messages:
InvalidAccessKeyId.NotFound- Wrong Access Key IDSignatureDoesNotMatch- Wrong Access Key SecretInvalidSecurityToken.Expired- STS token expired (for StsToken mode)Forbidden.RAM- Insufficient permissions
Debug Configuration
# Show current configuration
aliyun configure get
# Test with debug logging
aliyun ecs describe-regions --log-level=debug
# Check credential provider
aliyun configure get modeSecurity Best Practices
1. Use RAM Users (Not Root Account)
- Don't: Use Aliyun root account credentials
- Do: Create RAM users with specific permissions
# Create RAM user in console
# Attach only necessary policies
# Use RAM user's access keys2. Principle of Least Privilege
Grant only the minimum permissions needed:
# Example: Read-only ECS access
# Attach policy: AliyunECSReadOnlyAccess3. Rotate Access Keys Regularly
# Create new access key in RAM Console, then update configuration
aliyun configure set --access-key-id NEW_KEY --access-key-secret NEW_SECRET
# Delete old access key from console4. Use STS Tokens for Temporary Access
aliyun configure set --mode StsToken \
--access-key-id XXXX --access-key-secret XXXX \
--sts-token XXXX --region cn-hangzhou5. Use ECS RAM Roles When Possible
aliyun configure set --mode EcsRamRole --ram-role-name MyRole --region cn-hangzhou6. Never Commit Credentials
# Add to .gitignore
echo "~/.aliyun/config.json" >> .gitignore
# Use environment variables in CI/CD instead7. Secure Config File
# Restrict permissions
chmod 600 ~/.aliyun/config.jsonTroubleshooting
Issue: Command Not Found
# Check installation
which aliyun
# Check PATH
echo $PATH
# Reinstall or add to PATHIssue: Authentication Failed
# Verify configuration
aliyun configure get
# Test with debug
aliyun ecs describe-regions --log-level=debug
# Check credentials in console
# Verify access key is activeIssue: Permission Denied
# Error: Forbidden.RAM
# Check RAM user permissions
# Attach necessary policies in RAM console
# Example: AliyunECSFullAccess for ECS operationsIssue: STS Token Expired
# Error: InvalidSecurityToken.Expired
# Reconfigure with new token
aliyun configure set --mode StsToken \
--access-key-id XXXX --access-key-secret XXXX \
--sts-token NEW_TOKEN --region cn-hangzhouIssue: Wrong Region
# Some resources may not exist in the specified region
# Check available regions
aliyun ecs describe-regions
# Update default region
aliyun configure set region cn-shanghaiAdvanced Configuration
Custom Endpoint
# Use custom or private endpoint
export ALIBABA_CLOUD_ECS_ENDPOINT=ecs-vpc.cn-hangzhou.aliyuncs.comProxy Settings
# HTTP proxy
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080
# No proxy for specific domains
export NO_PROXY=localhost,127.0.0.1,.aliyuncs.comTimeout Settings
# Connection timeout (default: 10s)
export ALIBABA_CLOUD_CONNECT_TIMEOUT=30
# Read timeout (default: 10s)
export ALIBABA_CLOUD_READ_TIMEOUT=30Next Steps
After installation and configuration: 1. Install plugins for services you need (v3.3.3+ supports all published product plugins):
aliyun plugin install --names ecs vpc rds
# List all available plugins
aliyun plugin list-remote2. Explore commands:
aliyun ecs --help
aliyun fc --help3. Read documentation:
- Command Syntax Guide
- Global Flags Reference
- Common Scenarios
References
- Official Documentation: https://help.aliyun.com/zh/cli/
- RAM Console: https://ram.console.aliyun.com/
- Access Key Management: https://ram.console.aliyun.com/manage/ak
- Plugin Repository: https://github.com/aliyun/aliyun-cli
MetaQuery Query Condition Reference
This document provides query condition formats and examples for MetaQuery scalar and semantic queries.
For script command formats, refer to the Python SDK Scripts section in related-apis.md. For the complete workflow and environment setup, refer to SKILL.md.
---
Scalar Query Conditions
This condition performs an exact match query by filename:
{
"SubQueries":[
{
"Field":"Filename",
"Value":"video_1.mov",
"Operation":"eq"
}
],
"Operation":"and"
}Condition Description:
- Field:
"Filename"- Specifies the query field as filename - Value:
"video_1.mov"- The specific filename to search for, can be replaced with any filename - Operation:
"eq"- Equals operator, for exact matching
query_time - Time-Based Query Configuration
This configuration queries based on file creation time and modification time:
{
"SubQueries":[
{
"Field":"OSSTagging.CreateTime",
"Value":"1672588800000000000",
"Operation":"eq"
},{
"Field":"FileModifiedTime",
"Value":"2025-11-21T14:50:13.011643661+08:00",
"Operation":"eq"
}
],
"Operation":"and"
}Configuration Description:
- First sub-query - Creation time:
- Field:
"OSSTagging.CreateTime"- File creation time tag. The CreateTime tag can be optionally attached when uploading files via upload.py. - Value:
"1672588800000000000"- Nanosecond timestamp corresponding to the CreateTime tag added during upload - Operation:
"eq"- Exact match for this creation time - Second sub-query - Modification time:
- Field:
"FileModifiedTime"- File modification time - Value:
"2025-11-21T14:50:13.011643661+08:00"- Time in RFC3339Nano format - Operation:
"eq"- Exact match for this modification time - Operation:
"and"- Both conditions must be satisfied simultaneously - Use case: Filter files by time range, such as finding files uploaded or modified during a specific period
- Usage tips:
- Creation time uses nanosecond timestamp format
- Modification time must use RFC3339Nano format, or alternatively, you can use OSS Tags to carry the modification time during upload and search using the
OSSTagging.{TagName}format - Either time condition can be used independently or in combination
- Supports other operators like
"gt"(greater than),"lt"(less than), etc. for range queries
Fields supported by scalar index: For more supported fields and operators, refer to the official documentation: Fields and Operators Supported by Scalar Index
Semantic Query Conditions
Pure Vector Semantic Query Configuration
This configuration is used for intelligent search based on content semantics:
<MetaQuery>
<MediaTypes><MediaType>video</MediaType></MediaTypes>
<Query>person</Query>
</MetaQuery>Configuration Description:
- MediaTypes:
<MediaType>video</MediaType>- Restricts search to video-type media files - Available values:
video,image,audio,document - Query:
person- Semantic search keyword. The system analyzes video content to find videos containing "person" - Can be descriptive terms for objects, scenes, actions, etc.
query_body_with_basic - Combined Vector + Scalar Query Configuration
This configuration combines semantic search with attribute filtering for composite queries:
<MetaQuery>
<MediaTypes><MediaType>video</MediaType></MediaTypes>
<Query>person</Query>
<SimpleQuery>{
"SubQueries":[
{
"Field":"Size",
"Value":"30",
"Operation":"gt"
},
{
"Field":"OSSTagging.CreateTime",
"Value":"1763722586691406000",
"Operation":"eq"
}
],
"Operation":"and"
}</SimpleQuery>
</MetaQuery>Configuration Description:
- MediaTypes: Restricted to video type (same as pure vector query)
- Query:
person- Semantic search keyword (same as pure vector query) - SimpleQuery: Adds scalar query conditions for further filtering
- First condition - File size:
- Field:
"Size"- File size (bytes) - Value:
"30"- 30 bytes - Operation:
"gt"- Greater than operator, filters files larger than 30 bytes - Second condition - Creation time:
- Field:
"OSSTagging.CreateTime"- Creation time tag. The CreateTime tag can be optionally attached when uploading files via upload.py. - Value:
"1763722586691406000"- Specific nanosecond timestamp - Operation:
"eq"- Exact match for this creation time - Operation:
"and"- All conditions must be satisfied simultaneously - Use case: Precise queries that need to satisfy both content semantics and file attribute conditions
- Usage tips:
- Scalar query conditions can include file size, creation time, modification time, etc.
- Operators can be adjusted based on actual needs (eq, gt, lt, gte, lte, etc.)
- Suitable for scenarios requiring precise control over search results
Fields supported by vector index: For more supported fields and operators, refer to the official documentation: Fields and Operators Supported by Vector Index
---
Query Result Response Fields: Query results contain rich field information. For detailed descriptions, refer to the official documentation: DoMetaQuery Response Field Descriptions
ossutil Installation Guide
Overview
ossutil is a command-line tool for managing Alibaba Cloud OSS resources. This guide provides installation instructions for ossutil v2.2.1.
Download Links
Current latest version: 2.2.1
Linux
| System Architecture | Download Link | SHA256 Checksum |
|---|---|---|
| x86_32 | ossutil-2.2.1-linux-386.zip | 09726a85eb35f863fc584f4fa1ca5e6a8805729083bc29ec91e803f0eb64bcc7 |
| x86_64 | ossutil-2.2.1-linux-amd64.zip | fbf1026bd383a5d9bee051cd64a6226c730357ba569491f7c7b91af66560ef1d |
| arm32 | ossutil-2.2.1-linux-arm.zip | 30fed1691d774a3d1872cae0fc266122b8f9c68c990199361d974406f7d2ef5a |
| arm64 | ossutil-2.2.1-linux-arm64.zip | b7680e79aec0adc9d42a12b795612680a58efec1fad24b0ceb9e13b2390c6652 |
macOS
| System Architecture | Download Link | SHA256 Checksum |
|---|---|---|
| x86_64 | ossutil-2.2.1-mac-amd64.zip | a1bf1491037e138e52b0b92cdfd620decdc9e22d8dd1d8699226a8f2596b0cc2 |
| arm64 | ossutil-2.2.1-mac-arm64.zip | 326bff983e8e02142fc4e68d07f129475f9cbafb9777ed57cd7b6640edd8595c |
Windows
| System Architecture | Download Link | SHA256 Checksum |
|---|---|---|
| x86_32 | ossutil-2.2.1-windows-386.zip | 36043ddeed88188f36b41b631fae3c6909ffffb661d34bc1d5405863f9064d0c |
| x86_64 | ossutil-2.2.1-windows-amd64.zip | a7c22a0172fdca0e54cb8366f1ae8a869bc6bb64c1899352eb62d8eb9a1a9af0 |
| amd64 (Go 1.20) | ossutil-2.2.1-windows-amd64-go1.20.zip | 8670b88437be62053aa4b3d2da7695fa410f451693833534faa7b20e39c8eded |
Installation Steps
Linux/macOS
# 1. Download (example for Linux x86_64)
wget https://gosspublic.alicdn.com/ossutil/v2/2.2.1/ossutil-2.2.1-linux-amd64.zip
# 2. Extract
unzip ossutil-2.2.1-linux-amd64.zip
# 3. Move to PATH directory
chmod +x ossutil
sudo mv ossutil /usr/local/bin/
# 4. Verify installation
ossutil versionWindows
1. Download the appropriate version from the links above 2. Extract the zip archive 3. Add the extracted directory to your system PATH 4. Or copy the ossutil.exe file to a directory that's already in your PATH 5. Verify installation by running ossutil version in Command Prompt
Configuration
After installation, ossutil automatically obtains authentication information through the default credential chain without manual AK/SK configuration. The default credential chain looks for credentials in the following order:
1. Environment variables (ALIBABA_CLOUD_ACCESS_KEY_ID / ALIBABA_CLOUD_ACCESS_KEY_SECRET) 2. Credentials in configuration file (set via aliyun configure) 3. ECS instance RAM role (automatically obtained in ECS environments)
It is recommended to use instance RAM roles in cloud environments such as ECS, and use environment variables or aliyun configure for local development environments.
Configure endpoint information:
ossutil config --endpoint oss-cn-hangzhou.aliyuncs.comRAM Permission Policies
Required Permissions
The following RAM permissions are required for each feature of this project:
Bucket Basic Operations
oss:GetBucketInfo-- Query Bucket basic information (region, storage class, etc.)oss:ListObjects-- List files in the Bucket (V1)oss:ListObjectsV2-- List files in the Bucket (V2)
File Upload and Download
oss:GetObject-- Download (read) file contentoss:PutObject-- Upload (write) files to the Bucketoss:DeleteObject-- Delete files from the Bucket
Data Index and Semantic Search
oss:OpenMetaQuery-- Enable metadata management (includes AI content awareness configuration)oss:DoMetaQuery-- Execute metadata queries (scalar search / vector semantic search)oss:GetMetaQueryStatus-- Query data index statusoss:CloseMetaQuery-- Close data index
Minimum Permission Policy
The following is the minimum RAM permission policy JSON required for OSS vector search and AI content awareness features:
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": [
"oss:GetBucketInfo",
"oss:ListObjects",
"oss:ListObjectsV2",
"oss:GetObject",
"oss:PutObject",
"oss:OpenMetaQuery",
"oss:DoMetaQuery",
"oss:GetMetaQueryStatus",
"oss:CloseMetaQuery"
],
"Resource": [
"acs:oss:*:*:your-bucket-name",
"acs:oss:*:*:your-bucket-name/*"
]
}
]
}Read-Only Query Permissions
If the application only needs to perform semantic searches, the following minimum read-only permissions can be used:
oss:DoMetaQuery-- Execute metadata queriesoss:GetMetaQueryStatus-- Query data index statusoss:GetObject-- Download retrieved files
Corresponding policy JSON:
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": [
"oss:DoMetaQuery",
"oss:GetMetaQueryStatus"
],
"Resource": "acs:oss:*:*:your-bucket-name"
},
{
"Effect": "Allow",
"Action": "oss:GetObject",
"Resource": "acs:oss:*:*:your-bucket-name/*"
}
]
}Service Role Authorization
When using the data index feature for the first time, you need to authorize the OSS service role AliyunMetaQueryDefaultRole:
1. The OSS Console will automatically prompt for authorization when enabling data index 2. This role allows the OSS service to manage data indexes in the Bucket
Service Role Permission Scope:
- Read file content in the Bucket for AI analysis
- Build and manage vector indexes
- Process incremental file updates
Important Notes
1. Resource Scope: It is recommended to replace your-bucket-name with the specific Bucket name to avoid over-authorization 2. Region Restrictions: OSS resources are region-level; you can specify region restrictions in the Resource field 3. Regular Audits: Regularly review and clean up permissions that are no longer needed 4. Use STS: For temporary access scenarios, it is recommended to use STS temporary credentials instead of long-term AccessKeys
Reference Links
Related APIs and CLI Commands
Table of Contents
- ossutil Commands
- SDK API
- API Details
- OpenMetaQuery (Enable Metadata Management)
- GetMetaQueryStatus (Get Index Status)
- DoMetaQuery (Execute Query)
- CloseMetaQuery (Close Index)
- SDK Version Requirements
- Reference Links
---
ossutil Commands
| Command | Description | Example |
|---|---|---|
Python open_metaquery.py | Enable metadata management with content awareness | python scripts/open_metaquery.py --region <region> --bucket <bucket-name> |
get-meta-query-status | Get metadata index status | aliyun ossutil api get-meta-query-status --bucket <bucket-name> |
do-meta-query | Execute metadata query | aliyun ossutil api do-meta-query --bucket <bucket-name> --meta-query file://meta-query.xml --meta-query-mode semantic |
close-meta-query | Close metadata management | aliyun ossutil api close-meta-query --bucket <bucket-name> |
do-meta-query Query Conditions
The --meta-query parameter of do-meta-query requires an XML-formatted query condition file.
For detailed format, field descriptions, and complete examples (including semantic and scalar queries), see metaquery.md.
Python SDK Scripts (Alternative to aliyun ossutil)
When aliyun ossutil is unavailable, you can use the following Python scripts as alternatives. For the complete usage workflow, refer to the "Core Workflows" section in SKILL.md.
| Script | Description | Command Example |
|---|---|---|
create_bucket.py | Create an OSS bucket | python scripts/create_bucket.py --region <region> --bucket <bucket-name> |
open_metaquery.py | Enable MetaQuery with content awareness | python scripts/open_metaquery.py --region <region> --bucket <bucket-name> |
upload.py | Upload files to OSS | python scripts/upload.py --region <region> --bucket <bucket-name> --local-path <file> --remote-key <key> |
basic_query.py | Execute scalar queries (basic search) | python scripts/basic_query.py --region <region> --bucket <bucket-name> --scalar-query '<json>' |
semantic_query.py | Execute semantic queries (vector search) | python scripts/semantic_query.py --region <region> --bucket <bucket-name> --query <term> |
close_metaquery.py | Disable MetaQuery functionality | python scripts/close_metaquery.py --region <region> --bucket <bucket-name> |
API Details
OpenMetaQuery (Enable Metadata Management)
Request Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
| bucket | string | Yes | Bucket name |
| mode | string | Yes | Search mode: basic (scalar search), semantic (vector search) |
| role | string | No | RAM role name (required when configuring MNS notifications) |
| MetaQuery | object | No | Configuration container, includes WorkflowParameters, Filters, NotificationAttributes |
WorkflowParameters - AI Content Awareness Configuration:
| Parameter Name | Value | Description |
|---|---|---|
VideoInsightEnable | True/False | Video content awareness switch |
ImageInsightEnable | True/False | Image content awareness switch |
Filters - File Filtering Rules:
| Field | Type | Supported Operators | Example |
|---|---|---|---|
| Size | Integer | =, !=, >, >=, <, <= | Size > 1024 |
| Filename | String | =, !=, prefix, suffix, in, notin | Filename prefix (YWEvYmIv) |
| FileModifiedTime | String | =, !=, >, >=, <, <= | FileModifiedTime > 2025-06-03T09:20:47.999Z |
| OSSTagging.* | String | =, !=, !, exists, prefix, suffix, in, notin | OSSTagging.Zm9v == YWJj |
Note: Filename and OSSTagging values must be URL-safe Base64 encoded
---
GetMetaQueryStatus (Get Index Status)
Request Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
| bucket | string | Yes | Bucket name |
Response Fields:
| Field | Type | Description |
|---|---|---|
| State | string | Index status |
| Phase | string | Current scan phase |
| CreateTime | string | Creation time (RFC 3339 format) |
| UpdateTime | string | Update time (RFC 3339 format) |
| MetaQueryMode | string | Search mode: basic or semantic |
State Values:
| Value | Description |
|---|---|
Ready | Preparing after creation, data cannot be queried |
Running | Running |
Stop | Paused |
Retrying | Retrying after creation failure |
Failed | Creation failed |
Deleted | Deleted |
Phase (Scan Phase):
| Value | Description |
|---|---|
FullScanning | Full scan in progress |
IncrementalScanning | Incremental scan in progress |
---
DoMetaQuery (Execute Query)
Vector Search Mode (mode=semantic)
Request Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
| bucket | string | Yes | Bucket name |
| mode | string | Yes | semantic |
| Query | string | Yes | Semantic query content, e.g., "aerial view of a snow-covered forest" |
| MaxResults | int | No | Maximum number of results (0-100), default 100 |
| MediaTypes | array | Yes | Media type list |
| SimpleQuery | string | No | Additional filter conditions (JSON) |
Supported MediaType Values:
| Value | Description |
|---|---|
image | Image |
video | Video |
audio | Audio |
document | Document |
SimpleQuery Examples:
// File size greater than 30 bytes
{"Operation": "gt", "Field": "Size", "Value": "30"}
// Combined conditions
{
"Operation": "and",
"SubQueries": [
{"Operation": "gt", "Field": "Size", "Value": "1000"},
{"Operation": "prefix", "Field": "Filename", "Value": "videos/"}
]
}Scalar Search Mode (mode=basic)
Request Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
| Query | string | Yes | Query conditions (JSON format) |
| Sort | string | No | Sort field |
| Order | string | No | Sort order: asc (ascending), desc (descending, default) |
| Aggregations | array | No | Aggregation operations |
| NextToken | string | No | Pagination token |
Query Operators:
| Operator | Description |
|---|---|
eq | Equal to |
gt | Greater than |
gte | Greater than or equal to |
lt | Less than |
lte | Less than or equal to |
match | Fuzzy match |
prefix | Prefix match |
and | Logical AND |
or | Logical OR |
not | Logical NOT |
Aggregation Operations:
| Operator | Description |
|---|---|
min | Minimum value |
max | Maximum value |
average | Average |
sum | Sum |
count | Count |
distinct | Distinct count |
group | Group count |
Response Fields:
| Field | Type | Description |
|---|---|---|
| NextToken | string | Pagination token |
| Files | array | File list |
| Files[].Filename | string | Full file path |
| Files[].Size | int | File size (bytes) |
| Files[].FileModifiedTime | string | Modification time |
| Files[].OSSObjectType | string | Object type: Normal/Appendable/Multipart/Symlink |
| Files[].OSSStorageClass | string | Storage class: Standard/IA/Archive/ColdArchive |
| Files[].ETag | string | ETag value |
| Files[].OSSTagging | array | Tag list |
| Files[].OSSUserMeta | array | Custom metadata |
| Aggregations | array | Aggregation results |
---
CloseMetaQuery (Close Index)
Request Parameters:
| Parameter | Type | Required | Description |
|---|---|---|---|
| bucket | string | Yes | Bucket name |
---
Opening MetaQuery via Python Script
Opening MetaQuery must use the Python script. For detailed usage, WorkflowParameters XML configuration, and the enablement process, refer to "Rule 1" and "Task 2: Enable Vector Search & AI Content Awareness" sections in SKILL.md.
SDK Version Requirements
| SDK | Minimum Version | Package Name |
|---|---|---|
| Java SDK | 3.18.2+ | com.aliyun.oss:aliyun-sdk-oss |
| Python SDK (oss2) | - | oss2 |
| Go SDK V2 | - | github.com/aliyun/alibabacloud-oss-go-sdk-v2 |
| PHP SDK V2 | - | alibabacloud/oss-sdk-php |
Reference Links
Verification Methods
Table of Contents
- 1. Verify Bucket Creation
- 2. Verify File Upload
- 3. Verify Data Index Status
- 4. Verify Semantic Search Functionality
- 5. Verify AI Content Awareness Results
- 6. Complete Verification Flow
- Common Issue Troubleshooting
---
1. Verify Bucket Creation
CLI Verification
# Check if Bucket exists
aliyun ossutil api get-bucket-info --bucket <bucket-name> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
# View detailed Bucket information
aliyun ossutil api get-bucket-stat --bucket <bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquerySuccess Indicators:
- Command returns Bucket information without errors
- Displays Bucket creation time, region, storage class, and other information
---
2. Verify File Upload
CLI Verification
# List files in the Bucket
aliyun ossutil ls oss://<bucket-name>/<prefix>/ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
# View individual file information
aliyun ossutil api head-object --bucket <bucket-name> --key <object-key> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquerySuccess Indicators:
- File list contains the uploaded files
- File size and last modification time are correct
---
3. Verify Data Index Status
CLI Verification
aliyun ossutil api get-meta-query-status --bucket <bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquerySuccess Indicators:
- Command returns JSON result without errors
StateisRunningPhaseisFullScanning(full scan in progress) orIncrementalScanning(incremental scan, searchable)MetaQueryModeissemantic
---
4. Verify Semantic Search Functionality
CLI Verification
1. Create query condition file `meta-query.xml`:
<MetaQuery>
<MediaTypes>
<MediaType>video</MediaType>
<MediaType>image</MediaType>
</MediaTypes>
<Query>a yard with parked cars</Query>
</MetaQuery>2. Execute semantic search:
aliyun ossutil api do-meta-query --bucket <bucket-name> --meta-query file://meta-query.xml --meta-query-mode semantic --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquerySuccess Indicators:
- Command returns XML result without errors
- Result contains a
<Files>list - File entries contain
<OSSAIMeta>fields (with description and summary)
---
5. Verify AI Content Awareness Results
Check Returned AI Metadata
In the semantic search results (XML) from Step 4, check whether each file entry contains the <OSSAIMeta> field:
<File>
<Filename>test_medias/example.jpg</Filename>
...
<OSSAIMeta>
<Description>A yard with several parked cars, surrounded by walls and green plants...</Description>
<Summary>Yard with cars</Summary>
</OSSAIMeta>
</File>Success Indicators:
- Returned files contain the
<OSSAIMeta>field <Description>content is approximately 100 characters describing the file content<Summary>content is no more than 20 characters, a concise summary
---
6. Complete Verification Flow
Execute the following CLI commands in order for end-to-end verification:
# [1/4] Verify Bucket exists
aliyun ossutil api get-bucket-info --bucket <bucket-name> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
# [2/4] Verify files are uploaded
aliyun ossutil ls oss://<bucket-name>/<prefix>/ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
# [3/4] Verify data index status (confirm State=Running, MetaQueryMode=semantic)
aliyun ossutil api get-meta-query-status --bucket <bucket-name> --output-format json --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery
# [4/4] Execute semantic search (create meta-query.xml first, refer to Step 4)
aliyun ossutil api do-meta-query --bucket <bucket-name> --meta-query file://meta-query.xml --meta-query-mode semantic --user-agent AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaqueryVerification Pass Criteria: 1. Bucket information returns successfully 2. File list contains uploaded files 3. Index status State is Running, Phase is IncrementalScanning 4. Semantic search results contain <OSSAIMeta> (with Description and Summary)
---
Common Issue Troubleshooting
| Issue | Possible Cause | Solution |
|---|---|---|
| Index status query fails | Data index not enabled | Enable data index in Console |
| Semantic search returns no results | Index build not complete | Wait for index build to complete (may take hours) |
| No AI metadata | AI content awareness not enabled | Enable image/video content awareness in Console |
| Insufficient permissions | RAM permissions missing | Add oss:DoMetaQuery and other required permissions |
| Region not supported | Bucket not in a supported region | Use a region that supports AI content awareness |
import argparse
import oss2
from oss2.models import MetaQuery
from credentials import create_oss_auth, create_oss_bucket
from validation import validate_common_args
def parse_args():
parser = argparse.ArgumentParser(description='执行OSS基本查询(标量检索)')
parser.add_argument('--region', type=str, default='cn-beijing',
help='OSS region,例如:cn-beijing')
parser.add_argument('--bucket', type=str, required=True,
help='OSS bucket 名称')
parser.add_argument('--endpoint', type=str, default=None,
help='OSS endpoint,例如:https://oss-cn-beijing.aliyuncs.com(不指定则自动生成)')
parser.add_argument('--scalar-query', type=str, required=True,
help='完整的标量查询JSON字符串,例如: \'{"SubQueries":[{"Field":"Filename","Value":"test.jpg","Operation":"eq"}],"Operation":"and"}\'')
return parser.parse_args()
def main():
args = parse_args()
validate_common_args(args)
if args.endpoint is None:
args.endpoint = f'https://oss-{args.region}.aliyuncs.com'
auth = create_oss_auth()
bucket = create_oss_bucket(auth, args.endpoint, args.bucket, region=args.region)
query_request = MetaQuery(
max_results=100,
query=args.scalar_query,
sort='Size',
order='asc',
)
try:
result = bucket.do_bucket_meta_query(query_request)
print('查询结果:')
if result.files:
for file_info in result.files:
print(f' 文件: {file_info.file_name}')
print(f' 大小: {file_info.size}')
print(f' 类型: {file_info.oss_object_type}')
print(f' 存储类型: {file_info.oss_storage_class}')
print(f' ETag: {file_info.etag}')
print(' ---')
else:
print(' 无匹配结果')
except oss2.exceptions.OssError as e:
print(f'查询失败: {e.message}')
print(f'Error Code: {e.code}, EC: {e.ec}')
return False
if __name__ == "__main__":
main()import argparse
import oss2
from credentials import create_oss_auth, create_oss_bucket
from validation import validate_common_args
# 用于关闭 OSS MetaQuery 功能
def parse_args():
parser = argparse.ArgumentParser(description='关闭OSS MetaQuery功能')
parser.add_argument('--region', type=str, default='cn-shenzhen',
help='OSS region,例如:cn-shenzhen')
parser.add_argument('--bucket', type=str, required=True,
help='OSS bucket 名称')
parser.add_argument('--endpoint', type=str, default=None,
help='OSS endpoint,例如:https://oss-cn-shenzhen.aliyuncs.com(不指定则自动生成)')
return parser.parse_args()
def main():
args = parse_args()
validate_common_args(args)
if args.endpoint is None:
args.endpoint = f'https://oss-{args.region}.aliyuncs.com'
auth = create_oss_auth()
bucket = create_oss_bucket(auth, args.endpoint, args.bucket, region=args.region)
try:
bucket.close_bucket_meta_query()
print(f'{args.bucket} 关闭MetaQuery成功')
except oss2.exceptions.OssError as e:
print(f'关闭 {args.bucket} MetaQuery 失败: {e.message}')
print(f'Error Code: {e.code}, EC: {e.ec}')
return False
if __name__ == "__main__":
main()import argparse
import oss2
from credentials import create_oss_auth, create_oss_bucket
from validation import validate_common_args
# 用于创建 OSS 存储空间
def parse_args():
parser = argparse.ArgumentParser(description='创建OSS存储空间')
parser.add_argument('--region', type=str, default='cn-beijing',
help='OSS region,例如:cn-beijing')
parser.add_argument('--bucket', type=str, required=True,
help='OSS bucket 名称')
parser.add_argument('--endpoint', type=str, default=None,
help='OSS endpoint,例如:https://oss-cn-beijing.aliyuncs.com(不指定则自动生成)')
return parser.parse_args()
def main():
args = parse_args()
validate_common_args(args)
if args.endpoint is None:
args.endpoint = f'https://oss-{args.region}.aliyuncs.com'
auth = create_oss_auth()
bucket = create_oss_bucket(auth, args.endpoint, args.bucket, region=args.region)
try:
bucket.create_bucket()
print(f'创建存储空间 {args.bucket} 成功')
except oss2.exceptions.OssError as e:
print(f'创建 {args.bucket} 失败: {e.message}')
print(f'Error Code: {e.code}')
return False
if __name__ == "__main__":
main()import oss2
from alibabacloud_credentials.client import Client as CredentialClient
class _DefaultCredentialProvider(oss2.credentials.CredentialsProvider):
"""基于 alibabacloud_credentials 默认凭证链的 oss2 CredentialsProvider 实现。
CredentialClient 会按以下顺序自动查找凭证:
环境变量 → ~/.aliyun/config.json → IMDS(ECS 实例角色)等,
无需在代码中显式处理 AK/SK。
"""
def __init__(self):
self._client = CredentialClient()
def get_credentials(self):
ak = self._client.get_access_key_id()
sk = self._client.get_access_key_secret()
token = self._client.get_security_token()
return oss2.credentials.Credentials(ak, sk, token)
def create_oss_auth():
"""通过默认凭证链创建 oss2 V4 签名认证对象。
依赖 alibabacloud_credentials 的 CredentialClient 自动发现凭证,
支持 AK、STS、ECS 实例角色等多种认证方式,无需手动传入凭证。
"""
credentials_provider = _DefaultCredentialProvider()
return oss2.ProviderAuthV4(credentials_provider)
def create_oss_bucket(auth, endpoint, bucket_name, region=None):
"""创建带有 User-Agent 配置的 OSS Bucket 对象。
Args:
auth: 认证对象
endpoint: OSS endpoint
bucket_name: Bucket 名称
region: 区域(可选)
Returns:
配置好 User-Agent 的 Bucket 对象
"""
bucket = oss2.Bucket(auth, endpoint, bucket_name, region=region, connect_timeout=60)
bucket.user_agent = 'AlibabaCloud-Agent-Skills/alibabacloud-oss-manage-metaquery'
return bucket
import argparse
import oss2
from credentials import create_oss_auth, create_oss_bucket
from validation import validate_common_args
# 用于打开 MetaQuery 功能(向量模式 + 内容感知)
def parse_args():
parser = argparse.ArgumentParser(description='打开 OSS MetaQuery 功能')
parser.add_argument('--region', type=str, default='cn-shenzhen',
help='OSS region,例如:cn-shenzhen')
parser.add_argument('--bucket', type=str, required=True,
help='OSS bucket 名称')
parser.add_argument('--endpoint', type=str, default=None,
help='OSS endpoint,例如:https://oss-cn-shenzhen.aliyuncs.com(不指定则自动生成)')
return parser.parse_args()
def main():
args = parse_args()
validate_common_args(args)
if args.endpoint is None:
args.endpoint = f'https://oss-{args.region}.aliyuncs.com'
auth = create_oss_auth()
bucket = create_oss_bucket(auth, args.endpoint, args.bucket, region=args.region)
# oss2 的 open_bucket_meta_query() 不支持传递 WorkflowParameters,
# 因此使用底层 _do 方法发送自定义 XML body 以同时开启向量模式和内容感知
xml_body = '''<MetaQuery>
<WorkflowParameters>
<WorkflowParameter>
<Name>VideoInsightEnable</Name>
<Value>True</Value>
</WorkflowParameter>
<WorkflowParameter>
<Name>ImageInsightEnable</Name>
<Value>True</Value>
</WorkflowParameter>
</WorkflowParameters>
</MetaQuery>'''
try:
resp = bucket._do(
'POST',
args.bucket,
'',
params={'comp': 'add', 'mode': 'semantic', 'metaQuery': ''},
data=xml_body.encode('utf-8'),
)
print(f'{args.bucket} 开通MetaQuery成功')
print(f'status code: {resp.status}, request id: {resp.headers.get("x-oss-request-id", "N/A")}')
except oss2.exceptions.OssError as e:
print(f'{args.bucket} 开通MetaQuery失败: {e.message}')
print(f'Error Code: {e.code}, EC: {e.ec}')
return False
if __name__ == "__main__":
main()import argparse
import sys
from xml.sax.saxutils import escape as xml_escape
import oss2
from credentials import create_oss_auth, create_oss_bucket
from validation import validate_common_args
VALID_MEDIA_TYPES = {'image', 'video', 'audio', 'document'}
def parse_args():
parser = argparse.ArgumentParser(description='执行OSS语义查询(向量检索)')
parser.add_argument('--region', type=str, default='cn-beijing',
help='OSS region,例如:cn-beijing')
parser.add_argument('--bucket', type=str, required=True,
help='OSS bucket 名称')
parser.add_argument('--endpoint', type=str, default=None,
help='OSS endpoint,例如:https://oss-cn-beijing.aliyuncs.com(不指定则自动生成)')
parser.add_argument('--query', type=str, required=True,
help='语义查询内容,如"人"、"风景"等自然语言描述')
parser.add_argument('--media-types', type=str, nargs='+', default=['video'],
help='多媒体类型,支持: image, video, audio, document')
parser.add_argument('--scalar-query', type=str, help='完整的标量查询JSON字符串,用于向量+标量组合查询')
return parser.parse_args()
def main():
args = parse_args()
# 校验 media-types 是否在允许的枚举值范围内
invalid_types = set(args.media_types) - VALID_MEDIA_TYPES
if invalid_types:
print(f'错误: 不支持的媒体类型: {", ".join(invalid_types)}')
print(f'允许的值: {", ".join(sorted(VALID_MEDIA_TYPES))}')
sys.exit(1)
validate_common_args(args)
if args.endpoint is None:
args.endpoint = f'https://oss-{args.region}.aliyuncs.com'
auth = create_oss_auth()
bucket = create_oss_bucket(auth, args.endpoint, args.bucket, region=args.region)
# 构建媒体类型 XML(对用户输入进行 XML 转义)
media_types_xml = ''.join([f'<MediaType>{xml_escape(mt)}</MediaType>' for mt in args.media_types])
# 构建 SimpleQuery 部分(如果提供了标量查询,对输入进行 XML 转义)
simple_query_part = ''
if args.scalar_query:
simple_query_part = f'\n<SimpleQuery>{xml_escape(args.scalar_query)}</SimpleQuery>'
# 构建查询 XML body(对 query 参数进行 XML 转义)
# oss2 的 do_bucket_meta_query 不支持 semantic 模式,因此使用底层 _do 方法
xml_body = f'''<MetaQuery>
<MediaTypes>
{media_types_xml}
</MediaTypes>
<Query>{xml_escape(args.query)}</Query>{simple_query_part}
</MetaQuery>'''
try:
resp = bucket._do(
'POST',
args.bucket,
'',
params={'comp': 'query', 'mode': 'semantic', 'metaQuery': ''},
data=xml_body.encode('utf-8'),
)
content = resp.read()
print('查询结果:')
print(content.decode('utf-8'))
except oss2.exceptions.OssError as e:
print(f'查询失败: {e.message}')
print(f'Error Code: {e.code}, EC: {e.ec}')
return False
if __name__ == "__main__":
main()import argparse
import os
import time
import oss2
from credentials import create_oss_auth, create_oss_bucket
from validation import validate_common_args
# 用于上传文件到 OSS
def parse_args():
parser = argparse.ArgumentParser(description='上传文件到OSS')
parser.add_argument('--region', type=str, default='cn-beijing',
help='OSS region,例如:cn-beijing')
parser.add_argument('--bucket', type=str, required=True,
help='OSS bucket 名称')
parser.add_argument('--endpoint', type=str, default=None,
help='OSS endpoint,例如:https://oss-cn-beijing.aliyuncs.com(不指定则自动生成)')
parser.add_argument('--local-path', type=str, required=True,
help='本地文件路径')
parser.add_argument('--remote-key', type=str, required=True,
help='OSS上的文件名(key)')
return parser.parse_args()
def main():
args = parse_args()
validate_common_args(args)
if args.endpoint is None:
args.endpoint = f'https://oss-{args.region}.aliyuncs.com'
if not os.path.exists(args.local_path):
print(f'文件不存在: {args.local_path}')
return False
auth = create_oss_auth()
bucket = create_oss_bucket(auth, args.endpoint, args.bucket, region=args.region)
# 生成 OSS tag,记录文件创建时间纳秒时间戳,后续可用于搜索
tagging_header = f'CreatTime={time.time_ns()}'
try:
print(f'开始上传文件: {args.remote_key}')
with open(args.local_path, 'rb') as file_obj:
result = bucket.put_object(
args.remote_key,
file_obj,
headers={'x-oss-tagging': tagging_header},
)
print(f'上传 {args.remote_key} 成功')
print(f'status code: {result.status}, request id: {result.request_id}')
except oss2.exceptions.OssError as e:
print(f'上传 {args.remote_key} 失败: {e.message}')
print(f'Error Code: {e.code}')
return False
if __name__ == "__main__":
main()import re
import sys
# OSS 支持向量检索与 AI 内容感知的地域列表
VALID_REGIONS = {
'cn-hangzhou', # 华东1(杭州)
'cn-shanghai', # 华东2(上海)
'cn-qingdao', # 华北1(青岛)
'cn-beijing', # 华北2(北京)
'cn-zhangjiakou', # 华北3(张家口)
'cn-shenzhen', # 华南1(深圳)
'cn-guangzhou', # 华南3(广州)
'cn-chengdu', # 西南1(成都)
'cn-hongkong', # 中国香港
'ap-southeast-1', # 新加坡
'us-east-1', # 美国(弗吉尼亚)
}
# OSS Bucket 命名规则正则:3-63 个字符,仅允许小写字母、数字和短横线,
# 不能以短横线开头或结尾
_BUCKET_NAME_RE = re.compile(r'^[a-z0-9][a-z0-9\-]{1,61}[a-z0-9]$')
# Endpoint 合法格式正则:https://oss-{region}.aliyuncs.com 或
# https://{region}.oss[-internal].aliyuncs.com 等阿里云 OSS 域名
_ENDPOINT_RE = re.compile(
r'^https://[\w\-]+\.aliyuncs\.com$'
)
def validate_region(region):
"""校验 --region 参数是否为合法的 OSS 地域。"""
if region not in VALID_REGIONS:
print(f'错误: 不支持的地域: {region}')
print(f'允许的值: {", ".join(sorted(VALID_REGIONS))}')
sys.exit(1)
def validate_bucket_name(name):
"""校验 --bucket 参数是否符合 OSS Bucket 命名规则。
规则: 3-63 个字符,仅允许小写字母、数字和短横线,不能以短横线开头或结尾。
"""
if not _BUCKET_NAME_RE.match(name):
print(f'错误: Bucket 名称不合法: {name}')
print('Bucket 命名规则: 3-63 个字符,仅允许小写字母、数字和短横线,不能以短横线开头或结尾')
sys.exit(1)
def validate_endpoint(endpoint):
"""校验 --endpoint 参数是否为合法的阿里云 OSS 域名。"""
if not _ENDPOINT_RE.match(endpoint):
print(f'错误: endpoint 格式不合法: {endpoint}')
print('endpoint 必须为 https://<host>.aliyuncs.com 格式的阿里云 OSS 域名')
sys.exit(1)
def validate_common_args(args):
"""统一校验 --region、--bucket、--endpoint 参数。
应在各脚本 main() 函数开头、使用参数之前调用。
如果 endpoint 为 None(由 region 自动生成),则跳过 endpoint 校验。
"""
validate_region(args.region)
validate_bucket_name(args.bucket)
if args.endpoint is not None:
validate_endpoint(args.endpoint)