
Alibabacloud Milvus Manage
- 145 installs
- 208 repo stars
- Updated August 4, 2026
- aliyun/alibabacloud-aiops-skills
Administer Alibaba Cloud Milvus vector stores—collections, indexes, scaling, and connectivity—for RAG, semantic search, and embedding-heavy AI features in production.
About
alibabacloud-milvus-manage guides agents operating Alibaba Cloud Milvus for vector search and RAG: collections, indexes, scaling, endpoints, and routine health tasks. AI product teams rely on it to keep embedding pipelines reliable without deep manual Milvus expertise during incidents or capacity changes.
- Milvus collection and index administration
- Cluster scaling and resource tuning
- Endpoint and access configuration
- Embedding retrieval health checks
- Managed vector DB operational runbooks
Alibabacloud Milvus Manage by the numbers
- 145 all-time installs (skills.sh)
- Ranked #274 of 911 Databases skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/aliyun/alibabacloud-aiops-skills --skill alibabacloud-milvus-manageAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 145 |
|---|---|
| repo stars | ★ 208 |
| Last updated | August 4, 2026 |
| Repository | aliyun/alibabacloud-aiops-skills ↗ |
What it does
Administer Alibaba Cloud Milvus vector stores—collections, indexes, scaling, and connectivity—for RAG, semantic search, and embedding-heavy AI features in production.
Files
Alibaba Cloud Milvus Full-Stack Skill
Prerequisites
[MUST] CLI User-Agent — Every aliyun CLI command invocation must include: --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage
[MUST] PyMilvus User-Agent — Every pymilvus SDK connection must include: grpc_options={"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"}
Alibaba Cloud CLI version must be >= 3.3.3.
Handle two distinct planes:
- Control-plane: manage Alibaba Cloud managed Milvus instances with
aliyunCLI. - Data-plane: operate Milvus with
pymilvusPython code.
Treat SKILL.md as the router. Load references/*.md for detailed commands, parameters, and examples.
Scope
Use this skill for:
- Alibaba Cloud managed Milvus instance lifecycle: create, inspect, scale, rename, configure, network, whitelist.
- Milvus Python SDK workflows with
pymilvus: connect, collections, vectors, search, indexes, partitions, databases, RBAC. - Retrieval use cases built on Milvus: semantic search, hybrid search, full-text search, RAG patterns.
Do not use this skill for:
- self-hosted Milvus deployment on Docker, Helm, Kubernetes, or Milvus Operator,
- Milvus Java / Go / Node SDKs,
- other Alibaba Cloud products such as ECS, RDS, OSS, EMR, Kafka, StarRocks,
- other vector databases such as Zilliz Cloud, Pinecone, Qdrant, or Weaviate.
Route The Request
Control-plane
Route here when the user asks about:
- creating, scaling, renaming, or inspecting a Milvus instance,
- connection address, component spec, configuration, public network, whitelist,
- VPC/VSwitch prerequisites for Alibaba Cloud Milvus,
- Milvus REST-style CLI APIs, creation parameters, or control-plane troubleshooting.
Read:
- first-time flow: references/getting-started.md
- create / list / detail / scale / release: references/instance-lifecycle.md
- config / network / inspection / troubleshooting: references/operations.md
- creation field meanings and templates: references/create-params.md
- raw API field reference: references/api-reference.md
- RAM permissions: references/ram-policies.md
Data-plane
Route here when the user asks about:
- connecting to Milvus with Python,
- creating collections or schemas,
- inserting, upserting, querying, deleting, or searching vectors,
- hybrid search, BM25 full-text search, iterators, indexes,
- partitions, databases, users, roles, or privileges,
- Milvus-based RAG or semantic retrieval patterns.
Read:
- collection schema and lifecycle: references/collection.md
- vector CRUD, search, hybrid search, full-text search: references/vector.md
- index types and metrics: references/index.md
- partitions: references/partition.md
- databases: references/database.md
- RBAC: references/user-role.md
- common solution patterns: references/patterns.md
Shared Guardrails
- Decide the plane first. Do not mix control-plane instance operations with data-plane SDK code.
- Confirm destructive actions before execution.
- Validate untrusted user input before passing it into shell commands or code.
- Prefer loading a targeted reference doc instead of keeping large inline examples in this file.
Control-Plane Rules
Required Environment
- Reuse the configured
aliyunprofile. Verify credentials are configured before API calls. - Every
aliyunCLI invocation must include the required User-Agent flag:
aliyun ... --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage- Milvus OpenAPI calls through
aliyunmust include--force.
Preconditions
Before create or major modify operations:
1. Confirm RegionId with the user. 2. Verify VPC and VSwitch resources in that region. 3. For create, record ZoneId, VpcId, and VSwitchId. 4. If the request is ambiguous, ask whether the user wants dev/test standalone or production HA cluster.
Baseline decision rule:
standalone_prois the default for dev/test.- HA cluster is for production.
- In HA mode,
streaming,data,mix_coordinator, andquerymust use at least 4 CU;proxymust use at least 2 CU.
Detailed templates and field definitions live in references/instance-lifecycle.md and references/create-params.md.
CLI Calling Modes
Use the API's expected parameter mode. Do not improvise.
# get / delete: business params in URL query
aliyun milvus get "/path?RegionId=<region>&instanceId=<id>" --RegionId <region> --force --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage
# post / put with request body: business params in --body JSON
aliyun milvus post "/path?RegionId=<region>" --RegionId <region> --body '{...}' --force --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage
# post with query-style flags: business params as --Flag value
aliyun milvus post "/path" --RegionId <region> --InstanceId <id> --force --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manageRules:
- Always pass
--RegionId <region>. - For
CreateInstanceandUpdateInstance, use--body. - For query-style POST APIs such as detail, config, network, ACL, and rename operations, use
--Flag value. - Do not put user-provided raw text directly into a shell command unless it has been validated.
Runtime Safety
- Do not download and execute remote scripts or unaudited dependencies during control-plane work.
- Do not use
evalorsourcewith untrusted input. - Set reasonable timeouts on CLI calls. Prefer short timeouts for reads and bounded polling for long-running async operations.
- For list APIs, do not trust
totalblindly; inspect the returned array. - Read the full error message before retrying. Automatic retry is appropriate for throttling, not for arbitrary failures.
Forbidden Operations
- Instance deletion (DeleteInstance) is strictly forbidden through this Skill. If the user requests to delete/release a Milvus instance, do not execute the Milvus delete command through
aliyunCLI. Instead, instruct the user to delete the instance via the Alibaba Cloud Milvus Console.
Destructive Operations
Require explicit confirmation before:
- modifying instance config,
- disabling public network access.
Use this template:
About to execute:<API>, Target:<InstanceId>, Impact:<Description>. Continue?
For config change and network troubleshooting flows, read references/operations.md or references/instance-lifecycle.md first.
Output Style
- Summarize instance lists as a compact table.
- Highlight
instanceId,instanceName,status,dbVersion,ha,paymentType, and connection endpoints when relevant. - Convert timestamps to readable time.
- Use
--cli-queryorjqto trim noisy payloads when useful.
Data-Plane Rules
Connection First
Before writing any pymilvus code, ask for:
1. deployment type: Milvus Lite, self-hosted standalone/cluster, or Alibaba Cloud managed instance, 2. URI or endpoint, 3. authentication method and credentials if needed, 4. database name if not using default.
Do not assume connection parameters. Use Milvus Lite only when the user explicitly wants local embedded mode.
Minimal connection shape:
from pymilvus import MilvusClient
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
client = MilvusClient(
uri="<USER_URI>",
token="<USER_TOKEN>",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)- Every
MilvusClient(...)andconnections.connect(...)example must passgrpc_options=PYMILVUS_GRPC_OPTIONS. - Do not emit
pymilvusSDK connection code withoutgrpc_options=PYMILVUS_GRPC_OPTIONS.
For async usage, schema details, and deployment-specific patterns, load the relevant reference doc.
Data Safety And Correctness
- Never generate fake or placeholder vectors. Always use a real embedding model.
- The query embedding model must match the model used to create stored vectors.
- Vector dimensions must exactly match the collection schema.
- A collection must be loaded before search or query.
- Confirm destructive operations such as
drop_collection,drop_database, or large deletes before executing. - Prefer
AUTOINDEXunless the user has explicit performance requirements.
Minimal Workflow
For most SDK tasks:
1. load references/collection.md for schema and collection operations, 2. load references/vector.md for insert/search/query/delete patterns, 3. load references/index.md if the user cares about index type, metric, or tuning, 4. add partition/database/RBAC references only if the task actually needs them.
Common Patterns
- quick prototype collection: references/collection.md
- vector CRUD and similarity search: references/vector.md
- hybrid search or full-text search: references/vector.md
- RAG / semantic retrieval patterns: references/patterns.md
- index tuning: references/index.md
Suggested Response Flow
If control-plane
1. Confirm region and target instance scope. 2. Read the matching control-plane reference. 3. Run the command with the correct parameter mode. 4. Report the key fields, next state, and any follow-up wait conditions.
If data-plane
1. Ask for connection details first. 2. Read only the references needed for the requested SDK task. 3. Write or explain pymilvus code with real embeddings, real connection placeholders, and grpc_options=PYMILVUS_GRPC_OPTIONS. 4. Call out schema, load-state, index, and dimension pitfalls if they matter.
Reference Map
- references/getting-started.md: first Milvus instance from scratch
- references/instance-lifecycle.md: create, inspect, scale, rename, release
- references/operations.md: config, network, ACL, inspection, troubleshooting
- references/create-params.md: create body fields and component templates
- references/api-reference.md: raw API signatures and return fields
- references/collection.md: schema and collection lifecycle
- references/vector.md: insert, search, hybrid search, BM25, iterators
- references/index.md: index types and metric guidance
- references/partition.md: partition operations
- references/database.md: database operations
- references/user-role.md: users, roles, privileges
- references/patterns.md: RAG and semantic search patterns
- references/ram-policies.md: IAM/RAM policies
API Parameter Reference
All APIs version 2023-10-12, Endpoint is milvus.<RegionId>.aliyuncs.com.
Calling Method: Use aliyun CLI REST style, must add `--force` (bypass local path validation).
User-Agent requirement: EveryaliyunCLI invocation must include--user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage.
⚠️ Critical Limitation: Milvus API has two parameter passing methods, must choose according to API type:
- GET / DELETE: All business parameters concatenated to URL query string (e.g.,
"/path?RegionId=xx&instanceId=c-xxx") - POST / PUT (body type): Pass JSON with
--body '{...}'(CreateInstance, UpdateInstance) - POST (query type): Business parameters passed with
--Flag value(other POST APIs) - All requests keep
--RegionId <region>for endpoint routing
Table of Contents
- Instance Management: ListInstancesV2, GetInstance, GetInstanceDetail, CreateInstance, ~~DeleteInstance~~ (console only), UpdateInstance, UpdateInstanceName
- Configuration Management: DescribeInstanceConfigs, ModifyInstanceConfig
- Network and Security: UpdatePublicNetworkStatus, DescribeAccessControlList, UpdateAccessControlList
- Resource Group: ChangeResourceGroup
- Others: CreateDefaultRole
- Network Resource Query: DescribeVpcs, DescribeVSwitches, DescribeSecurityGroups
---
Instance Management
ListInstancesV2 — Query Instance List
Path: GET /webapi/instance/list
Request Parameters (CLI flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| RegionId | String | Yes | Region ID |
| instanceName | String | No | Filter by instance name |
| instanceId | String | No | Filter by instance ID |
| pageNumber | Integer | No | Page number, default 1 |
| pageSize | Integer | No | Page size, default 10, max 100 |
Key Return Fields: instances[] (instanceId, instanceName, regionId, zoneId, status, dbVersion, ha, paymentType, createTime, vpcId)
⚠️ Note: Returned total field may be inaccurate (returns 0 but actually has data), should directly check instances array.
aliyun milvus get "/webapi/instance/list?RegionId=cn-hangzhou&pageNumber=1&pageSize=50" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --force---
GetInstance — Query Instance Basic Info
Path: GET /webapi/instance/get
Request Parameters (CLI flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| RegionId | String | Yes | Region ID |
| instanceId | String | Yes | Instance ID |
Key Return Fields: instance (instanceId, instanceName, regionId, zoneId, status, dbVersion, ha, paymentType, createTime, vpcId)
aliyun milvus get "/webapi/instance/get?RegionId=cn-hangzhou&instanceId=c-xxx" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --force---
GetInstanceDetail — Query Instance Details
Get component specs, connection addresses (intranet/public), storage usage, HA config and other detailed info.
Path: POST /webapi/cluster/detail
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| InstanceId | String | Yes | Instance ID |
Key Return Fields:
Data.InstanceId/ClusterName/RegionId/ZoneId/InstanceStatusData.Version/EnableHa/PayType(0=PayAsYouGo, 1=Subscription)Data.ClusterInfo.IntranetUrl/InternetUrl/ProxyPort/AttuPortData.ClusterInfo.TotalCuNum/TotalDiskSize/OssStorageSizeData.ClusterInfo.MilvusResourceInfoList[](ComponentType, Replica, CuNum, DiskSize)
aliyun milvus post "/webapi/cluster/detail" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force---
CreateInstance — Create Instance
Path: POST /webapi/instance/create
Request Parameters (RegionId is CLI flag, others are body camelCase):
| Parameter | Location | Type | Required | Description |
|---|---|---|---|---|
| RegionId | CLI flag | String | Yes | Region ID |
| clientToken | CLI flag | String | No | Idempotent token, max 64 ASCII characters |
| regionId | body | String | Yes | Region ID, must match RegionId in CLI flag |
| zoneId | body | String | Yes | Primary availability zone |
| instanceName | body | String | Yes | Instance name |
| dbVersion | body | String | Yes | Kernel version: 2.3 / 2.4 / 2.5 / 2.6 |
| vpcId | body | String | Yes | VPC ID |
| vSwitchIds | body | Array | Yes | VSwitch list, see structure description |
| paymentType | body | String | Yes | PayAsYouGo / Subscription |
| ha | body | Boolean | Yes | false=standalone, true=cluster |
| components | body | Array | Yes | Component config list, see structure description |
| dbAdminPassword | body | String | Yes | Admin password |
| autoBackup | body | Boolean | No | Auto backup, default false |
| loadReplicas | body | Integer | No | Load replica count, default 1 |
| encrypted | body | Boolean | No | Data encryption, default false |
| isMultiAzStorage | body | Boolean | No | Multi-AZ storage, default true |
| multiZoneMode | body | String | No | single (default) / Active-Active |
| aiFunction | body | Boolean | No | Enable AI embedding functions (auto true when dbVersion is 2.6) |
| autoRenew | body | Boolean | No | Auto renew (Subscription only) |
vSwitchIds Structure: [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}]
components Structure: [{"type":"...","replica":N,"cuNum":N,"cuType":"general","diskSizeType":"Normal"}]
- type options:
standalone_pro(standalone) /proxy/mix_coordinator/data/query/streaming(cluster) - ⚠️ streaming/data/mix_coordinator/query minimum 4 CU, proxy minimum 2 CU
Key Return Fields: data.instanceId, data.orderId, requestId
# Standalone (Development & Testing)
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "milvus-dev",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"paymentType": "PayAsYouGo",
"ha": false,
"components": [{"type":"standalone_pro","replica":1,"cuNum":4,"cuType":"general"}],
"dbAdminPassword": "YourPass@123",
"autoBackup": true,
"aiFunction": true
}' \
--force
# Cluster (Production, 36 CU)
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "milvus-prod",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"paymentType": "PayAsYouGo",
"ha": true,
"components": [
{"type":"streaming", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"data", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"proxy", "replica":2,"cuNum":2,"cuType":"general"},
{"type":"mix_coordinator", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"query", "replica":2,"cuNum":4,"cuType":"general","diskSizeType":"Normal"}
],
"dbAdminPassword": "YourPass@123",
"autoBackup": true,
"aiFunction": true
}' \
--force---
DeleteInstance — Release Instance
🚫 This API is NOT available through this Skill. Instance deletion must be performed via the Alibaba Cloud Milvus Console. Do not execute Milvus delete commands through aliyun CLI.---
UpdateInstance — Update Instance (Scaling)
Path: PUT /webapi/instance/update
Request body (camelCase):
| Parameter | Type | Required | Description |
|---|---|---|---|
| instanceId | String | Yes | Instance ID |
| instanceName | String | No | New instance name |
| ha | Boolean | No | Enable high availability |
| components | Array | No | Updated component config list |
aliyun milvus put "/webapi/instance/update?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"instanceId": "c-xxx",
"components": [
{"type":"query","replica":3,"cuNum":8,"cuType":"cap","diskSizeType":"Normal"}
]
}' \
--force---
UpdateInstanceName — Modify Instance Name
Path: POST /webapi/cluster/update_name
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| InstanceId | String | Yes | Instance ID |
| ClusterName | String | Yes | New instance name |
aliyun milvus post "/webapi/cluster/update_name" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--ClusterName new-name \
--force---
Configuration Management
DescribeInstanceConfigs — Get Instance Custom Config
Path: POST /webapi/config/describe_milvus_user_config
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| InstanceId | String | Yes | Instance ID |
Return Fields: Data (YAML format config string), Success
aliyun milvus post "/webapi/config/describe_milvus_user_config" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force---
ModifyInstanceConfig — Update Instance Config
Path: POST /webapi/config/modify_milvus_config
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| InstanceId | String | Yes | Instance ID |
| Reason | String | Yes | Update reason |
| UserConfig | String | No | YAML format user custom config |
aliyun milvus post "/webapi/config/modify_milvus_config" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--Reason "Adjust proxy max task count" \
--UserConfig "proxy:
maxTaskNum: 1024
" \
--force---
Network and Security
UpdatePublicNetworkStatus — Enable/Disable Public Network Access
Path: POST /webapi/network/updatePublicNetworkStatus
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| InstanceId | String | Yes | Instance ID |
| ComponentType | String | Yes | Component type, enter Proxy |
| PublicNetworkEnabled | Boolean | Yes | true=enable, false=disable |
| Cidr | String | No | Allowed access CIDR (recommended to fill when enabling) |
# Enable public network access and set whitelist
aliyun milvus post "/webapi/network/updatePublicNetworkStatus" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--ComponentType Proxy \
--PublicNetworkEnabled true \
--Cidr "10.0.0.0/8" \
--force---
DescribeAccessControlList — Query Public Network Whitelist
Path: POST /webapi/milvus/describe_access_control_list
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| InstanceId | String | Yes | Instance ID |
Return Fields: Data (AclId, Cidr[])
aliyun milvus post "/webapi/milvus/describe_access_control_list" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force---
UpdateAccessControlList — Update Public Network Whitelist
Path: POST /webapi/milvus/update_access_control_list
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| InstanceId | String | Yes | Instance ID |
| AclId | String | Yes | Public network access control ID (obtain via DescribeAccessControlList) |
| Cidr | String | No | CIDR block |
aliyun milvus post "/webapi/milvus/update_access_control_list" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--Cidr "192.168.1.0/24" \
--AclId acl-xxx \
--force---
Resource Group
ChangeResourceGroup — Transfer Resource Group
Path: POST /webapi/resourceGroup/change
Request Parameters (query type, pass with --Flag):
| Parameter | Type | Required | Description |
|---|---|---|---|
| NewResourceGroupId | String | Yes | Target resource group ID |
| ResourceId | String | No | Resource ID |
| RegionId | String | No | Region ID |
aliyun milvus post "/webapi/resourceGroup/change" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--NewResourceGroupId rg-xxx \
--ResourceId c-xxx \
--force---
Others
CreateDefaultRole — Create Service Role
Path: POST /webapi/user/create_default_role
Creates service role needed for Milvus to access other cloud products (like OSS), no request parameters.
aliyun milvus post "/webapi/user/create_default_role" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --force---
Network Resource Query
DescribeVpcs — Query VPC List
Product: vpc, Version: 2016-04-28
aliyun vpc describe-vpcs --biz-region-id cn-hangzhou --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manageKey Return Fields: Vpcs.Vpc[] (VpcId, VpcName, CidrBlock, Status)
---
DescribeVSwitches — Query VSwitch List
Product: vpc, Version: 2016-04-28
aliyun vpc describe-vswitches --biz-region-id cn-hangzhou --vpc-id vpc-xxx --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manageKey Return Fields: VSwitches.VSwitch[] (VSwitchId, VSwitchName, ZoneId, CidrBlock, AvailableIpAddressCount)
---
DescribeSecurityGroups — Query Security Group List
Product: ecs, Version: 2014-05-26
aliyun ecs describe-security-groups --biz-region-id cn-hangzhou --vpc-id vpc-xxx --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manageCollection Management — Detailed Reference
Supported Data Types
Scalar Types
| DataType | Notes |
|---|---|
DataType.BOOL | Boolean |
DataType.INT8 / INT16 / INT32 / INT64 | Integers |
DataType.FLOAT / DOUBLE | Floating point |
DataType.VARCHAR | String (requires max_length) |
DataType.JSON | JSON object |
DataType.ARRAY | Array (requires element_type, max_capacity) |
Vector Types
| DataType | Notes |
|---|---|
DataType.FLOAT_VECTOR | Float32 vector (requires dim) |
DataType.FLOAT16_VECTOR | Float16 vector (requires dim) |
DataType.BFLOAT16_VECTOR | BFloat16 vector (requires dim) |
DataType.BINARY_VECTOR | Binary vector (requires dim) |
DataType.SPARSE_FLOAT_VECTOR | Sparse vector (no dim needed) |
DataType.INT8_VECTOR | Int8 vector (requires dim) |
add_field Parameters
schema.add_field(
field_name="my_field",
datatype=DataType.VARCHAR,
is_primary=False,
auto_id=False,
max_length=256, # Required for VARCHAR
dim=768, # Required for vector types (except sparse)
element_type=DataType.INT64, # Required for ARRAY
max_capacity=100, # Required for ARRAY
nullable=False,
default_value=None,
is_partition_key=False,
description=""
)All Collection Operations
# List all collections
collections = client.list_collections()
# Describe a collection
info = client.describe_collection(collection_name="my_collection")
# Check if collection exists
exists = client.has_collection(collection_name="my_collection")
# Rename a collection
client.rename_collection(old_name="old_name", new_name="new_name")
# Drop a collection
client.drop_collection(collection_name="my_collection")
# Truncate a collection (delete all data, keep schema and index)
client.truncate_collection(collection_name="my_collection")
# Load collection into memory (required before search/query)
client.load_collection(collection_name="my_collection")
# Release collection from memory
client.release_collection(collection_name="my_collection")
# Get load state
state = client.get_load_state(collection_name="my_collection")
# Get collection statistics
stats = client.get_collection_stats(collection_name="my_collection")Function (Embedding Function)
Requires Milvus ≥ 2.6. Embedding functions are not available on earlier versions.
Functions allow Milvus to automatically generate vector embeddings from scalar fields during insert and search, eliminating the need to manually compute vectors.
Imports
from pymilvus import MilvusClient, DataType, Function, FunctionTypeFunction Parameters
Function(
name="my_embedding_func", # Unique identifier for this function
function_type=FunctionType.TEXTEMBEDDING, # Function type
input_field_names=["text_field"], # Scalar field(s) to embed
output_field_names=["vector_field"], # Vector field(s) to store embeddings
params={
"provider": "aliyun_milvus", # Embedding model provider
"model_name": "text-embedding-v4" # Model name
}
)| Parameter | Type | Description |
|---|---|---|
name | String | Unique identifier for this embedding function |
function_type | FunctionType | Type of function, e.g. FunctionType.TEXTEMBEDDING |
input_field_names | List[str] | Scalar field names to use as input (e.g. VARCHAR fields) |
output_field_names | List[str] | Vector field names to store generated embeddings |
params | Dict | Provider-specific parameters (provider, model_name, dim, etc.) |
Adding Function to Schema
schema.add_function(my_function)Complete Example: Text Embedding + Multimodal Embedding
from pymilvus import MilvusClient, DataType, Function, FunctionType
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
client = MilvusClient(
uri="http://<endpoint>:19530",
token="root:password",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)
schema = client.create_schema()
# Define fields
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("mm_value", DataType.VARCHAR, max_length=9000, nullable=True)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
schema.add_field("dense_mm", DataType.FLOAT_VECTOR, dim=1024)
# Text embedding function
text_embedding_function = Function(
name="text_embedding_func",
function_type=FunctionType.TEXTEMBEDDING,
input_field_names=["document"],
output_field_names=["dense"],
params={
"provider": "aliyun_milvus",
"model_name": "text-embedding-v4"
}
)
# Multimodal embedding function
mm_embedding_function = Function(
name="mm_embedding_func",
function_type=FunctionType.TEXTEMBEDDING,
input_field_names=["document"],
output_field_names=["dense_mm"],
params={
"provider": "aliyun_milvus",
"model_name": "qwen3-vl-embedding",
"dim": "1024"
}
)
# Add functions to schema
schema.add_function(text_embedding_function)
schema.add_function(mm_embedding_function)
# Create indexes
index_params = client.prepare_index_params()
index_params.add_index(field_name="dense", index_type="AUTOINDEX", metric_type="COSINE")
index_params.add_index(field_name="dense_mm", index_type="AUTOINDEX", metric_type="COSINE")
# Create collection
client.create_collection(
collection_name="my_collection",
schema=schema,
index_params=index_params
)Search with Function (Text/URL as Input)
When a collection has embedding functions, search data accepts raw text or URLs instead of vectors:
# Text search via text embedding function
results = client.search(
collection_name="my_collection",
data=["How does Milvus handle semantic search?"], # Raw text, not vector
anns_field="dense",
limit=5,
output_fields=["document", "mm_value"],
)
# Image URL search via multimodal embedding function
results = client.search(
collection_name="my_collection",
data=["https://example.com/image.jpeg"], # Image URL, not vector
anns_field="dense_mm",
limit=5,
output_fields=["document", "mm_value"],
)Insert with Function
When inserting data, only provide the scalar input fields — vector fields are auto-generated by the function:
client.insert("my_collection", [
{"id": 1, "document": "A description of an image.", "mm_value": "https://example.com/image.jpeg"},
{"id": 2, "document": "Vector embeddings convert text into numeric data.", "mm_value": "https://example.com/another.jpeg"},
{"id": 3, "document": "Semantic search helps users find relevant info."}, # mm_value is nullable
])Supported Providers and Models
| Provider | Model Name | Description |
|---|---|---|
aliyun_milvus | text-embedding-v4 | Alibaba Cloud text embedding |
aliyun_milvus | text-embedding-v3 | Alibaba Cloud text embedding |
aliyun_milvus | text-embedding-v2 | Alibaba Cloud text embedding |
aliyun_milvus | qwen3-vl-embedding | Alibaba Cloud multimodal (vision-language) embedding, requires dim param |
Key Notes
- The
input_field_namesfield must reference an existing scalar field (typically VARCHAR) in the schema. - The
output_field_namesfield must reference an existing vector field in the schema with matchingdim. - For multimodal models like
qwen3-vl-embedding, thedimparameter inparamsis optional — if omitted, it defaults to thedimof the corresponding vector field inoutput_field_names. - Multiple functions can be added to a single schema, each mapping different input/output field pairs.
- When searching, set
anns_fieldto the specific vector field corresponding to the desired embedding function.
Guidance
- Quick create is best for prototyping; use custom schema for production.
- A collection must be loaded before search or query operations.
- Before dropping a collection, confirm with the user — this deletes all data.
- Use
enable_dynamic_field=Trueto allow inserting fields not defined in the schema. - Use
truncate_collectionto clear all data while preserving the collection structure.
CreateInstance Parameter Reference
Path: POST /webapi/instance/create, version 2023-10-12.
Calling method: RegionId must be placed in both URL query string (?RegionId=<region>) and CLI flag (--RegionId <region>), other parameters placed in --body JSON (camelCase).
User-Agent requirement: EveryaliyunCLI invocation must include--user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage.
Table of Contents
1. Required Parameters 2. Component Spec Configuration 3. Network Configuration 4. Payment and High Availability 5. Other Parameters 6. Typical Configuration Examples
---
Required Parameters
| body field | Type | Description |
|---|---|---|
regionId | String | Region ID (e.g., cn-hangzhou), must match RegionId in CLI flag and URL |
zoneId | String | Primary availability zone (e.g., cn-hangzhou-j) |
instanceName | String | Instance name |
dbVersion | String | Kernel version: 2.3 / 2.4 / 2.5 / 2.6 (recommend 2.6) |
vpcId | String | VPC ID |
vSwitchIds | Array | VSwitch list, see structure description |
paymentType | String | Payment type: PayAsYouGo / Subscription |
ha | Boolean | false=standalone, true=cluster |
components | Array | Component config list, see structure description |
dbAdminPassword | String | Admin password |
aiFunction | Boolean | Enable AI embedding functions (auto true when dbVersion is 2.6) |
CLI flag:
--RegionId: Region ID (required)--clientToken: Idempotent token, max 64 ASCII characters (optional, prevent duplicate creation)
Component Specification Configuration
⚠️ Component CU Minimum Limit (Important!)
When creating cluster instances, each component has minimum CU requirements:
| Component | Minimum CU | Notes |
|---|---|---|
| streaming | 4 CU | Does not support 2 CU |
| data | 4 CU | Does not support 2 CU |
| proxy | 2 CU | Supports 2 CU |
| mix_coordinator | 4 CU | Does not support 2 CU |
| query | 4 CU | Does not support 2 CU |
Error Example: If using 2 CU configuration for streaming/data/mix_coordinator/query, you will get an error:
Error.InternalError code: 500, pricing plan price result not foundStandalone Version (standalone_pro, suitable for dev/test)
When ha=false creates standalone version, component type is standalone_pro:
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "milvus-dev",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"paymentType": "PayAsYouGo",
"ha": false,
"components": [{"type":"standalone_pro","replica":1,"cuNum":4,"cuType":"general"}],
"dbAdminPassword": "YourPassword@123",
"aiFunction": true
}' \
--forceCU Spec Reference:
| cuNum | Memory | Applicable Scenario |
|---|---|---|
| 4 | ~16GB | Personal dev/test (default) |
| 8 | ~32GB | Small-medium scale |
| 16 | ~64GB | Medium scale |
| 32 | ~128GB | Large scale |
Cluster Version (HA mode, suitable for production)
When ha=true creates cluster version, need to configure 5 components:
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "milvus-prod",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"paymentType": "PayAsYouGo",
"ha": true,
"components": [
{"type":"streaming", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"data", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"proxy", "replica":2,"cuNum":2,"cuType":"general"},
{"type":"mix_coordinator", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"query", "replica":2,"cuNum":4,"cuType":"general","diskSizeType":"Normal"}
],
"dbAdminPassword": "YourPassword@123",
"autoBackup": true,
"aiFunction": true
}' \
--forceTotal CU = 4×2 + 4×2 + 2×2 + 4×2 + 4×2 = 36 CU
Component Type Description
| Type | Responsibility | Scaling Trigger |
|---|---|---|
proxy | Request entry point, load balancing | High request QPS |
mix_coordinator | Coordination node (RootCoord + QueryCoord + DataCoord merged) | Many metadata operations |
query | Vector search execution (memory-intensive) | Memory watermark > 70% or high search latency |
data | Data write and flush (CPU-intensive) | CPU watermark > 90% |
streaming | Stream message processing (WAL / message queue replacement layer) | High write throughput |
cuType Options
| Value | Description | Applicable Scenario |
|---|---|---|
general | General type (CPU:Memory = 1:4) | Default, most scenarios |
perf | Performance type (CPU-intensive) | Index building, high-concurrency writes |
cap | Capacity type (large memory) | QueryNode large data search |
diskSizeType (query component only)
| Value | Description |
|---|---|
Normal | Normal disk (default) |
Large | Large disk |
Network Configuration
Single Availability Zone (multiZoneMode: single)
{
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"multiZoneMode": "single"
}Multi Availability Zone (multiZoneMode: Active-Active)
Need to specify one VSwitch in each of two availability zones:
{
"vSwitchIds": [
{"vswId":"vsw-primary","zoneId":"cn-hangzhou-j"},
{"vswId":"vsw-secondary","zoneId":"cn-hangzhou-b"}
],
"isMultiAzStorage": true,
"multiZoneMode": "Active-Active"
}Network Resource Discovery
Before creating instance, query available network resources:
# List VPCs
aliyun vpc describe-vpcs --biz-region-id cn-hangzhou --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage
# List VSwitches (includes availability zone and available IP count)
aliyun vpc describe-vswitches --biz-region-id cn-hangzhou --vpc-id vpc-xxx --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-managePayment and High Availability
paymentType
| Value | Description | Notes |
|---|---|---|
PayAsYouGo | Pay-as-you-go | Release anytime, suitable for testing |
Subscription | Annual/monthly subscription | Need console refund to release, can use autoRenew: true |
autoBackup
When autoBackup: true is enabled, data is automatically backed up to OSS daily.
loadReplicas
loadReplicas: N (default 1), load replica count, improves search concurrency performance.
Other Parameters
| body field | Default | Description |
|---|---|---|
autoBackup | false | Auto backup |
loadReplicas | 1 | Load replica count |
encrypted | false | Data encryption switch |
kmsKeyId | — | KMS Key ID used for encryption |
isMultiAzStorage | true | Multi-AZ storage |
multiZoneMode | single | Multi availability zone mode |
autoRenew | false | Auto renew (Subscription only) |
Complete Creation Template
For complete creation templates (dev/test / production cluster / search-intensive), please refer to Instance Full Lifecycle.
---
API Endpoint Reference
- Endpoint:
milvus.<RegionId>.aliyuncs.com - API Version:
2023-10-12 - OpenAPI Meta:
https://api.aliyun.com/meta/v1/products/milvus/versions/2023-10-12/api-docs.json
Database Management
# Create a database
client.create_database(db_name="my_database")
# List all databases
databases = client.list_databases()
# Returns: ["default", "my_database"]
# Switch to a database
client.using_database(db_name="my_database")
# Drop a database (must drop all collections first)
client.drop_database(db_name="my_database")
# Or connect to a specific database at init (use the user's actual URI and credentials)
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
client = MilvusClient(
uri="<USER_URI>",
token="<USER_TOKEN>",
db_name="my_database",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)Guidance
- Every Milvus instance has a
"default"database. - Before dropping a database, all collections in it must be dropped first.
Quick Start: Create Your First Milvus Instance from Scratch
This guide helps first-time users complete: prerequisite check → create first instance → verify running → get connection info → cleanup resources.
Prerequisites
1. CLI Environment
- Alibaba Cloud CLI installed, version
>= 3.3.3. - Credentials configured in the active CLI profile.
- API-calling CLI commands in this guide include the required
--user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manageflag.
2. Network Resources
Creating Milvus instance requires VPC and VSwitch. Before execution confirm RegionId with user (e.g., cn-hangzhou, cn-beijing, cn-shanghai, etc.):
# Check if available VPC exists
aliyun vpc describe-vpcs --biz-region-id <RegionId> --region <RegionId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage
# Check if VSwitch exists under VPC, record ZoneId
aliyun vpc describe-vswitches --biz-region-id <RegionId> --vpc-id vpc-xxx --region <RegionId> --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manageDon't have these resources? Please first create VPC and VSwitch via Alibaba Cloud console or CLI.
3. Confirm Availability Zone Info
Record the following info, will be used when creating instance:
- RegionId (e.g.,
cn-hangzhou) - ZoneId (e.g.,
cn-hangzhou-j, from VSwitch's availability zone) - VpcId, VSwitchId (can prepare two VSwitches in different availability zones for multi-AZ)
Step 1: Create Test Instance
Below creates a standalone version (standalone_pro) minimal instance, 4 CU, pay-as-you-go:
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "my-first-milvus",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"paymentType": "PayAsYouGo",
"ha": false,
"components": [{"type":"standalone_pro","replica":1,"cuNum":4,"cuType":"general"}],
"dbAdminPassword": "YourPassword@123",
"autoBackup": true,
"aiFunction": true,
"encrypted": false,
"isMultiAzStorage": false,
"multiZoneMode": "single"
}' \
--forceReturn contains instanceId (e.g., c-xxx), record it for subsequent operations.
Note: Creating instance incurs cost. Standalone 4 CU pay-as-you-go suitable for dev/test, don't use for production.
Step 2: Verify Instance Status
Instance creation is async operation, usually takes 5-15 minutes.
# View instance status
aliyun milvus get "/webapi/instance/get?RegionId=cn-hangzhou&instanceId=c-xxx" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --forceStatus Transition: creating → running
Wait until status becomes running to indicate instance ready.
Step 3: Get Connection Info
After instance ready, view connection address and component details:
aliyun milvus post "/webapi/cluster/detail" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--forceFocus on key fields in return:
Data.ClusterInfo.IntranetUrl: Intranet connection addressData.ClusterInfo.InternetUrl: Public network connection address (if enabled)Data.ClusterInfo.ProxyPort: Milvus service port (default 19530)Data.ClusterInfo.AttuPort: Attu visual management port
Connection example (pymilvus):
from pymilvus import connections
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
connections.connect(
host="c-xxx.milvus.aliyuncs.com",
port=19530,
user="root",
password="YourPassword@123",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)Step 4: View Instance List
# View all instances under current region
aliyun milvus get "/webapi/instance/list?RegionId=cn-hangzhou&pageSize=50" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --forceCleanup: Release Test Instance
🚫 Instance deletion is NOT available through this Skill. To release/delete a test instance, please go to the Alibaba Cloud Milvus Console. Release promptly after use to avoid ongoing billing.
Common Creation Failure Reasons
| Symptom | Possible Reason | Troubleshooting Method |
|---|---|---|
| Creation failed | VPC/VSwitch doesn't exist or not in same availability zone | Check if VPC and VSwitch exist and in specified availability zone |
| Creation failed | VSwitch available IP insufficient | Switch to a VSwitch with sufficient available IPs |
| Creation failed | Account balance insufficient | Recharge and retry |
| Creation failed | RAM permission insufficient | Confirm AccessKey has milvus:CreateInstance permission |
| Long time Creating | Backend resource scheduling | Wait 15-30 minutes, if timeout contact support |
Next Steps
- Need production-grade instance? → Refer to Instance Full Lifecycle production config template
- Detailed creation parameters? → Refer to Create Parameter Reference
- Daily operations? → Refer to Operations Manual
- API parameter query? → Refer to API Parameter Reference
Index Management — Detailed Reference
Create Index
index_params = client.prepare_index_params()
# Vector index
index_params.add_index(
field_name="embedding",
index_type="HNSW", # See index types table below
metric_type="COSINE", # "COSINE", "L2", "IP"
params={"M": 16, "efConstruction": 256}
)
# Optional: scalar index
index_params.add_index(
field_name="text",
index_type="" # Auto-select for scalars
)
client.create_index(
collection_name="my_collection",
index_params=index_params
)Common Index Types
| Index Type | For | Key Params | Notes |
|---|---|---|---|
AUTOINDEX | Dense vectors | Auto-tuned | Recommended for most cases |
FLAT | Dense vectors | None | Brute force, 100% recall |
IVF_FLAT | Dense vectors | nlist | Good balance |
IVF_SQ8 | Dense vectors | nlist | Compressed, less memory |
HNSW | Dense vectors | M, efConstruction | High recall, more memory |
DISKANN | Dense vectors | None | Disk-based, large datasets |
SPARSE_INVERTED_INDEX | Sparse vectors | drop_ratio_build | For sparse vectors |
SPARSE_WAND | Sparse vectors | drop_ratio_build | Faster sparse search |
Metric Types
| Metric | Description | Use With |
|---|---|---|
"COSINE" | Cosine similarity (larger = more similar) | Dense vectors |
"L2" | Euclidean distance (smaller = more similar) | Dense vectors |
"IP" | Inner product (larger = more similar) | Dense & Sparse vectors |
"BM25" | BM25 relevance scoring | Full-text search (sparse vectors from built-in tokenizer) |
Other Index Operations
# List indexes
indexes = client.list_indexes(collection_name="my_collection")
# Describe an index
info = client.describe_index(collection_name="my_collection", index_name="my_index")
# Drop an index
client.drop_index(collection_name="my_collection", index_name="my_index")Guidance
AUTOINDEXis recommended for most use cases.- An index is required before loading a collection.
- After creating an index, load the collection before searching.
- Sparse vectors only support
"IP"metric type. - For full-text search, use
"BM25"metric type withSPARSE_INVERTED_INDEXorSPARSE_WAND.
Instance Full Lifecycle: Planning → Creation → Query → Scaling → Release
Table of Contents
- 1. Planning Phase: Version selection, component planning, network preparation
- 2. Creation Phase: Dev/test / Production cluster / Custom component three templates
- 3. Query and Monitoring: Instance list, details, state machine
- 4. Scaling and Management: Scale up/down, rename
- 5. Release Instance: Console only (not available via Skill)
1. Planning Phase
Version Selection
| Instance Version | Applicable Scenario | Components | Recommended Config |
|---|---|---|---|
| Standalone (standalone_pro) | Dev/test, feature verification, small data | 1 | 4-8 CU (general) |
| Cluster (HA) | Production, large data, high concurrency | 5 | 30 CU minimum |
Not sure which to choose? Use standalone for dev/test (low cost), cluster for production (high availability).
Network Planning
Before creation confirm target RegionId and check network resources. Every `aliyun` CLI invocation must include `--user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage`:
# List available VPCs
aliyun vpc describe-vpcs --biz-region-id cn-hangzhou --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage
# List VSwitches under VPC (record ZoneId and available IP count)
aliyun vpc describe-vswitches --biz-region-id cn-hangzhou --vpc-id vpc-xxx --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage
# List security groups (for reference only, CreateInstance doesn't require passing security group)
aliyun ecs describe-security-groups --biz-region-id cn-hangzhou --vpc-id vpc-xxx --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manageRecommended Practice: List VPC → User selects → List VSwitches → User selects → Create instance.
For multi-AZ scenarios select one VSwitch in each of different availability zones to improve availability.
Payment Decision
| Payment Method | Applicable Scenario | Release Method |
|---|---|---|
| PayAsYouGo | Dev/test, short-term use | API direct release |
| Subscription | Production, long-term running | Need to request refund in console to release |
2. Creation Phase
Template 1: Dev/Test Instance (Standalone, Minimum Cost)
Standalone + pay-as-you-go + 4 CU, suitable for feature verification and dev debugging.
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "milvus-dev",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"paymentType": "PayAsYouGo",
"ha": false,
"components": [{"type":"standalone_pro","replica":1,"cuNum":4,"cuType":"general"}],
"dbAdminPassword": "YourPassword@123",
"autoBackup": true,
"aiFunction": true,
"encrypted": false,
"isMultiAzStorage": false,
"multiZoneMode": "single"
}' \
--forceTemplate 2: Production Instance (Cluster HA, 5 Components 36 CU)
Cluster + 5-component distributed deployment, suitable for production environment.
⚠️ Note: streaming, data, mix_coordinator, query minimum 4 CU.
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "milvus-prod",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"}],
"paymentType": "PayAsYouGo",
"ha": true,
"components": [
{"type":"streaming", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"data", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"proxy", "replica":2,"cuNum":2,"cuType":"general"},
{"type":"mix_coordinator", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"query", "replica":2,"cuNum":4,"cuType":"general","diskSizeType":"Normal"}
],
"dbAdminPassword": "YourPassword@123",
"autoBackup": true,
"aiFunction": true,
"encrypted": false,
"isMultiAzStorage": false,
"multiZoneMode": "single"
}' \
--forceTotal CU = 4×2 + 4×2 + 2×2 + 4×2 + 4×2 = 36 CU
Template 3: Custom Component (Query Large Spec, Multi-AZ)
Suitable for search-intensive scenarios, QueryNode uses cap type large memory, dual-AZ high-availability storage.
aliyun milvus post "/webapi/instance/create?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"regionId": "cn-hangzhou",
"zoneId": "cn-hangzhou-j",
"instanceName": "milvus-search-heavy",
"dbVersion": "2.6",
"vpcId": "vpc-xxx",
"vSwitchIds": [
{"vswId":"vsw-xxx","zoneId":"cn-hangzhou-j"},
{"vswId":"vsw-yyy","zoneId":"cn-hangzhou-b"}
],
"paymentType": "PayAsYouGo",
"ha": true,
"components": [
{"type":"streaming", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"data", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"proxy", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"mix_coordinator", "replica":2,"cuNum":4,"cuType":"general"},
{"type":"query", "replica":3,"cuNum":8,"cuType":"cap","diskSizeType":"Normal"}
],
"dbAdminPassword": "YourPassword@123",
"autoBackup": true,
"aiFunction": true,
"isMultiAzStorage": true,
"multiZoneMode": "Active-Active"
}' \
--forceFor complete parameter description refer to Create Parameter Reference
3. Query and Monitoring
Instance List
aliyun milvus get "/webapi/instance/list?RegionId=cn-hangzhou&pageSize=50" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --force⚠️ Important: total field may be inaccurate (returns 0 but actually has data), directly check instances array.
Instance Basic Info
aliyun milvus get "/webapi/instance/get?RegionId=cn-hangzhou&instanceId=c-xxx" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --forceFocus on return fields: instanceId, instanceName, status, dbVersion, ha, paymentType, createTime, vpcId, zoneId.
Instance Details (Component Specs, Connection Addresses, Storage Usage)
aliyun milvus post "/webapi/cluster/detail" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--forceFocus on return fields:
Data.ClusterInfo.IntranetUrl/InternetUrl: Connection addressesData.ClusterInfo.ProxyPort: Service port (19530)Data.ClusterInfo.TotalCuNum: Total CU countData.ClusterInfo.MilvusResourceInfoList: Each component spec detailsData.ClusterInfo.OssStorageSize: OSS storage usage
Instance State Machine
| State | Meaning | Follow-up Action |
|---|---|---|
creating | Creating | Wait, usually 5-15 minutes |
running | Instance ready | Can use normally |
updating | Scaling (scale up/down) | Wait to return to running |
modifying_config | Modifying config | Wait to return to running |
enable_public_network | Enabling public network | Wait to return to running |
deleting | Releasing | Wait |
deleted | Released | No action needed |
Note: In transitional state (updating / modifying_config / enable_public_network), cannot execute other write operations, need to wait instance returns to running before operating.
4. Scaling and Management
Scale Up/Down (UpdateInstance)
Adjust component CU count or replica count via UpdateInstance. Before scaling use GetInstanceDetail to confirm current specs.
# 1. View current component specs
aliyun milvus post "/webapi/cluster/detail" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force
# 2. Scale up query component to 3 replicas × 8 CU
aliyun milvus put "/webapi/instance/update?RegionId=cn-hangzhou" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--body '{
"instanceId": "c-xxx",
"components": [
{"type":"query","replica":3,"cuNum":8,"cuType":"cap","diskSizeType":"Normal"}
]
}' \
--forceScaling Notes:
- Only need to pass components to modify, unpassed components remain unchanged
- ⚠️ streaming/data/mix_coordinator/query minimum 4 CU, proxy minimum 2 CU
- During scaling instance status briefly becomes non-running, wait to recover before operating
- Before scaling down confirm current load can handle fewer resources
Modify Instance Name
aliyun milvus post "/webapi/cluster/update_name" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--ClusterName new-name \
--force5. Release Instance
🚫 Instance deletion is NOT available through this Skill. To release/delete a Milvus instance, please use the Alibaba Cloud Milvus Console.
Before Releasing (Checklist)
1. Data Backup: Confirm important data backed up (Collection data, indexes) 2. Confirm Dependencies: No other services depend on this instance's connection address 3. Instance Status: Confirm instance status is running
Related Documentation
- Quick Start — Simplified process for first-time instance creation
- Create Parameter Reference — Complete creation parameter description
- Operations Manual — Configuration, network management and troubleshooting
- API Parameter Reference — Complete API documentation
Daily Operations: Inspection, Configuration, Network, Troubleshooting
Table of Contents
- 1. Instance Inspection: Quick inspection checklist
- 2. Configuration Management: View config, modify config
- 3. Network Management: Public network access, whitelist
- 4. Resource Group Management: Resource transfer
- 5. Troubleshooting: Creation failure, instance abnormality, operation rejected
User-Agent requirement: EveryaliyunCLI invocation must include--user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage.
1. Instance Inspection
Quick Inspection Checklist
# 1. View all instance status (focus on non-running status)
aliyun milvus get "/webapi/instance/list?RegionId=cn-hangzhou&pageSize=50" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --force
# 2. View specific instance details (component specs, connection addresses, storage usage)
aliyun milvus post "/webapi/cluster/detail" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force
# 3. Confirm connection address available (extract IntranetUrl)
aliyun milvus post "/webapi/cluster/detail" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force | jq '.Data.ClusterInfo.IntranetUrl'Inspection Focus Points
| Check Item | Normal Standard | Action When Abnormal |
|---|---|---|
| Instance status | running | Non-running needs troubleshooting |
| Connection address | IntranetUrl not empty | Empty means instance may not be ready |
| Component CU | Matches expected config | Mismatch may mean scaling incomplete |
| Storage usage | No abnormal growth | Continuous growth needs data cleanup attention |
2. Configuration Management
View Instance Config
aliyun milvus post "/webapi/config/describe_milvus_user_config" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--forceReturns Data field as YAML format user custom config.
Modify Instance Config
⚠️ Note: Config changes may affect service stability, before modifying must confirm current config and understand change impact.
# 1. First view current config
aliyun milvus post "/webapi/config/describe_milvus_user_config" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force
# 2. Modify config (need to fill change reason)
aliyun milvus post "/webapi/config/modify_milvus_config" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--Reason "Adjust proxy max task count" \
--UserConfig "proxy:
maxTaskNum: 1024
" \
--force3. Network Management
Public Network Access
# View public network access status and whitelist
aliyun milvus post "/webapi/milvus/describe_access_control_list" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force
# Enable public network access and set whitelist
aliyun milvus post "/webapi/network/updatePublicNetworkStatus" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--ComponentType Proxy \
--PublicNetworkEnabled true \
--Cidr "10.0.0.0/8" \
--force
# ⚠️ Disable public network access (confirm no external services depend on public network address before operation)
aliyun milvus post "/webapi/network/updatePublicNetworkStatus" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--ComponentType Proxy \
--PublicNetworkEnabled false \
--forceWhitelist Management
# View current whitelist
aliyun milvus post "/webapi/milvus/describe_access_control_list" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force
# Update whitelist (AclId required, first obtain via DescribeAccessControlList)
aliyun milvus post "/webapi/milvus/update_access_control_list" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--AclId acl-xxx \
--Cidr "192.168.1.0/24" \
--force4. Resource Group Management
# Transfer instance to target resource group
aliyun milvus post "/webapi/resourceGroup/change" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--NewResourceGroupId rg-xxx \
--ResourceId c-xxx \
--force5. Troubleshooting
Instance Creation Failure
# View instance status and error info
aliyun milvus get "/webapi/instance/get?RegionId=cn-hangzhou&instanceId=c-xxx" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --force| Common Reason | Troubleshooting Method |
|---|---|
| VPC/VSwitch doesn't exist or not in same availability zone | aliyun vpc describe-vswitches --biz-region-id cn-hangzhou --vpc-id vpc-xxx --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage to confirm |
| VSwitch available IP exhausted | Check AvailableIpAddressCount field |
| RAM permission insufficient | Confirm AccessKey has milvus:CreateInstance permission |
| Account balance insufficient | Recharge and retry |
| Kernel version not supported | Confirm dbVersion is 2.3/2.4/2.5/2.6 |
| Component config invalid (pricing plan price result not found) | HA mode must have 5 components, streaming/data/mix_coordinator/query minimum 4 CU |
| Region not supported | Confirm RegionId is in supported list |
| InternalError (general server error) | 1. Confirm account has enabled Milvus service (check console access) 2. Confirm account balance sufficient and not overdue 3. Record RequestId and submit ticket for investigation |
Instance Cannot Connect
# 1. Confirm instance status
aliyun milvus get "/webapi/instance/get?RegionId=cn-hangzhou&instanceId=c-xxx" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou --force
# 2. Get connection address
aliyun milvus post "/webapi/cluster/detail" --user-agent AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage \
--RegionId cn-hangzhou \
--InstanceId c-xxx \
--force| Common Reason | Troubleshooting Method |
|---|---|
| Instance status not running | Wait for instance ready |
| Network unreachable | Confirm client and instance in same VPC, or public network access enabled |
| Password error | Confirm using dbAdminPassword set during creation |
| Port incorrect | Use ProxyPort (default 19530) |
| Public network not enabled | Enable public network access via UpdatePublicNetworkStatus |
| Whitelist not allowing | Check whitelist config via DescribeAccessControlList |
| Security group rule not allowing | Confirm VPC security group allows port 19530 |
Operation Rejected
| Error | Reason | Solution |
|---|---|---|
| OperationDenied | Instance status doesn't allow current operation | Wait for instance to become running then retry |
| OperationDenied.Subscription | Annual/monthly instance limitation | Need to operate in console |
| Forbidden.RAM | RAM permission insufficient | Contact admin for authorization |
API Rate Limiting
| Error | Description | Solution |
|---|---|---|
| Throttling | Request rate exceeded | Wait 5-10 seconds then retry, max 3 times |
Related Documentation
- Instance Full Lifecycle — Create, scale and release instances
- Quick Start — First time creating instance
- Create Parameter Reference — Complete creation parameters
- API Parameter Reference — Complete API documentation
Partition Management
# Create a partition
client.create_partition(collection_name="my_collection", partition_name="partition_A")
# List partitions
partitions = client.list_partitions(collection_name="my_collection")
# Returns: ["_default", "partition_A"]
# Check if partition exists
exists = client.has_partition(collection_name="my_collection", partition_name="partition_A")
# Load specific partitions
client.load_partitions(collection_name="my_collection", partition_names=["partition_A"])
# Release specific partitions
client.release_partitions(collection_name="my_collection", partition_names=["partition_A"])
# Drop a partition
client.drop_partition(collection_name="my_collection", partition_name="partition_A")Guidance
- Every collection has a
_defaultpartition. - Use
is_partition_key=Trueon a field to enable automatic partitioning by field value. - A partition must be loaded before search.
- Before dropping a partition, confirm with the user — all data in it will be deleted.
Common Patterns
Note: All patterns below use<USER_URI>and<USER_TOKEN>as connection placeholders. Always ask the user for their actual connection details before writing code. Every PyMilvus connection must includegrpc_options={"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"}. For local development, use Milvus Lite (uri="./milvus.db") only if the user explicitly requests it.
RAG Pipeline Pattern
from pymilvus import MilvusClient, DataType, model
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
# 1. Connect (ask user for URI and credentials)
client = MilvusClient(
uri="<USER_URI>",
token="<USER_TOKEN>",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)
# 2. Set up embedding model
embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
# 3. Create collection (dim must match embedding model output)
schema = client.create_schema(auto_id=True, enable_dynamic_field=True)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("text", DataType.VARCHAR, max_length=2048)
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=384) # all-MiniLM-L6-v2 outputs 384-dim
schema.add_field("source", DataType.VARCHAR, max_length=256)
index_params = client.prepare_index_params()
index_params.add_index(field_name="embedding", index_type="AUTOINDEX", metric_type="COSINE")
client.create_collection(collection_name="knowledge_base", schema=schema, index_params=index_params)
# 4. Insert documents — generate real vectors from text chunks
chunks = ["Milvus is a vector database...", "RAG combines retrieval and generation..."]
vectors = embedding_fn.encode_documents(chunks)
client.insert("knowledge_base", data=[
{"text": chunk, "embedding": vec, "source": "doc1.pdf"}
for chunk, vec in zip(chunks, vectors)
])
# 5. Retrieve relevant context — use the SAME embedding model
query = "What is a vector database?"
query_vectors = embedding_fn.encode_queries([query])
results = client.search(
collection_name="knowledge_base",
data=query_vectors,
limit=5,
output_fields=["text", "source"],
search_params={"metric_type": "COSINE"}
)Quick Semantic Search Pattern
from pymilvus import MilvusClient, model
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
# Simplest possible setup (Milvus Lite for local dev)
client = MilvusClient(uri="./search.db", grpc_options=PYMILVUS_GRPC_OPTIONS)
embedding_fn = model.dense.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
# Prepare data — vectors come from embedding model
texts = ["first document", "second document", "third document"]
vectors = embedding_fn.encode_documents(texts)
client.create_collection(collection_name="docs", dimension=384)
client.insert("docs", data=[
{"id": i, "vector": vec, "text": txt}
for i, (vec, txt) in enumerate(zip(vectors, texts))
])
# Search — encode query with the same model
query_vectors = embedding_fn.encode_queries(["search query"])
results = client.search("docs", data=query_vectors, limit=10, output_fields=["text"])Hybrid Search Pattern (Dense + Sparse)
from pymilvus import MilvusClient, DataType, AnnSearchRequest, RRFRanker
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
# Ask user for connection details
client = MilvusClient(
uri="<USER_URI>",
token="<USER_TOKEN>",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)
# Schema with both dense and sparse vectors
schema = client.create_schema(auto_id=True)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("text", DataType.VARCHAR, max_length=2048)
schema.add_field("dense_embedding", DataType.FLOAT_VECTOR, dim=768)
schema.add_field("sparse_embedding", DataType.SPARSE_FLOAT_VECTOR)
index_params = client.prepare_index_params()
index_params.add_index(field_name="dense_embedding", index_type="AUTOINDEX", metric_type="COSINE")
index_params.add_index(field_name="sparse_embedding", index_type="SPARSE_INVERTED_INDEX", metric_type="IP")
client.create_collection(collection_name="hybrid_col", schema=schema, index_params=index_params)
# Search with both vectors and fuse results
# dense_query_vector and sparse_query_vector come from your respective embedding models
req1 = AnnSearchRequest(data=[dense_query_vector], anns_field="dense_embedding",
param={"metric_type": "COSINE"}, limit=20)
req2 = AnnSearchRequest(data=[sparse_query_vector], anns_field="sparse_embedding",
param={"metric_type": "IP"}, limit=20)
results = client.hybrid_search(
collection_name="hybrid_col",
reqs=[req1, req2],
ranker=RRFRanker(k=60),
limit=10,
output_fields=["text"]
)Full-Text Search Pattern (BM25)
from pymilvus import MilvusClient, DataType, Function, FunctionType
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
# Ask user for connection details
client = MilvusClient(
uri="<USER_URI>",
token="<USER_TOKEN>",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)
schema = client.create_schema(auto_id=True)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("title", DataType.VARCHAR, max_length=512)
schema.add_field("body", DataType.VARCHAR, max_length=4096, enable_analyzer=True)
schema.add_field("body_sparse", DataType.SPARSE_FLOAT_VECTOR)
schema.add_function(Function(
name="body_bm25",
input_field_names=["body"],
output_field_names=["body_sparse"],
function_type=FunctionType.BM25,
))
index_params = client.prepare_index_params()
index_params.add_index(field_name="body_sparse", index_type="AUTOINDEX", metric_type="BM25")
client.create_collection(collection_name="articles", schema=schema, index_params=index_params)
# Insert — only provide text, sparse vector is auto-generated by BM25 function
client.insert("articles", data=[
{"title": "Intro to ML", "body": "Machine learning is a subset of artificial intelligence..."},
])
# Search with raw text — no embedding model needed, Milvus handles BM25 tokenization
results = client.search(
collection_name="articles",
data=["machine learning fundamentals"],
anns_field="body_sparse",
limit=10,
output_fields=["title", "body"]
)RAM Permission Statement
This Skill calls Alibaba Cloud Milvus and related product APIs, requires the following RAM permissions.
Required Permissions
{
"Version": "1",
"Statement": [
{
"Effect": "Allow",
"Action": [
"milvus:ListInstancesV2",
"milvus:GetInstance",
"milvus:GetInstanceDetail",
"milvus:CreateInstance",
"milvus:UpdateInstance",
"milvus:UpdateInstanceName",
"milvus:DescribeInstanceConfigs",
"milvus:ModifyInstanceConfig",
"milvus:UpdatePublicNetworkStatus",
"milvus:DescribeAccessControlList",
"milvus:UpdateAccessControlList",
"milvus:ChangeResourceGroup",
"milvus:CreateDefaultRole"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"vpc:DescribeVpcs",
"vpc:DescribeVSwitches"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ecs:DescribeSecurityGroups"
],
"Resource": "*"
}
]
}Permission Description
| Permission Action | Purpose |
|---|---|
milvus:ListInstancesV2 | Query instance list |
milvus:GetInstance | Query instance basic info |
milvus:GetInstanceDetail | Query instance details |
milvus:CreateInstance | Create instance |
milvus:UpdateInstance | Change instance config |
milvus:UpdateInstanceName | Modify instance name |
milvus:DescribeInstanceConfigs | Query instance config |
milvus:ModifyInstanceConfig | Modify instance config |
milvus:UpdatePublicNetworkStatus | Enable/disable public network access |
milvus:DescribeAccessControlList | Query access control list |
milvus:UpdateAccessControlList | Update access control list |
milvus:ChangeResourceGroup | Change resource group |
milvus:CreateDefaultRole | Create default role |
vpc:DescribeVpcs | Query VPC list (instance creation prerequisite check) |
vpc:DescribeVSwitches | Query VSwitch list (instance creation prerequisite check) |
ecs:DescribeSecurityGroups | Query security group list (instance creation prerequisite check) |
Minimum Permission Principle
If following minimum permission principle, can limit Resource field to specific instance ARN:
"Resource": [
"acs:milvus:<region>:<account-id>:instance/<instance-id>"
]User & Role Management (RBAC)
User Operations
# Create a user
client.create_user(user_name="analyst", password="SecureP@ss123")
# List users
users = client.list_users()
# Describe a user (shows assigned roles)
info = client.describe_user(user_name="analyst")
# Update password
client.update_password(user_name="analyst", old_password="SecureP@ss123", new_password="NewP@ss456")
# Grant role to user
client.grant_role(user_name="analyst", role_name="read_only")
# Revoke role from user
client.revoke_role(user_name="analyst", role_name="read_only")
# Drop a user
client.drop_user(user_name="analyst")Role Operations
# Create a role
client.create_role(role_name="read_only")
# List roles
roles = client.list_roles()
# Grant privilege (v2 API — recommended)
client.grant_privilege_v2(
role_name="read_only",
privilege="Search", # e.g., "Search", "Insert", "Query", "Delete"
collection_name="my_collection", # Use "*" for all collections
db_name="default" # Use "*" for all databases
)
# Built-in privilege groups
client.grant_privilege_v2(
role_name="admin_role",
privilege="ClusterAdmin", # See privilege groups below
collection_name="*",
db_name="*"
)
# Revoke privilege
client.revoke_privilege_v2(
role_name="read_only",
privilege="Search",
collection_name="my_collection",
db_name="default"
)
# Describe role (see granted privileges)
info = client.describe_role(role_name="read_only")
# Drop a role
client.drop_role(role_name="read_only")Built-in Privilege Groups
| Group | Scope |
|---|---|
ClusterAdmin | Full cluster access |
ClusterReadOnly | Read-only cluster access |
ClusterReadWrite | Read-write cluster access |
DatabaseAdmin | Full database access |
DatabaseReadOnly | Read-only database access |
DatabaseReadWrite | Read-write database access |
CollectionAdmin | Full collection access |
CollectionReadOnly | Read-only collection access |
CollectionReadWrite | Read-write collection access |
Common Individual Privileges
Search, Query, Insert, Delete, Upsert, CreateIndex, DropIndex, CreateCollection, DropCollection, Load, Release, CreatePartition, DropPartition
Guidance
- Recommended workflow: create role -> grant privileges -> create user -> assign role.
- Use
"*"for collection_name/db_name to grant on all resources. - Before dropping a user or role, confirm with the user.
Vector Operations — Detailed Reference
Target collection must exist and be loaded.
Never use fake or placeholder vectors (e.g.,[0.1, 0.2, ...]). Use Milvus built-in embedding functions (aliyun_milvusprovider, requires ≥ 2.6) to automatically generate vectors — no manual embedding code needed.
Embedding Function Setup (aliyun_milvus)
Requires Milvus ≥ 2.6. Embedding functions are not available on earlier versions.
Milvus can automatically generate vector embeddings from scalar fields via Function. You do not need to install any external embedding library or call any embedding API manually.
from pymilvus import MilvusClient, DataType, Function, FunctionType
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
client = MilvusClient(
uri="http://<endpoint>:19530",
token="root:password",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)
schema = client.create_schema()
# Define fields
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=True)
schema.add_field("text", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
# Text embedding function — vectors are auto-generated on insert and search
text_embedding_function = Function(
name="text_embedding_func",
function_type=FunctionType.TEXTEMBEDDING,
input_field_names=["text"],
output_field_names=["dense"],
params={
"provider": "aliyun_milvus",
"model_name": "text-embedding-v4"
}
)
schema.add_function(text_embedding_function)
# Create index and collection
index_params = client.prepare_index_params()
index_params.add_index(field_name="dense", index_type="AUTOINDEX", metric_type="COSINE")
client.create_collection(
collection_name="my_collection",
schema=schema,
index_params=index_params
)Multimodal Embedding Function (Vision-Language)
For multimodal content (text + images), use qwen3-vl-embedding. Add a nullable mm_value field to carry image/video/audio references:
schema.add_field("mm_value", DataType.VARCHAR, max_length=9000, nullable=True)
schema.add_field("dense_mm", DataType.FLOAT_VECTOR, dim=1024)
mm_embedding_function = Function(
name="mm_embedding_func",
function_type=FunctionType.TEXTEMBEDDING,
input_field_names=["mm_value"],
output_field_names=["dense_mm"],
params={
"provider": "aliyun_milvus",
"model_name": "qwen3-vl-embedding",
"dim": "1024"
}
)
schema.add_function(mm_embedding_function)
index_params.add_index(field_name="dense_mm", index_type="AUTOINDEX", metric_type="COSINE")Multimodal Content Handling
Different media types require different handling strategies for the mm_value field:
| Media Type | Size | Strategy | mm_value Example |
|---|---|---|---|
| Small image (< 60KB) | Small | Convert to base64 data URI | "data:image/jpeg;base64,/9j/4AAQ..." |
| Large image (≥ 60KB) | Large | Upload to OSS (public read), pass URL | "https://your-bucket.oss-cn-hangzhou.aliyuncs.com/img/large.jpg" |
| Video | Any | Upload to OSS (public read), pass URL | "https://your-bucket.oss-cn-hangzhou.aliyuncs.com/video/demo.mp4" |
| Audio | Any | Upload to OSS (public read), pass URL | "https://your-bucket.oss-cn-hangzhou.aliyuncs.com/audio/speech.wav" |
import base64
# Small image → base64
with open("small_photo.jpg", "rb") as f:
base64_str = base64.b64encode(f.read()).decode("utf-8")
mm_value_base64 = f"data:image/jpeg;base64,{base64_str}"
# Large video/audio/image → upload to OSS first, then use public-read URL
mm_value_oss_url = "https://your-bucket.oss-cn-hangzhou.aliyuncs.com/video/demo.mp4"Supported Providers and Models
| Provider | Model Name | Description |
|---|---|---|
aliyun_milvus | text-embedding-v4 | Alibaba Cloud text embedding |
aliyun_milvus | text-embedding-v3 | Alibaba Cloud text embedding |
aliyun_milvus | text-embedding-v2 | Alibaba Cloud text embedding |
aliyun_milvus | qwen3-vl-embedding | Alibaba Cloud multimodal (vision-language) embedding, requires dim param |
Insert
When a collection has embedding functions, only provide the scalar input fields — vector fields are auto-generated by the function:
# Text-only insert (vector auto-generated from "text" field)
data = [
{"id": 1, "text": "AI advances in 2024"},
{"id": 2, "text": "ML basics for beginners"},
]
res = client.insert(collection_name="my_collection", data=data)
# Returns: {"insert_count": 2, "ids": [1, 2]}
# With multimodal content (mm_value is nullable)
import base64
# Small image → base64 data URI
with open("small_photo.jpg", "rb") as f:
base64_str = base64.b64encode(f.read()).decode("utf-8")
mm_base64 = f"data:image/jpeg;base64,{base64_str}"
data_with_mm = [
{"id": 3, "text": "A cat sitting on a sofa", "mm_value": mm_base64},
{"id": 4, "text": "Product demo video", "mm_value": "https://your-bucket.oss-cn-hangzhou.aliyuncs.com/video/demo.mp4"},
{"id": 5, "text": "Pure text document"}, # mm_value is nullable, can be omitted
]
res = client.insert(collection_name="my_collection", data=data_with_mm)Upsert (insert or update if PK exists)
data = [
{"id": 1, "text": "Updated: AI advances in 2025"},
{"id": 2, "text": "Updated: Deep learning fundamentals"},
]
res = client.upsert(collection_name="my_collection", data=data)
# Returns: {"upsert_count": 2}Search (vector similarity)
When a collection has embedding functions, search data accepts raw text or URLs instead of vectors:
# Text search — pass raw text, embedding is auto-generated
results = client.search(
collection_name="my_collection",
data=["What is artificial intelligence?"], # Raw text, not vector
anns_field="dense", # Vector field name (output of text embedding function)
limit=10, # Top-K
output_fields=["text", "id"], # Fields to return
filter='age > 20 and status == "active"', # Optional scalar filter
)
# Returns: List[List[dict]]
# Each hit: {"id": 1, "distance": 0.95, "entity": {"text": "AI advances..."}}
# Multimodal search — pass image URL or base64
results = client.search(
collection_name="my_collection",
data=["https://example.com/image.jpeg"], # Image URL, not vector
anns_field="dense_mm", # Vector field name (output of multimodal embedding function)
limit=5,
output_fields=["text", "mm_value"],
)Hybrid Search (multi-vector with reranking)
When using embedding functions, pass raw text/URLs as data in AnnSearchRequest:
from pymilvus import AnnSearchRequest, RRFRanker, WeightedRanker
# Dense text search — raw text, auto-embedded by aliyun_milvus function
req1 = AnnSearchRequest(
data=["What is artificial intelligence?"], # Raw text
anns_field="dense",
param={"metric_type": "COSINE"},
limit=10
)
# Sparse BM25 full-text search — raw text, auto-tokenized by BM25 function
req2 = AnnSearchRequest(
data=["artificial intelligence"], # Raw text for BM25
anns_field="sparse",
param={"metric_type": "BM25"},
limit=10
)
# RRF reranking
results = client.hybrid_search(
collection_name="my_collection",
reqs=[req1, req2],
ranker=RRFRanker(k=60),
limit=10,
output_fields=["text"]
)
# Or weighted reranking
results = client.hybrid_search(
collection_name="my_collection",
reqs=[req1, req2],
ranker=WeightedRanker(0.7, 0.3),
limit=10
)Full-Text Search
Full-text search uses Milvus's built-in BM25 tokenizer to convert text into sparse vectors automatically.
Setup: Collection with Full-Text Search
from pymilvus import MilvusClient, DataType, Function, FunctionType
PYMILVUS_GRPC_OPTIONS = {
"grpc.primary_user_agent": "AlibabaCloud-Agent-Skills/alibabacloud-milvus-manage"
}
client = MilvusClient(
uri="<USER_URI>",
token="<USER_TOKEN>",
grpc_options=PYMILVUS_GRPC_OPTIONS,
)
schema = client.create_schema()
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=True)
schema.add_field("text", DataType.VARCHAR, max_length=1000, enable_analyzer=True)
schema.add_field("sparse", DataType.SPARSE_FLOAT_VECTOR)
# Define BM25 function to auto-convert text -> sparse vector
bm25_function = Function(
name="text_bm25",
input_field_names=["text"],
output_field_names=["sparse"],
function_type=FunctionType.BM25,
)
schema.add_function(bm25_function)
index_params = client.prepare_index_params()
index_params.add_index(field_name="sparse", index_type="AUTOINDEX", metric_type="BM25")
client.create_collection(collection_name="full_text_col", schema=schema, index_params=index_params)Search with Text
results = client.search(
collection_name="full_text_col",
data=["machine learning algorithms"], # Raw text query
anns_field="sparse",
limit=10,
output_fields=["text"]
)Search Iterator (paginated search over large results)
iterator = client.search_iterator(
collection_name="my_collection",
data=["search query text"], # Raw text, auto-embedded by aliyun_milvus function
anns_field="dense",
batch_size=100,
limit=10000,
output_fields=["text"],
search_params={"metric_type": "COSINE"}
)
results = []
while True:
batch = iterator.next()
if not batch:
break
results.extend(batch)
iterator.close()Query Iterator (paginated filter-based retrieval)
iterator = client.query_iterator(
collection_name="my_collection",
filter='age > 20',
output_fields=["text", "age"],
batch_size=100,
limit=10000
)
results = []
while True:
batch = iterator.next()
if not batch:
break
results.extend(batch)
iterator.close()Query (filter-based retrieval)
results = client.query(
collection_name="my_collection",
filter='id in [1, 2, 3]',
output_fields=["text"],
limit=100
)Get (by primary key)
results = client.get(
collection_name="my_collection",
ids=[1, 2, 3],
output_fields=["text"]
)Delete
# By primary keys
client.delete(collection_name="my_collection", ids=[1, 2, 3])
# By filter expression
client.delete(collection_name="my_collection", filter='status == "obsolete"')Filter Expression Syntax
| Expression | Example |
|---|---|
| Comparison | age > 20 |
| Equality | status == "active" |
| IN list | id in [1, 2, 3] |
| AND/OR | age > 20 and status == "active" |
| String match | text like "hello%" |
| Array contains | ARRAY_CONTAINS(tags, "ml") |
| JSON field | json_field["key"] > 100 |
| Match all | id > 0 |
Guidance
- Never use fake or placeholder vectors. Use
aliyun_milvusembedding functions to auto-generate vectors from text or multimodal content. - When a collection has embedding functions, pass raw text or URLs as
datain search — do not manually compute vectors. - For text embedding, use
provider: "aliyun_milvus"withmodel_name: "text-embedding-v4". - For multimodal embedding (images/video/audio), use
model_name: "qwen3-vl-embedding"withdimparam. - Small images (< 60KB): convert to base64 data URI (
data:image/jpeg;base64,...) and pass directly in themm_valuefield. - Large images / video / audio: upload to OSS with public-read access, then pass the URL in the
mm_valuefield. - For full-text search, pass raw text strings as
data— Milvus handles tokenization via BM25. - For large inserts, batch data into chunks (e.g., 1000 rows per batch).
- Always specify
output_fieldsto control which fields are returned. - For large result sets, use
search_iteratororquery_iteratorinstead of increasinglimit. - Always call
iterator.close()when done to release server resources.