
Byted Escloud
- 2 installs
- 408 repo stars
- Updated August 3, 2026
- volcengine/agentkit-samples
Manages Volcengine ESCloud and CloudSearch clusters for lifecycle operations plus Elasticsearch/OpenSearch indexing, querying, and aggregation.
About
Manages Volcengine ESCloud and CloudSearch clusters, covering control-plane lifecycle actions and Elasticsearch/OpenSearch data-plane workflows. A developer uses it to create, scale, and inspect clusters and to index, query, and aggregate data with guardrails on destructive operations.
- Splits control-plane lifecycle and data-plane indexing/query workflows via bundled CLIs
- Requires explicit confirmation before deletes, reindex, or bulk mutations
Byted Escloud by the numbers
- 2 all-time installs (skills.sh)
- Ranked #916 of 1,039 Cloud & Infrastructure skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/volcengine/agentkit-samples --skill byted-escloudAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 2 |
|---|---|
| repo stars | ★ 408 |
| Last updated | August 3, 2026 |
| Repository | volcengine/agentkit-samples ↗ |
What it does
Manages Volcengine ESCloud and CloudSearch clusters for lifecycle operations plus Elasticsearch/OpenSearch indexing, querying, and aggregation.
Files
Volcano Engine ESCloud
Use when
- The user needs to manage ESCloud or CloudSearch on Volcano Engine / Volcengine.
- The task is a control-plane lifecycle action such as create, inspect, scale, expose, restart, or delete.
- The task is an Elasticsearch/OpenSearch data-plane workflow such as cluster validation, inspection, indexing, querying, aggregation, or bulk guidance.
Route
- Control-plane lifecycle workflows ->
scripts/control.pyand CONTROL_PLANE.md - Control-plane fallback tools ->
scripts/control_tools.pyand CONTROL_TOOLS.md - Data-plane workflows -> DATA_PLANE.md
Guardrails
- Always run the bundled CLIs with
{baseDir}/venv/bin/python. - Before any data-plane read or write, run
data.py --endpoint <endpoint> info; if it fails, stop and fix endpoint exposure, allowlist, credentials, or TLS first. - Use
data.py smoke_testfor new-cluster validation; if the test index already exists, pass--reuse-existingor choose a different--index; delete its test index only with--cleanup --confirm. - For non-routine data-plane work, prefer ad-hoc
curlor small Python snippets grounded in the data-plane references instead of expandingdata.py. - Require explicit confirmation before destructive or high-impact operations such as deletes, alias cutovers, bulk mutations,
_delete_by_query,_update_by_query, or_reindex.
Control Plane - Goal-Based Cluster Management
Primary lifecycle guide for ESCloud operations via control.py; shared guardrails from SKILL.md apply here.
Command prefix
{baseDir}/venv/bin/python {baseDir}/scripts/control.py <command>Output contract
control.py returns JSON with status, goal, data, and steps_completed.
Operator rules:
status: success-> continue to validation or next step.status: error-> stop immediately and report the error.status: timeout-> do not blindly rerun the same mutation; inspect current instance state first.
Global execution rules
- Use instance ID (
--id) for all mutating operations. - Do not mutate instances in transitional states such as
Creating,Updating,Scaling, orReleasing. - For transitional states, wait 30-60 seconds and re-check status.
- Scale one node type at a time and wait for
Runningbefore the next scale. - Enable or disable public access only when explicitly requested.
Timeout recovery
When status == timeout: 1. Check steps_completed. 2. If the main mutation step was already sent, do not resend it immediately. 3. Inspect with control.py detail --id <instance-id>. 4. If the instance is still transitional, wait 30-60 seconds and check again. 5. Continue only after the instance returns to a stable state, typically Running.
Workflows
List instances
- When: user asks to list ESCloud instances.
- Run:
control.py list [--page-number <N>] [--page-size <N>]- Validate: ensure the output includes the expected instances and pagination fields.
Provision / create instance
- When: user asks to create an ESCloud instance.
- Preconditions: collect required inputs such as VPC, subnet, version, node specs, storage, and admin password.
- Discover options:
control.py provision-info- Run:
control.py provision \
--name <instance-name> \
--vpc-id <vpc-id> \
--subnet-id <subnet-id> \
--admin-password <password> \
--version <version> \
--hot-spec <spec> \
--hot-storage-spec <storage-spec> \
--hot-storage-size <GiB> \
[--charge-type PostPaid|PrePaid] \
[--master-spec <spec>] [--master-count <N>] [--master-storage-spec <storage-spec>] [--master-storage-size <GiB>] \
[--kibana-spec <spec>] [--kibana-count <N>] \
[--poll-interval <seconds>] [--timeout <seconds>]- Validate: confirm the command succeeds and the instance reaches
Running. - Next: optionally configure public access or IP allowlist.
Inspect status / detail
- When: user asks for instance status, details, or endpoint information.
- Run:
control.py detail --id <instance-id>- Validate: surface normalized status, endpoints, and any transitional state.
- Next: if the instance is transitional, wait before any further mutation.
Scale instance
- When: user asks to change node count, node spec, or storage for one node type.
- Preconditions: instance must be
Running. - Run:
control.py scale --id <instance-id> \
--node-type <Master|Hot|Kibana|...> \
--spec-name <spec> \
--count <N> \
[--storage-spec-name <storage-spec>] [--storage-size <GiB>] \
[--poll-interval <seconds>] [--timeout <seconds>]- Validate: wait until the instance returns to
Running. - Next: if additional node types must change, repeat sequentially.
Manage public access
- When: user asks to enable or disable internet access, or expose the endpoint.
- Preconditions: instance should be
Running. - Run:
control.py public-access --id <instance-id> --enable true|false \
[--eip-id <eip-id>] [--eip-bandwidth <Mbps>] [--eip-billing-type <type>] [--eip-isp <isp>] [--eip-auto-reuse true|false] \
[--poll-interval <seconds>] [--timeout <seconds>]- Validate: confirm endpoint/EIP state in status output.
- Next: if enabling public access, configure public IP allowlist before data-plane operations.
- Prefer reusing an existing available EIP before allocating a new one.
- After public endpoint changes, update allowlists before data-plane work.
Manage IP allowlist
- When: user asks to permit client IPs for private or public access.
- Preconditions: identify the correct allowlist type for the target endpoint.
- Run:
control.py allowlist --id <instance-id> \
--ips "<ip1>,<ip2>" \
[--group-name <name>] [--type PRIVATE_ES|PUBLIC_ES] \
[--poll-interval <seconds>] [--timeout <seconds>]- Validate: confirm the instance returns to
Runningand the allowlist reflects the requested IP set. - Next: retry connectivity checks if this was done to unblock data-plane access.
Reset admin password
- When: user asks to rotate or reset the admin password.
- Preconditions: instance should be
Running. - Run:
control.py reset-password --id <instance-id> --admin-password <password> \
[--poll-interval <seconds>] [--timeout <seconds>]- Validate: wait until the instance returns to
Running. - Next: if the password was needed for data-plane access, retry endpoint connectivity.
Maintenance window
- When: user asks to set maintenance days or time ranges.
- Run:
control.py maintenance --id <instance-id> --day "Mon,Wed" --time "02:00-06:00"- Validate: confirm the new maintenance policy in follow-up status/detail output.
Rename instance
- When: user asks to rename the instance.
- Run:
control.py rename --id <instance-id> --name <new-name>- Validate: confirm the updated name from status/detail output.
Restart node
- When: user asks to restart a specific node.
- Preconditions: identify the exact target node.
- Run:
control.py restart-node --id <instance-id> --node-id <node-id> [--force] \
[--poll-interval <seconds>] [--timeout <seconds>]- Validate: wait until the instance returns to
Running.
Delete / deprovision instance
- When: user asks to delete an instance.
- Preconditions: show the exact target name and ID; require explicit user confirmation before executing.
- Run:
control.py deprovision --id <instance-id> --confirm <instance-id> [--force] \
[--poll-interval <seconds>] [--timeout <seconds>]- Validate: stop on error; if deletion protection blocks the action, rerun with the supported force option.
- Next: none.
Control Tools - Low-Level ESCloud API Operations
Fallback control-plane reference for operations not covered by goal-based control.py; shared guardrails from SKILL.md apply here.
Command prefix
{baseDir}/venv/bin/python {baseDir}/scripts/control_tools.py <command>Output contract
Commands return JSON success payloads under status/data, or error payloads under error/details.
Fallback-only rules
- Prefer
control.pyfor normal lifecycle workflows. - Use this file for granular, API-shaped, or uncovered operations.
- For destructive operations, resolve and show the exact target first, require explicit user confirmation, then pass the required confirmation flag.
Common intents
| Intent | Command(s) |
|---|---|
| List instances (fallback) | list |
| Inspect one instance | detail --id <id> |
| Discover VPCs/subnets/zones/specs | vpc, subnet --vpc-id <id>, zones, node_specs, versions |
| Create an instance | create ... |
| Scale one node type | scale ... |
| Manage allowlists | ip_allowlist_get, ip_allowlist_set |
| Manage public network/EIP | public_network, eip_list, eip_allocate, eip_release |
| Reset password | reset_password |
| Rename / maintenance / restart | rename, maintenance_set, restart_node |
| Delete instance | delete --id <id> --confirm |
Discovery commands
list
control_tools.py list [--page-number <n>] [--page-size <n>]detail
control_tools.py detail --id <instance-id>vpc
control_tools.py vpcsubnet
control_tools.py subnet --vpc-id <vpc-id>zones
control_tools.py zonesnode_specs
control_tools.py node_specsversions
Best-effort list derived from node specs; prefer this over hard-coded version guesses.
control_tools.py versionsInstance mutation commands
create
control_tools.py create \
--name <name> \
--version <value-from-versions> \
--vpc-id <vpc-id> \
--subnet-id <subnet-id> \
--admin-password <password> \
--master-spec <resourceSpecName> \
[--master-count 3] \
[--master-storage-spec <storageSpecName>] \
[--master-storage-size <GiB>] \
--hot-spec <resourceSpecName> \
--hot-storage-spec <storageSpecName> \
--hot-storage-size <GiB> \
[--hot-count 2] \
[--kibana-spec <resourceSpecName>] [--kibana-count 1] \
[--charge-type PostPaid|PrePaid] \
[--https true|false] \
[--deletion-protection true|false] \
[--pure-master true|false]Notes:
- Use
vpc,subnet,zones,node_specs, andversionsfirst when values are missing. - Present valid options to the user instead of guessing IDs or spec names.
scale
control_tools.py scale \
--id <instance-id> \
--node-type <Master|Hot|Warm|Cold|Coordinator|Kibana|Other> \
--spec-name <resourceSpecName> \
--count <n> \
[--storage-spec-name <storageSpecName>] \
[--storage-size <GiB>]Notes:
- Run only when the instance is
Running. - Scale one node type at a time.
delete
control_tools.py delete --id <instance-id> --confirmIf deletion is blocked by deletion protection:
control_tools.py deletion_protection_set --id <instance-id> --enabled falseNetwork and EIP commands
eip_list
control_tools.py eip_list [--status Available|Attached|...]eip_allocate
control_tools.py eip_allocate \
[--bandwidth <Mbps>] \
[--billing-type PostPaidByTraffic|PostPaidByBandwidth|PrePaid] \
[--name <name>] \
[--isp BGP]eip_release
control_tools.py eip_release --allocation-id <id>public_network
control_tools.py public_network --id <id> --enable true|false [--eip-id <id>]ip_allowlist_get
control_tools.py ip_allowlist_get --id <instance-id>ip_allowlist_set
control_tools.py ip_allowlist_set --id <instance-id> --group-name <name> --ips '["1.2.3.4/32","5.6.7.0/24"]' [--type PRIVATE_ES|PUBLIC_ES]Note:
- Use
--type PUBLIC_ESfor public endpoint allowlists.
Other low-level commands
reset_password
control_tools.py reset_password --id <instance-id> --admin-password <new-password>nodes
control_tools.py nodes --id <instance-id>plugins
control_tools.py plugins --id <instance-id>rename
control_tools.py rename --id <instance-id> --name <new-name>maintenance_set
control_tools.py maintenance_set --id <instance-id> --day "Mon,Wed" --time "02:00-06:00"deletion_protection_set
control_tools.py deletion_protection_set --id <instance-id> --enabled true|falserestart_node
control_tools.py restart_node --id <instance-id> --node-id <node-id>Data Plane - Quickstart, Inspection, and Reference-Driven Workflows
Use scripts/data.py for endpoint validation, smoke-testing a new cluster, and safe inspection. Use references/*.md plus ad-hoc curl or Python snippets for richer Elasticsearch/OpenSearch workflows.
Command prefix
{baseDir}/venv/bin/python {baseDir}/scripts/data.py --endpoint <endpoint> <command>Decision rule
- Use
data.pyfor routine connectivity validation, new-cluster smoke tests, and common inspection. - Use
references/*.mdplus ad-hoc snippets for task-specific search, aggregation, alias, bulk, and migration work.
Reference routing
- Not sure where to start, or the task spans multiple steps -> references/patterns.md
- Connectivity, auth, TLS, privilege, or endpoint reachability -> references/connectivity.md
- Cluster health, nodes, shards, or safe diagnostics -> references/diagnostics.md
- Indices, mappings, settings, or aliases -> references/index.md
- Single-document create/get/update/delete patterns -> references/documents.md
- Search, filters, highlighting, or pagination -> references/search.md
- Aggregations, grouped counts, trends, or percentiles -> references/aggregation.md
- Bulk ingest, broad writes, or by-query/reindex-like operations -> references/write.md
Connection and auth
Auth options:
- Basic auth:
--username <user> --password <pass> - API key:
--api-key <value> - Bearer token:
--bearer-token <value>
Use exactly one auth mode per invocation.
TLS options:
--ca-certs <path>for a custom CA bundle--insecureto disable certificate verification
Environment variable defaults:
ESCLOUD_ENDPOINTESCLOUD_USERNAMEESCLOUD_PASSWORDESCLOUD_CA_CERTSESCLOUD_INSECUREESCLOUD_API_KEYESCLOUD_BEARER_TOKEN
Connectivity preflight
Before any data-plane read or write, run:
data.py --endpoint <https://domain:9200> infoIf info fails, stop and verify endpoint reachability, allowlist/public exposure, credentials, and TLS settings.
Commands
info
data.py --endpoint <endpoint> infosmoke_test
Creates a disposable test index, indexes one sample document, reads it back, and runs a simple search.
data.py --endpoint <endpoint> smoke_testOptional flags:
--index <name>(default:escloud-smoke-test)--doc-id <id>--reuse-existing--cleanup --confirm
If the test index already exists, smoke_test refuses to proceed unless --reuse-existing is passed.
index_exists
data.py --endpoint <endpoint> index_exists --index <name>index_list
data.py --endpoint <endpoint> index_listindex_get
Returns settings, mappings, and aliases for an index.
data.py --endpoint <endpoint> index_get --index <name>cluster_health
data.py --endpoint <endpoint> cluster_healthcat_nodes
data.py --endpoint <endpoint> cat_nodescat_shards
data.py --endpoint <endpoint> cat_shardsAgent guidance
- Do not add fixed
data.pysubcommands for one-off data-plane tasks by default. - Prefer the smallest useful
curlcommand or Python snippet for the user's actual schema, filters, and auth mode. - Require explicit confirmation before destructive or high-impact requests.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
Volcano Engine ESCloud / CloudSearch Skill
byted-escloud helps an agent operate Volcano Engine (Volcengine) ESCloud / CloudSearch clusters and work with their Elasticsearch/OpenSearch data plane.
What this skill can do
Control plane
Use this skill to manage cluster lifecycle tasks such as:
- create a cluster
- inspect cluster details and status
- scale or reconfigure a cluster
- enable public access and manage related networking settings
- restart, rename, or delete a cluster safely
Data plane
Use this skill to help with Elasticsearch/OpenSearch work such as:
- validate connectivity, auth, and TLS
- smoke-test a new cluster
- inspect cluster health, nodes, shards, and index metadata
- create or inspect mappings, settings, aliases, documents, queries, aggregations, and bulk patterns
- generate task-specific
curlcommands or short Python snippets for one-off workflows
How to use this skill
In OpenClaw
Install this repo under your OpenClaw skills directory, for example:
.openclaw/skills/byted-escloud/
Then enable it in .openclaw/openclaw.json:
{
"skills": {
"entries": {
"byted-escloud": {
"enabled": true,
"env": {
"VOLCENGINE_ACCESS_KEY": "...",
"VOLCENGINE_SECRET_KEY": "...",
"VOLCENGINE_REGION": "cn-beijing"
}
}
}
}
}Required and optional configuration
Control-plane operations use:
VOLCENGINE_ACCESS_KEYVOLCENGINE_SECRET_KEY- optional:
VOLCENGINE_REGION(defaults tocn-beijing)
Data-plane helper commands can use:
ESCLOUD_ENDPOINTESCLOUD_USERNAME,ESCLOUD_PASSWORDESCLOUD_API_KEYESCLOUD_BEARER_TOKENESCLOUD_CA_CERTSESCLOUD_INSECURE
Local helper CLI usage
This repo includes helper CLIs under scripts/. Create a virtualenv and install dependencies:
python3 -m venv venv
./venv/bin/pip install -r requirements.txtCommon examples:
# Control plane
./venv/bin/python ./scripts/control.py list
./venv/bin/python ./scripts/control.py detail --id <instance-id>
# Data-plane validation and inspection
./venv/bin/python ./scripts/data.py --endpoint <https://domain:9200> info
./venv/bin/python ./scripts/data.py --endpoint <https://domain:9200> cluster_health
./venv/bin/python ./scripts/data.py --endpoint <https://domain:9200> index_list
# Smoke test a new cluster
./venv/bin/python ./scripts/data.py --endpoint <https://domain:9200> smoke_test
./venv/bin/python ./scripts/data.py --endpoint <https://domain:9200> smoke_test --cleanup --confirmHow the data-plane part works
- Use
scripts/data.pyfor routine validation, smoke tests, and safe inspection. - Use the docs in `references/` for richer Elasticsearch/OpenSearch workflows.
- For variant needs, let the agent generate ad-hoc
curlor Python snippets instead of forcing everything into a fixed CLI.
A good entry point for richer workflows is references/patterns.md.
Example prompts
- "List my Volcengine ESCloud instances."
- "Show details for ESCloud instance
<id>." - "Create an ESCloud cluster named
search-prodin VPC<vpc>subnet<subnet>with 3 masters and 2 hot nodes." - "Open public access for this cluster and add my IP to the allowlist."
- "Quickly test whether
<endpoint>is usable, then show cluster health and index metadata." - "Build a query for timeout logs from the last 24 hours."
- "Write a short Python script to bulk ingest these NDJSON documents."
Important safety rules
- Before any data-plane read or write, run
data.py --endpoint <endpoint> infofirst. - Use
data.py smoke_testwhen validating a new cluster; if the test index already exists, pass--reuse-existingor choose a different--index. - Cleanup for the smoke test is destructive and requires
--cleanup --confirm. - Destructive or high-impact operations should always be explicitly confirmed.
More documentation
- SKILL.md - skill routing and guardrails
- CONTROL_PLANE.md - lifecycle workflows
- CONTROL_TOOLS.md - fine-grained control-plane tools
- DATA_PLANE.md - helper CLI and data-plane operating model
- references/patterns.md - detailed ES/OpenSearch workflow references
Aggregation Reference
Use this reference when the user wants grouped counts, trends, percentiles, or rollups instead of raw documents.
When to use
- counts by service, host, tenant, or status
- hourly or daily trends
- percentiles for latency or size metrics
- nested breakdowns such as service -> error type
- summary answers that are cheaper than exporting raw documents
Core guidance
- Prefer
size: 0when only aggregations matter. - Use
.keywordfields for terms aggregations when available. - Add time filters for logs and metrics.
- Keep bucket sizes bounded.
- Prefer summaries over raw exports for large datasets.
- These examples use aggregation APIs shared by Elasticsearch and OpenSearch.
Terms aggregation
Method / Endpoint:
POST /logs-*/_search
Content-Type: application/jsonBody:
{
"size": 0,
"aggs": {
"services": {
"terms": {
"field": "service.keyword",
"size": 10
}
}
}
}Time-series aggregation
{
"size": 0,
"query": {
"bool": {
"filter": [
{ "term": { "level.keyword": "ERROR" } },
{ "range": { "@timestamp": { "gte": "now-24h" } } }
]
}
},
"aggs": {
"errors_over_time": {
"date_histogram": {
"field": "@timestamp",
"fixed_interval": "1h"
}
}
}
}Nested breakdown
{
"size": 0,
"query": {
"bool": {
"filter": [
{ "term": { "level.keyword": "ERROR" } },
{ "range": { "@timestamp": { "gte": "now-24h" } } }
]
}
},
"aggs": {
"by_service": {
"terms": {
"field": "service.keyword"
},
"aggs": {
"by_error_type": {
"terms": {
"field": "error_type.keyword"
}
}
}
}
}
}Percentiles
{
"size": 0,
"aggs": {
"endpoints": {
"terms": {
"field": "endpoint.keyword",
"size": 20
},
"aggs": {
"latency_percentiles": {
"percentiles": {
"field": "response_time_ms",
"percents": [50, 95, 99]
}
}
}
}
}
}Expected result shape
{
"aggregations": {
"by_service": {
"buckets": [
{ "key": "api", "doc_count": 120 },
{ "key": "worker", "doc_count": 73 }
]
}
}
}Connectivity Reference
Use this reference for remote connection safety, authentication, TLS, privileges, and connectivity preflight.
Default posture
- Prefer HTTPS for remote clusters.
- Prefer least-privilege credentials.
- Use explicit auth rather than assuming anonymous access.
- Treat production clusters as sensitive by default.
- Separate safe reads from routine writes and destructive writes when discussing permissions.
Supported configuration
Base URL
ES_URL=https://search.example.com
# or
OPENSEARCH_URL=https://search.example.comAuth
# Basic auth
ES_USERNAME=app_user
ES_PASSWORD=secret
# or API key auth
ES_API_KEY=base64-or-raw-api-keyOptional CA certificate
ES_CA_CERT=/absolute/path/to/ca.crtConnectivity preflight
Use a lightweight request before broader reads or writes.
Method / Endpoint:
GET /Confirm:
- the base URL is reachable
- TLS succeeds with the expected certificate chain
- credentials are accepted
- the response reveals engine/version details that may affect later guidance
Privilege guidance
_searchandGET /<index>/_doc/<id>usually need index read permissions._mapping,_settings,_alias, and_cat/*may need broader monitor or admin-style visibility.- document writes usually need write permissions.
- index create/delete and alias changes usually need index admin permissions.
- security plugins and managed-service policies can change exact privilege names and failure modes.
Common auth failures
401 Unauthorized: invalid credentials, rejected API key, or missing auth header.403 Forbidden: valid credentials but insufficient privileges._searchsuccess with_mappingfailure often means the role is too narrow.- read operations succeeding does not imply delete or alias permissions are available.
TLS guidance
- use
ES_CA_CERTwhen the endpoint is signed by a private CA - certificate errors often mean missing CA trust, hostname mismatch, or proxy interception
- avoid telling users to disable TLS verification unless they explicitly accept the risk
Diagnostics Reference
Use this reference for non-destructive cluster diagnostics.
When to use
- the user asks for cluster health
- searches are timing out and you need high-level status
- you need a safe first look before deeper troubleshooting
- you want to identify engine/version before planning data-plane requests
Safety and compatibility guidance
- Keep this reference non-destructive.
- Privilege required: cluster visibility endpoints may need monitor or admin permissions depending on the deployment.
- Engine/version discovery, health, and cat APIs are useful preflight checks before deeper troubleshooting.
Root endpoint for preflight
Method / Endpoint:
GET /Typical result shape:
{
"name": "search-node-1",
"cluster_name": "production-search",
"version": {
"number": "2.11.0"
},
"tagline": "The OpenSearch Project: https://opensearch.org/"
}Cluster health
GET /_cluster/healthTypical result shape:
{
"cluster_name": "production-search",
"status": "green",
"number_of_nodes": 6,
"active_primary_shards": 128
}Cat nodes
GET /_cat/nodes?v&format=jsonCat shards
GET /_cat/shards?v&format=jsonInterpretation guidance
greenmeans primaries and replicas are allocated.yellowusually means unassigned replicas.redmeans one or more primary shards are unavailable.- the root endpoint often reveals engine flavor and version, which can explain compatibility or auth differences.
Safety guidance
- Use health and cat APIs before suggesting invasive remediation.
- Avoid cluster setting changes unless the user explicitly asks for them.
Documents Reference
Canonical reference for single-document create, get, update, upsert, and delete workflows.
When to use
- the user wants to insert one document
- the user wants to fetch a document by ID
- the user wants to update part of a document
- the user wants upsert behavior when a document may not exist yet
- the user wants to delete one document safely
Task-specific notes
- Use
_docpaths for cross-engine compatibility. - Routine writes are fine when the target index and document scope are explicit.
- Deletes are low-freedom operations: confirm the exact target before proceeding.
- Use
refresh=trueonly when immediate visibility is required.
Index a document with an explicit ID
Method / Endpoint:
PUT /products/_doc/sku-1001
Content-Type: application/jsonBody:
{
"sku": "sku-1001",
"name": "Mechanical Keyboard",
"price": 99.0,
"created_at": "2026-04-15T10:30:15Z"
}Typical result shape:
{
"_index": "products",
"_id": "sku-1001",
"result": "created"
}Create a document with an auto-generated ID
Method / Endpoint:
POST /products/_doc
Content-Type: application/jsonBody:
{
"sku": "sku-1002",
"name": "Trackpad",
"price": 129.0,
"created_at": "2026-04-15T10:35:00Z"
}Typical result shape:
{
"_index": "products",
"_id": "Q9f...generated-id",
"result": "created"
}Get a document by ID
Method / Endpoint:
GET /products/_doc/sku-1001Typical result shape:
{
"_index": "products",
"_id": "sku-1001",
"found": true,
"_source": {
"sku": "sku-1001",
"name": "Mechanical Keyboard",
"price": 99.0
}
}Update part of a document
Method / Endpoint:
POST /products/_update/sku-1001
Content-Type: application/jsonBody:
{
"doc": {
"price": 89.0,
"updated_at": "2026-04-15T11:00:00Z"
}
}Typical result shape:
{
"_index": "products",
"_id": "sku-1001",
"result": "updated"
}Upsert when the document may not exist
Method / Endpoint:
POST /products/_update/sku-1003
Content-Type: application/jsonBody:
{
"doc": {
"name": "USB-C Dock",
"price": 149.0,
"updated_at": "2026-04-15T11:05:00Z"
},
"doc_as_upsert": true
}Typical result shape:
{
"_index": "products",
"_id": "sku-1003",
"result": "created"
}Write with immediate refresh
Method / Endpoint:
PUT /products/_doc/sku-1004?refresh=true
Content-Type: application/jsonDelete a document
Destructive operation.
Method / Endpoint:
DELETE /products/_doc/sku-1001Typical result shape:
{
"_index": "products",
"_id": "sku-1001",
"result": "deleted"
}Index Reference
Use this reference for index CRUD, mappings, settings, aliases, and compatibility-safe cutovers.
When to use
- the user wants to list or inspect indices
- the user needs to create an index with mappings or settings
- the user needs field types before writing a query
- the user needs shard or replica settings
- the user wants to delete an index or switch an alias safely
Safety and compatibility guidance
- These examples use shared Elasticsearch/OpenSearch REST APIs.
- Privilege required: index inspection usually needs index read or monitor permissions; create/update/delete usually needs index admin permissions.
- Prefer aliases for cutovers or logical renames; direct rename flows are not universal.
- Require explicit confirmation before deleting an index.
List indices
Method / Endpoint:
GET /_cat/indices?v&format=jsonTypical result shape:
[
{
"health": "green",
"status": "open",
"index": "products-v1",
"docs.count": "124556",
"store.size": "1.2gb"
}
]Check whether an index exists
Method / Endpoint:
HEAD /productsExpected result:
200 OKif the index exists404 Not Foundif the index does not exist
Create an index
Method / Endpoint:
PUT /products
Content-Type: application/jsonBody:
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1
},
"mappings": {
"properties": {
"sku": { "type": "keyword" },
"name": { "type": "text" },
"price": { "type": "float" },
"created_at": { "type": "date" }
}
}
}Typical result shape:
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "products"
}Get index details
Get mapping
Method / Endpoint:
GET /products/_mappingTypical result shape:
{
"products": {
"mappings": {
"properties": {
"sku": { "type": "keyword" },
"name": { "type": "text" },
"price": { "type": "float" },
"created_at": { "type": "date" }
}
}
}
}Get settings
Method / Endpoint:
GET /products/_settingsGet aliases
Method / Endpoint:
GET /_aliasUpdate index settings
Use this for safe/common dynamic settings such as replica count. Do not imply that every setting is dynamically updateable.
Method / Endpoint:
PUT /products/_settings
Content-Type: application/jsonBody:
{
"index": {
"number_of_replicas": 2
}
}Typical result shape:
{
"acknowledged": true
}Safe alias cutover
Use aliases when the user wants a logical name to point to a new backing index.
Method / Endpoint:
POST /_aliases
Content-Type: application/jsonBody:
{
"actions": [
{ "remove": { "index": "products-v1", "alias": "products" } },
{ "add": { "index": "products-v2", "alias": "products" } }
]
}Typical result shape:
{
"acknowledged": true
}Delete an index
Destructive operation. Require explicit confirmation before suggesting or executing it.
Method / Endpoint:
DELETE /productsTypical result shape:
{
"acknowledged": true
}Interpretation guidance
textfields are analyzed and suited to full-text search.keywordfields are suited to exact filtering and aggregations.- Mappings explain why
termvsmatchbehaves differently. - Settings help identify shard counts, replicas, and custom analyzers.
Patterns Reference
Use this reference as a workflow index for common Elasticsearch/OpenSearch tasks. For exact request bodies, open the referenced canonical file.
1. Start with connectivity and engine preflight
Use when the cluster is new, credentials may be wrong, or compatibility is unknown.
1. Check GET /. 2. Confirm reachability, auth success, TLS success, and engine/version. 3. If that fails, use connectivity.md first.
Primary reference: connectivity.md Secondary reference: diagnostics.md
2. Create or inspect an index
Use when the user wants to create an index, inspect mappings, update safe settings, or manage aliases.
1. If the index may already exist, check existence first. 2. For new indices, use the canonical create-index body from index.md. 3. For cutovers or logical renames, use aliases instead of rename-style assumptions. 4. Require explicit confirmation before delete.
Primary reference: index.md
3. Read or write one known document
Use when the user knows the target index and document ID, or wants a single-document create/update/delete.
1. Use _doc lookup when the ID is known. 2. Use explicit-ID index or partial update for routine writes. 3. Use upsert only when create-if-missing is intended. 4. Require explicit confirmation before delete.
Primary reference: documents.md
4. Search for matching documents
Use when the user wants matching documents rather than one known ID.
1. Inspect mappings first if field behavior is unclear. 2. Use match/multi_match for analyzed text and term/terms for exact values. 3. Add bounded filters, explicit sort, and limited _source. 4. Use search_after instead of deep paging when the user needs the next page.
Primary reference: search.md Supporting references: index.md, connectivity.md
5. Return summaries instead of raw documents
Use when the user wants grouped counts, trends, percentiles, or rollups.
1. Prefer size: 0. 2. Add time filters when the data is time-based. 3. Use .keyword fields for exact grouping when available. 4. Keep bucket sizes bounded.
Primary reference: aggregation.md
6. Bulk ingest or broad write operations
Use when the user needs NDJSON bulk ingestion or asks for by-query/reindex-like writes.
1. Keep target indices explicit. 2. Use the canonical NDJSON structure from write.md. 3. Inspect per-item bulk results instead of assuming success. 4. For by-query or reindex-like changes, preview scope first and require explicit confirmation.
Primary reference: write.md Supporting references: documents.md
7. Diagnose cluster-level issues safely
Use when searches time out, shard health is suspicious, or broader diagnostics are needed.
1. Start with GET / and GET /_cluster/health. 2. Use cat APIs only for visibility, not remediation. 3. Keep the workflow non-destructive unless the user explicitly asks for deeper operational changes.
Primary reference: diagnostics.md Supporting reference: connectivity.md
Search Reference
Canonical reference for search, filtering, highlighting, and deterministic pagination.
When to use
- the user wants matching documents
- the user needs recent logs, traces, or events
- the user needs field-level filters or sorted results
- the user wants matched snippets or a focused sample set
Task-specific notes
- Adapt field names, filters, and
_sourceto the user's schema. - Use
match,match_phrase, ormulti_matchfor analyzed text fields. - Use
termortermsfor exact values, usually on.keywordfields. - Keep pagination deterministic with explicit sorts.
- If the user already knows the ID, route to
documents.mdinstead of search.
Basic text search
Method / Endpoint:
POST /logs-*/_search
Content-Type: application/jsonBody:
{
"size": 10,
"query": {
"match": {
"message": "error"
}
},
"_source": ["@timestamp", "level", "service", "message"]
}Filtered query
{
"size": 20,
"sort": [
{ "@timestamp": "desc" },
{ "_id": "asc" }
],
"query": {
"bool": {
"filter": [
{ "term": { "level.keyword": "ERROR" } },
{ "term": { "service.keyword": "api" } },
{ "range": { "@timestamp": { "gte": "now-24h" } } }
]
}
},
"_source": ["@timestamp", "service", "message", "trace_id"]
}Highlighting
{
"size": 10,
"query": {
"match_phrase": {
"message": "connection timeout"
}
},
"highlight": {
"fields": {
"message": {}
}
}
}Multi-field search
{
"size": 10,
"query": {
"multi_match": {
"query": "timeout error",
"fields": ["message", "error_details", "stack_trace"],
"type": "best_fields"
}
}
}Deterministic pagination with search_after
First page body:
{
"size": 20,
"sort": [
{ "@timestamp": "desc" },
{ "_id": "asc" }
],
"query": {
"range": {
"@timestamp": { "gte": "now-24h" }
}
}
}Next page body:
{
"size": 20,
"sort": [
{ "@timestamp": "desc" },
{ "_id": "asc" }
],
"search_after": ["2026-04-15T10:30:15Z", "A1b2C3"],
"query": {
"range": {
"@timestamp": { "gte": "now-24h" }
}
}
}Expected result shape
{
"hits": {
"total": { "value": 145 },
"hits": [
{
"_index": "logs-2026.04.15",
"_id": "A1b2C3",
"sort": ["2026-04-15T10:30:15Z", "A1b2C3"],
"_source": {
"@timestamp": "2026-04-15T10:30:15Z",
"level": "ERROR",
"service": "api",
"message": "Connection timeout"
}
}
]
}
}Write Reference
Canonical reference for bulk ingest and other broad-impact write workflows.
When to use
- the user wants to ingest many documents efficiently
- the user wants NDJSON bulk examples
- the user wants guidance on write safety or refresh behavior
- the user asks about by-query or reindex-like write workflows
Task-specific notes
- Keep target indices explicit.
- Bulk, by-query, and reindex-like operations are low-freedom workflows: follow the documented sequence.
- Require explicit confirmation for broad or destructive mutations.
Bulk ingest with NDJSON
Method / Endpoint:
POST /_bulk
Content-Type: application/x-ndjsonBody:
{ "index": { "_index": "products", "_id": "sku-1001" } }
{ "sku": "sku-1001", "name": "Mechanical Keyboard", "price": 99.0 }
{ "index": { "_index": "products", "_id": "sku-1002" } }
{ "sku": "sku-1002", "name": "Trackpad", "price": 129.0 }Typical result shape:
{
"errors": false,
"items": [
{
"index": {
"_index": "products",
"_id": "sku-1001",
"result": "created",
"status": 201
}
}
]
}Bulk update example
Method / Endpoint:
POST /_bulk
Content-Type: application/x-ndjsonBody:
{ "update": { "_index": "products", "_id": "sku-1001" } }
{ "doc": { "price": 89.0 } }
{ "update": { "_index": "products", "_id": "sku-1003" } }
{ "doc": { "price": 149.0 }, "doc_as_upsert": true }Refresh guidance
- default write behavior is usually enough for normal ingestion
- use
refresh=trueonly when the user explicitly needs immediate search visibility - avoid forcing refresh on large bulk loads unless the user accepts the throughput tradeoff
High-impact write categories
Require explicit confirmation before suggesting or executing:
DELETE /<index>DELETE /<index>/_doc/<id>when the user did not clearly identify the exact documentPOST /<index>/_delete_by_queryPOST /<index>/_update_by_queryPOST /_reindex
By-query mutation guidance
1. restate the target index pattern and filter scope 2. preview the scope with a normal search first 3. warn that broad filters can touch many documents 4. require explicit confirmation before the mutation request
Error handling guidance
errors: truein bulk responses means at least one item failed even if the request returned200 OK- inspect per-item
statusanderrorfields instead of assuming all writes succeeded - partial failures are common in bulk flows; summarize successes and failures separately
volcengine-python-sdk==5.0.19
opensearch-py>=2.0.0
requests==2.32.5
# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from abc import ABC, abstractmethod
from typing import Any
class ApiError(Exception):
pass
class EsCloudApi(ABC):
@abstractmethod
def _call(self, method: str, action: str, body: Any) -> Any:
pass
def describe_instances(self, body: Any) -> Any:
return self._call("POST", "DescribeInstances", body)
def describe_instance(self, body: Any) -> Any:
return self._call("POST", "DescribeInstance", body)
def describe_zones(self, body: Any) -> Any:
return self._call("POST", "DescribeZones", body)
def describe_node_available_specs(self, body: Any) -> Any:
return self._call("POST", "DescribeNodeAvailableSpecs", body)
def create_instance_in_one_step(self, body: Any) -> Any:
return self._call("POST", "CreateInstanceInOneStep", body)
def modify_node_spec_in_one_step(self, body: Any) -> Any:
return self._call("POST", "ModifyNodeSpecInOneStep", body)
def release_instance(self, body: Any) -> Any:
return self._call("POST", "ReleaseInstance", body)
def describe_ip_allow_list(self, body: Any) -> Any:
return self._call("POST", "DescribeIpAllowList", body)
def modify_ip_allow_list_v2(self, body: Any) -> Any:
return self._call("POST", "ModifyIpAllowListV2", body)
def reset_admin_password(self, body: Any) -> Any:
return self._call("POST", "ResetAdminPassword", body)
def describe_instance_nodes(self, body: Any) -> Any:
return self._call("POST", "DescribeInstanceNodes", body)
def describe_instance_plugins(self, body: Any) -> Any:
return self._call("POST", "DescribeInstancePlugins", body)
def rename_instance(self, body: Any) -> Any:
return self._call("POST", "RenameInstance", body)
def modify_maintenance_setting(self, body: Any) -> Any:
return self._call("POST", "ModifyMaintenanceSetting", body)
def modify_deletion_protection(self, body: Any) -> Any:
return self._call("POST", "ModifyDeletionProtection", body)
def restart_node(self, body: Any) -> Any:
return self._call("POST", "RestartNode", body)
def create_public_address(self, body: Any) -> Any:
return self._call("POST", "CreatePublicAddress", body)
def release_public_address(self, body: Any) -> Any:
return self._call("POST", "ReleasePublicAddress", body)
class VpcApi(ABC):
@abstractmethod
def _call(self, method: str, action: str, body: Any) -> Any:
pass
def describe_vpcs(self, body: Any) -> Any:
return self._call("GET", "DescribeVpcs", body)
def describe_subnets(self, body: Any) -> Any:
return self._call("GET", "DescribeSubnets", body)
def describe_eip_addresses(self, body: Any) -> Any:
return self._call("GET", "DescribeEipAddresses", body)
def allocate_eip_address(self, body: Any) -> Any:
return self._call("GET", "AllocateEipAddress", body)
def release_eip_address(self, body: Any) -> Any:
return self._call("GET", "ReleaseEipAddress", body)
# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import os
import sys
import urllib.parse
import urllib.request
from typing import Any, Dict
from api import ApiError, EsCloudApi, VpcApi
class _SimpleResponse:
def __init__(self, payload: Dict[str, Any]):
self._payload = payload
def to_dict(self) -> Dict[str, Any]:
return self._payload
api_host = os.environ.get("ARK_SKILL_API_BASE")
api_key = os.environ.get("ARK_SKILL_API_KEY")
def check_is_ark_env() -> bool:
return bool(api_host and api_key)
def _do_http_call(
service_name: str,
action: str,
version: str,
method: str,
body_dict: Dict[str, Any] | None = None,
) -> Dict[str, Any]:
if not check_is_ark_env():
raise ApiError(
"ARK_SKILL_API_BASE and ARK_SKILL_API_KEY must be set for ark_shim"
)
url = f"{api_host.rstrip('/')}/?Action={action}&Version={version}"
headers = {
"ServiceName": service_name,
"Authorization": f"Bearer {api_key}",
"Accept": "application/json",
}
if method.upper() == "GET":
if body_dict:
query = urllib.parse.urlencode(body_dict, doseq=True)
url = f"{url}&{query}"
data = None
else:
headers["Content-Type"] = "application/json"
data = json.dumps(body_dict or {}).encode("utf-8")
req = urllib.request.Request(url, data=data, headers=headers, method=method.upper())
try:
with urllib.request.urlopen(req) as resp:
return json.loads(resp.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
error_body = exc.read().decode("utf-8")
raise ApiError(f"HTTP {exc.code}: {exc.reason} - {error_body}")
except Exception as exc:
raise ApiError(f"Network error: {str(exc)}")
class ESCloudHttpShim(EsCloudApi):
def _call(self, method: str, action: str, body: Any) -> Any:
payload = body.to_dict() if hasattr(body, "to_dict") else (body or {})
return _SimpleResponse(
_do_http_call("ESCloud", action, "2023-01-01", method, payload)
)
class VpcHttpShim(VpcApi):
def _call(self, method: str, action: str, body: Any) -> Any:
payload = body.to_dict() if hasattr(body, "to_dict") else (body or {})
return _SimpleResponse(
_do_http_call("vpc", action, "2020-04-01", method, payload)
)
def get_clients():
try:
region = os.environ.get("VOLCENGINE_REGION", "cn-beijing")
return ESCloudHttpShim(), VpcHttpShim(), None, region
except ApiError as exc:
print(
json.dumps(
{"error": "Initialization Error", "details": str(exc)},
ensure_ascii=False,
)
)
sys.exit(1)
# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import json
import sys
from typing import Any, Dict, List, Optional, Set, Tuple
from api import ApiError, EsCloudApi, VpcApi
def print_result(data: Any) -> None:
print(json.dumps({"status": "success", "data": data}, ensure_ascii=False))
def print_error(msg: str, details: Optional[str] = None) -> None:
err: Dict[str, Any] = {"error": msg}
if details:
err["details"] = details
print(json.dumps(err, ensure_ascii=False))
sys.exit(1)
def api_call(fn):
try:
response = fn()
try:
result = response.to_dict()
except AttributeError:
result = str(response)
print_result(result)
except ApiError as e:
msg = str(e)
instr = ""
if "TaskIsRunning" in msg:
instr = "An operation is already in progress. Wait a few minutes and retry, or check status via 'detail'."
elif any(
k in msg
for k in ["BadRequest", "InvalidParameter", "NotFound", "Unauthorized"]
):
instr = "Verify IDs/specs and permissions. Use 'vpc', 'subnet', and 'node_specs' to fetch valid options."
details = f"{msg}\n\nInstruction: {instr}" if instr else msg
print_error("API Error", details)
except Exception as e:
print_error(
"Unexpected Error",
f"{str(e)}\n\nInstruction: Check network connectivity and credentials.",
)
def str_to_bool(v: str) -> bool:
if v.lower() in ("true", "1", "yes", "y"):
return True
if v.lower() in ("false", "0", "no", "n"):
return False
raise argparse.ArgumentTypeError(f"Boolean value expected, got '{v}'")
def _get_clients() -> Tuple[EsCloudApi, VpcApi, Any, str]:
import ark_shim
if ark_shim.check_is_ark_env():
return ark_shim.get_clients()
import sdk_shim
return sdk_shim.get_clients()
def get_zone_id_by_subnet(vpc_api: VpcApi, subnet_id: str) -> str:
resp = vpc_api.describe_subnets({"SubnetIds": [subnet_id]})
resp_dict = (
resp.to_dict()
if hasattr(resp, "to_dict")
else (resp if isinstance(resp, dict) else {})
)
result = resp_dict.get("Result") or resp_dict.get("result") or resp_dict
subnets = result.get("Subnets") or result.get("subnets") or []
if not subnets:
print_error(
"Subnet Not Found",
f"Subnet ID '{subnet_id}' does not exist or is not in the current region. Instruction: Run 'vpc' and 'subnet --vpc-id <ID>' to list valid options.",
)
first = subnets[0] if isinstance(subnets, list) and subnets else {}
zone_id = first.get("ZoneId") or first.get("zone_id") or ""
if not zone_id:
print_error(
"Zone Not Found",
"Failed to derive zone_id from the subnet. Try a different subnet.",
)
return zone_id
def _resolve_eip_billing_type(val):
"""
EIP billing type constants.
"""
EIP_BILL_TYPES = {
"PostPaidByBandwidth": 1,
"PostPaidByTraffic": 2,
"PrePaid": 3,
}
return EIP_BILL_TYPES.get(val, val)
def cmd_list(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {"PageNumber": args.page_number, "PageSize": args.page_size}
api_call(lambda: es_api.describe_instances(body))
def cmd_detail(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {"InstanceId": args.id}
api_call(lambda: es_api.describe_instance(body))
def cmd_zones(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {}
api_call(lambda: es_api.describe_zones(body))
def cmd_node_specs(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
body = {}
api_call(lambda: es_api.describe_node_available_specs(body))
def cmd_vpc(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {}
api_call(lambda: vpc_api.describe_vpcs(body))
def cmd_subnet(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {"VpcId": args.vpc_id}
api_call(lambda: vpc_api.describe_subnets(body))
def cmd_eip_list(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {}
if args.status:
body["Status"] = args.status
api_call(lambda: vpc_api.describe_eip_addresses(body))
def cmd_eip_allocate(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
billing_type = (
_resolve_eip_billing_type(args.billing_type) or 2
) # Default to PostPaidByTraffic
body = {
"Bandwidth": args.bandwidth,
"BillingType": billing_type,
"Name": args.name,
"Description": args.description,
"ISP": args.isp,
}
api_call(lambda: vpc_api.allocate_eip_address(body))
def cmd_eip_release(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
body = {"AllocationId": args.allocation_id}
api_call(lambda: vpc_api.release_eip_address(body))
def cmd_create(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
zone_id = get_zone_id_by_subnet(vpc_api, args.subnet_id)
node_specs = []
if args.master_spec:
node_specs.append(
{
"Type": "Master",
"Number": args.master_count,
"ResourceSpecName": args.master_spec,
"StorageSpecName": args.master_storage_spec or args.hot_storage_spec,
"StorageSize": args.master_storage_size or 20,
}
)
node_specs.append(
{
"Type": "Hot",
"Number": args.hot_count,
"ResourceSpecName": args.hot_spec,
"StorageSpecName": args.hot_storage_spec,
"StorageSize": args.hot_storage_size,
}
)
if args.kibana_spec:
node_specs.append(
{
"Type": "Kibana",
"Number": args.kibana_count,
"ResourceSpecName": args.kibana_spec,
}
)
instance_conf = {
"InstanceName": args.name,
"Version": args.version,
"AdminPassword": args.admin_password,
"ChargeType": args.charge_type,
"DeletionProtection": args.deletion_protection,
"EnableHttps": args.https,
"EnablePureMaster": args.pure_master
if args.pure_master is not None
else (True if args.master_spec else False),
"ProjectName": "default",
"RegionId": region,
"ZoneId": zone_id,
"VPC": {"VpcId": args.vpc_id},
"Subnet": {"SubnetId": args.subnet_id},
"NodeSpecsAssigns": node_specs,
}
body = {"InstanceConfiguration": instance_conf}
api_call(lambda: es_api.create_instance_in_one_step(body))
def cmd_scale(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
assign_kwargs = {
"Type": args.node_type,
"Number": args.count,
"ResourceSpecName": args.spec_name,
}
if args.storage_spec_name:
assign_kwargs["StorageSpecName"] = args.storage_spec_name
if args.storage_size is not None:
assign_kwargs["StorageSize"] = args.storage_size
assign = dict(assign_kwargs)
body = {
"InstanceId": args.id,
"NodeSpecsAssigns": [assign],
}
api_call(lambda: es_api.modify_node_spec_in_one_step(body))
def cmd_delete(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
if not getattr(args, "confirm", False):
print_error(
"Confirmation Required",
"Refusing to delete without --confirm. Ask the user to explicitly confirm, then rerun with: control_tools.py delete --id <instance-id> --confirm",
)
body = {"InstanceId": args.id}
api_call(lambda: es_api.release_instance(body))
def cmd_ip_allowlist_get(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
body = {"InstanceId": args.id}
api_call(lambda: es_api.describe_ip_allow_list(body))
def parse_json_array(raw: str, name: str) -> list:
try:
data = json.loads(raw)
except json.JSONDecodeError as e:
print_error("Invalid JSON", f"Invalid JSON in {name}: {str(e)}")
if not isinstance(data, list):
print_error(
"Invalid JSON", f"{name} must be a JSON array, e.g. '[\"1.2.3.4/32\"]'"
)
return data
def extract_supported_versions_from_node_specs(
node_specs_dict: Dict[str, Any],
) -> List[str]:
versions: Set[str] = set()
def walk(obj: Any) -> None:
if isinstance(obj, dict):
for k, v in obj.items():
if isinstance(v, str) and k.lower() in (
"version",
"versionid",
"version_id",
):
versions.add(v)
walk(v)
elif isinstance(obj, list):
for item in obj:
walk(item)
walk(node_specs_dict)
def sort_key(s: str) -> Tuple[int, str]:
# Best-effort sorting: prefer known prefixes, then lexical.
if s.startswith("V"):
return (0, s)
if s.startswith("OPEN_SEARCH_"):
return (1, s)
return (2, s)
return sorted(versions, key=sort_key)
def cmd_versions(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
# ESCloud does not expose a dedicated "describe_versions" API in this SDK.
# Derive versions from DescribeNodeAvailableSpecs output (best-effort), so results stay current.
body = {}
def call():
return es_api.describe_node_available_specs(body)
try:
resp = call()
resp_dict = resp.to_dict() if hasattr(resp, "to_dict") else {}
versions = extract_supported_versions_from_node_specs(resp_dict)
if not versions:
print_result(
{
"versions": [],
"note": "No versions found in DescribeNodeAvailableSpecs response. Use 'node_specs' to inspect raw output or consult the console.",
}
)
return
print_result({"versions": versions})
except ApiError as e:
print_error("API Error", str(e))
except Exception as e:
print_error("Unexpected Error", str(e))
def cmd_ip_allowlist_set(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
ips = parse_json_array(args.ips, "--ips")
allowlist_type = (args.type or "PRIVATE_ES").strip().upper()
group = {
"Name": args.group_name,
"AllowList": ",".join(ips),
}
allowlist = {
"Groups": [group],
"Type": allowlist_type,
"AllowList": "",
}
body = {
"InstanceId": args.id,
"EsIpAllowList": allowlist,
}
api_call(lambda: es_api.modify_ip_allow_list_v2(body))
def cmd_public_network_set(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
body = {
"InstanceId": args.id,
}
if args.enable:
if not args.eip_id:
print_error(
"Missing Parameters", "--eip-id is required when enabling public access"
)
body["EsEip"] = {"IsOpen": True, "EipId": args.eip_id}
api_call(lambda: es_api.create_public_address(body))
else:
body["EsEip"] = {"IsOpen": False}
api_call(lambda: es_api.release_public_address(body))
def cmd_reset_password(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
body = {
"InstanceId": args.id,
"NewPassword": args.admin_password,
}
api_call(lambda: es_api.reset_admin_password(body))
def cmd_nodes(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {"InstanceId": args.id}
api_call(lambda: es_api.describe_instance_nodes(body))
def cmd_plugins(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {"InstanceId": args.id}
api_call(lambda: es_api.describe_instance_plugins(body))
def cmd_rename(args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str):
body = {"InstanceId": args.id, "NewName": args.name}
api_call(lambda: es_api.rename_instance(body))
def cmd_maintenance_set(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
# Keep a stable CLI surface even if upstream docs vary:
# allow either JSON object {"MaintenanceDay":[...],"MaintenanceTime":"..."} or
# explicit flags --day/--time.
maintenance_day = args.day
maintenance_time = args.time
if args.setting:
setting = json.loads(args.setting)
if not isinstance(setting, dict):
print_error("Invalid --setting", "--setting must be a JSON object")
if "MaintenanceDay" in setting and not maintenance_day:
maintenance_day = setting.get("MaintenanceDay")
if "MaintenanceTime" in setting and not maintenance_time:
maintenance_time = setting.get("MaintenanceTime")
if "maintenance_day" in setting and not maintenance_day:
maintenance_day = setting.get("maintenance_day")
if "maintenance_time" in setting and not maintenance_time:
maintenance_time = setting.get("maintenance_time")
if not maintenance_time:
print_error(
"Missing Parameters",
"Provide --time or --setting containing MaintenanceTime.",
)
if isinstance(maintenance_day, str) and maintenance_day:
# Accept common user inputs (Mon/Tue, Monday/Tuesday, MONDAY/TUESDAY) and normalize to
# full uppercase day names required by the API.
day_map = {
"MON": "MONDAY",
"MONDAY": "MONDAY",
"TUE": "TUESDAY",
"TUES": "TUESDAY",
"TUESDAY": "TUESDAY",
"WED": "WEDNESDAY",
"WEDNESDAY": "WEDNESDAY",
"THU": "THURSDAY",
"THUR": "THURSDAY",
"THURS": "THURSDAY",
"THURSDAY": "THURSDAY",
"FRI": "FRIDAY",
"FRIDAY": "FRIDAY",
"SAT": "SATURDAY",
"SATURDAY": "SATURDAY",
"SUN": "SUNDAY",
"SUNDAY": "SUNDAY",
}
raw_days = [d.strip().upper() for d in maintenance_day.split(",") if d.strip()]
maintenance_day = [day_map.get(d, d) for d in raw_days]
body = {
"InstanceId": args.id,
"MaintenanceDay": maintenance_day,
"MaintenanceTime": maintenance_time,
}
api_call(lambda: es_api.modify_maintenance_setting(body))
def cmd_deletion_protection_set(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
body = {
"InstanceId": args.id,
"DeletionProtection": args.enabled,
}
api_call(lambda: es_api.modify_deletion_protection(body))
def cmd_restart_node(
args, es_api: EsCloudApi, vpc_api: VpcApi, configuration, region: str
):
body = {
"InstanceId": args.id,
"NodeName": args.node_id,
"Force": args.force,
}
api_call(lambda: es_api.restart_node(body))
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="Volcano Engine ESCloud control tools CLI"
)
subparsers = parser.add_subparsers(dest="command", required=True)
p = subparsers.add_parser("list", help="List ESCloud instances")
p.add_argument("--page-number", type=int, default=1)
p.add_argument("--page-size", type=int, default=10)
p.set_defaults(func=cmd_list)
p = subparsers.add_parser("detail", help="Get instance details")
p.add_argument("--id", required=True, help="Instance ID")
p.set_defaults(func=cmd_detail)
p = subparsers.add_parser("create", help="Create a new instance (one step)")
p.add_argument("--name", required=True, help="Instance name")
p.add_argument(
"--version",
required=True,
help="Version (e.g. V7_10, V8_18, OPEN_SEARCH_2_9, OPEN_SEARCH_3_3)",
)
p.add_argument("--vpc-id", required=True, help="VPC ID")
p.add_argument("--subnet-id", required=True, help="Subnet ID (used to derive zone)")
p.add_argument("--admin-password", required=True, help="Admin password")
p.add_argument(
"--charge-type",
default="PostPaid",
help="Charge type (PostPaid or PrePaid). Default: PostPaid",
)
p.add_argument(
"--https", type=str_to_bool, default=True, help="Enable HTTPS (default: true)"
)
p.add_argument(
"--deletion-protection",
type=str_to_bool,
default=True,
help="Deletion protection (default: true)",
)
p.add_argument(
"--master-spec",
default="",
help="Optional master node resource spec name. If provided, pure master nodes are enabled.",
)
p.add_argument(
"--master-count", type=int, default=3, help="Master node count (default: 3)"
)
p.add_argument(
"--master-storage-spec",
default="",
help="Optional master node storage spec name.",
)
p.add_argument(
"--master-storage-size",
type=int,
default=None,
help="Optional master node storage size in GiB.",
)
p.add_argument(
"--pure-master",
type=str_to_bool,
default=None,
help="Explicitly enable/disable pure master nodes.",
)
p.add_argument("--hot-spec", required=True, help="Hot node resource spec name")
p.add_argument(
"--hot-count", type=int, default=2, help="Hot node count (default: 2)"
)
p.add_argument(
"--hot-storage-spec", required=True, help="Hot node storage spec name"
)
p.add_argument(
"--hot-storage-size",
type=int,
required=True,
help="Hot node storage size in GiB",
)
p.add_argument(
"--kibana-spec", default="", help="Optional Kibana node resource spec name"
)
p.add_argument(
"--kibana-count", type=int, default=1, help="Kibana node count (default: 1)"
)
p.set_defaults(func=cmd_create)
p = subparsers.add_parser("scale", help="Scale node spec/count (one step)")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument(
"--node-type",
required=True,
help="Node type (Master, Hot, Warm, Cold, Coordinator, Kibana, Other)",
)
p.add_argument("--spec-name", required=True, help="Target resource spec name")
p.add_argument("--count", type=int, required=True, help="Target node count")
p.add_argument(
"--storage-spec-name", default="", help="Optional target storage spec name"
)
p.add_argument(
"--storage-size",
type=int,
default=None,
help="Optional target storage size in GiB",
)
p.set_defaults(func=cmd_scale)
p = subparsers.add_parser("delete", help="Delete an instance")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument(
"--confirm", action="store_true", help="Required safety flag for deletion"
)
p.set_defaults(func=cmd_delete)
p = subparsers.add_parser("vpc", help="List VPCs")
p.set_defaults(func=cmd_vpc)
p = subparsers.add_parser("subnet", help="List subnets in a VPC")
p.add_argument("--vpc-id", required=True, help="VPC ID")
p.set_defaults(func=cmd_subnet)
p = subparsers.add_parser("zones", help="List available zones")
p.set_defaults(func=cmd_zones)
p = subparsers.add_parser("node_specs", help="List node available specs")
p.set_defaults(func=cmd_node_specs)
p = subparsers.add_parser(
"versions",
help="Best-effort list of supported versions (derived from node_specs)",
)
p.set_defaults(func=cmd_versions)
p = subparsers.add_parser("ip_allowlist_get", help="Get IP allowlist")
p.add_argument("--id", required=True, help="Instance ID")
p.set_defaults(func=cmd_ip_allowlist_get)
p = subparsers.add_parser("ip_allowlist_set", help="Set IP allowlist")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--group-name", required=True, help="Group name")
p.add_argument("--ips", required=True, help="JSON array of CIDRs/IPs")
p.add_argument(
"--type",
default="PRIVATE_ES",
help="Allowlist type (default: PRIVATE_ES). Use PUBLIC_ES when managing a public endpoint allowlist, if supported by your instance.",
)
p.set_defaults(func=cmd_ip_allowlist_set)
p = subparsers.add_parser("reset_password", help="Reset admin password")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--admin-password", required=True, help="New admin password")
p.set_defaults(func=cmd_reset_password)
p = subparsers.add_parser("nodes", help="Describe nodes in an instance")
p.add_argument("--id", required=True, help="Instance ID")
p.set_defaults(func=cmd_nodes)
p = subparsers.add_parser("plugins", help="Describe installed plugins")
p.add_argument("--id", required=True, help="Instance ID")
p.set_defaults(func=cmd_plugins)
p = subparsers.add_parser("rename", help="Rename instance display name")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--name", required=True, help="New instance name")
p.set_defaults(func=cmd_rename)
p = subparsers.add_parser(
"maintenance_set", help="Set maintenance window/setting (JSON pass-through)"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument(
"--day", default=None, help="Maintenance days (comma-separated) e.g. Mon,Tue"
)
p.add_argument(
"--time",
default="",
help="Maintenance time window (required if --setting omitted)",
)
p.add_argument(
"--setting",
default="",
help="Optional JSON object containing MaintenanceDay/MaintenanceTime",
)
p.set_defaults(func=cmd_maintenance_set)
p = subparsers.add_parser(
"deletion_protection_set", help="Enable/disable deletion protection"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--enabled", type=str_to_bool, required=True, help="true or false")
p.set_defaults(func=cmd_deletion_protection_set)
p = subparsers.add_parser("restart_node", help="Restart a specific node")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--node-id", required=True, help="Node name")
p.add_argument(
"--force",
type=str_to_bool,
default=False,
help="Force restart (default: false)",
)
p.set_defaults(func=cmd_restart_node)
p = subparsers.add_parser(
"public_network", help="Enable/disable public network access (EIP)"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument(
"--enable",
type=str_to_bool,
default=True,
help="Enable or disable public access (default: true)",
)
p.add_argument(
"--eip-id", default="", help="EIP allocation ID (required when enabling)"
)
p.set_defaults(func=cmd_public_network_set)
p = subparsers.add_parser("eip_list", help="List EIP addresses")
p.add_argument(
"--status",
choices=["Available", "Attaching", "Attached", "Detaching", "Releasing"],
help="Optional filter by status",
)
p.set_defaults(func=cmd_eip_list)
p = subparsers.add_parser("eip_allocate", help="Allocate (create) a new EIP")
p.add_argument(
"--bandwidth", type=int, default=1, help="Bandwidth in Mbps (default: 1)"
)
p.add_argument(
"--billing-type",
default="PostPaidByTraffic",
help="Billing type (PostPaidByBandwidth, PostPaidByTraffic, PrePaid). Default: PostPaidByTraffic",
)
p.add_argument("--name", default="", help="Optional EIP name")
p.add_argument("--description", default="", help="Optional EIP description")
p.add_argument("--isp", default="BGP", help="ISP (BGP, etc.). Default: BGP")
p.set_defaults(func=cmd_eip_allocate)
p = subparsers.add_parser("eip_release", help="Release (delete) an EIP")
p.add_argument("--allocation-id", required=True, help="EIP Allocation ID")
p.set_defaults(func=cmd_eip_release)
return parser
def main() -> None:
parser = build_parser()
args = parser.parse_args()
es_api, vpc_api, configuration, region = _get_clients()
args.func(args, es_api, vpc_api, configuration, region)
if __name__ == "__main__":
main()
# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import json
import sys
import time
from typing import Any, Callable, Tuple
from api import ApiError, EsCloudApi, VpcApi
# EIP Billing Types mapping (human strings to API integers)
EIP_BILLING_TYPES = {
"PrePaid": 1,
"PostPaid": 2, # Postpaid by Bandwidth
"PostPaidByTraffic": 3, # Postpaid by Traffic (Recommended)
}
class WorkflowError(Exception):
def __init__(
self,
error: str,
details: str = "",
data: dict[str, Any] | None = None,
steps_completed: list[str] | None = None,
):
super().__init__(details or error)
self.error = error
self.details = details
self.data = data or {}
self.steps_completed = list(steps_completed or [])
def emit(payload: dict[str, Any], exit_code: int = 0) -> None:
print(json.dumps(payload, default=str, ensure_ascii=False))
sys.exit(exit_code)
def first_present(obj: Any, keys: list[str]) -> Any:
if isinstance(obj, dict):
for key in keys:
if key in obj and obj[key] not in (None, ""):
return obj[key]
for value in obj.values():
found = first_present(value, keys)
if found not in (None, ""):
return found
elif isinstance(obj, list):
for item in obj:
found = first_present(item, keys)
if found not in (None, ""):
return found
return None
def get_list(obj: dict[str, Any], *keys: str) -> list[Any]:
for key in keys:
value = obj.get(key)
if isinstance(value, list):
return value
result = obj.get("Result") or obj.get("result")
if isinstance(result, dict):
for key in keys:
value = result.get(key)
if isinstance(value, list):
return value
return []
def normalize_detail(detail: dict[str, Any]) -> dict[str, Any]:
result = detail.get("Result") or detail.get("result") or detail
inst_id = first_present(result, ["InstanceId", "instance_id", "Id", "id"])
inst_conf = result.get("InstanceConfiguration", {})
name = first_present(inst_conf, ["InstanceName", "instance_name"])
status = first_present(result, ["Status", "status"])
return {
"id": inst_id,
"name": name,
"status": status,
"raw": detail,
}
def str_to_bool(v: str) -> bool:
if isinstance(v, bool):
return v
if v.lower() in ("true", "1", "yes", "y"):
return True
if v.lower() in ("false", "0", "no", "n"):
return False
raise argparse.ArgumentTypeError(f"Boolean value expected, got '{v}'")
def add_eip_alloc_args(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--eip-id", default="", help="Existing EIP allocation id")
parser.add_argument(
"--eip-bandwidth",
type=int,
default=10,
help="EIP bandwidth for auto-allocation",
)
parser.add_argument(
"--eip-billing-type",
type=str,
default="PostPaidByTraffic",
choices=list(EIP_BILLING_TYPES.keys()),
help=f"EIP billing type for auto-allocation (Allowed: {', '.join(EIP_BILLING_TYPES.keys())})",
)
parser.add_argument("--eip-isp", type=str, default="BGP", help="Optional EIP ISP")
parser.add_argument(
"--eip-auto-reuse",
type=str_to_bool,
default=True,
help="If true, reuse an existing Available EIP before allocating a new one",
)
def _get_clients() -> Tuple[EsCloudApi, VpcApi, Any, str]:
import ark_shim
if ark_shim.check_is_ark_env():
return ark_shim.get_clients()
import sdk_shim
return sdk_shim.get_clients()
class ControlPlane:
def __init__(self, debug: bool = False):
self.es_api, self.vpc_api, self.configuration, self.region = _get_clients()
self.debug = debug
def _run_api(
self, fn: Callable[[], Any], context_data: dict[str, Any] | None = None
) -> Any:
try:
resp = fn()
# Normalize to dict for inspection
data = resp
try:
if hasattr(resp, "to_dict"):
data = resp.to_dict()
except Exception:
pass
# check for nested errors even if no exception was raised
if isinstance(data, dict):
error_msg = self._extract_error(data)
if error_msg:
if not self.debug:
# Strip duplicated Error blocks from the raw response to keep output concise
if "Result" in data and isinstance(data["Result"], dict):
data["Result"].pop("Error", None)
if "ResponseMetadata" in data and isinstance(
data["ResponseMetadata"], dict
):
data["ResponseMetadata"].pop("Error", None)
raise WorkflowError("API Error", error_msg, {"response": data})
return data
except WorkflowError:
raise
except ApiError as exc:
msg = str(exc)
if not self.debug:
# Try to extract a more precise message from the exception string if it contains JSON
extracted = self._extract_error(msg)
msg = extracted if extracted else msg
instruction = ""
if "TaskIsRunning" in msg:
instruction = (
"An operation is already in progress for this instance. "
"Wait for the instance to return to Running and retry."
)
elif any(
key in msg
for key in [
"BadRequest",
"InvalidParameter",
"NotFound",
"Unauthorized",
"InvalidAction",
]
):
instruction = (
"Verify that all IDs and parameters are valid and that the current credentials "
"have access to the target VPC, subnet, and instance."
)
details = f"{msg}\n\nInstruction: {instruction}" if instruction else msg
error_data = context_data if self.debug else None
raise WorkflowError("API Error", details, error_data)
except Exception as exc:
error_data = context_data if self.debug else None
raise WorkflowError(
"Unexpected Error",
f"{exc}\n\nInstruction: Internal script error. Check credentials, network, and parameters.",
error_data,
)
def _extract_error(self, source: Any) -> str | None:
"""Parses API response or error string for a precise root cause message."""
if not source:
return None
data = source
if isinstance(source, str):
try:
# Some ApiError messages contain the JSON response
start = source.find("{")
end = source.rfind("}")
if start != -1 and end != -1:
data = json.loads(source[start : end + 1])
else:
return None
except Exception:
return None
if not isinstance(data, dict):
return None
# Target locations for Volcengine ESCloud errors
# 1. Result.Error
# 2. ResponseMetadata.Error
res_err = (
data.get("Result", {}).get("Error", {})
if isinstance(data.get("Result"), dict)
else {}
)
meta_err = (
data.get("ResponseMetadata", {}).get("Error", {})
if isinstance(data.get("ResponseMetadata"), dict)
else {}
)
raw_msg = res_err.get("Message") or meta_err.get("Message")
if not raw_msg:
return None
# Keep original message without cropping details
return raw_msg
def _success(
self, goal: str, data: dict[str, Any], steps_completed: list[str]
) -> dict[str, Any]:
return {
"status": "success",
"goal": goal,
"data": data,
"steps_completed": steps_completed,
}
def _error(
self,
goal: str,
error: str,
details: str,
steps_completed: list[str],
data: dict[str, Any] | None = None,
status: str = "error",
) -> dict[str, Any]:
error_label = error
details_label = details
if error_label in ("API Error", "Unexpected Error") and details_label:
error_label = details_label
details_label = ""
payload = {
"status": status,
"goal": goal,
"error": error_label,
}
if details_label:
payload["details"] = details_label
payload["data"] = data or {}
payload["steps_completed"] = steps_completed
return payload
def _fetch_detail(self, instance_id: str) -> dict[str, Any]:
body = {"InstanceId": instance_id}
return self._run_api(
lambda: self.es_api.describe_instance(body), {"instance_id": instance_id}
)
def _wait_for_condition(
self,
fetch_fn: Callable[[], dict[str, Any]],
condition_fn: Callable[[dict[str, Any]], bool],
poll_interval: int,
timeout: int,
consecutive_successes: int = 1,
) -> tuple[bool, dict[str, Any]]:
deadline = time.time() + timeout
last_detail: dict[str, Any] = {}
success_count = 0
while True:
last_detail = fetch_fn()
if condition_fn(last_detail):
success_count += 1
if success_count >= consecutive_successes:
return True, last_detail
else:
success_count = 0
if time.time() >= deadline:
return False, last_detail
time.sleep(poll_interval)
def detail(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
detail = self._fetch_detail(args.id)
steps.append("fetch_detail")
normalized = normalize_detail(detail)
return self._success(
"detail",
{
"instance_id": normalized["id"],
"instance_name": normalized["name"],
"status": normalized["status"],
"detail": detail,
},
steps,
)
def list(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
body = {"PageNumber": args.page_number, "PageSize": args.page_size}
instances = self._run_api(
lambda: self.es_api.describe_instances(body), {"request": body}
)
steps.append("fetch_instances")
return self._success("list", instances, steps)
def provision_info(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
vpcs_resp = self._run_api(lambda: self.vpc_api.describe_vpcs({}))
steps.append("fetch_vpcs")
specs_resp = self._run_api(
lambda: self.es_api.describe_node_available_specs({})
)
steps.append("fetch_specs")
subnets_by_vpc: dict[str, Any] = {}
for vpc in get_list(vpcs_resp, "Vpcs", "vpcs"):
vpc_id = vpc.get("VpcId") or vpc.get("vpc_id")
if not vpc_id:
continue
subnet_resp = self._run_api(
lambda: self.vpc_api.describe_subnets({"VpcId": vpc_id}),
{"vpc_id": vpc_id},
)
subnets_by_vpc[vpc_id] = subnet_resp
steps.append("fetch_subnets")
zones_resp = self._run_api(lambda: self.es_api.describe_zones({}))
steps.append("fetch_zones")
return self._success(
"provision-info",
{
"region": self.region,
"vpcs": vpcs_resp,
"subnets_by_vpc": subnets_by_vpc,
"specs": specs_resp,
"zones": zones_resp,
},
steps,
)
def provision(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
subnet_resp = self._run_api(
lambda: self.vpc_api.describe_subnets({"SubnetIds": [args.subnet_id]}),
{"subnet_id": args.subnet_id},
)
subnets = get_list(subnet_resp, "Subnets", "subnets")
if not subnets:
raise WorkflowError(
"Subnet Not Found",
f"Subnet ID '{args.subnet_id}' does not exist or is not in the current region.",
steps_completed=steps,
)
zone_id = subnets[0].get("ZoneId") or subnets[0].get("zone_id")
if not zone_id:
raise WorkflowError(
"Zone Not Found",
"Failed to derive zone_id from the subnet.",
steps_completed=steps,
)
steps.append("validate_subnet")
node_specs = []
if args.master_spec:
node_specs.append(
{
"Type": "Master",
"Number": args.master_count,
"ResourceSpecName": args.master_spec,
"StorageSpecName": args.master_storage_spec
or args.hot_storage_spec,
"StorageSize": args.master_storage_size or 20,
}
)
node_specs.append(
{
"Type": "Hot",
"Number": args.hot_count,
"ResourceSpecName": args.hot_spec,
"StorageSpecName": args.hot_storage_spec,
"StorageSize": args.hot_storage_size,
}
)
if getattr(args, "kibana_spec", ""):
node_specs.append(
{
"Type": "Kibana",
"Number": getattr(args, "kibana_count", 1),
"ResourceSpecName": getattr(args, "kibana_spec", ""),
}
)
instance_conf = {
"InstanceName": args.name,
"Version": args.version,
"AdminPassword": args.admin_password,
"ChargeType": args.charge_type,
"DeletionProtection": True,
"EnableHttps": True,
"EnablePureMaster": True if args.master_spec else False,
"ProjectName": "default",
"RegionId": self.region,
"ZoneId": zone_id,
"VPC": {"VpcId": args.vpc_id},
"Subnet": {"SubnetId": args.subnet_id},
"NodeSpecsAssigns": node_specs,
}
body = {"InstanceConfiguration": instance_conf}
create_resp = self._run_api(
lambda: self.es_api.create_instance_in_one_step(body), {"request": body}
)
steps.append("create_instance")
instance_id = first_present(create_resp, ["InstanceId", "instance_id", "Id"])
if not instance_id:
# This should normally be caught by _run_api now, but keeping a fallback
raise WorkflowError(
"API Error",
"CreateInstanceInOneStep returned no instance identifier.",
{"create_response": create_resp},
steps_completed=steps,
)
ok, detail = self._wait_for_condition(
fetch_fn=lambda: self._fetch_detail(instance_id),
condition_fn=lambda d: normalize_detail(d)["status"] == "Running",
poll_interval=args.poll_interval,
timeout=args.timeout,
consecutive_successes=2,
)
steps.append("poll_status")
data = {
"instance_id": instance_id,
"create_response": create_resp,
"final_detail": detail,
}
if ok:
return self._success("provision", data, steps)
return self._error(
"provision",
"Timeout",
f"Instance '{instance_id}' did not reach Running within {args.timeout} seconds.",
steps,
data,
status="timeout",
)
def deprovision(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
if args.confirm != args.id:
raise WorkflowError(
"Confirmation required",
f"--confirm must exactly match the instance id {args.id!r}.",
steps_completed=steps,
)
detail = self._fetch_detail(args.id)
steps.append("fetch_detail")
inst_info = detail.get("Result", detail).get("InstanceInfo", {})
if inst_info.get("DeletionProtection"):
if not getattr(args, "force", False):
raise WorkflowError(
"Deletion Protection Enabled",
"Instance has deletion protection. Use --force to disable it and delete.",
steps_completed=steps,
)
self._run_api(
lambda: self.es_api.modify_deletion_protection(
{"InstanceId": args.id, "DeletionProtection": False}
),
{"instance_id": args.id},
)
steps.append("disable_deletion_protection")
release_resp = self._run_api(
lambda: self.es_api.release_instance({"InstanceId": args.id}),
{"instance_id": args.id},
)
steps.append("release_instance")
def _fetch_safe():
try:
return self._fetch_detail(args.id)
except WorkflowError as e:
if (
"NotFound" in str(e)
or "not exist" in str(e).lower()
or "InvalidParameter" in str(e)
):
return {"is_deleted": True}
raise e
ok, detail_after = self._wait_for_condition(
fetch_fn=_fetch_safe,
condition_fn=lambda d: d.get("is_deleted")
or normalize_detail(d).get("status") == "Deleted",
poll_interval=args.poll_interval,
timeout=args.timeout,
)
steps.append("poll_deletion")
data = {
"instance_id": args.id,
"release_response": release_resp,
"final_detail": detail_after,
}
if ok:
return self._success("deprovision", data, steps)
return self._error(
"deprovision", "Timeout", "Instance was not deleted in time.", steps, data
)
def scale(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
current_detail = self._fetch_detail(args.id)
steps.append("fetch_detail")
current_status = normalize_detail(current_detail).get("status")
if current_status != "Running":
raise WorkflowError(
"Invalid Instance State",
f"Instance '{args.id}' must be Running before scaling. Current status: {current_status!r}.",
{"detail": current_detail},
steps_completed=steps,
)
steps.append("validate_running")
assign_kwargs = {
"Type": args.node_type,
"Number": args.count,
"ResourceSpecName": args.spec_name,
}
if args.storage_spec_name:
assign_kwargs["StorageSpecName"] = args.storage_spec_name
if args.storage_size is not None:
assign_kwargs["StorageSize"] = args.storage_size
body = {"InstanceId": args.id, "NodeSpecsAssigns": [assign_kwargs]}
scale_resp = self._run_api(
lambda: self.es_api.modify_node_spec_in_one_step(body), {"request": body}
)
steps.append("scale_instance")
ok, detail = self._wait_for_condition(
fetch_fn=lambda: self._fetch_detail(args.id),
condition_fn=lambda d: normalize_detail(d).get("status") == "Running",
poll_interval=args.poll_interval,
timeout=args.timeout,
consecutive_successes=2,
)
steps.append("poll_status")
data = {
"instance_id": args.id,
"scale_response": scale_resp,
"final_detail": detail,
}
if ok:
return self._success("scale", data, steps)
return self._error(
"scale",
"Timeout",
f"Instance '{args.id}' did not return to Running within {args.timeout} seconds.",
steps,
data,
status="timeout",
)
def public_access(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
detail_before = self._fetch_detail(args.id)
steps.append("fetch_detail")
current_status = normalize_detail(detail_before).get("status")
if current_status != "Running":
raise WorkflowError(
"Invalid Instance State",
f"Instance '{args.id}' must be Running before managing public endpoint.",
{"detail": detail_before},
steps_completed=steps,
)
body = {
"InstanceId": args.id,
}
if args.enable:
eip_id = self._ensure_eip(args)
body["EsEip"] = {"IsOpen": True, "EipId": eip_id}
steps.append("ensure_eip")
modify_resp = self._run_api(
lambda: self.es_api.create_public_address(body), {"request": body}
)
steps.append("create_public_address")
else:
body["EsEip"] = {"IsOpen": False}
modify_resp = self._run_api(
lambda: self.es_api.release_public_address(body), {"request": body}
)
steps.append("release_public_address")
ok, detail_after = self._wait_for_condition(
fetch_fn=lambda: self._fetch_detail(args.id),
condition_fn=lambda d: normalize_detail(d).get("status") == "Running",
poll_interval=args.poll_interval,
timeout=args.timeout,
consecutive_successes=2,
)
steps.append("poll_endpoint")
data = {
"instance_id": args.id,
"enable": args.enable,
"modify_response": modify_resp,
"detail_before": detail_before,
"detail_after": detail_after,
}
if ok:
return self._success("public-access", data, steps)
return self._error(
"public-access",
"Timeout",
"Endpoint change did not complete in time.",
steps,
data,
status="timeout",
)
def allowlist(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
detail_before = self._fetch_detail(args.id)
steps.append("fetch_detail")
current_status = normalize_detail(detail_before).get("status")
if current_status != "Running":
raise WorkflowError(
"Invalid Instance State",
f"Instance '{args.id}' must be Running before modifying allowlist.",
{"detail": detail_before},
steps_completed=steps,
)
ips_list = args.ips.split(",") if args.ips else []
ips_list = [ip.strip() for ip in ips_list if ip.strip()]
allowlist_type = (args.type or "PRIVATE_ES").strip().upper()
group = {
"Name": args.group_name,
"AllowList": ",".join(ips_list),
}
allowlist_conf = {
"Groups": [group],
"Type": allowlist_type,
"AllowList": "",
}
body = {
"InstanceId": args.id,
"EsIpAllowList": allowlist_conf,
}
modify_resp = self._run_api(
lambda: self.es_api.modify_ip_allow_list_v2(body), {"request": body}
)
steps.append("modify_allowlist")
ok, detail_after = self._wait_for_condition(
fetch_fn=lambda: self._fetch_detail(args.id),
condition_fn=lambda d: normalize_detail(d).get("status") == "Running",
poll_interval=args.poll_interval,
timeout=args.timeout,
consecutive_successes=2,
)
steps.append("poll_status")
data = {
"instance_id": args.id,
"modify_response": modify_resp,
"detail_after": detail_after,
}
if ok:
return self._success("allowlist", data, steps)
return self._error(
"allowlist",
"Timeout",
"Allowlist change did not complete in time.",
steps,
data,
status="timeout",
)
def reset_password(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
detail_before = self._fetch_detail(args.id)
steps.append("fetch_detail")
current_status = normalize_detail(detail_before).get("status")
if current_status != "Running":
raise WorkflowError(
"Invalid Instance State",
f"Instance '{args.id}' must be Running before resetting password.",
{"detail": detail_before},
steps_completed=steps,
)
body = {
"InstanceId": args.id,
"NewPassword": args.admin_password,
}
modify_resp = self._run_api(
lambda: self.es_api.reset_admin_password(body), {"request": body}
)
steps.append("reset_password")
ok, detail_after = self._wait_for_condition(
fetch_fn=lambda: self._fetch_detail(args.id),
condition_fn=lambda d: normalize_detail(d).get("status") == "Running",
poll_interval=args.poll_interval,
timeout=args.timeout,
consecutive_successes=2,
)
steps.append("poll_status")
data = {
"instance_id": args.id,
"modify_response": modify_resp,
"detail_after": detail_after,
}
if ok:
return self._success("reset-password", data, steps)
return self._error(
"reset-password",
"Timeout",
"Password reset did not complete in time.",
steps,
data,
status="timeout",
)
def _ensure_eip(self, args: argparse.Namespace) -> str:
if args.eip_id:
return args.eip_id
if args.eip_auto_reuse:
try:
resp = self._run_api(
lambda: self.vpc_api.describe_eip_addresses({"Status": "Available"})
)
eips = get_list(resp, "EipAddresses", "eip_addresses")
for eip in eips:
alloc_id = eip.get("AllocationId") or eip.get("allocation_id")
if alloc_id and eip.get("Status") == "Available":
return alloc_id
except Exception:
pass
# Allocate new EIP
billing_type = EIP_BILLING_TYPES.get(args.eip_billing_type, 3)
body = {
"Bandwidth": args.eip_bandwidth,
"BillingType": billing_type,
"ISP": args.eip_isp or "BGP",
}
alloc_resp = self._run_api(
lambda: self.vpc_api.allocate_eip_address(body), {"request": body}
)
eip_id = first_present(alloc_resp, ["AllocationId", "allocation_id"])
if not eip_id:
raise WorkflowError(
"API Error",
f"AllocateEipAddress returned no AllocationId: {alloc_resp!r}",
)
return eip_id
def maintenance(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
maintenance_day = args.day
maintenance_time = args.time
if not maintenance_time:
raise WorkflowError(
"Missing Parameters", "Provide --time.", steps_completed=steps
)
day_map = {
"MON": "MONDAY",
"MONDAY": "MONDAY",
"TUE": "TUESDAY",
"TUES": "TUESDAY",
"TUESDAY": "TUESDAY",
"WED": "WEDNESDAY",
"WEDNESDAY": "WEDNESDAY",
"THU": "THURSDAY",
"THUR": "THURSDAY",
"THURS": "THURSDAY",
"THURSDAY": "THURSDAY",
"FRI": "FRIDAY",
"FRIDAY": "FRIDAY",
"SAT": "SATURDAY",
"SATURDAY": "SATURDAY",
"SUN": "SUNDAY",
"SUNDAY": "SUNDAY",
}
raw_days = [d.strip().upper() for d in maintenance_day.split(",") if d.strip()]
maintenance_day_parsed = [day_map.get(d, d) for d in raw_days]
body = {
"InstanceId": args.id,
"MaintenanceDay": maintenance_day_parsed,
"MaintenanceTime": maintenance_time,
}
modify_resp = self._run_api(
lambda: self.es_api.modify_maintenance_setting(body), {"request": body}
)
steps.append("modify_maintenance")
data = {"instance_id": args.id, "modify_response": modify_resp}
return self._success("maintenance", data, steps)
def rename(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
body = {"InstanceId": args.id, "NewName": args.name}
modify_resp = self._run_api(
lambda: self.es_api.rename_instance(body), {"request": body}
)
steps.append("rename_instance")
data = {"instance_id": args.id, "modify_response": modify_resp}
return self._success("rename", data, steps)
def restart_node(self, args: argparse.Namespace) -> dict[str, Any]:
steps: list[str] = []
detail_before = self._fetch_detail(args.id)
steps.append("fetch_detail")
current_status = normalize_detail(detail_before).get("status")
if current_status != "Running":
raise WorkflowError(
"Invalid Instance State",
f"Instance '{args.id}' must be Running before restarting nodes.",
{"detail": detail_before},
steps_completed=steps,
)
body = {
"InstanceId": args.id,
"NodeName": args.node_id,
"Force": args.force,
}
modify_resp = self._run_api(
lambda: self.es_api.restart_node(body), {"request": body}
)
steps.append("restart_node")
ok, detail_after = self._wait_for_condition(
fetch_fn=lambda: self._fetch_detail(args.id),
condition_fn=lambda d: normalize_detail(d).get("status") == "Running",
poll_interval=args.poll_interval,
timeout=args.timeout,
consecutive_successes=2,
)
steps.append("poll_status")
data = {
"instance_id": args.id,
"modify_response": modify_resp,
"detail_after": detail_after,
}
if ok:
return self._success("restart-node", data, steps)
return self._error(
"restart-node",
"Timeout",
"Restart did not complete in time.",
steps,
data,
status="timeout",
)
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="Volcano Engine ESCloud goal-based control plane CLI"
)
parser.add_argument(
"--debug", action="store_true", help="Include context data in error payloads"
)
subparsers = parser.add_subparsers(dest="command", required=True)
p = subparsers.add_parser(
"provision-info",
help="Collect necessary info for provisioning (VPCs, subnets, specs, zones)",
)
p.set_defaults(goal="provision-info")
p = subparsers.add_parser("list", help="List ESCloud instances")
p.add_argument("--page-number", type=int, default=1)
p.add_argument("--page-size", type=int, default=10)
p.set_defaults(goal="list")
p = subparsers.add_parser(
"provision", help="Create an ESCloud instance and wait until Running"
)
p.add_argument("--name", required=True, help="Instance name")
p.add_argument("--vpc-id", required=True, help="VPC ID")
p.add_argument("--subnet-id", required=True, help="Subnet ID")
p.add_argument("--admin-password", required=True, help="Admin password")
p.add_argument("--version", required=True, help="ESCloud version (e.g. V7_10)")
p.add_argument(
"--charge-type", default="PostPaid", help="Charge type (PostPaid or PrePaid)"
)
p.add_argument("--master-spec", default="", help="Master node resource spec name")
p.add_argument("--master-count", type=int, default=3, help="Master node count")
p.add_argument(
"--master-storage-spec", default="", help="Master node storage spec name"
)
p.add_argument(
"--master-storage-size",
type=int,
default=None,
help="Master node storage size in GiB",
)
p.add_argument("--hot-spec", required=True, help="Hot node resource spec name")
p.add_argument("--hot-count", type=int, default=2, help="Hot node count")
p.add_argument(
"--hot-storage-spec", required=True, help="Hot node storage spec name"
)
p.add_argument(
"--hot-storage-size",
type=int,
required=True,
help="Hot node storage size in GiB",
)
p.add_argument("--kibana-spec", default="", help="Kibana node resource spec name")
p.add_argument("--kibana-count", type=int, default=1, help="Kibana node count")
p.add_argument(
"--poll-interval", type=int, default=5, help="Polling interval in seconds"
)
p.add_argument(
"--timeout", type=int, default=600, help="Polling timeout in seconds"
)
p.set_defaults(goal="provision")
p = subparsers.add_parser(
"deprovision", help="Delete an ESCloud instance and wait until destroyed"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument(
"--confirm", required=True, help="Must exactly match the instance ID to confirm"
)
p.add_argument(
"--force",
action="store_true",
help="Force deletion by disabling deletion protection if enabled",
)
p.add_argument(
"--poll-interval", type=int, default=5, help="Polling interval in seconds"
)
p.add_argument(
"--timeout", type=int, default=600, help="Polling timeout in seconds"
)
p.set_defaults(goal="deprovision")
p = subparsers.add_parser(
"detail", help="Fetch status and raw details for an ESCloud instance"
)
p.add_argument("--id", required=True, help="Instance ID")
p.set_defaults(goal="detail")
p = subparsers.add_parser(
"scale", help="Scale ESCloud node spec or count and wait until Running"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument(
"--node-type", required=True, help="Node type (Master, Hot, Kibana, etc.)"
)
p.add_argument("--spec-name", required=True, help="Target resource spec name")
p.add_argument("--count", type=int, required=True, help="Target node count")
p.add_argument(
"--storage-spec-name", default="", help="Optional target storage spec name"
)
p.add_argument(
"--storage-size",
type=int,
default=None,
help="Optional target storage size in GiB",
)
p.add_argument(
"--poll-interval", type=int, default=5, help="Polling interval in seconds"
)
p.add_argument(
"--timeout", type=int, default=600, help="Polling timeout in seconds"
)
p.set_defaults(goal="scale")
p = subparsers.add_parser(
"public-access", help="Toggle public access for ESCloud endpoint"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--enable", type=str_to_bool, required=True, help="true/false")
add_eip_alloc_args(p)
p.add_argument(
"--poll-interval", type=int, default=5, help="Polling interval in seconds"
)
p.add_argument(
"--timeout", type=int, default=600, help="Polling timeout in seconds"
)
p.set_defaults(goal="public-access")
p = subparsers.add_parser(
"allowlist", help="Set IP allowlist and wait until Running"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--group-name", default="default", help="Group name")
p.add_argument("--ips", required=True, help="Comma-separated CIDRs/IPs")
p.add_argument(
"--type",
default="PRIVATE_ES",
help="Allowlist type (e.g. PRIVATE_ES, PUBLIC_ES)",
)
p.add_argument(
"--poll-interval", type=int, default=5, help="Polling interval in seconds"
)
p.add_argument(
"--timeout", type=int, default=600, help="Polling timeout in seconds"
)
p.set_defaults(goal="allowlist")
p = subparsers.add_parser(
"reset-password", help="Reset admin password and wait until Running"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--admin-password", required=True, help="New admin password")
p.add_argument(
"--poll-interval", type=int, default=5, help="Polling interval in seconds"
)
p.add_argument(
"--timeout", type=int, default=600, help="Polling timeout in seconds"
)
p.set_defaults(goal="reset-password")
p = subparsers.add_parser("maintenance", help="Configure maintenance window")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument(
"--day", required=True, help="Maintenance days (comma-separated, e.g. Mon,Tue)"
)
p.add_argument(
"--time", required=True, help="Maintenance time window (e.g. 02:00-06:00)"
)
p.set_defaults(goal="maintenance")
p = subparsers.add_parser("rename", help="Rename instance display name")
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--name", required=True, help="New instance name")
p.set_defaults(goal="rename")
p = subparsers.add_parser(
"restart-node", help="Restart a specific node and wait until Running"
)
p.add_argument("--id", required=True, help="Instance ID")
p.add_argument("--node-id", required=True, help="Node name to restart")
p.add_argument("--force", action="store_true", help="Force restart")
p.add_argument(
"--poll-interval", type=int, default=5, help="Polling interval in seconds"
)
p.add_argument(
"--timeout", type=int, default=600, help="Polling timeout in seconds"
)
p.set_defaults(goal="restart-node")
return parser
def dispatch(control_plane: ControlPlane, args: argparse.Namespace) -> dict[str, Any]:
goal = args.goal
if goal == "provision-info":
return control_plane.provision_info(args)
elif goal == "list":
return control_plane.list(args)
elif goal == "provision":
return control_plane.provision(args)
elif goal == "deprovision":
return control_plane.deprovision(args)
elif goal == "detail":
return control_plane.detail(args)
elif goal == "scale":
return control_plane.scale(args)
elif goal == "public-access":
return control_plane.public_access(args)
elif goal == "allowlist":
return control_plane.allowlist(args)
elif goal == "reset-password":
return control_plane.reset_password(args)
elif goal == "maintenance":
return control_plane.maintenance(args)
elif goal == "rename":
return control_plane.rename(args)
elif goal == "restart-node":
return control_plane.restart_node(args)
raise WorkflowError("Invalid Command", f"Unsupported goal {args.goal!r}.")
def main() -> None:
try:
parser = build_parser()
args = parser.parse_args()
control_plane = ControlPlane(debug=args.debug)
resp = dispatch(control_plane, args)
emit(resp)
except WorkflowError as exc:
error_label = exc.error
details_label = exc.details or str(exc)
if error_label in ("API Error", "Unexpected Error") and details_label:
error_label = details_label
details_label = ""
payload = {
"status": "error",
"goal": getattr(args, "goal", ""),
"error": error_label,
}
if details_label:
payload["details"] = details_label
payload["data"] = exc.data
payload["steps_completed"] = exc.steps_completed
emit(payload, exit_code=1)
except Exception as exc:
import traceback
print(
f"FATAL: Unhandled exception: {type(exc).__name__}: {exc}", file=sys.stderr
)
traceback.print_exc(file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
# Copyright (c) 2025 Beijing Volcano Engine Technology Co., Ltd. and/or its affiliates.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import os
import sys
from typing import Any, Dict
import volcenginesdkcore
from volcenginesdkcore.universal import UniversalApi, UniversalInfo
from volcenginesdkcore.rest import ApiException
from api import ApiError, EsCloudApi, VpcApi
class _SimpleResponse:
def __init__(self, payload: Dict[str, Any]):
self._payload = payload
def to_dict(self) -> Dict[str, Any]:
return self._payload
class _UniversalShim:
def __init__(self):
ak = os.environ.get("VOLCENGINE_ACCESS_KEY")
sk = os.environ.get("VOLCENGINE_SECRET_KEY")
region = os.environ.get("VOLCENGINE_REGION", "cn-beijing")
if not ak or not sk:
raise ApiError(
"Missing Credentials: VOLCENGINE_ACCESS_KEY or VOLCENGINE_SECRET_KEY is not set. "
"Ask the user to provide their Volcano Engine Access Key and Secret Key."
)
configuration = volcenginesdkcore.Configuration()
configuration.ak = ak
configuration.sk = sk
configuration.region = region
configuration.client_side_validation = False
self._api = UniversalApi(volcenginesdkcore.ApiClient(configuration))
self.region = region
def do_call(
self, service: str, action: str, version: str, method: str, body: Any
) -> Dict[str, Any]:
info = UniversalInfo(
method=method.upper(), service=service, version=version, action=action
)
if method.upper() != "GET":
info.content_type = "application/json"
try:
resp = self._api.do_call(info, body)
if isinstance(resp, dict):
return resp
if hasattr(resp, "to_dict"):
return resp.to_dict()
return {"Result": resp}
except ApiException as exc:
raise ApiError(str(exc))
except Exception as exc:
raise ApiError(f"SDK Error: {str(exc)}")
class ESCloudSdkShim(EsCloudApi):
def __init__(self, bridge: _UniversalShim):
self._bridge = bridge
def _call(self, method: str, action: str, body: Any) -> Any:
payload = body.to_dict() if hasattr(body, "to_dict") else (body or {})
return _SimpleResponse(
self._bridge.do_call("ESCloud", action, "2023-01-01", method, payload)
)
class VpcSdkShim(VpcApi):
def __init__(self, bridge: _UniversalShim):
self._bridge = bridge
def _call(self, method: str, action: str, body: Any) -> Any:
payload = body.to_dict() if hasattr(body, "to_dict") else (body or {})
return _SimpleResponse(
self._bridge.do_call("vpc", action, "2020-04-01", method, payload)
)
def get_clients():
try:
bridge = _UniversalShim()
return ESCloudSdkShim(bridge), VpcSdkShim(bridge), None, bridge.region
except ApiError as exc:
print(
json.dumps(
{"error": "Initialization Error", "details": str(exc)},
ensure_ascii=False,
)
)
sys.exit(1)