
Alicloud Data Lake Dlf Test
- 294 installs
- 396 repo stars
- Updated July 18, 2026
- cinience/alicloud-skills
alicloud-data-lake-dlf-test is a Claude Code skill that validates Alibaba Cloud Data Lake Formation pipelines, catalogs, and ingestion jobs with repeatable agent-driven test procedures before production analytics go-live
About
alicloud-data-lake-dlf-test is an AliCloud skills entry for Data Lake Formation (DLF) platforms on Alibaba Cloud. The skill structures verification of lake catalogs, ingestion jobs, and pipeline stages so data teams catch misconfigured tables or failed jobs pre-production. Developers reach for alicloud-data-lake-dlf-test when standing up or upgrading DLF-backed analytics and need agent-executable checklists instead of ad hoc console clicking.
- DLF pipeline validation steps
- Catalog and table metadata checks
- Ingestion job smoke tests
- Failure and retry verification
- Pre-production data lake readiness
Alicloud Data Lake Dlf Test by the numbers
- 294 all-time installs (skills.sh)
- Ranked #179 of 911 Databases skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/cinience/alicloud-skills --skill alicloud-data-lake-dlf-testAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 294 |
|---|---|
| repo stars | ★ 396 |
| Last updated | July 18, 2026 |
| Repository | cinience/alicloud-skills ↗ |
How do you test Alibaba Cloud DLF pipelines?
Validate Alibaba Cloud Data Lake Formation (DLF) pipelines, catalogs, and ingestion jobs with repeatable agent-driven test procedures before production analytics go-live.
Who is it for?
Data engineers on Alibaba Cloud DLF who need repeatable pipeline and catalog validation before analytics production cutover.
Skip if: Non-AliCloud data stacks or batch jobs with no DLF catalog or ingestion components.
When should I use this skill?
User asks to test, validate, or verify Alibaba Cloud Data Lake Formation pipelines, catalogs, or ingestion.
What you get
DLF validation checklists, ingestion job status reports, catalog verification logs, and pre-go-live test results.
- validation report
- job status summary
Files
Category: service
Cloud Backup
Use Alibaba Cloud OpenAPI (RPC) with official SDKs or OpenAPI Explorer to manage resources for Cloud Backup.
Workflow
1) Confirm region, resource identifiers, and desired action. 2) Discover API list and required parameters (see references). 3) Call API with SDK or OpenAPI Explorer. 4) Verify results with describe/list APIs.
AccessKey priority (must follow)
1) Environment variables: ALIBABACLOUD_ACCESS_KEY_ID / ALIBABACLOUD_ACCESS_KEY_SECRET / ALIBABACLOUD_REGION_ID Region policy: ALIBABACLOUD_REGION_ID is an optional default. If unset, decide the most reasonable region for the task; if unclear, ask the user. 2) Shared config file: ~/.alibabacloud/credentials
API discovery
- Product code:
hbr - Default API version:
2017-09-08 - Use OpenAPI metadata endpoints to list APIs and get schemas (see references).
High-frequency operation patterns
1) Inventory/list: prefer List* / Describe* APIs to get current resources. 2) Change/configure: prefer Create* / Update* / Modify* / Set* APIs for mutations. 3) Status/troubleshoot: prefer Get* / Query* / Describe*Status APIs for diagnosis.
Minimal executable quickstart
Use metadata-first discovery before calling business APIs:
python scripts/list_openapi_meta_apis.pyOptional overrides:
python scripts/list_openapi_meta_apis.py --product-code <ProductCode> --version <Version>The script writes API inventory artifacts under the skill output directory.
Output policy
If you need to save responses or generated artifacts, write them under: output/aliyun-hbr-backup/
Validation
mkdir -p output/aliyun-hbr-backup
for f in skills/backup/aliyun-hbr-backup/scripts/*.py; do
python3 -m py_compile "$f"
done
echo "py_compile_ok" > output/aliyun-hbr-backup/validate.txtPass criteria: command exits 0 and output/aliyun-hbr-backup/validate.txt is generated.
Output And Evidence
- Save artifacts, command outputs, and API response summaries under
output/aliyun-hbr-backup/. - Include key parameters (region/resource id/time range) in evidence files for reproducibility.
Prerequisites
- Configure least-privilege Alibaba Cloud credentials before execution.
- Prefer environment variables:
ALIBABACLOUD_ACCESS_KEY_ID,ALIBABACLOUD_ACCESS_KEY_SECRET, optionalALIBABACLOUD_REGION_ID. - If region is unclear, ask the user before running mutating operations.
References
- Sources:
references/sources.md
interface:
display_name: "Alibaba Cloud Backup HBR"
short_description: "Cloud Backup vault and job workflows"
default_prompt: "Use $aliyun-hbr-backup to complete this backup task on Alibaba Cloud."
Sources
- OpenAPI product page:
https://api.aliyun.com/product/hbr - API list (metadata):
https://api.aliyun.com/meta/v1/products/hbr/versions/2017-09-08/api-docs.json - API definition (single API):
https://api.aliyun.com/meta/v1/products/hbr/versions/2017-09-08/apis/{ApiName}/api.json
#!/usr/bin/env python3
"""Fetch OpenAPI metadata API list for one product/version and save to output/.
Env:
- OPENAPI_META_TIMEOUT (seconds, default: 20)
"""
from __future__ import annotations
import argparse
import json
import os
import pathlib
import urllib.request
DEFAULT_PRODUCT_CODE = "hbr"
DEFAULT_VERSION = "2017-09-08"
OUTPUT_DIR = pathlib.Path("output/aliyun-hbr-backup")
def fetch_json(url: str, timeout: int) -> dict:
req = urllib.request.Request(url, headers={"User-Agent": "codex-skill"})
with urllib.request.urlopen(req, timeout=timeout) as resp:
return json.loads(resp.read().decode("utf-8"))
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--product-code", default=DEFAULT_PRODUCT_CODE)
parser.add_argument("--version", default=DEFAULT_VERSION)
parser.add_argument("--output-dir", default=str(OUTPUT_DIR))
args = parser.parse_args()
timeout = int(os.getenv("OPENAPI_META_TIMEOUT", "20"))
output_dir = pathlib.Path(args.output_dir)
output_dir.mkdir(parents=True, exist_ok=True)
url = (
f"https://api.aliyun.com/meta/v1/products/{args.product_code}"
f"/versions/{args.version}/api-docs.json"
)
payload = fetch_json(url, timeout)
raw_apis = payload.get("apis", {})
if isinstance(raw_apis, dict):
api_names = sorted(raw_apis.keys())
elif isinstance(raw_apis, list):
names = []
for item in raw_apis:
if isinstance(item, dict):
name = item.get("name") or item.get("apiName")
if name:
names.append(name)
elif isinstance(item, str):
names.append(item)
api_names = sorted(set(names))
else:
api_names = []
json_file = output_dir / f"{args.product_code}_{args.version}_api_docs.json"
md_file = output_dir / f"{args.product_code}_{args.version}_api_list.md"
json_file.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
md_lines = [
f"# {args.product_code} {args.version} API List",
"",
f"- Source: {url}",
f"- API count: {len(api_names)}",
"",
]
md_lines.extend([f"- `{name}`" for name in api_names])
md_file.write_text("\n".join(md_lines) + "\n", encoding="utf-8")
print(f"Saved: {json_file}")
print(f"Saved: {md_file}")
if __name__ == "__main__":
main()
Related skills
FAQ
What Alibaba Cloud service does alicloud-data-lake-dlf-test cover?
alicloud-data-lake-dlf-test covers Alibaba Cloud Data Lake Formation (DLF)—catalogs, ingestion jobs, and pipelines. The skill runs repeatable tests so analytics platforms do not go live with broken lake configuration.
When should data teams use this skill?
Data teams should use alicloud-data-lake-dlf-test before production analytics go-live or after major DLF changes. Skip it if the workload does not use Alibaba Cloud Data Lake Formation.