
Alibabacloud Sls Query
- 900 installs
- 208 repo stars
- Updated August 4, 2026
- aliyun/alibabacloud-aiops-skills
alibabacloud-sls-query is an official Alibaba Cloud agent skill that authors, executes, and troubleshoots Simple Log Service (SLS) index search, SQL, and SPL queries through the Aliyun CLI for developers who need to inve
About
alibabacloud-sls-query is an official Alibaba Cloud agent skill from aliyun/alibabacloud-aiops-skills that turns natural-language log questions into executable SLS statements and runs them via the Aliyun CLI. The skill follows a 7-step workflow: read Logstore index config with get-index, pick among 4 query modes (index search, SQL, SQL scan, SPL), build the statement, resolve Unix --from/--to timestamps, execute get-logs-v2, extract rows with jq or JMESPath, and present copy-paste CLI output. It bundles 9 reference markdown guides and 49 source-of-truth YAML specs covering query routing, SPL pipelines, and SQL functions. Install with npx skills add aliyun/alibabacloud-aiops-skills --skill alibabacloud-sls-query; Aliyun CLI 3.3.8+ and configured credentials are required. Reach for alibabacloud-sls-query during on-call investigations when you need fast, index-aware SLS queries without hand-writing syntax from scratch.
- SLS query syntax help
- Log investigation patterns
- On-call troubleshooting
- Metric extraction from logs
- Centralized observability
Alibabacloud Sls Query by the numbers
- 900 all-time installs (skills.sh)
- +120 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #200 of 1,435 DevOps & CI/CD skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/aliyun/alibabacloud-aiops-skills --skill alibabacloud-sls-queryAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 900 |
|---|---|
| repo stars | ★ 208 |
| Last updated | August 4, 2026 |
| Repository | aliyun/alibabacloud-aiops-skills ↗ |
How do you query Alibaba Cloud SLS logs from CLI?
Author and debug Alibaba Cloud Log Service SLS queries so agents can investigate outages, trace requests, and extract metrics from centralized logs during on-call investigations.
Who is it for?
Developers on Alibaba Cloud who run on-call investigations and want an agent to author, execute, and debug SLS queries against centralized logstores.
Skip if: Developers not using Alibaba Cloud SLS or teams that only need ad-hoc console queries without CLI execution, index validation, or agent-driven troubleshooting.
When should I use this skill?
A developer asks to query, analyze, optimize, execute, or troubleshoot Alibaba Cloud SLS logs, index search, SQL analytics, or SPL scan statements.
What you get
Copy-paste-ready aliyun sls get-logs-v2 CLI commands, validated SLS index-search/SQL/SPL statements, and formatted log rows or aggregation results extracted from get-logs-v2 JSON responses.
- Copy-paste-ready aliyun sls get-logs-v2 CLI commands
- Validated SLS index-search, SQL, or SPL query statements
- Formatted log rows or aggregation results from get-logs-v2 responses
By the numbers
- Bundles 9 reference markdown guides and 49 source-of-truth YAML specification files
- Supports 4 SLS query modes: index search, SQL, SQL scan, and SPL
- Requires Aliyun CLI version 3.3.8 or newer and follows a 7-step query workflow
Files
Alibaba Cloud SLS Query & Analysis
Scenario Description
Use this skill when the user wants to:
- Explain, rewrite, optimize or execute an existing query
- Translate a natural-language requirement into an SLS index query, SQL, or SPL statement
---
Prerequisites
Install Aliyun CLI
Run aliyun version to verify if version >= 3.3.8. If not installed or outdated, follow the doc references/cli-installation-guide.md to install or update.
Ensure AI Mode Enabled
Before executing any CLI commands, enable AI-Mode, set User-Agent, and update plugins:
aliyun configure ai-mode enable
aliyun configure ai-mode set-user-agent --user-agent "AlibabaCloud-Agent-Skills/alibabacloud-sls-query"
aliyun plugin updateCheck Alibaba Cloud credentials configured
Run aliyun configure list to check if credentials configured. If no valid profile is shown, STOP here and ask the user to run aliyun configure outside of this session.
Security rules:
- NEVER read, echo, or print AK/SK values
- NEVER ask the user to paste AK/SK into the conversation
- ONLY use
aliyun configure listto check credential status
---
RAM Permission Requirements
| API | CLI | Action | Purpose |
|---|---|---|---|
| GetLogsV2 | get-logs-v2 | log:GetLogStoreLogs | Run query / SQL / SPL and read results |
| GetIndex | get-index | log:GetIndex | Read index config to verify prerequisites |
For the minimum and complete RAM policy JSON, see references/ram-policies.md.
Permission failure handling: If a call returns Unauthorized permission error, stop and surface references/ram-policies.md to the user. Do not retry with a different account without explicit user confirmation.---
Core Workflow
1. Read index configuration (GetIndex) 2. Pick query mode 3. Build statement 4. Resolve time range 5. Execute query 6. Extract data from response 7. Present CLI command and results
Step 1: Read the Index Configuration (Mandatory)
Always call get-index first — the index config decides which query modes are available in Step 2.
aliyun sls get-index \
--project <project> --logstore <logstore>Two sections in the response drive every later decision:
| Section | Meaning |
|---|---|
line | Full-text index — absence means full-text search is disabled |
keys | Field indexes — map of field → { type, doc_value, token, caseSensitive, chn, ... }. doc_value: true means statistics are enabled on that field |
If the call returns IndexConfigNotExist (HTTP 404), or the response has neither line nor keys populated, the Logstore has no index at all — stop immediately and tell the user they must create an index before any query / SQL / SPL can run.
- The response can be large — extract only the fields relevant to the current query. Cache per
logstoreand reuse within the session.
For field types, tokenization, and how get-index maps to capabilities, see references/related-apis.md and references/query-analysis.md.
---
Step 2: Pick the Query Mode (Critical)
The query statement takes one of the following forms:
| Priority | Mode | Statement Form | Use when | Requires |
|---|---|---|---|---|
| 1 | Index search | <index-search> | Filtering raw logs; return time-ordered and paginated logs | Full-text (line) or any field index (keys.<field>) |
| 2 | SQL | `<index-search> \ | <SQL>` | Aggregation, GROUP BY, sort, window, top-N, projection, and other analytical operations |
| 3 | SQL scan | `<index-search> \ | <SQL scan>` | User requested |
| 4 | SPL | `<index-search> \ | <SPL>` | User requested |
Selection rule:
- Always prefer Index search for fastest speed.
- Use Index search + SQL when the user needs analytical operations or field projection rather than full raw-log retrieval, such as aggregation,
GROUP BY, sorting, window analysis, top-N, or returning only the required fields/columns. - Do not proactively choose SQL scan or SPL; use them only when the user explicitly requests.
For the full decision guide, see references/query-analysis.md.
---
Step 3: Write the Statement
3.1 Build the index-search segment first (left of |)
Collect every filter that can be expressed in index-search syntax and place it before the first |. Use * if no filter applies.
* and "payment failed" and status: "500" and not path: "/healthz"*matches all;"..."is full-text (needs full-text index).key: "value"is a field filter (needs field index).- Combine with
and/or/not; group with parentheses. key: *means field exists. Range (>,>=,[a, b]) works only onlong/double.
If the requirement can be fully answered without aggregation or row-level processing, stop here — this is already a complete index search. For full index-search syntax, see references/query-analysis.md.
3.2 Append SQL — for aggregation / analytics
status: 500 | SELECT date_trunc('minute', __time__) AS minute,
count(*) AS errors
FROM log
GROUP BY minute
ORDER BY minute- Read references/query-analysis.md for Query & SQL rules
- Table name is
log(recommended to omit). - SQL respects the indexed field type from
get-index— along/doublefield can be compared directly (status >= 500). Cast only when a field is indexed astextbut numeric semantics are needed (try_castto suppress errors). - Read references/functions-guide.md for unusual Function selection (aggregate, JSON, regex, datetime, IP geo …)
3.3 Append SPL — for row-level processing / flexible filtering
status: 500 and service: payment
| where try_cast(latency as BIGINT) > 1000
| extend latency_ms = try_cast(latency as BIGINT)
| project service, latency_ms, messageFor SPL syntax, pipeline commands, and field-handling rules, read references/spl-guide.md.
3.4 Append SQL scan — fallback when the target field has no index / statistics
Syntax follows regular SQL (see 3.2), with one difference: every field is `varchar`, so always cast() / try_cast() before numeric comparison or arithmetic. See references/query-analysis.md for scan semantics.
* | set session mode=scan; SELECT api, count(1) AS pv FROM log GROUP BY api---
Step 4: Resolve the Time Range
Generate --from / --to as Unix timestamps in seconds before building the CLI command. --from is inclusive and --to is exclusive.
Choose one of three input patterns:
1. Relative time — user says "recent / last N minutes|hours|days". 2. Natural-language absolute time without timezone — normalize to YYYY-MM-DD HH:MM:SS, then parse using the machine's local timezone. 3. Absolute time with explicit timezone — parse using the customer-provided timezone or UTC offset.
1. Relative time
# recent 15 minutes
FROM=$(($(date +%s) - 900))
TO=$(date +%s)2. Natural-language absolute time without timezone
If the user gives a date/time but no timezone, use the machine's local timezone. First normalize natural language such as 2026年3月13日12点 to 2026-03-13 12:00:00, then parse it as local time.
# Example: 2026年3月13日12点 -> 2026-03-13 12:00:00
# Linux (GNU date): local timezone
FROM=$(date -d "2026-03-13 12:00:00" +%s)
# macOS (BSD date): local timezone
FROM=$(date -j -f "%Y-%m-%d %H:%M:%S" "2026-03-13 12:00:00" +%s)For a time range such as "2026年3月13日12点到13点", compute both endpoints the same way. For a single point-in-time request, infer a practical window from the user's intent; if unclear, ask for the range before executing.
3. Absolute time with explicit timezone
To convert a local date/time to a Unix timestamp: parse the input as UTC with date -u, then subtract the timezone's UTC offset in seconds.
Formula: unix_ts = date_utc_parse(input) − (UTC_offset_hours × 3600)
# Example: 2025-01-15 10:30:00 Beijing Time (UTC+8)
# Beijing is UTC+8, so subtract 8 × 3600 = 28800
# Linux (GNU date)
FROM=$(( $(date -u -d "2025-01-15 10:30:00" +%s) - 28800 ))
# macOS (BSD date)
FROM=$(( $(date -u -j -f "%Y-%m-%d %H:%M:%S" "2025-01-15 10:30:00" +%s) - 28800 ))# Example: 2025-01-15 10:30:00 New York Time (UTC-5)
# New York is UTC-5, so subtract -5 × 3600 = subtract -18000 = add 18000
# Linux (GNU date)
FROM=$(( $(date -u -d "2025-01-15 10:30:00" +%s) + 18000 ))
# macOS (BSD date)
FROM=$(( $(date -u -j -f "%Y-%m-%d %H:%M:%S" "2025-01-15 10:30:00" +%s) + 18000 ))Common UTC offsets (value to subtract):
| Timezone | UTC offset hours | Seconds to subtract |
|---|---|---|
| Beijing (UTC+8) | +8 | 28800 |
| Tokyo (UTC+9) | +9 | 32400 |
| London (UTC) | 0 | 0 |
| New York (UTC-5) | -5 | -18000 |
---
Step 5: Execute via get-logs-v2
Use aliyun sls get-logs-v2 to execute queries. Run aliyun help sls get-logs-v2 to see CLI parameter usage; read references/related-apis.md for detailed API parameter descriptions.
Required CLI flags:
--project: SLS project name--logstore: Logstore name within the project--from: Start of time range, Unix timestamp in seconds (inclusive)--to: End of time range, Unix timestamp in seconds (exclusive)--query: Statement built in Step 3
Pagination works differently depending on whether the statement has a |:
5.1 Index-search only — paginate with --offset / --line
aliyun sls get-logs-v2 \
--project my-project --logstore my-logstore \
--from 1740000000 --to 1740003600 \
--query '* and "payment failed" and status: "500"' \
--line 100 --offset 0 --reverse true- Pagination:
--lineis page size (1–100, required);--offsetis the start row (optional, default0). - Ordering:
--reverse truereturns newest first; defaultfalseis oldest first.
5.2 With SQL — paginate with LIMIT inside the statement
aliyun sls get-logs-v2 \
--project my-project --logstore my-logstore \
--from 1740000000 --to 1740003600 \
--query 'status: "500" | SELECT request_uri, count(*) AS cnt FROM log GROUP BY request_uri ORDER BY cnt DESC LIMIT 20'- SQL default result cap is 100 rows. To get more results or paginate:
LIMIT count— raise the cap (e.g.,LIMIT 500returns up to 500 rows)LIMIT offset, count— paginate (e.g.,LIMIT 20, 20for rows 21–40;LIMIT 40, 20for rows 41–60). Max offset+count is 1000000.- Do not use
LIMIT count OFFSET offsetsyntax — it is not supported. Always useLIMIT offset, count. - Ordering: use
ORDER BY <field> DESC/ASCto sort.
Result completeness check: every response contains meta.progress. If it is Incomplete, re-issue the same request until it returns Complete.
---
Step 6: Extract Data from the Response
get-logs-v2 returns:
{
"meta": { "progress": "Complete", "count": 10, ... },
"data": [ { "field1": "value1", ... }, ... ]
}| Field | Meaning |
|---|---|
meta.progress | Complete or Incomplete (see Step 5) |
meta.count | Number of rows returned |
data | Array of log entries or aggregation rows; may contain __time__ (Unix seconds, string) |
Use jq (preferred) or --cli-query (JMESPath) to extract the fields the user needs:
| Extract | jq | --cli-query (JMESPath) |
|---|---|---|
| Data rows | `\ | jq '.data'` |
| Progress | `\ | jq '.meta.progress'` |
| Row count | `\ | jq '.meta.count'` |
| Specific fields | `\ | jq '.data[] \ |
---
Step 7: Present the CLI Command and Results
CLI command — always show the full, copy-paste-ready aliyun sls get-logs-v2 ... command. Redact any AK/SK. If the query was not executed (write / explain scenario), present the command the user should run.
Results — when a query was executed, use Step 6 to extract data and format according to the user's request (table, list, summary, etc.). Append one sentence explaining the query mode choice.
---
Cleanup
Whether operations succeed or fail, you MUST disable AI-Mode before ending the session:
aliyun configure ai-mode disable---
Global Rules
- Always prefer Index search for fastest raw-log retrieval, and use Index search + SQL for analysis or field projection.
- When the user only needs specific fields, use `SELECT` to project them rather than fetching full raw logs — this reduces network overhead. Requires
doc_value: trueon the target fields (confirmed in Step 1). - Do not hard-code
__time__filters — pass time range via--from/--to. - Deprecated API: never call
get-logs; always useget-logs-v2.
---
Troubleshooting
When the user reports "no data", "wrong result", or a CLI error, walk through the checklist in this exact order:
1. Time range — wrong --from/--to? Milliseconds instead of seconds? Recent writes still indexing? 2. Index configuration — field index missing? Full-text index off? Target field not in keys? 3. Field type / statistics — range query on a text field? SQL on a field without doc_value? 4. Syntax — mixed SQL and SPL? Leading * in fuzzy match? SPL string escaping? 5. Mode choice — scanning when an index-based query would do? Aggregating in SPL instead of SQL? 6. Completeness — meta.progress = Incomplete, caller did not retry (see Step 5). 7. ProjectNotExist — region or endpoint is wrong. See references/regions.md. 8. Network failure (timeout, connection refused) — try switching to internal endpoint. See references/regions.md.
For the full catalog of failure modes and error codes, see references/troubleshooting.md and the Common Errors table in references/related-apis.md.
---
Reference Documents
| Document | Description |
|---|---|
| references/query-analysis.md | Mode decision, index-search / SQL rules, scan semantics |
| references/spl-guide.md | SPL pipeline syntax, common commands, field handling |
| references/functions-guide.md | Function categories, SQL/SPL differences, templates |
| references/troubleshooting.md | "No data / wrong result / error" playbook |
| references/related-apis.md | GetLogsV2 and GetIndex API & CLI reference |
| references/ram-policies.md | Minimum and complete RAM policies |
| references/cli-installation-guide.md | Aliyun CLI install, auth modes, profiles |
| references/regions.md | Region / endpoint configuration, internal endpoint, ProjectNotExist troubleshooting |
| references/acceptance-criteria.md | CLI invocation acceptance tests |
references/query_analysis/*.yaml · references/spl/*.yaml · references/functions/*.yaml | Source-of-truth YAMLs bundled with this skill |
Acceptance Criteria: sls-query-analysis
Scenario: SLS Log Query & Analysis Purpose: Skill testing acceptance criteria
---
Correct CLI Invocation Patterns
1. Command Format — verify product and API name
CORRECT
aliyun sls get-logs-v2 \
--project my-project \
--logstore my-logstore \
--from 1740000000 \
--to 1740003600 \
--query '* and status: "500"' \
--line 100INCORRECT — Wrong product name
aliyun log get-logs-v2 --project my-project --logstore my-logstoreWhy: Product name is sls, not log, logservice, aliyunlog, or aliyun-sls.
2. Parameter Format
CORRECT — Kebab-case CLI sub-command and flags
aliyun sls get-logs-v2 \
--project my-project \
--logstore my-logstore \
--from 1740000000 \
--to 1740003600 \
--query '* | select count(*) as total from log' \
--line 100 \
--offset 0 \
--reverse trueINCORRECT — PascalCase sub-command or flags
# Sub-command in PascalCase
aliyun sls GetLogsV2 --project my-project --logstore my-logstore
aliyun sls GetIndex --project my-project --logstore my-logstore
# Flags in PascalCase
aliyun sls get-logs-v2 --Project my-project --Logstore my-logstore --From 1740000000 --To 1740003600Why: The SLS plugin uses kebab-case for both sub-commands (get-logs-v2, get-index) and flags (--project, --logstore, --from, --to, --query).
INCORRECT — Using --region-id instead of --region
aliyun sls get-logs-v2 --region-id cn-hangzhou --project p --logstore l --from 1 --to 2Why: The CLI global flag is --region, not --region-id.
INCORRECT — JSON --params string (old SDK pattern)
aliyun sls get-logs-v2 --params '{"Project":"my-project","Logstore":"my-logstore","From":"1740000000","To":"1740003600"}'Why: The CLI takes individual flags, not a JSON --params blob.
3. Authentication — never expose credentials
CORRECT — Verify credential profile via default credential chain
aliyun configure listINCORRECT — Passing AK/SK directly in the command
aliyun sls get-logs-v2 \
--access-key-id LTAI5tXXXX \
--access-key-secret 8dXXXX \
--project p --logstore l --from 1740000000 --to 1740003600Why: Credentials must come from the configured profile, environment variables, STS, or RAM role — never be typed into the command line.
INCORRECT — Reading or printing raw credentials
aliyun configure get # FORBIDDEN: may expose credential details
cat ~/.aliyun/config.json # FORBIDDEN: may expose credential detailsINCORRECT — Any command that prints environment credentials
echo $ALIBABA_CLOUD_ACCESS_KEY_ID # FORBIDDEN: example of secret output
printenv | grep -i credential # FORBIDDEN: may reveal secrets
env | grep -i access_key # FORBIDDEN: may reveal secrets4. API Names — verify exact sub-command
CORRECT
get-logs-v2 # OpenAPI Action: GetLogsV2
get-index # OpenAPI Action: GetIndexINCORRECT
GetLogsV2 # PascalCase is the Action name, not the CLI sub-command
GetIndex # PascalCase is the Action name, not the CLI sub-command
getLogsV2 # Wrong casing
get_logs_v2 # Wrong separator (snake_case)
getlogsv2 # Missing separators
get-logs # Deprecated — use get-logs-v2
get-logs-2 # Wrong suffix (v2, not 2)
describe-index # Wrong verb — SLS uses get-, not describe-
get-log-index # Not a real sub-command — use get-index5. Region Parameter
CORRECT
--region cn-hangzhou
--region cn-shanghai
--region ap-southeast-1
--region us-west-1INCORRECT
--region hangzhou # Missing country prefix
--region cn-hangzhou-1 # Not a real region IDWhy: Only valid Alibaba Cloud region IDs are accepted (e.g., cn-hangzhou, ap-southeast-1). The project is region-scoped — a region mismatch returns ProjectNotExist.
6. Time Parameters
CORRECT — Unix timestamp in seconds
--from 1711324800 --to 1711411200INCORRECT — Millisecond timestamps
--from 1711324800000 --to 1711411200000Why: --from / --to are Unix seconds, not milliseconds.
INCORRECT — Date or ISO strings
--from "2024-03-25" --to "2024-03-26"
--from "2024-03-25T00:00:00Z" --to "2024-03-26T00:00:00Z"Why: Only integer seconds are accepted; date strings must be converted first (e.g., date -d "2024-03-25 00:00:00 UTC" +%s).
Aliyun CLI Installation & Configuration Guide
Complete guide for installing and configuring Aliyun CLI.
Aliyun CLI 3.3.8+: Supports installing and using all published Alibaba Cloud product plugins. Make sure to upgrade to 3.3.8 or later for full plugin ecosystem coverage.
Installation
macOS
Using Homebrew (Recommended)
brew install aliyun-cli
# Upgrade to latest
brew upgrade aliyun-cli
# Verify version (>= 3.3.8)
aliyun versionUsing Binary
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz
# Extract
tar -xzf aliyun-cli-macosx-latest-amd64.tgz
# Move to PATH
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionLinux
Debian/Ubuntu
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionCentOS/RHEL
# Download
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-amd64.tgz
sudo mv aliyun /usr/local/bin/
# Verify
aliyun versionARM64 Architecture
# Download ARM64 version
wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-arm64.tgz
# Extract and install
tar -xzf aliyun-cli-linux-latest-arm64.tgz
sudo mv aliyun /usr/local/bin/Windows
Using Binary
1. Download from: <https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip> 2. Extract the ZIP file 3. Add the directory to your PATH environment variable 4. Open new Command Prompt or PowerShell 5. Verify: aliyun version
Using PowerShell
# Download
Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip"
# Extract
Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli
# Add to PATH (requires admin privileges)
$env:Path += ";C:\aliyun-cli"
[Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::Machine)
# Verify
aliyun versionConfiguration
Quick Start
aliyun configure set \
--mode AK \
--access-key-id <your-access-key-id> \
--access-key-secret <your-access-key-secret> \
--region cn-hangzhouAll aliyun configure commands support non-interactive flags, which is the recommended approach — it works in scripts, CI/CD pipelines, and agent-driven automation without hanging on stdin prompts.
Where to Get Access Keys
1. Log in to Aliyun Console: <https://ram.console.aliyun.com/> 2. Navigate to: AccessKey Management 3. Create a new AccessKey pair 4. Save the secret immediately — it's only shown once
Configuration Modes
Aliyun CLI supports 6 authentication modes. All examples below use non-interactive flags.
1. AK Mode (Access Key)
Most common mode for personal accounts and scripts.
aliyun configure set \
--mode AK \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--region cn-hangzhouConfiguration is stored in ~/.aliyun/config.json:
{
"current": "default",
"profiles": [
{
"name": "default",
"mode": "AK",
"access_key_id": "LTAI5tXXXXXXXX",
"access_key_secret": "8dXXXXXXXXXXXXXXXXXXXXXXXX",
"region_id": "cn-hangzhou",
"output_format": "json",
"language": "en"
}
]
}2. StsToken Mode (Temporary Credentials)
For short-lived access (tokens expire in 1-12 hours).
aliyun configure set \
--mode StsToken \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--sts-token v1.0:XXXXXXXXXXXXXXXX \
--region cn-hangzhouUse cases: CI/CD pipelines, temporary access for external contractors, cross-account access.
3. RamRoleArn Mode (Assume RAM Role)
Assume a RAM role for elevated or cross-account access.
aliyun configure set \
--mode RamRoleArn \
--access-key-id LTAI5tXXXXXXXX \
--access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \
--ram-role-arn acs:ram::123456789012:role/AdminRole \
--role-session-name my-session \
--region cn-hangzhouUse cases: cross-account resource access, temporary elevated privileges, role-based access control.
4. EcsRamRole Mode (ECS Instance RAM Role)
Use the RAM role attached to an ECS instance — no credentials needed.
aliyun configure set \
--mode EcsRamRole \
--ram-role-name MyEcsRole \
--region cn-hangzhouRequirements: must be running on an ECS instance with a RAM role attached.
Use cases: scripts and automation running on ECS instances.
5. RsaKeyPair Mode (RSA Key Pair)
Use RSA key pair for authentication (generate key pair in Aliyun Console first).
aliyun configure set \
--mode RsaKeyPair \
--private-key /path/to/private-key.pem \
--key-pair-name my-key-pair \
--region cn-hangzhou6. RamRoleArnWithEcs Mode (ECS + RAM Role)
Combine ECS instance role with RAM role assumption for cross-account access from ECS.
aliyun configure set \
--mode RamRoleArnWithEcs \
--ram-role-name MyEcsRole \
--ram-role-arn acs:ram::123456789012:role/TargetRole \
--role-session-name my-session \
--region cn-hangzhouEnvironment Variables
Highest priority - overrides config file
Access Key Mode
export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_REGION_ID=cn-hangzhouSTS Token Mode
export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
export ALIBABA_CLOUD_SECURITY_TOKEN=your_sts_token
export ALIBABA_CLOUD_REGION_ID=cn-hangzhouECS RAM Role Mode
export ALIBABA_CLOUD_ECS_METADATA=role_nameUse Case:
- CI/CD pipelines
- Docker containers
- Temporary credential override
Managing Multiple Profiles
Create Named Profiles
aliyun configure set --profile projectA \
--mode AK \
--access-key-id LTAI5tAAAAAAAA \
--access-key-secret 8dAAAAAAAAAAAAAAAAAAAAAAAA \
--region cn-hangzhou
aliyun configure set --profile projectB \
--mode AK \
--access-key-id LTAI5tBBBBBBBB \
--access-key-secret 8dBBBBBBBBBBBBBBBBBBBBBBBB \
--region cn-shanghaiUse Specific Profile
aliyun ecs describe-instances --profile projectA
export ALIBABA_CLOUD_PROFILE=projectA
aliyun ecs describe-instances # Uses projectAList and Switch Profiles
aliyun configure list # List all profiles
aliyun configure set --current projectA # Switch default profileCredential Priority
Credentials are loaded in this order (first found wins):
1. Command-line flag: --profile <name> 2. Environment variable: ALIBABA_CLOUD_PROFILE 3. Environment credentials: ALIBABA_CLOUD_ACCESS_KEY_ID, etc. 4. Configuration file: ~/.aliyun/config.json (current profile) 5. ECS Instance RAM Role: If running on ECS with attached role
Verification
Test Authentication
# Basic test - list regions
aliyun ecs describe-regions
# Expected output: JSON array of regionsIf successful, you'll see:
{
"Regions": {
"Region": [
{
"RegionId": "cn-hangzhou",
"RegionEndpoint": "ecs.cn-hangzhou.aliyuncs.com",
"LocalName": "华东 1(杭州)"
},
...
]
},
"RequestId": "..."
}If failed, you'll see error messages:
InvalidAccessKeyId.NotFound- Wrong Access Key IDSignatureDoesNotMatch- Wrong Access Key SecretInvalidSecurityToken.Expired- STS token expired (for StsToken mode)Forbidden.RAM- Insufficient permissions
Debug Configuration
# Show current configuration
aliyun configure get
# Test with debug logging
aliyun ecs describe-regions --log-level=debug
# Check credential provider
aliyun configure get modeSecurity Best Practices
1. Use RAM Users (Not Root Account)
❌ Don't: Use Aliyun root account credentials ✅ Do: Create RAM users with specific permissions
# Create RAM user in console
# Attach only necessary policies
# Use RAM user's access keys2. Principle of Least Privilege
Grant only the minimum permissions needed:
# Example: Read-only ECS access
# Attach policy: AliyunECSReadOnlyAccess3. Rotate Access Keys Regularly
# Create new access key in RAM Console, then update configuration
aliyun configure set --access-key-id NEW_KEY --access-key-secret NEW_SECRET
# Delete old access key from console4. Use STS Tokens for Temporary Access
aliyun configure set --mode StsToken \
--access-key-id XXXX --access-key-secret XXXX \
--sts-token XXXX --region cn-hangzhou5. Use ECS RAM Roles When Possible
aliyun configure set --mode EcsRamRole --ram-role-name MyRole --region cn-hangzhou6. Never Commit Credentials
# Add to .gitignore
echo "~/.aliyun/config.json" >> .gitignore
# Use environment variables in CI/CD instead7. Secure Config File
# Restrict permissions
chmod 600 ~/.aliyun/config.jsonTroubleshooting
Issue: Command Not Found
# Check installation
which aliyun
# Check PATH
echo $PATH
# Reinstall or add to PATHIssue: Authentication Failed
# Verify configuration
aliyun configure get
# Test with debug
aliyun ecs describe-regions --log-level=debug
# Check credentials in console
# Verify access key is activeIssue: Permission Denied
# Error: Forbidden.RAM
# Check RAM user permissions
# Attach necessary policies in RAM console
# Example: AliyunECSFullAccess for ECS operationsIssue: STS Token Expired
# Error: InvalidSecurityToken.Expired
# Reconfigure with new token
aliyun configure set --mode StsToken \
--access-key-id XXXX --access-key-secret XXXX \
--sts-token NEW_TOKEN --region cn-hangzhouIssue: Wrong Region
# Some resources may not exist in the specified region
# Check available regions
aliyun ecs describe-regions
# Update default region
aliyun configure set region cn-shanghaiAdvanced Configuration
Custom Endpoint
# Use custom or private endpoint
export ALIBABA_CLOUD_ECS_ENDPOINT=ecs-vpc.cn-hangzhou.aliyuncs.comProxy Settings
# HTTP proxy
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080
# No proxy for specific domains
export NO_PROXY=localhost,127.0.0.1,.aliyuncs.comTimeout Settings
# Connection timeout (default: 10s)
export ALIBABA_CLOUD_CONNECT_TIMEOUT=30
# Read timeout (default: 10s)
export ALIBABA_CLOUD_READ_TIMEOUT=30Next Steps
After installation and configuration:
1. Install plugins for services you need (v3.3.1+ supports all published product plugins):
aliyun plugin install --names ecs vpc rds
# List all available plugins
aliyun plugin list-remote2. Explore commands:
aliyun sls --help
aliyun fc --help3. Read documentation:
- Command Syntax Guide
- Global Flags Reference
- Common Scenarios
References
- Official Documentation: <https://help.aliyun.com/zh/cli/>
- RAM Console: <https://ram.console.aliyun.com/>
- Access Key Management: <https://ram.console.aliyun.com/manage/ak>
- Plugin Repository: <https://github.com/aliyun/aliyun-cli>
函数选型指南
先按场景选函数分类,再回读 skill 内部的对应 YAML。
高频分类
- 数据统计:
./functions/aggregate.yaml - 字符串处理:
./functions/string.yaml - 正则匹配:
./functions/regex.yaml - 时间处理:
./functions/datetime.yaml - 类型转换:
./functions/type_conversion.yaml - 条件判断:
./functions/conditional.yaml - JSON 提取:
./functions/json.yaml - 数值计算:
./functions/math.yaml - URL 解析:
./functions/url.yaml - 数组 / Map:
./functions/array.yaml、./functions/map.yaml - 窗口分析:
./functions/window.yaml - 漏斗分析:
./functions/window_funnel.yaml - Lambda 表达式:
./functions/lambda.yaml
语言差异
- SQL + SPL 都支持:字符串、正则、时间、类型转换、条件、JSON、数学、URL、编码、哈希、数组、Map 等大部分基础函数
- 仅 SQL:窗口函数、位运算、空间函数、HyperLogLog、统计函数、漏斗函数等
- 仅 SPL:
ip_to_province、ip_to_city、ip_to_country、ip_to_geo
高频提醒
- 数值比较前必须先
cast()或try_cast() - 想避免转换失败时整条报错,用
try_cast() - 时间分组优先
date_trunc()或date_format() - JSON 字段优先
json_extract()/json_extract_scalar() - SPL 做转义相关处理,优先看
ascii_escape、ascii_unescape、unicode_unescape - SPL 中某些函数能力和 SQL 不完全对齐,拿不准时回读对应函数 YAML
常见模板
类型转换
* | SELECT count(*) FROM log WHERE cast(status as BIGINT) >= 500* | where try_cast(status as BIGINT) >= 500JSON 提取
* | SELECT json_extract_scalar(payload, '$.user.id') AS user_id, count(*) FROM log GROUP BY user_id正则提取
* | SELECT regexp_extract(message, 'code:(\d+)', 1) AS code, count(*) FROM log GROUP BY codeSPL 地域分析
* | extend province = ip_to_province(client_ip) | stats pv = count(*) by province本地源文档
./functions/overview.yaml./functions/README.md./functions/*.yaml
category: aggregate_functions
name: 聚合函数
description: 对数据进行汇总计算,通常与GROUP BY配合使用
support:
sql: true
spl: false
functions:
- name: count
syntax: "count(*) 或 count(x)"
description: 统计日志条数
example: "* | SELECT count(*) AS pv"
note: "count(*) 统计所有,count(x) 统计x非NULL的数量"
- name: sum
syntax: "sum(x)"
description: 计算总和
example: "* | SELECT sum(cast(response_size as BIGINT)) AS total_size"
- name: avg
syntax: "avg(x)"
description: 计算平均值
example: "* | SELECT avg(cast(request_time as DOUBLE)) AS avg_time"
- name: max
syntax: "max(x)"
description: 返回最大值
example: "* | SELECT max(cast(response_time as BIGINT)) AS max_time"
- name: min
syntax: "min(x)"
description: 返回最小值
example: "* | SELECT min(cast(response_time as BIGINT)) AS min_time"
- name: count_if
syntax: "count_if(condition)"
description: 统计满足条件的日志数
example: "* | SELECT count_if(cast(status as BIGINT) >= 500) AS error_count"
- name: arbitrary
syntax: "arbitrary(x)"
description: 返回任意一个非空值
example: "* | SELECT status, arbitrary(request_time) GROUP BY status"
note: 用于GROUP BY时获取非分组字段的值
category: approximate_functions
name: 估算函数
description: 基于数据预测或填充缺失值的近似计算
support:
sql: true
spl: false
functions:
- name: approx_distinct
syntax: "approx_distinct(x)"
description: 估算唯一值的个数,使用HyperLogLog算法
returns: 近似计数
example: "* | SELECT approx_distinct(client_ip) AS uv"
note: 比count(distinct x)更快,但是近似值
- name: approx_percentile
syntax: "approx_percentile(x, percentage)"
description: 计算近似百分位数
params:
- x: 列名
- percentage: 百分位,取值0~1
example: "* | SELECT approx_percentile(cast(request_time as double), 0.99) AS p99"
- name: approx_percentile (with array)
syntax: "approx_percentile(x, array[p1, p2,...])"
description: 同时计算多个百分位数
example: "* | SELECT approx_percentile(cast(request_time as double), array[0.5, 0.95, 0.99]) AS percentiles"
- name: numeric_histogram
syntax: "numeric_histogram(bucket_count, x)"
description: 按照bucket数量统计x列的近似直方图
params:
- bucket_count: 桶的数量
- x: 数值列
returns: Map类型,键为桶的代表值,值为该桶的近似计数
example: "* | SELECT numeric_histogram(10, cast(request_time as double))"
- name: numeric_histogram_u
syntax: "numeric_histogram_u(bucket_count, x)"
description: 按照bucket数量统计x列的近似直方图,返回多行格式
example: "* | SELECT numeric_histogram_u(10, cast(request_time as double))"
use_cases:
- 快速估算UV(独立访客)
- 计算性能指标的P50、P95、P99
- 生成数值分布直方图
- 大数据量下的快速统计
important_notes:
- 估算函数牺牲精度换取性能
- approx_distinct使用HyperLogLog算法,标准误差约2.3%
- approx_percentile误差在1%以内
- 适用于大数据量场景
category: array_functions
name: 数组函数和运算符
description: 对数组进行增删改查、遍历和转换操作
support:
sql: true
spl: partial
functions:
- name: array_distinct
syntax: "array_distinct(x)"
description: 删除数组中重复的元素
examples:
sql: "* | SELECT array_distinct(cast(json_parse(number) as array(bigint)))"
spl: "* | extend unique_arr = array_distinct(arr_field)"
- name: array_intersect
syntax: "array_intersect(x, y)"
description: 计算两个数组的交集
examples:
sql: "* | SELECT array_intersect(array[1,2,3,4,5], array[1,3,5,7])"
spl: "* | extend intersection = array_intersect(arr1, arr2)"
- name: array_union
syntax: "array_union(x, y)"
description: 计算两个数组的并集
examples:
sql: "* | SELECT array_union(array[1,2,3,4,5], array[1,3,5,7])"
note: 仅支持SQL
- name: array_except
syntax: "array_except(x, y)"
description: 计算两个数组的差集
examples:
sql: "* | SELECT array_except(array[1,2,3,4,5], array[1,3,5,7])"
spl: "* | extend diff = array_except(arr1, arr2)"
- name: array_join
syntax: "array_join(x, delimiter [, null_replacement])"
description: 使用指定连接符将数组元素拼接为字符串
params:
- x: 数组
- delimiter: 连接符
- null_replacement: 可选,用于替换null元素的字符串
examples:
sql: "* | SELECT array_join(array[null,'Log','Service'], ' ', 'Alicloud')"
spl: "* | extend joined = array_join(arr_field, ',')"
note: 返回结果最大1KB,超出会被截断
- name: array_max
syntax: "array_max(x)"
description: 获取数组中的最大值
examples:
sql: "* | SELECT array_max(try_cast(json_parse(number) as array(bigint))) AS max_number"
- name: array_min
syntax: "array_min(x)"
description: 获取数组中的最小值
examples:
sql: "* | SELECT array_min(try_cast(json_parse(number) as array(bigint))) AS min_number"
- name: array_position
syntax: "array_position(x, element)"
description: 获取指定元素的下标(从1开始),不存在返回0
examples:
sql: "* | SELECT array_position(array[49,45,47], 45)"
- name: array_remove
syntax: "array_remove(x, element)"
description: 删除数组中指定的元素
examples:
sql: "* | SELECT array_remove(array[49,45,47], 45)"
- name: array_sort
syntax: "array_sort(x)"
description: 对数组元素进行升序排序,null元素排在最后
examples:
sql: "* | SELECT array_sort(array['b','d',null,'c','a'])"
- name: cardinality
syntax: "cardinality(x)"
description: 计算数组中元素的个数
examples:
sql: "* | SELECT cardinality(cast(json_parse(number) as array(bigint)))"
- name: contains
syntax: "contains(x, element)"
description: 判断数组中是否包含指定元素
returns: boolean类型
examples:
sql: "* | SELECT contains(cast(json_parse(region) as array(varchar)), 'cn-beijing')"
- name: reverse
syntax: "reverse(x)"
description: 对数组中的元素进行反向排列
examples:
sql: "* | SELECT reverse(array[1,2,3,4,5])"
spl: "* | extend reversed = reverse(arr_field)"
- name: slice
syntax: "slice(x, start, length)"
description: 获取数组的子集
params:
- start: 索引开始位置(负数从末尾开始,正数从头部开始)
- length: 子集元素个数
examples:
sql: "* | SELECT slice(array[1,2,4,5,6,7,7], 3, 2)"
- name: filter
syntax: "filter(x, lambda_expression)"
description: 结合Lambda表达式过滤数组元素
examples:
sql: "* | SELECT filter(array[5,-6,null,7], x -> x > 0)"
spl: "* | extend filtered = filter(arr_field, x -> x > 0)"
- name: transform
syntax: "transform(x, lambda_expression)"
description: 将Lambda表达式应用到数组的每个元素
examples:
sql: "* | SELECT transform(array[5,6], x -> x + 1)"
spl: "* | extend transformed = transform(arr_field, x -> x * 2)"
- name: reduce
syntax: "reduce(x, lambda_expression)"
description: 根据Lambda表达式对数组元素进行累加计算
examples:
sql: "* | SELECT reduce(array[5,20,50], 0, (s, x) -> s + x, s -> s)"
- name: sequence
syntax: "sequence(x, y [, step])"
description: 返回起始值范围内连续递增的数组
params:
- x: 起始值
- y: 结束值
- step: 可选,递增间隔(默认为1)
examples:
sql: "* | SELECT sequence(0, 10, 2)"
spl: "* | extend seq = sequence(1, 100)"
- name: zip
syntax: "zip(x, y...)"
description: 将多个数组合并为二维数组
examples:
sql: "* | SELECT zip(array[1,2,3], array['1b',null,'3b'], array[1,2,3])"
important_notes:
- 数组下标从1开始
- array_join返回结果最大1KB
- 使用Lambda表达式可以实现复杂的数组处理逻辑
- 配合cast和json_parse处理JSON格式的数组字段
category: binary_functions
name: 二进制函数
description: 处理二进制类型的数据,进行编码和解码
support:
sql: true
spl: partial
functions:
- name: from_base64
syntax: "from_base64(x)"
description: 对Base64编码的字符串进行解码
returns: varbinary类型
example: "* | SELECT from_base64('aGVsbG8=')"
- name: to_base64
syntax: "to_base64(x)"
description: 将二进制数据编码为Base64字符串
returns: varchar类型
example: "* | SELECT to_base64(cast('hello' as varbinary))"
- name: from_hex
syntax: "from_hex(x)"
description: 将十六进制字符串转换为二进制
example: "* | SELECT from_hex('68656C6C6F')"
- name: to_hex
syntax: "to_hex(x)"
description: 将二进制数据转换为十六进制字符串
example: "* | SELECT to_hex(cast('hello' as varbinary))"
- name: from_big_endian_64
syntax: "from_big_endian_64(x)"
description: 将大端序的8字节二进制转为bigint
example: "* | SELECT from_big_endian_64(from_hex('0000000000000001'))"
- name: to_big_endian_64
syntax: "to_big_endian_64(x)"
description: 将bigint转为大端序的8字节二进制
example: "* | SELECT to_big_endian_64(1)"
- name: md5
syntax: "md5(x)"
description: 计算MD5哈希值,返回二进制
example: "* | SELECT to_hex(md5(cast('hello' as varbinary)))"
- name: sha1
syntax: "sha1(x)"
description: 计算SHA1哈希值,返回二进制
example: "* | SELECT to_hex(sha1(cast('hello' as varbinary)))"
- name: sha256
syntax: "sha256(x)"
description: 计算SHA256哈希值,返回二进制
example: "* | SELECT to_hex(sha256(cast('hello' as varbinary)))"
- name: sha512
syntax: "sha512(x)"
description: 计算SHA512哈希值,返回二进制
example: "* | SELECT to_hex(sha512(cast('hello' as varbinary)))"
use_cases:
- Base64编解码
- 哈希值计算
- 二进制数据处理
- 数据校验
important_notes:
- 配合to_hex可以将二进制结果转为可读的十六进制
- 哈希函数返回二进制,通常需要to_hex转换
- 注意cast类型转换
category: bitwise_functions
name: 位运算函数
description: 直接操作二进制位的运算函数
support:
sql: true
spl: false
functions:
- name: bit_count
syntax: "bit_count(x, bits)"
description: 统计二进制表示中1的个数
params:
- x: bigint类型的数值
- bits: 位数(32或64)
example: "* | SELECT bit_count(5, 64)"
- name: bitwise_and
syntax: "bitwise_and(x, y)"
description: 按位与运算
example: "* | SELECT bitwise_and(5, 3)"
- name: bitwise_or
syntax: "bitwise_or(x, y)"
description: 按位或运算
example: "* | SELECT bitwise_or(5, 3)"
- name: bitwise_xor
syntax: "bitwise_xor(x, y)"
description: 按位异或运算
example: "* | SELECT bitwise_xor(5, 3)"
- name: bitwise_not
syntax: "bitwise_not(x)"
description: 按位取反运算
example: "* | SELECT bitwise_not(5)"
- name: bitwise_left_shift
syntax: "bitwise_left_shift(x, n)"
description: 按位左移n位
example: "* | SELECT bitwise_left_shift(5, 2)"
- name: bitwise_right_shift
syntax: "bitwise_right_shift(x, n)"
description: 按位右移n位
example: "* | SELECT bitwise_right_shift(5, 1)"
- name: bitwise_right_shift_arithmetic
syntax: "bitwise_right_shift_arithmetic(x, n)"
description: 算术右移n位(保留符号位)
example: "* | SELECT bitwise_right_shift_arithmetic(-8, 2)"
use_cases:
- 权限位掩码操作
- 标志位检查
- 位图运算
- 低级别数据处理
important_notes:
- 所有位运算函数参数必须为bigint类型
- 需要先cast转换为bigint
- 位运算结果也是bigint类型
category: color_functions
name: 颜色函数
description: 颜色表示与转换,用于可视化展示
support:
sql: true
spl: false
functions:
- name: bar
syntax: "bar(x, width [, low, high])"
description: 生成ASCII条形图
params:
- x: 数值
- width: 条形图宽度
- low: 最小值(可选)
- high: 最大值(可选)
example: "* | SELECT request_time, bar(cast(request_time as double), 20) as bar"
- name: color
syntax: "color(string [, color])"
description: 为字符串添加颜色标记(ANSI颜色码)
params:
- string: 要着色的字符串
- color: 颜色名称(可选)
example: "* | SELECT color('ERROR', 'red')"
- name: render
syntax: "render(x, color)"
description: 使用指定颜色渲染布尔值
example: "* | SELECT render(cast(status as bigint) >= 400, 'red')"
- name: rgb
syntax: "rgb(red, green, blue)"
description: 根据RGB值创建颜色
params:
- red: 红色分量(0-255)
- green: 绿色分量(0-255)
- blue: 蓝色分量(0-255)
example: "* | SELECT rgb(255, 0, 0)"
use_cases:
- 控制台输出美化
- 日志级别着色
- 可视化标记
- ASCII图表
important_notes:
- 主要用于控制台输出
- 支持标准ANSI颜色
- 在Web界面可能不显示颜色
category: comparison_functions
name: 同比和环比函数
description: 计算时间序列数据的相对变化,用于同比环比分析
support:
sql: true
spl: false
functions:
- name: compare
syntax: "compare(x, n)"
description: 对比当前时间周期内的计算结果与n秒之前时间周期内的计算结果
params:
- x: double或long类型的计算结果
- n: 时间窗口,单位为秒。如3600(1小时)、86400(1天)、604800(1周)
returns: 数组格式 [当前结果, n秒前结果, 比值]
examples:
sql: "* | SELECT compare(PV, 86400) FROM (SELECT count(*) AS PV FROM log)"
note: 对比的时间必须相同,支持对比当前1小时与昨天同时段,不支持对比当前1小时与上1小时
- name: compare (multiple)
syntax: "compare(x, n1, n2, n3...)"
description: 对比当前时间周期与多个历史时间周期的计算结果
params:
- x: double或long类型的计算结果
- n1, n2, n3: 多个时间窗口,单位为秒
returns: 数组格式,包含当前结果和各历史时间点的结果及比值
examples:
sql: "* | SELECT status, request_method, compare(PV, 3600) FROM (SELECT status, request_method, count(*) AS PV FROM log GROUP BY status, request_method) GROUP BY status, request_method"
- name: ts_compare
syntax: "ts_compare(x, n)"
description: 对比当前时间周期内的计算结果与n秒之前时间周期内的计算结果,必须按照时间列分组
params:
- x: double或long类型的计算结果
- n: 时间窗口,单位为秒
returns: 数组格式 [当前结果, n秒前结果, 比值, n秒前的Unix时间戳]
examples:
sql: "* | SELECT time, ts_compare(PV, 86400) as diff FROM (SELECT count(*) as PV, date_trunc('hour', __time__) AS time FROM log GROUP BY time) GROUP BY time ORDER BY time"
note: 必须按照时间列进行分组(GROUP BY),不支持嵌套使用
important_notes:
- compare函数要求对比的时间周期必须相同
- ts_compare函数必须按照时间列进行GROUP BY
- compare和ts_compare不支持嵌套使用
- 返回结果为数组,可使用下标[1][2][3]获取具体值
use_cases:
- 对比今天与昨天同时段的访问量
- 计算周同比增长率
- 分析各时段流量趋势
- 监控关键指标的环比变化
category: conditional_functions
name: 条件函数
description: 条件判断和分支
support:
sql: true
spl: true
functions:
- name: if
syntax: "if(condition, true_value, false_value)"
description: 条件表达式
examples:
sql: "* | SELECT if(cast(status as BIGINT) >= 500, 'error', 'normal') AS level"
spl: "* | extend level = if(cast(status as BIGINT) >= 500, 'error', 'normal')"
- name: case
syntax: "CASE WHEN condition THEN result [...] ELSE default END"
description: 多条件分支
examples:
sql: |
* | SELECT CASE
WHEN cast(status as BIGINT) >= 500 THEN 'error'
WHEN cast(status as BIGINT) >= 400 THEN 'client_error'
ELSE 'success'
END AS status_type
spl: |
* | extend status_type = CASE
WHEN cast(status as BIGINT) >= 500 THEN 'error'
WHEN cast(status as BIGINT) >= 400 THEN 'client_error'
ELSE 'success'
END
- name: coalesce
syntax: "coalesce(value1, value2, ..., default)"
description: 返回第一个非NULL值
examples:
sql: "* | SELECT coalesce(user_name, user_id, 'anonymous') AS user"
spl: "* | extend user = coalesce(user_name, user_id, 'anonymous')"
- name: nullif
syntax: "nullif(value1, value2)"
description: 相等则返回NULL,否则返回第一个值
examples:
sql: "* | SELECT nullif(status, '')"
spl: "* | extend valid_status = nullif(status, '')"
category: conversion_functions
name: 单位换算函数
description: 换算数据量或时间间隔的单位
support:
sql: true
spl: false
functions:
- name: convert_data_size
syntax: "convert_data_size(size, unit)"
description: 将数据大小转换为指定单位
params:
- size: 数据大小(字节)
- unit: 目标单位(B、KB、MB、GB、TB、PB)
returns: 转换后的数据大小(保留两位小数)
example: "* | SELECT convert_data_size(cast(body_bytes_sent as double), 'MB') AS size_mb"
- name: format_duration
syntax: "format_duration(duration)"
description: 将毫秒数转换为可读的时间格式
params:
- duration: 时间长度(毫秒)
returns: 格式化的时间字符串(如 1d 2h 3m 4s)
example: "* | SELECT format_duration(cast(request_time as bigint) * 1000) AS duration"
use_cases:
- 数据大小可读化展示
- 时间间隔格式化
- 性能指标展示
- 报表生成
important_notes:
- convert_data_size输入单位为字节(Byte)
- format_duration输入单位为毫秒
- 返回结果为字符串类型
- 适合用于结果展示,不适合用于计算
examples:
- desc: 将响应大小从字节转换为MB
sql: "* | SELECT convert_data_size(cast(body_bytes_sent as double), 'MB')"
- desc: 格式化请求时间
sql: "* | SELECT format_duration(cast(request_time as bigint))"
category: datetime_functions
name: 日期时间函数
description: 处理日期和时间
support:
sql: true
spl: true
functions:
- name: date_format
syntax: "date_format(timestamp, format)"
description: 格式化时间戳
format_examples:
- "%Y-%m-%d": "2024-01-01"
- "%Y-%m-%d %H:%i:%s": "2024-01-01 12:30:45"
- "%H:%i": "12:30"
examples:
sql: "* | SELECT date_format(__time__, '%Y-%m-%d %H:%i:%s') AS time"
spl: "* | extend time_str = date_format(__time__, '%Y-%m-%d %H:%i:%s')"
note: __time__是日志时间戳(Unix秒)
- name: date_parse
syntax: "date_parse(string, format)"
description: 解析时间字符串为时间戳
examples:
sql: "* | SELECT date_parse(time_str, '%Y-%m-%d %H:%i:%s')"
spl: "* | extend timestamp = date_parse(time_str, '%Y-%m-%d %H:%i:%s')"
- name: from_unixtime
syntax: "from_unixtime(unix_timestamp)"
description: Unix时间戳转时间对象
examples:
sql: "* | SELECT from_unixtime(__time__)"
spl: "* | extend time_obj = from_unixtime(__time__)"
- name: to_unixtime
syntax: "to_unixtime(timestamp)"
description: 时间对象转Unix时间戳
examples:
sql: "* | SELECT to_unixtime(cast(time_str as TIMESTAMP))"
spl: "* | extend unix_ts = to_unixtime(cast(time_str as TIMESTAMP))"
- name: date_trunc
syntax: "date_trunc(unit, timestamp)"
description: 时间截断到指定粒度
units: [second, minute, hour, day, week, month, quarter, year]
examples:
sql: "* | SELECT date_trunc('hour', __time__) AS hour"
spl: "* | extend hour = date_trunc('hour', __time__)"
note: 常用于按时间分组统计
- name: current_timestamp
syntax: "current_timestamp"
description: 获取当前时间戳
examples:
sql: "* | SELECT current_timestamp"
spl: "* | extend now = current_timestamp"
category: encoding_functions
name: 编码解码函数
description: 数据编码和解码
support:
sql: true
spl: true
functions:
- name: url_encode
syntax: "url_encode(str)"
description: URL编码
examples:
sql: "* | SELECT url_encode(query)"
spl: "* | extend encoded_query = url_encode(query)"
- name: url_decode
syntax: "url_decode(str)"
description: URL解码
examples:
sql: "* | SELECT url_decode(encoded_param)"
spl: "* | extend decoded_param = url_decode(encoded_param)"
- name: base64_encode
syntax: "base64_encode(str)"
description: Base64编码
examples:
sql: "* | SELECT base64_encode(data)"
spl: "* | extend encoded_data = base64_encode(data)"
- name: base64_decode
syntax: "base64_decode(str)"
description: Base64解码
examples:
sql: "* | SELECT base64_decode(encoded_data)"
spl: "* | extend decoded_data = base64_decode(encoded_data)"
category: geo_functions
name: 地理函数
description: 地理位置分析与地图计算
support:
sql: true
spl: false
functions:
- name: geohash
syntax: "geohash(latitude, longitude)"
description: 将经纬度编码为geohash字符串
returns: varchar类型
example: "* | SELECT geohash(39.9075, 116.3972)"
- name: geohash (with precision)
syntax: "geohash(latitude, longitude, precision)"
description: 将经纬度编码为指定精度的geohash字符串
params:
- precision: 精度,取值1-12,默认为12
example: "* | SELECT geohash(39.9075, 116.3972, 8)"
- name: geohash_decode
syntax: "geohash_decode(geohash_string)"
description: 将geohash字符串解码为经纬度
returns: 包含latitude和longitude的Row类型
example: "* | SELECT geohash_decode('wx4g0s')"
use_cases:
- 地理位置编码
- 附近位置查询
- 地理聚类
- 热力图展示
important_notes:
- geohash精度越高,表示的区域越小
- 常用精度:6位约±0.61km,8位约±19m
- geohash具有前缀特性,前缀相同表示位置接近
category: geospatial_functions
name: 空间几何函数
description: 处理空间几何体和地理位置数据
support:
sql: true
spl: false
functions:
- name: ST_Point
syntax: "ST_Point(longitude, latitude)"
description: 根据经纬度创建一个点
example: "* | SELECT ST_Point(120.13, 30.26)"
- name: ST_GeometryFromText
syntax: "ST_GeometryFromText(wkt_string)"
description: 从WKT(Well-Known Text)字符串创建几何对象
example: "* | SELECT ST_GeometryFromText('POINT(120.13 30.26)')"
- name: ST_AsText
syntax: "ST_AsText(geometry)"
description: 将几何对象转换为WKT字符串
example: "* | SELECT ST_AsText(ST_Point(120.13, 30.26))"
- name: ST_Contains
syntax: "ST_Contains(geometry1, geometry2)"
description: 判断geometry1是否完全包含geometry2
returns: boolean
example: "* | SELECT ST_Contains(ST_GeometryFromText('POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))'), ST_Point(5, 5))"
- name: ST_Distance
syntax: "ST_Distance(geometry1, geometry2)"
description: 计算两个几何对象之间的距离
example: "* | SELECT ST_Distance(ST_Point(120.13, 30.26), ST_Point(121.47, 31.23))"
- name: ST_Intersects
syntax: "ST_Intersects(geometry1, geometry2)"
description: 判断两个几何对象是否相交
returns: boolean
example: "* | SELECT ST_Intersects(ST_GeometryFromText('POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))'), ST_Point(5, 5))"
- name: ST_Boundary
syntax: "ST_Boundary(geometry)"
description: 返回几何对象的边界
example: "* | SELECT ST_Boundary(ST_GeometryFromText('POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))'))"
- name: ST_Buffer
syntax: "ST_Buffer(geometry, distance)"
description: 返回距离几何对象指定距离内的区域
example: "* | SELECT ST_Buffer(ST_Point(0, 0), 1.0)"
- name: ST_Area
syntax: "ST_Area(geometry)"
description: 计算几何对象的面积
example: "* | SELECT ST_Area(ST_GeometryFromText('POLYGON((0 0, 10 0, 10 10, 0 10, 0 0))'))"
use_cases:
- 地理围栏判断
- 距离计算
- 区域分析
- 空间查询
important_notes:
- 支持WKT格式的几何对象
- 坐标系统为WGS84
- 距离单位与坐标单位一致
category: hash_functions
name: 哈希函数
description: 计算哈希值
support:
sql: true
spl: true
functions:
- name: md5
syntax: "md5(str)"
description: 计算MD5哈希
returns: 32位十六进制字符串
examples:
sql: "* | SELECT md5(user_id)"
spl: "* | extend user_id_hash = md5(user_id)"
- name: sha1
syntax: "sha1(str)"
description: 计算SHA1哈希
examples:
sql: "* | SELECT sha1(token)"
spl: "* | extend token_hash = sha1(token)"
- name: sha256
syntax: "sha256(str)"
description: 计算SHA256哈希
examples:
sql: "* | SELECT sha256(token)"
spl: "* | extend token_hash = sha256(token)"
category: hyperloglog_functions
name: HyperLogLog函数
description: 对大数据集进行统计处理,牺牲精度以节省内存
support:
sql: true
spl: false
functions:
- name: approx_set
syntax: "approx_set(x)"
description: 将x的值转换为HyperLogLog对象
returns: HyperLogLog类型
example: "* | SELECT approx_set(client_ip)"
- name: cardinality
syntax: "cardinality(hll)"
description: 计算HyperLogLog对象中唯一值的估算数量
returns: bigint类型
example: "* | SELECT cardinality(approx_set(client_ip)) AS uv"
- name: empty_approx_set
syntax: "empty_approx_set()"
description: 创建一个空的HyperLogLog对象
example: "* | SELECT empty_approx_set()"
- name: merge
syntax: "merge(hll)"
description: 合并多个HyperLogLog对象
returns: HyperLogLog类型
example: "* | SELECT cardinality(merge(approx_set(client_ip))) FROM log GROUP BY status"
use_cases:
- UV(独立访客)统计
- 去重计数
- 多维度UV分析
- 实时大数据去重
important_notes:
- HyperLogLog是一种概率数据结构
- 标准误差约2.3%
- 内存占用固定约12KB
- 适合大数据量的去重统计
- 比count(distinct)更高效
related:
- approx_distinct函数是HyperLogLog的便捷封装
category: ip_geo_functions
name: IP地理位置函数
description: IP地址转换为地理位置
support:
sql: false
spl: true
note: 这些函数仅在SPL模式下可用
functions:
- name: ip_to_province
syntax: "ip_to_province(ip)"
description: IP转省份
returns: 省份名称(中文)
example: "* | extend province = ip_to_province(client_ip)"
- name: ip_to_city
syntax: "ip_to_city(ip)"
description: IP转城市
returns: 城市名称(中文)
example: "* | extend city = ip_to_city(client_ip)"
- name: ip_to_country
syntax: "ip_to_country(ip)"
description: IP转国家
returns: 国家名称(中文)
example: "* | extend country = ip_to_country(client_ip)"
- name: ip_to_geo
syntax: "ip_to_geo(ip)"
description: IP转经纬度
returns: 经纬度字符串(格式:纬度,经度)
example: "* | extend geo = ip_to_geo(client_ip)"
category: json_functions
name: JSON函数
description: 处理JSON数据
support:
sql: true
spl: true
functions:
- name: json_extract
syntax: "json_extract(json_string, json_path)"
description: 提取JSON值(返回JSON)
path_syntax:
- "$": 根节点
- "$.key": 对象属性
- "$[index]": 数组索引
- "$[*]": 数组所有元素
examples:
sql: "* | SELECT json_extract(payload, '$.user.id')"
spl: "* | extend user_id = json_extract(payload, '$.user.id')"
- name: json_extract_scalar
syntax: "json_extract_scalar(json_string, json_path)"
description: 提取JSON标量值(返回字符串)
examples:
sql: "* | SELECT json_extract_scalar(payload, '$.user.name') AS user_name"
spl: "* | extend user_name = json_extract_scalar(payload, '$.user.name')"
note: 提取后可能需要cast转换为其他类型
category: lambda_expressions
name: Lambda表达式
description: 定义Lambda表达式并传递给指定函数,丰富函数表达
support:
sql: true
spl: partial
functions:
- name: filter
syntax: "filter(array, lambda_expression)"
description: 过滤数组元素,只保留满足条件的元素
lambda_syntax: "x -> condition"
example: "* | SELECT filter(array[5, -6, null, 7], x -> x > 0)"
spl_example: "* | extend filtered = filter(arr, x -> x > 0)"
- name: transform
syntax: "transform(array, lambda_expression)"
description: 对数组每个元素应用转换
lambda_syntax: "x -> expression"
example: "* | SELECT transform(array[5, 6], x -> x + 1)"
spl_example: "* | extend transformed = transform(arr, x -> x * 2)"
- name: reduce
syntax: "reduce(array, initial_value, combine_function, final_function)"
description: 累加计算数组元素
lambda_syntax: "(accumulator, element) -> expression"
example: "* | SELECT reduce(array[5, 20, 50], 0, (s, x) -> s + x, s -> s)"
- name: any_match
syntax: "any_match(array, lambda_expression)"
description: 判断数组中是否有元素满足条件
returns: boolean
example: "* | SELECT any_match(array[1, 2, 3], x -> x > 2)"
- name: all_match
syntax: "all_match(array, lambda_expression)"
description: 判断数组中所有元素是否都满足条件
returns: boolean
example: "* | SELECT all_match(array[1, 2, 3], x -> x > 0)"
- name: map_filter
syntax: "map_filter(map, lambda_expression)"
description: 过滤Map中的键值对
lambda_syntax: "(key, value) -> condition"
example: "* | SELECT map_filter(map(array[1,2,3], array['a','b','c']), (k, v) -> k > 1)"
- name: zip_with
syntax: "zip_with(array1, array2, lambda_expression)"
description: 将两个数组元素两两组合并计算
lambda_syntax: "(x, y) -> expression"
example: "* | SELECT zip_with(array[1, 2], array[3, 4], (x, y) -> x + y)"
lambda_syntax:
single_parameter:
format: "x -> expression"
example: "x -> x * 2"
description: 单参数Lambda表达式
multiple_parameters:
format: "(x, y, ...) -> expression"
example: "(x, y) -> x + y"
description: 多参数Lambda表达式
complex_expression:
format: "x -> complex_expression"
example: "x -> if(x > 0, x, 0)"
description: 复杂条件表达式
use_cases:
- 数组元素过滤
- 数组元素转换
- 累加计算
- 复杂条件判断
- Map数据处理
important_notes:
- Lambda表达式是匿名函数
- 参数名可以自定义(如x, y, element等)
- 表达式可以包含条件、运算等
- 配合数组和Map函数使用
- SPL中部分支持Lambda表达式
examples:
- desc: 过滤正数
sql: "* | SELECT filter(array[5, -6, 7], x -> x > 0)"
- desc: 元素翻倍
sql: "* | SELECT transform(array[1, 2, 3], x -> x * 2)"
- desc: 计算数组和
sql: "* | SELECT reduce(array[1, 2, 3], 0, (s, x) -> s + x, s -> s)"
- desc: 过滤Map
sql: "* | SELECT map_filter(my_map, (k, v) -> v is not null)"
category: map_functions
name: Map映射函数和运算符
description: 操作键值对数据结构,进行Map的创建、查询、合并等操作
support:
sql: true
spl: partial
functions:
- name: map
syntax: "map() 或 map(x, y)"
description: 返回空Map或将两个数组映射为Map
params:
- x: 键数组(可选)
- y: 值数组(可选)
examples:
sql: "* | SELECT map()"
sql_with_arrays: "* | SELECT map(try_cast(json_parse(class) AS array(varchar)), try_cast(json_parse(number) AS array(bigint)))"
spl: "* | extend mapped = map(keys_arr, values_arr)"
- name: element_at
syntax: "element_at(x, key)"
description: 获取Map中指定键的值
examples:
sql: "* | SELECT element_at(histogram(request_method), 'DELETE') AS count"
spl: "* | extend value = element_at(map_field, 'key1')"
- name: cardinality
syntax: "cardinality(x)"
description: 计算Map的大小(键值对数量)
examples:
sql: "* | SELECT cardinality(histogram(request_method)) AS kinds"
- name: map_keys
syntax: "map_keys(x)"
description: 提取Map中所有的键,以数组形式返回
examples:
sql: "* | SELECT map_keys(try_cast(json_parse(etl_context) AS map(varchar, varchar)))"
spl: "* | extend keys = map_keys(map_field)"
- name: map_values
syntax: "map_values(x)"
description: 提取Map中所有键的值,以数组形式返回
examples:
sql: "* | SELECT map_values(try_cast(json_parse(etl_context) AS map(varchar, varchar)))"
spl: "* | extend values = map_values(map_field)"
- name: map_concat
syntax: "map_concat(x, y...)"
description: 将多个Map合并为一个Map
examples:
sql: "* | SELECT map_concat(cast(json_parse(etl_context) AS map(varchar, varchar)), cast(json_parse(progress) AS map(varchar, varchar)))"
spl: "* | extend merged = map_concat(map1, map2)"
- name: map_filter
syntax: "map_filter(x, lambda_expression)"
description: 结合Lambda表达式过滤Map中的元素
examples:
sql: "* | SELECT map_filter(map(array[10, 20, 30], array['a', NULL, 'c']), (k, v) -> v is not null)"
spl: "* | extend filtered = map_filter(map_field, (k, v) -> v > 100)"
- name: histogram
syntax: "histogram(x)"
description: 对数据进行分组,返回JSON格式的Map
examples:
sql: "* | SELECT histogram(request_method) AS request_method"
note: 类似于GROUP BY,返回Map格式
- name: histogram_u
syntax: "histogram_u(x)"
description: 对数据进行分组,返回多行多列格式
examples:
sql: "* | SELECT histogram_u(request_method) as request_method"
note: 适合用柱状图展示结果
- name: map_agg
syntax: "map_agg(x, y)"
description: 将x和y映射为Map,x为键,y为键值。当y有多个值时随机选一个
examples:
sql: "* | SELECT map_agg(request_method, request_time)"
- name: multimap_agg
syntax: "multimap_agg(x, y)"
description: 将x和y映射为Map,x为键,y为键值(数组格式)。当y有多个值时全部保留
examples:
sql: "* | SELECT multimap_agg(request_method, request_time)"
use_cases:
- 分析请求方法分布
- 统计状态码出现次数
- 合并多个配置Map
- 从JSON字段提取Map结构
- 对Map进行条件过滤
important_notes:
- 使用下标运算符[key]可直接获取值
- histogram函数适合快速分组统计
- 配合cast和json_parse处理JSON格式的Map字段
- Map的键和值类型需要一致
category: math_functions
name: 数学函数
description: 数学运算
support:
sql: true
spl: true
functions:
- name: abs
syntax: "abs(x)"
description: 绝对值
examples:
sql: "* | SELECT abs(cast(value1 as BIGINT) - cast(value2 as BIGINT))"
spl: "* | extend diff_abs = abs(cast(value1 as BIGINT) - cast(value2 as BIGINT))"
- name: round
syntax: "round(x [, decimals])"
description: 四舍五入
examples:
sql: "* | SELECT round(cast(price as DOUBLE), 2)"
spl: "* | extend price_rounded = round(cast(price as DOUBLE), 2)"
- name: floor
syntax: "floor(x)"
description: 向下取整
examples:
sql: "* | SELECT floor(cast(value as DOUBLE))"
spl: "* | extend value_floor = floor(cast(value as DOUBLE))"
- name: ceil
syntax: "ceil(x)"
description: 向上取整
examples:
sql: "* | SELECT ceil(cast(value as DOUBLE))"
spl: "* | extend value_ceil = ceil(cast(value as DOUBLE))"
- name: power
syntax: "power(x, y)"
description: 幂运算 x^y
examples:
sql: "* | SELECT power(2, 10)"
spl: "* | extend result = power(2, 10)"
- name: sqrt
syntax: "sqrt(x)"
description: 平方根
examples:
sql: "* | SELECT sqrt(cast(value as DOUBLE))"
spl: "* | extend value_sqrt = sqrt(cast(value as DOUBLE))"
- name: mod
syntax: "mod(x, y)"
description: 取模运算
examples:
sql: "* | SELECT mod(cast(count as BIGINT), 2)"
spl: "* | extend is_even = mod(cast(count as BIGINT), 2)"
category: mobile_functions
name: 电话号码函数
description: 分析中国内地地域电话号码的归属地、运营商等信息
support:
sql: true
spl: false
functions:
- name: mobile_carrier
syntax: "mobile_carrier(mobile_number)"
description: 获取手机号码的运营商
returns: varchar类型(中国移动、中国联通、中国电信)
example: "* | SELECT mobile_carrier('13900000000')"
- name: mobile_city
syntax: "mobile_city(mobile_number)"
description: 获取手机号码的归属城市
returns: varchar类型
example: "* | SELECT mobile_city('13900000000')"
- name: mobile_province
syntax: "mobile_province(mobile_number)"
description: 获取手机号码的归属省份
returns: varchar类型
example: "* | SELECT mobile_province('13900000000')"
use_cases:
- 用户地域分析
- 运营商分布统计
- 号码归属地查询
- 区域用户画像
important_notes:
- 仅支持中国内地手机号码
- 号码格式需为11位数字字符串
- 数据库会定期更新
- 对于无效号码返回NULL
category: operators
name: 比较与逻辑运算符
description: 判断参数的大小关系,组合多个布尔条件
support:
sql: true
spl: true
comparison_operators:
- name: "="
description: 等于
example: "status = 200"
- name: "!= 或 <>"
description: 不等于
example: "status != 200"
- name: "<"
description: 小于
example: "cast(request_time as bigint) < 100"
- name: "<="
description: 小于等于
example: "cast(request_time as bigint) <= 100"
- name: ">"
description: 大于
example: "cast(request_time as bigint) > 100"
- name: ">="
description: 大于等于
example: "cast(request_time as bigint) >= 100"
- name: BETWEEN
syntax: "x BETWEEN min AND max"
description: 判断x是否在min和max之间(包含边界)
example: "* | SELECT * WHERE cast(status as bigint) BETWEEN 200 AND 299"
- name: IN
syntax: "x IN (value1, value2, ...)"
description: 判断x是否在值列表中
example: "* | SELECT * WHERE status IN ('200', '201', '204')"
- name: IS NULL
description: 判断是否为NULL
example: "* | SELECT * WHERE user_name IS NULL"
- name: IS NOT NULL
description: 判断是否不为NULL
example: "* | SELECT * WHERE user_name IS NOT NULL"
- name: LIKE
syntax: "x LIKE pattern"
description: 字符串模式匹配,%匹配任意字符,_匹配单个字符
example: "* | SELECT * WHERE request_uri LIKE '/api/%'"
logical_operators:
- name: AND
description: 逻辑与,所有条件都为true时返回true
example: "cast(status as bigint) >= 200 AND cast(status as bigint) < 300"
- name: OR
description: 逻辑或,任一条件为true时返回true
example: "status = '404' OR status = '500'"
- name: NOT
description: 逻辑非,对布尔值取反
example: "NOT status = '200'"
important_notes:
- 字符串比较区分大小写
- NULL参与的比较运算结果为NULL
- 使用IS NULL / IS NOT NULL判断NULL值
- SPL中字段默认为VARCHAR类型,数值比较前需要cast
- 组合多个条件时注意运算符优先级,必要时使用括号
use_cases:
- 过滤特定状态码的日志
- 筛选特定时间范围的数据
- 组合多个条件查询
- NULL值处理
# SLS 函数参考索引
description: |
本文件是当前目录下函数文档的索引。
当前目录包含 SLS SQL/SPL 查询支持的所有函数,按功能分类存放。
需要具体函数详情时,请查阅对应的 YAML 文件。
# 重要说明
important_notes:
- SPL中字段默认为VARCHAR类型,数值运算前必须用cast()或try_cast()转换
- SQL语法:* | SELECT ... (使用SELECT、WHERE、GROUP BY)
- SPL语法:* | extend/where/stats ... (使用extend、where、stats)
- 字符串必须用单引号'',双引号""或无引号表示字段名
- 正则表达式中反斜杠不需要双重转义,\d 直接写 \d
# 函数分类目录
function_files:
- file: aggregate.yaml
name: 聚合函数
functions: [count, sum, avg, max, min, count_if, arbitrary]
support: SQL
description: 数据统计和汇总,配合GROUP BY使用
- file: string.yaml
name: 字符串函数
functions: [concat, substr, lower, upper, trim, length, split, replace, position]
support: SQL + SPL
description: 文本处理、拼接、截取、查找替换
- file: regex.yaml
name: 正则表达式函数
functions: [regexp_like, regexp_extract, regexp_replace]
support: SQL + SPL
description: 正则匹配、提取、替换(SPL使用RE2引擎)
- file: datetime.yaml
name: 日期时间函数
functions: [date_format, date_parse, date_trunc, from_unixtime, to_unixtime, current_timestamp]
support: SQL + SPL
description: 时间格式化、解析、截断、转换
- file: type_conversion.yaml
name: 类型转换函数
functions: [cast, try_cast]
support: SQL + SPL
description: 数据类型转换(必须转换后才能进行数值运算)
- file: conditional.yaml
name: 条件函数
functions: [if, case, coalesce, nullif]
support: SQL + SPL
description: 条件判断、分支逻辑、NULL处理
- file: json.yaml
name: JSON函数
functions: [json_extract, json_extract_scalar]
support: SQL + SPL
description: 从JSON字符串提取数据
- file: math.yaml
name: 数学函数
functions: [abs, round, floor, ceil, power, sqrt, mod]
support: SQL + SPL
description: 数学运算和计算
- file: url.yaml
name: URL函数
functions: [url_extract_host, url_extract_path, url_extract_parameter, url_extract_protocol]
support: SQL + SPL
description: URL解析和参数提取
- file: ip_geo.yaml
name: IP地理位置函数
functions: [ip_to_province, ip_to_city, ip_to_country, ip_to_geo]
support: 仅SPL
description: IP地址转地理位置(省份、城市、国家、经纬度)
- file: encoding.yaml
name: 编码解码函数
functions: [url_encode, url_decode, base64_encode, base64_decode]
support: SQL + SPL
description: URL和Base64编码解码
- file: hash.yaml
name: 哈希函数
functions: [md5, sha1, sha256]
support: SQL + SPL
description: 哈希值计算
- file: comparison.yaml
name: 同比和环比函数
functions: [compare, ts_compare]
support: SQL + SPL
description: 计算时间序列数据的相对变化,用于同比环比分析
- file: array.yaml
name: 数组函数和运算符
functions: [array_distinct, array_intersect, array_join, array_union, reverse, slice, contains]
support: SQL + SPL
description: 对数组进行增删改查、遍历和转换
- file: map.yaml
name: Map映射函数和运算符
functions: [cardinality, element_at, histogram, histogram_u, map, map_keys, map_values]
support: SQL + SPL
description: 操作键值对数据结构
- file: statistical.yaml
name: 数学统计函数
functions: [corr, covar_pop, covar_samp, stddev, variance, regr_intercept, regr_slope]
support: SQL
description: 数据分布分析、相关性分析与数值统计计算
- file: window.yaml
name: 窗口函数
functions: [row_number, rank, dense_rank, lag, lead, first_value, last_value]
support: SQL
description: 基于数据窗口的聚合、排序和分析
- file: approximate.yaml
name: 估算函数
functions: [approx_distinct, approx_percentile, numeric_histogram, numeric_histogram_u]
support: SQL
description: 基于数据预测或填充缺失值的近似计算
- file: binary.yaml
name: 二进制函数
functions: [from_base64, to_base64, from_hex, to_hex, from_big_endian_64, to_big_endian_64]
support: SQL + SPL
description: 处理二进制类型的数据
- file: bitwise.yaml
name: 位运算函数
functions: [bit_count, bitwise_and, bitwise_or, bitwise_xor, bitwise_not]
support: SQL
description: 直接操作二进制位
- file: geospatial.yaml
name: 空间几何函数
functions: [ST_AsText, ST_GeometryFromText, ST_Point, ST_Contains, ST_Distance, ST_Intersects]
support: SQL
description: 处理空间几何体和地理位置数据
- file: geo.yaml
name: 地理函数
functions: [geohash, geohash_decode]
support: SQL
description: 地理位置分析与地图计算
- file: color.yaml
name: 颜色函数
functions: [bar, color, render, rgb]
support: SQL
description: 颜色表示与转换,用于可视化展示
- file: hyperloglog.yaml
name: HyperLogLog函数
functions: [approx_set, cardinality, empty_approx_set, merge]
support: SQL
description: 对大数据集进行统计处理,牺牲精度以节省内存
- file: mobile.yaml
name: 电话号码函数
functions: [mobile_carrier, mobile_city, mobile_province]
support: SQL
description: 分析中国内地地域电话号码的归属地、运营商等信息
- file: operators.yaml
name: 比较与逻辑运算符
functions: [AND, OR, NOT, BETWEEN, IN, LIKE, IS NULL, IS NOT NULL]
support: SQL + SPL
description: 判断参数的大小关系,组合多个布尔条件
- file: conversion.yaml
name: 单位换算函数
functions: [convert_data_size, format_duration]
support: SQL
description: 换算数据量或时间间隔的单位
- file: window_funnel.yaml
name: 窗口漏斗函数
functions: [window_funnel]
support: SQL
description: 分析用户行为、APP流量、产品目标转化等数据
- file: lambda.yaml
name: Lambda表达式
functions: [filter, reduce, transform, any_match, all_match]
support: SQL + SPL
description: 定义Lambda表达式并传递给指定函数,丰富函数表达
# 常见场景快速查找
quick_lookup:
数据统计: aggregate.yaml
文本处理: string.yaml
正则匹配: regex.yaml
时间处理: datetime.yaml
类型转换: type_conversion.yaml
条件判断: conditional.yaml
JSON解析: json.yaml
数值计算: math.yaml
URL解析: url.yaml
IP分析: ip_geo.yaml
编码解码: encoding.yaml
哈希计算: hash.yaml
同比环比: comparison.yaml
数组操作: array.yaml
Map操作: map.yaml
统计分析: statistical.yaml
窗口分析: window.yaml
近似计算: approximate.yaml
二进制处理: binary.yaml
位运算: bitwise.yaml
空间几何: geospatial.yaml
地理分析: geo.yaml
颜色处理: color.yaml
大数据统计: hyperloglog.yaml
手机号分析: mobile.yaml
逻辑运算: operators.yaml
单位转换: conversion.yaml
漏斗分析: window_funnel.yaml
Lambda表达式: lambda.yaml
# 关键使用提示
usage_tips:
类型转换: 数值比较必须先cast,如 cast(status as BIGINT) >= 500
安全转换: 用try_cast避免转换失败,如 try_cast(value as DOUBLE)
时间分组: 用date_trunc按时间粒度统计,如 date_trunc('hour', __time__)
GROUP BY: SELECT只能包含分组字段或聚合函数,非分组字段用arbitrary()
NULL处理: 用coalesce提供默认值,如 coalesce(user_name, 'anonymous')
# 相关文档
related_docs:
- ../query_analysis/sql.yaml: SQL查询完整语法
- ../spl/overview.yaml: SPL查询基础语法
- ../query_analysis/indexSearch.yaml: 索引查询和关键字搜索
SLS 函数参考文档
本目录包含 SLS SQL 和 SPL 分析语句支持的所有函数,按功能分类。
目录结构
| 文件 | 类别 | 描述 | 支持 |
|---|---|---|---|
aggregate.yaml | 聚合函数 | count、sum、avg、max、min 等统计函数 | SQL |
string.yaml | 字符串函数 | 文本拼接、截取、大小写转换、查找替换 | SQL + SPL |
regex.yaml | 正则表达式函数 | 正则匹配、提取、替换 | SQL + SPL |
datetime.yaml | 日期时间函数 | 时间格式化、解析、截断、转换 | SQL + SPL |
type_conversion.yaml | 类型转换函数 | cast、try_cast 类型转换 | SQL + SPL |
conditional.yaml | 条件函数 | if、case、coalesce 条件判断 | SQL + SPL |
json.yaml | JSON函数 | JSON 数据提取和解析 | SQL + SPL |
math.yaml | 数学函数 | 数值计算、取整、幂运算 | SQL + SPL |
url.yaml | URL函数 | URL 解析和参数提取 | SQL + SPL |
ip_geo.yaml | IP地理位置函数 | IP 转省份、城市、国家、经纬度 | 仅SPL |
encoding.yaml | 编码解码函数 | URL、Base64 编码解码 | SQL + SPL |
hash.yaml | 哈希函数 | MD5、SHA1、SHA256 哈希计算 | SQL + SPL |
使用说明
1. 查找函数
- 按功能查找:根据上表选择对应的分类文件
- 按名称查找:在对应分类的 YAML 文件中查找具体函数
- 按场景查找:参考
overview.yaml中的常见场景示例
2. 查看函数详情
每个 YAML 文件包含以下信息:
functions:
- name: 函数名
syntax: 函数语法
description: 功能描述
examples:
sql: SQL 示例
spl: SPL 示例
note: 注意事项(可选)3. 重要提示
类型转换
- 字段默认为 VARCHAR 类型
- 数值比较和运算前必须使用
cast()或try_cast()转换 - SPL 中尤其需要注意类型转换
示例:
-- ✅ 正确
* | SELECT * WHERE cast(status as BIGINT) >= 500
-- ❌ 错误
* | SELECT * WHERE status >= 500SQL vs SPL
- SQL:使用 SELECT、WHERE、GROUP BY 语法
- SPL:使用 extend、where、stats 语法
- 聚合函数主要用于 SQL
- IP地理函数仅用于 SPL
正则表达式
- SPL 使用 RE2 正则引擎
- 不支持:后向引用(\1)、环视(?<=...)等
- 反斜杠不需要双重转义:
\d直接写\d
快速示例
统计分析
-- 按状态码统计
* | SELECT status, count(*) AS pv GROUP BY status ORDER BY pv DESC时间分组
-- 按小时统计
* | SELECT date_trunc('hour', __time__) AS hour, count(*) AS pv GROUP BY hour正则提取
-- 提取错误码
* | SELECT regexp_extract(message, 'code:(\d+)', 1) AS error_code, count(*) GROUP BY error_codeJSON 解析
-- 提取 JSON 字段
* | SELECT json_extract_scalar(payload, '$.user.name') AS user, count(*) GROUP BY userIP 地理分析(SPL)
# 按省份统计
* | extend province = ip_to_province(client_ip) | stats pv = count(*) by province相关文档
- 函数索引 - 函数分类索引和使用指南
- SQL 查询语法 - SQL 查询完整语法
- SPL 基础语法 - SPL 查询基础
- 索引查询 - 关键字搜索语法
category: regex_functions
name: 正则表达式函数
description: 使用正则表达式处理字符串
support:
sql: true
spl: true
notes:
- SPL使用RE2正则引擎,某些正则特性受到限制,RE2不支持的特性包括:后向引用(\1)、环视(?<=...)、贪婪模式修饰符等,建议使用RE2兼容的正则表达式语法
- 特别注意,正则表达式中的反斜杠不需要双重转义,比如\d 直接写 \d即可,不需要写成\\d
functions:
- name: regexp_like
syntax: "regexp_like(x, pattern)"
description: 检查是否匹配正则, 正则表达式中的反斜杠不需要双重转义,比如\d 直接写 \d即可
returns: 布尔值
examples:
sql: "* | SELECT * WHERE regexp_like(message, 'ERROR|FATAL')"
spl: "* | where regexp_like(message, 'ERROR|FATAL')"
- name: regexp_extract
syntax: "regexp_extract(x, pattern [, group])"
description: 提取匹配的内容
params:
- group: 捕获组索引(默认0)
examples:
sql: "* | SELECT regexp_extract(message, 'code:(\d+)', 1) AS error_code"
spl: "* | extend error_code = regexp_extract(message, 'code:(\d+)', 1)"
- name: regexp_replace
syntax: "regexp_replace(x, pattern, replacement)"
description: 正则替换
examples:
sql: "* | SELECT regexp_replace(phone, '(\d{3})\d{4}(\d{4})', '$1****$2')"
spl: "* | extend phone_masked = regexp_replace(phone, '(\d{3})\d{4}(\d{4})', '$1****$2')"
category: statistical_functions
name: 数学统计函数
description: 数据分布分析、相关性分析与数值统计计算
support:
sql: true
spl: false
functions:
- name: corr
syntax: "corr(y, x)"
description: 计算两列的相关系数(皮尔逊相关系数)
returns: double类型,范围[-1, 1]
example: "* | SELECT corr(request_time, request_length)"
- name: covar_pop
syntax: "covar_pop(y, x)"
description: 计算两列的总体协方差
example: "* | SELECT covar_pop(request_time, request_length)"
- name: covar_samp
syntax: "covar_samp(y, x)"
description: 计算两列的样本协方差
example: "* | SELECT covar_samp(request_time, request_length)"
- name: stddev
syntax: "stddev(x)"
description: 计算标准差
example: "* | SELECT stddev(cast(request_time as double))"
- name: stddev_pop
syntax: "stddev_pop(x)"
description: 计算总体标准差
example: "* | SELECT stddev_pop(cast(request_time as double))"
- name: stddev_samp
syntax: "stddev_samp(x)"
description: 计算样本标准差
example: "* | SELECT stddev_samp(cast(request_time as double))"
- name: variance
syntax: "variance(x)"
description: 计算方差
example: "* | SELECT variance(cast(request_time as double))"
- name: var_pop
syntax: "var_pop(x)"
description: 计算总体方差
example: "* | SELECT var_pop(cast(request_time as double))"
- name: var_samp
syntax: "var_samp(x)"
description: 计算样本方差
example: "* | SELECT var_samp(cast(request_time as double))"
- name: regr_intercept
syntax: "regr_intercept(y, x)"
description: 计算线性回归的截距
example: "* | SELECT regr_intercept(cast(request_time as double), cast(body_bytes_sent as double))"
- name: regr_slope
syntax: "regr_slope(y, x)"
description: 计算线性回归的斜率
example: "* | SELECT regr_slope(cast(request_time as double), cast(body_bytes_sent as double))"
use_cases:
- 分析请求时间和响应大小的相关性
- 计算性能指标的波动情况
- 线性回归预测
- 数据质量评估
category: string_functions
name: 字符串函数
description: 处理文本数据
support:
sql: true
spl: true
functions:
- name: concat
syntax: "concat(x, y, ...)"
description: 拼接字符串
examples:
sql: "* | SELECT concat(region, '-', status)"
spl: "* | extend full_info = concat(region, '-', status)"
- name: substr
syntax: "substr(x, start [, length])"
description: 提取子字符串
params:
- start: 起始位置(从1开始)
- length: 提取长度(可选)
examples:
sql: "* | SELECT substr(message, 1, 100)"
spl: "* | extend short_msg = substr(message, 1, 100)"
- name: lower
syntax: "lower(x)"
description: 转换为小写
examples:
sql: "* | SELECT lower(method)"
spl: "* | extend method_lower = lower(method)"
- name: upper
syntax: "upper(x)"
description: 转换为大写
examples:
sql: "* | SELECT upper(method)"
spl: "* | extend method_upper = upper(method)"
- name: trim
syntax: "trim(x)"
description: 去除首尾空格
examples:
sql: "* | SELECT trim(user_name)"
spl: "* | extend user_clean = trim(user_name)"
- name: ltrim
syntax: "ltrim(x)"
description: 去除左侧空格
examples:
sql: "* | SELECT ltrim(user_name)"
spl: "* | extend user_clean = ltrim(user_name)"
- name: rtrim
syntax: "rtrim(x)"
description: 去除右侧空格
examples:
sql: "* | SELECT rtrim(user_name)"
spl: "* | extend user_clean = rtrim(user_name)"
- name: length
syntax: "length(x)"
description: 返回字符串长度
examples:
sql: "* | SELECT length(message) AS msg_len"
spl: "* | extend msg_len = length(message)"
- name: split
syntax: "split(x, delimiter)"
description: 按分隔符分割字符串
returns: 数组类型
examples:
sql: "* | SELECT split(tags, ',')"
spl: "* | extend tags_array = split(tags, ',')"
- name: replace
syntax: "replace(x, search, replacement)"
description: 替换字符串
examples:
sql: "* | SELECT replace(url, 'http://', 'https://')"
spl: "* | extend url_https = replace(url, 'http://', 'https://')"
- name: position
syntax: "position(substring in string)"
description: 返回子串位置
examples:
sql: "* | SELECT position('error' in message)"
spl: 暂时还没支持
- name: ascii_escape
syntax: "ascii_escape(x)"
description: 将字符串转换为ASCII码
examples:
sql: 暂时还没支持
spl: "* | extend message_ascii = ascii_escape(message)"
- name: ascii_unescape
syntax: "ascii_unescape(x)"
description: 执行ASCII码转义,支持\n、\r、\t、\b、\f、\xXX等转义
examples:
sql: 暂时还没支持
spl: "* | extend message_ascii = ascii_unescape(message)"
- name: unicode_unescape
syntax: "unicode_unescape(x)"
description: 执行Unicode码转义,支持\uXXXX等转义
examples:
sql: 暂时还没支持
spl: "* | extend message_unicode = unicode_unescape(message)" category: type_conversion_functions
name: 类型转换函数
description: 数据类型转换
support:
sql: true
spl: true
functions:
- name: cast
syntax: "cast(value as TYPE)"
description: 类型转换(失败抛出错误)
types: [BIGINT, INTEGER, DOUBLE, VARCHAR, BOOLEAN, TIMESTAMP]
examples:
sql: "* | SELECT cast(status as BIGINT) WHERE cast(status as BIGINT) >= 500"
spl: "* | extend status_num = cast(status as BIGINT) | where status_num >= 500"
note: 字段默认为VARCHAR,数值运算前必须转换。SPL中尤其需要注意类型转换
- name: try_cast
syntax: "try_cast(value as TYPE)"
description: 安全类型转换(失败返回NULL)
examples:
sql: "* | SELECT try_cast(status as BIGINT) AS status_num"
spl: "* | extend status_num = try_cast(status as BIGINT)"
note: 推荐使用,避免转换错误导致查询失败
category: url_functions
name: URL函数
description: 解析URL
support:
sql: true
spl: true
functions:
- name: url_extract_host
syntax: "url_extract_host(url)"
description: 提取主机名
examples:
sql: "* | SELECT url_extract_host(request_url)"
spl: "* | extend host = url_extract_host(request_url)"
- name: url_extract_path
syntax: "url_extract_path(url)"
description: 提取路径
examples:
sql: "* | SELECT url_extract_path(request_url)"
spl: "* | extend path = url_extract_path(request_url)"
- name: url_extract_parameter
syntax: "url_extract_parameter(url, param_name)"
description: 提取查询参数
examples:
sql: "* | SELECT url_extract_parameter(request_url, 'user_id')"
spl: "* | extend user_id = url_extract_parameter(request_url, 'user_id')"
- name: url_extract_protocol
syntax: "url_extract_protocol(url)"
description: 提取协议
examples:
sql: "* | SELECT url_extract_protocol(request_url)"
spl: "* | extend protocol = url_extract_protocol(request_url)"
category: window_funnel_function
name: 窗口漏斗函数
description: 分析用户行为、APP流量、产品目标转化等数据
support:
sql: true
spl: false
functions:
- name: window_funnel
syntax: "window_funnel(window, mode, timestamp, event1, event2, ...)"
description: 在指定时间窗口内分析用户行为序列,计算漏斗转化
params:
- window: 时间窗口大小(秒)
- mode: 模式,可选值为default(默认)、strict(严格模式)
- timestamp: 时间戳列
- event1, event2, ...: 事件条件(布尔表达式)
returns: 用户完成的最大事件级数
example: |
* | SELECT
window_funnel(
3600,
'default',
cast(__time__ as bigint),
action='page_view',
action='add_to_cart',
action='purchase'
) AS level,
count(*) AS user_count
FROM log
GROUP BY user_id
use_cases:
- 电商购买转化漏斗
- APP使用流程分析
- 注册转化分析
- 用户行为路径分析
mode_options:
default:
description: 默认模式,事件可以不连续,中间可以有其他事件
example: 用户可以在浏览、加购之间进行其他操作
strict:
description: 严格模式,事件必须连续发生,中间不能有其他事件
example: 用户必须依次完成浏览、加购,中间不能有其他操作
important_notes:
- 返回值范围为0到事件数量
- 返回值表示用户完成到第几个步骤
- 需要按用户ID分组
- 时间窗口单位为秒
- 事件需要按时间戳排序
examples:
- desc: 分析3个步骤的购买漏斗
sql: |
* | SELECT
window_funnel(1800, 'default', cast(__time__ as bigint),
page='product',
action='add_cart',
action='payment'
) AS step,
count(distinct user_id) AS users
GROUP BY step
ORDER BY step
category: window_functions
name: 窗口函数
description: 基于数据窗口的聚合、排序和分析
support:
sql: true
spl: false
functions:
- name: row_number
syntax: "row_number() OVER ([PARTITION BY col] ORDER BY col)"
description: 为每行分配唯一的行号
example: "* | SELECT request_time, row_number() OVER (ORDER BY request_time DESC) AS row_num"
- name: rank
syntax: "rank() OVER ([PARTITION BY col] ORDER BY col)"
description: 计算排名,相同值排名相同,后续排名跳跃
example: "* | SELECT status, count(*) as cnt, rank() OVER (ORDER BY count(*) DESC) AS rank FROM log GROUP BY status"
- name: dense_rank
syntax: "dense_rank() OVER ([PARTITION BY col] ORDER BY col)"
description: 计算排名,相同值排名相同,后续排名连续
example: "* | SELECT status, count(*) as cnt, dense_rank() OVER (ORDER BY count(*) DESC) AS rank FROM log GROUP BY status"
- name: lag
syntax: "lag(col [, offset] [, default]) OVER ([PARTITION BY col] ORDER BY col)"
description: 获取当前行之前第offset行的值
params:
- offset: 偏移量,默认为1
- default: 默认值,当偏移超出范围时返回
example: "* | SELECT time, pv, lag(pv, 1, 0) OVER (ORDER BY time) AS prev_pv FROM (SELECT date_trunc('hour', __time__) as time, count(*) as pv FROM log GROUP BY time)"
- name: lead
syntax: "lead(col [, offset] [, default]) OVER ([PARTITION BY col] ORDER BY col)"
description: 获取当前行之后第offset行的值
params:
- offset: 偏移量,默认为1
- default: 默认值,当偏移超出范围时返回
example: "* | SELECT time, pv, lead(pv, 1, 0) OVER (ORDER BY time) AS next_pv FROM (SELECT date_trunc('hour', __time__) as time, count(*) as pv FROM log GROUP BY time)"
- name: first_value
syntax: "first_value(col) OVER ([PARTITION BY col] ORDER BY col)"
description: 返回窗口中第一个值
example: "* | SELECT time, pv, first_value(pv) OVER (ORDER BY time) AS first_pv FROM (SELECT date_trunc('hour', __time__) as time, count(*) as pv FROM log GROUP BY time)"
- name: last_value
syntax: "last_value(col) OVER ([PARTITION BY col] ORDER BY col)"
description: 返回窗口中最后一个值
example: "* | SELECT time, pv, last_value(pv) OVER (ORDER BY time) AS last_pv FROM (SELECT date_trunc('hour', __time__) as time, count(*) as pv FROM log GROUP BY time)"
- name: ntile
syntax: "ntile(n) OVER ([PARTITION BY col] ORDER BY col)"
description: 将有序数据分为n个桶,返回当前行所在桶号
example: "* | SELECT request_time, ntile(4) OVER (ORDER BY request_time) AS quartile"
use_cases:
- TOP N查询
- 计算排名
- 环比分析(对比相邻时间段)
- 累计计算
- 移动平均
important_notes:
- 窗口函数必须配合OVER子句使用
- PARTITION BY用于分组,ORDER BY用于排序
- 窗口函数不能在WHERE子句中使用
- 可以在窗口函数中使用聚合函数
concept:
- name: 索引
description:
- 类似数据库索引的概念,不过SLS中字段索引需要开启后才能进行查询分析
- name: 索引配置
description:
- 索引配置由下面多个配置组成,包括下面的全文索引、字段所哟、分词、中文分词、字段类型
- name: 全文索引
description:
- 当需要对整个日志建立索引,并通过不指定字段的方式进行查询时候,需要开启全文索引
- name: 字段索引
description:
- 当需要对日志中的某个字段建立索引,并通过指定字段的方式进行查询和分析时候,需要开启字段索引
- 对于字段索引,有text、long、double和JSON四种类型
- name: 字段类型
description:
- 字段类型有text、long、double和JSON四种类型
- name: 分词与分词符
description:
- 分词符由一堆ASCII字符组成
- 分词的作用是将日志拆成多个term,分词的过程相当于是把内容按照预设的分词符(一组ASCII码)进行split
- 在建立索引的时候, 会根据分词符将日志拆成多个term, 然后建立term的倒排索引
- 在查询的时候,query会按照分词配置进行分词,然后匹配对应的term,然后查倒排索引,从而找到对应的日志
- 这里说的分词是指ASCII分词,中文分词有单独的开关
- name: 中文分词
description:
- 中文分词是针对中文的特殊处理,在索引配置有单独的开关
- 开启中文分词后索引的性能有所下降
- name: 开启统计
description:
- 开启统计后,才能进行SQL分析
- 目前 查询型logstore 不支持开启统计,因此也不支持跑SQL分析
- 对于 标准型logstore 开启统计分析没有额外的费用
- name: 统计字段(text)最大长度
description:
- 统计字段(text)最大长度是SQL分析时,默认截取一定长度,日志服务的默认配置为 2048 字节(2KB)。
- 可以在页面上调大到16KB, 调大索引字段最大长度没有额外的费用
- name: 重建索引
description:
- 索引配置提交后会对后面新写入的日志生效
- 如果希望对历史数据生效,需要使用重建索引功能
- name: 日志聚类(LogReduce)
description:
- 打开日志聚类开关后,日志服务在采集文本日志时会自动聚合相似度高的日志,提取共同的日志模式。
- name: 自动生成索引
description:
- 这个是前端提供的自动抽样一部分日志来推导出索引配置的功能,客户可以根据推导出的索引配置点击追加或者覆盖现有索引配置
- 由于自动生成索引的时候是抽样了一部分日志,所以不一定能推导出所有字段。客户可以按需再追剧新的字段
- name: 索引流量
description:
- 索引流量决定了索引的费用,按每GB为单位收费
- 如果开启了全文索引后,索引流量等同日志流量
- name: 开启索引后默认可查的字段
description:
- 开启索引后,默认会为 __time__、__topic__、__source__
- 高精度时间的纳秒部分如果需要在SQL中分析使用,需要额外加字段索引,`__time__ns_part__` 类型为bigint
api:
- name: CreateIndex
aliyun_doc: https://help.aliyun.com/zh/sls/developer-reference/create-index
description: 注意Index的创建需要传入全量的Index配置,不支持对单个字段进行操作
- name: DeleteIndex
aliyun_doc: https://help.aliyun.com/zh/sls/developer-reference/delete-index
- name: GetIndexConfig
aliyun_doc: https://help.aliyun.com/zh/sls/developer-reference/get-index-config
- name: UpdateIndex
aliyun_doc: https://help.aliyun.com/zh/sls/developer-reference/update-index
description: 注意Index的更新需要传入全量的Index配置,不支持对单个字段进行操作
faq:
- question: 之前 logstore 没有开启索引配置,后来才开启的,可以在控制台或 GetLogs 接口查到日志吗?
answer: 由于索引配置只对配置后写入的数据生效,所以之前没有开启索引配置的日志无法被查询到。如需查询历史日志,请使用重建索引功能。
- question: 即使在控制台不加任何查询或者分析条件,如果我想在控制台查到日志也要求开索引配置吗?
answer: 是的,即使在控制台不加任何查询或者分析条件,如果想在控制台查到日志也要求开索引配置。索引是查询和分析的前提条件
- question: 调大统计分析的字段长度(默认限制 2KB)调大到最大 16KB,会有什么影响吗?
answer: 调大后对新写入的数据生效,如果历史数据要生效可以重建索引。调大字段长度没有额外的费用
- question: 如何在日志中搜索包含空格的关键字?
answer:
- 一般情况下空格是一个分词符,可以通过SPL过滤的方式来处理,比如要查的是Hello World这个词可以这样写 "Hello World" | where content like '%Hello World%'
- 在| 前面加上Hello World过滤是为了减少SPL处理的数据量,加速查询过程
- question: 为什么查询和分析时,字段值会被截断?
answer:
- 查询时,单个字段值最大长度为 512 KB,超出部分不参与查询。
- 分析时,默认支持的字段值最大长度为 2 KB,最大可调整为 16 KB。
- 可以通过修改索引配置中的"统计字段(text)最大长度"来调整,该配置修改仅对新增采集的日志数据生效。
- question: 如何分析非索引字段?
answer:
- 如果是分析新写入的日志,则直接为目标字段创建索引且开启统计功能。
- 如果是分析历史日志,则需要对历史日志重建索引且开启统计功能。
- 如果无法创建索引,可以打开扫描(Scan)模式,通过扫描分析功能分析日志。
- question: 为什么我的字段索引看起来没有生效
answer:
- 可能的原因1, 字段索引刚配置,只能对新写入的数据生效,历史数据需要重建索引
- 另一个可能的原因是日志中的字段名和索引配置的字段名大小写可能不一致,比如配置的字段索引是Status,但是日志中的字段名是status
limitation:
- 索引配置的大小上限是64KBindex_search:
# 适用范围
target:
- logstore
- storeview
# 前置条件
prerequisites:
- requirement: 必须配置索引
description: 索引查询依赖索引配置,索引只对配置后写入的数据生效
scope: logstore级别
- requirement: 字段索引配置
description: 需要查询的字段必须在索引配置中开启(全文索引或字段索引)
note: 未配置索引的字段无法被查询
- 关于索引配置的问题,可以参考 `indexConfig.yaml`
# 费用说明
pricing:
free: true
description: 普通索引查询功能是免费的
# 整体限制
limitations:
- concurrent_queries: 100 # 单个Project最多并发查询数
- max_keywords: 128 # 每次查询最多关键词数(除布尔逻辑符外)
- page_size: 100 # 每页最多返回结果数
- field_value_size: "512 KB" # 单个字段值最大为512KB,超出部分不参与查询
- sort_order: "按时间倒序(秒级或纳秒级)"
- fuzzy_query_limit: 100 # 模糊查询最多查询到符合条件的100个词
- 索引查询不支持注释,如果写注释会被当作索引查询关键词的进行查询,最终可能导致结果错误
# 查询语法
query_syntax_base:
description: 查询语句用于指定日志查询时的过滤规则,返回符合条件的日志
basic_rules:
- 查询条件可使用关键词、数值、数值范围、空格、* 等
- 如果为空格或 *,表示无过滤条件
- 查询语句中建议不超过 128 个条件
- 支持布尔运算符(AND、OR、NOT)、括号分组、模糊查询、短语查询等
aliyun_doc: https://help.aliyun.com/zh/sls/query-syntax
query_syntax:
# 大小写是否敏感?
- type: case_sensitivity
description: 大小写不敏感
# 布尔运算
- type: and_or_not
description: 使用AND/OR/NOT组合条件
limitations:
- 最多30个条件(除布尔逻辑符外)
examples:
- query: "status: 200 and request_method: GET"
description: 查询status为200且request_method为GET的日志
# 范围查询
- type: range_query
description: 使用范围查询, field in [min max] 或者 field in (min max)
limitations:
- 只对索引字段为long或double的日志有效
examples:
- query: "status in [200 300]"
description: 查询status为200到300的日志
# 模糊查询
- type: fuzzy_query
description: 使用模糊查询, * 表示匹配任意字符, ? 表示匹配单个字符
limitations:
- 最多匹配100个term
- 通配符*不能出现在开头
examples:
- query: "status: 200*"
description: 查询status为200开头的日志
# 关键词查询
- type: keyword_query
description: 使用关键词查询
limitations:
- 索引配置中的分词配置决定了keyword最终的分词方式
examples:
- query: "host: www.example.com"
description: 查询host为www.example.com的日志
note: 如果.是分词符号,会查询www、example、com
# 全文查询
- type: full_text_query
description: 不指定key,直接输入查询字符串
limitations:
- 全文查询会使用全文索引对查询进行分词,如果目标内容所在的key单独配置了索引,并且两者索引配置不同(如分词符不一致、是否支持中文不一致),可能会查询不到结果,这时候推荐使用关键词查询
- 当没有配置全文索引时,会使用默认分词符"! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ ] { }"对输入进行分词
examples:
- query: '"www.example.com"'
description: 如果.是分词符,查询准确包含www、example、com的日志
# 短语查询
- type: phrase_query
description: 使用短语查询,精确匹配
limitations:
- 不能在NOT语句中使用
examples:
- query: '#"www.example.com"'
description: 查询准确包含www.example.com的日志
# 关于时间范围查询
time_filter:
- description: SDK查询请在GetLogs接口中指定开始结束时间,控制台查询请直接选好具体查询时间范围。不需要也无法在query中指定时间范围
# 使用示例
usage_examples:
- query: "status: 200 and request_method: GET"
description: 查询status为200且request_method为GET的日志
- query: "NOT status: 200 NOT request_method: GET"
description: 查询status不为200且request_method不为GET的日志
- query: "request_method: GE* AND status: 200"
description: 查询request_method为GET且status为200的日志
# 高级功能
advanced_features:
- name: 字段分析(Field Analysis)
description: 提供字段分布、统计指标及 TOP5 时间序列图,帮助理解数据
features:
- 可以快速了解字段的取值分布、数量统计等信息
- 支持对数值型字段进行统计分析,包括最大值、最小值、平均值等
- 在查询结果页面,点击字段名称可以查看字段分析
aliyun_doc: https://help.aliyun.com/zh/sls/field-analysis
- name: 上下文查询(Context Query)
description: 支持查看指定日志的上下文信息,方便故障排查和问题定位
usage: 在日志详情页面,点击"上下文"按钮
aliyun_doc: https://help.aliyun.com/zh/sls/contextual-query
- name: LiveTail(实时监控)
description: 实时监控线上日志,减轻运维压力
features:
- 类似 tail -f 命令,可以实时查看最新产生的日志
- 适用于实时监控、故障诊断等场景
aliyun_doc: https://help.aliyun.com/zh/sls/livetail
FAQ:
- question: 为什么模糊查询结果不全
answer: 模糊查询默认只匹配100个term,如果需要完整匹配,可以使用SPL语句,例如
```
* | where column like '%keyword%'
```
- question: 为什么查不到数据
answer:
- 确定一下时间范围是否正确
- 确定一下查询语句是否正确
- 如果索引配置刚打开,索引配置只对新写入的数据生效,历史数据需要重建索引
- 如果查询中包含了中文,需要确认一下索引配置中是否开启了中文索引
- 可能是索引分词配置导致,需要确认一下索引分词配置是否正确
- 如果是SDK查询的话,需要确认一下offset和topic是否设置正确
- 需要确认是否使用全文查询查询字段,并且配置不一样,参考full_text_query
- 如果是使用全文查询,需要确认全文索引和字段索引配置是否一致,参考full_text_query
- 检查数据写入延迟:日志写入SLS后,构建索引通常有几秒的延迟(取决于集群负载),刚写入的日志可能无法立即被检索到
- 检查逻辑运算符优先级:AND 的优先级高于 OR,例如 "a or b and c" 会被解析为 "a or (b and c)",如果预期是 "(a or b) and c",必须使用括号显式分组
- 检查Logtail采集状态:确认Logtail心跳正常且无解析错误(Logtail配置错误导致日志根本没发到SLS,自然查不到)
- 检查是否存在不可见字符:日志中可能包含不可见的控制字符(如颜色代码 \033 等),导致关键词匹配失败,建议先用模糊查询排查
- 检查数值类型字段:如果字段在JSON中是数值(如 age: 20),但在索引配置中配成了“文本”类型,由于构建索引失败会导致查不到
- question: 查询(Search)和 分析(Analysis)有什么区别?
answer: |
- 查询(Search):仅用于过滤和返回原始日志内容。语法如 `status:200`。
- 分析(Analysis):基于 SQL 92 语法对日志进行聚合、统计、计算。语法如 `* | SELECT method, count(1) group by method`。
- 两者通过管道符 `|` 分割,管道符左边是查询语句,右边是分析语句。
- question: 为什么数值范围查询(如 latency > 100)报错或结果不符合预期?
answer: |
- **索引类型错误**:范围查询要求该字段在索引配置中必须设置为 **数值类型(Long/Double)**。如果配置的是 **文本类型(Text)**,即便是数字内容,SLS 也会将其视为字符串处理,不支持数学比较运算。
- question: 为什么 JSON 日志无法通过 `key.subkey: value` 的方式查询?
answer: |
- **未开启 JSON 索引**:默认情况下,如果字段索引配置为“文本”,SLS 会将整个 JSON 对象视为一长串字符串。你只能通过全文搜索查询其中的片段。
- **配置层级**:要支持 `object.field: value` 这种 key-value 对齐的查询,必须在索引配置中,将该父字段的数据类型设置为 **JSON**,并开启“索引所有文本字段(针对string的叶子节点)”或手动逐个添加子字段的索引。
- question: 为什么我搜索 `User-Agent: Chrome` 查不到数据,但直接搜 `Chrome` 能查到?
answer: |
- **字段大小写不一致**:检查索引配置中,该字段的 Key 配置日志实际的 Key是否不一致,比如日志实际的 Key 是 `user_agent` 而索引配置中配置的 `User-Agent`。
- question: 为什么精确搜索(例如 `id: "123-456"`)匹配到了不该匹配的日志(例如 `123-456-789`)?
answer: |
- **分词符问题**:这是最容易被忽视的配置。如果索引配置中,连字符 `-` 被设置为了分词符:
- 输入内容 `123-456` 会被切分为 `123` 和 `456`。
- 目标日志 `123-456-789` 也会被切分为 `123`, `456`, `789`。
- 因为同时包含 `123` 和 `456`,所以匹配成功。
- **解决方法**:如果需要绝对精确匹配,应从分词符列表中移除该符号,或者将该字段配置为“不分词”(只有完全相等才匹配),或者使用spl或sql。
- question: 为什么使用通配符查询(如 `*error`)报错?
answer: |
- **前缀通配符限制**:为了保障集群性能,SLS 的查询语法(Search)**禁止**通配符出现在查询词的**开头**(如 `*keyword`)。
- **后缀通配符**:支持后缀通配符(如 `keyword*`),但注意后缀通配符匹配的 Term 数量有限制(默认 100 个),如果匹配到的词过多,可能会导致部分结果丢失。
- question: 如何查询包含引号等特殊字符的内容?
answer: |
- **转义查询**:如果查询的值中包含双引号 `"`,需要使用反斜杠转义,例如 `content: "error message with \"quote\""`,注意在查询语句中,串用双引号包裹,单引号没有任何特殊含义
- question: 如何查询“存在某个字段”的日志(即字段非空)?
answer: |
- 使用通配符语法:`key: *`。
- 例如 `user_id: *` 可以筛选出所有包含 `user_id` 字段且不为空的日志。
- 注意:这依赖于 `user_id` 字段必须配置了索引。
- question: 为什么我修改了索引的分词符配置,原来的日志还是查不到?
answer: |
- **不可追溯性**:这是 SLS 索引机制的核心特性。索引是写入时构建的(Write-Once)。
- **修改后果**:修改索引配置(如增加分词符、改变字段类型)**仅对修改后新写入**的日志生效。
- **解决方案**:如果必须查询历史数据,唯一的办法是使用“重建索引(Reindex)”功能或者将数据导出重导。
trouble_shooting:
- name: 怀疑索引配置相关问题导致查不到数据
description:
- 参考 `indexConfig.yaml` 文档排查索引配置相关问题
- 结合 上述索引查询相关问题给出排查结论
concept:
- name: 查询
description:
- 一般通过若干关键词组合来过滤出符合条件的日志,我们叫做查询
- name: 分析
description:
- 一般带有聚合指令、函数的查询语句就是分析
- 客户有可能分不清楚查询和分析的区别,可能把分析也叫查询
- name: 索引查询
description:
- 索引查询是SLS的查询能力,通过关键词的and/or/not组合查询,过滤出符合条件的日志
- name: SQL
description:
- SLS 提供了SQL分析能力,语法符合PrestoSQL语法
- SQL一般是跟在索引查询之后,通过竖线 | 分割
- name: SPL
description:
- SPL可以在查询分析场景使用,一般跟在索引查询之后,通过竖线 | 分割
- name: 不精确
description:
- 当数据量过大的时候,索引查询或者SQL/SPL可能返回部分结果数据(部分查询结果或者部分数据的聚合结果)
- 一般情况下通过重试多次可以获得精确结果
- 控制台页面在显示日志柱状图的上方有显示"查询不精确"的提示
- SDK使用的话一般对于GetLogs的结果有isComplete字段来标识是否精确
- name: 索引配置
description:
- 需要开启索引配置后才能使用查询分析的功能
- name: 数据集/Storeview
description:
- 默认情况下查询分析只能在单个logstore/metricstore中使用
- Storeview提供了将多个logsrore或metricstore的数据联合起来查询的能力
- Storeview定义的时候关联到的store要么都是logstore,要么都是metricstore
- name: SCAN模式
description:
- SCAN场景是为了满足不建索引的情况下进行查询分析的需求,一般情况下性能会低
- SCAN场景要求至少建有__time__的索引, 也就是默认情况下开启索引即可(不开字段索引和全文索引的情况)
- SCAN模式下可以用SPL或者SQL进行查询分析, 对于| 前面的部分是查询语句(这部分依赖具体建了哪些索引),后面的部分是分析语句
- name: 上下文查询
description:
- logtail或者producer上报日志的时候会带上上下文信息,从而可以看到每个上报的日志前后的日志,这就是上下文
- 上下文信息依赖__pack_id__(这个由客户端上报)和__pack_meta__(服务端生成)这两个字段
- name: Livetail
description:
- 可以理解为tail -f命令,可以实时查看最新产生的日志,在SLS控制台可以使用
- name: 增强SQL
description:
- 对于并发能力或者单Logstore有较大数据量的情况,可以开启增强SQL来提升查询分析的性能
- name: 完全精确SQL
description:
- 对于需要完全精确结果的场景,可以开启完全精确SQL来提升查询分析的性能
- name: Shard和查询分析的性能关系
description:
- shard即是logstore中的存储数据的分区,也决定了查询分析的并发能力
- 一般情况下SQL查询能不足的时候,可以通过分裂shard来解决。但分裂shard一般对后面生成的数据有效
- 如果对历史数据shard还没分裂,可以使用增强SQL来提升查询分析的性能
- name: 日志聚类
description:
- 日志聚类是SLS的日志聚类能力,通过日志聚类可以快速发现日志的模式
- name: 外表
description:
- SQL提供了外表关联的能力,可以将外表和logstore进行关联分析
- 外表需要创建后才能在SLS中和Logstore进行Join
- 目前支持的外表有MySQL、OSS、Postgres
- 除此之外,还支持CSV数据托管,即将CSV数据通过SDK上传到SLS的external_store(当前只支持csv上传)
- name: 物化视图
description:
- 物化视图是SQL的能力,通过提供物化视图提升查询分析的性能
- name: 定时SQL
description:
- 定时SQL支持将SQL分析的结果存储到目标Logstore中
- name: 短语查询
description:
- sls的索引查询的原理是通过倒排来实现的,对于一个query会通过分词来匹配对应的日志;由于分词后不关心词的顺序,所以短语查询无法使用索引查询
- 为了精准匹配一个keyword,短语查询的语法是:#"www.example.com",也就是在字符串前面加#
- 目前短语查询只支持加在and条件中,如果有or或者not无法生效; 如果希望在or not中生效,可以使用SPL来做,比如 * | where column like '%www.example.com%'
- name: 大小写敏感问题
description:
- 在查询分析场景下,关键词以及字段名称都是大小写不敏感的
- 索引配置里的字段名称是大小写敏感的,如果索引配置中的字段名和日志中的不一致,会导致索引不生效
- name: 时间字段过滤
description:
- 在SLS中时间是一个特殊的内置字段(__time__), 一般情况下在查询分析的时候不需要在query中过滤时间范围,而是通过接口参数或者控制台界面指定时间范围
- 在SDK查询的时候GetLogs接口需要指定from_time和to_time参数来指定时间范围
- 在控制台查询的时候直接选择时间范围即可
related_docs:
- name: 索引配置细节参考 `indexConfig.yaml`
- name: 索引查询细节参考 `indexSearch.yaml`
- name: SQL细节参考 `sql.yaml`
faq:
- question: 为什么写入日志后查询不到日志?
answer:
- 检查已设置的分词符是否符合要求。
- 索引配置只对新增日志生效,如果要查询和分析历史数据,请使用重建索引功能。
- 确认是否已经开启索引配置。
- question: SQL的select * 为什么不返回所有字段?
answer:
- SQL的select * 返回的结果是所有索引字段的内容,而不是日志中所有字段。如果要返回所有字段需要用索引查询或者用扫描查询进行SCAN
查询分析路由
1. 先分清三种能力
- 查询:核心是不做聚合,返回的是日志级结果
- 索引查询负责高效过滤日志,语法能力相对有限
- 如果查询命中的日志还要继续做字段增减、字段筛选、parse/extend/project 等逐行处理,这部分属于 SPL 能力,不属于索引查询本身
- 分析:核心是带聚合过程,通常会使用聚合函数、分组、统计计算
- 索引查询:典型的查询语法,用来过滤日志,返回原始日志。典型形式:
status: 200 and method: GET - SQL 分析:典型的分析语法,用来聚合、分组、统计。典型形式:
status: 200 | SELECT count(*) AS pv FROM log - SPL:在查询分析场景下,主要作为查询能力的补充,用来处理扫描模式下的复杂过滤、逐行处理、字段增减、字段筛选,以及在需要时做
stats聚合。典型形式:* | where status = '500' | extend latency_ms = cast(latency as BIGINT) | project latency_ms
2. 选择规则
- 查询场景优先索引查询:效率最高,但语法能力相对有限
- 如果查询场景里需要复杂条件过滤、逐行处理、parse-json/parse-regexp/parse-kv/extend 等能力:用 SPL 作为索引查询的补充
- 分析场景优先 SQL:适合 count/sum/avg/group by/topN/窗口分析等聚合计算
- 分析场景优先普通 SQL;如果发现分析字段没有索引,普通 SQL 无法直接完成,再考虑 SQL SCAN
- SPL 虽然也支持
stats、sort、limit等聚合相关指令,但除非用户明确要求 SPL,或者问题本身就是扫描查询/流水线处理场景,否则分析语句默认优先生成 SQL
3. 管道分层规则
- 只要语句里有
|,第一级管道前面的部分就是索引查询 - 这一级索引查询对后面的 SPL 和 SQL 都有效
- 作用是先用索引快速过滤,提前缩小后续处理的数据范围
- 因此只要某个过滤条件可以用索引查询表达,就应该尽量前置到
|前面 - 对 SQL 来说,这通常是执行更快的关键:能前置的过滤条件尽量不要只写在后面的 SQL 里
- 对 SPL 来说也是一样:能前置的索引过滤先前置,再在后面做逐行处理
4. 前置条件
- 查询分析都依赖索引配置
- SQL 依赖字段开启统计
- 索引和统计通常只对配置后新写入的数据生效,历史数据需要重建索引
- 查询型 Logstore 不支持统计,因此不支持 SQL 分析
- SCAN 也不是“完全无索引”,查询部分仍依赖可用索引,至少要满足最小索引条件
5. 时间范围
- 不要默认把时间条件写进 query / SQL
- 时间范围通过
--from/--toflag 指定,值为 Unix 时间戳(秒) - SQL 中的
__time__更适合做格式化、分组,而不是作为主要过滤手段
6. 索引查询高频规则
- 大小写不敏感
- 支持
AND/OR/NOT - 范围查询只适用于
long/double类型字段 - 模糊查询不能以前缀
*开头 - 短语精确匹配用
#"..." key: *表示字段存在且非空
7. SQL 高价值提醒
- 语法是
查询语句 | 分析语句 - SLS SQL 基于 Presto SQL
- 在 Logstore 场景一般表名默认是
log - SQL does not support
LIMIT count OFFSET offsetsyntax; useLIMIT offset, countinstead for pagination (e.g.,LIMIT 20, 20for rows 21–40) - 默认返回最多 100 行;更大结果要配合
LIMIT - SQL 后不能继续接 SPL
- 如果过滤条件可以在
|前用索引查询表达,优先前置,不要只放在 SQL 里
8. SCAN 模式
- 用于未建索引字段的兜底分析,不是优先方案
- 查询场景:优先索引查询,其次 SPL
- 分析场景:优先普通 SQL,其次 SQL SCAN
- 典型写法:
* | set session mode = scan; SELECT count(1) AS pv, api FROM log GROUP BY api - 限制:
- 查询部分仍受索引影响
- 所有字段视为
varchar - 单 Shard 扫描条数和总扫描行数有限制
- 性能明显低于索引分析模式
- 更适合临时分析或补救场景,不适合默认生成
9. 常见回答模板
只查日志
status: 500 and service: payment统计错误数
status: 500 | SELECT count(*) AS error_count FROM log按小时聚合
status: 500 | SELECT date_format(__time__, '%Y-%m-%d %H:00:00') AS hour, count(*) AS error_count FROM log GROUP BY hour ORDER BY hour先前置索引过滤,再做 SPL 处理
status: 500 and service: payment | where cast(latency as BIGINT) > 1000 | extend latency_ms = cast(latency as BIGINT) | project service, latency_ms, message本地源文档
./query_analysis/overview.yaml./query_analysis/indexSearch.yaml./query_analysis/indexConfig.yaml./query_analysis/sql.yaml
Related skills
How it compares
Choose alibabacloud-sls-query over generic log-query skills when you need index-aware SLS mode routing, Aliyun CLI get-logs-v2 execution, and bundled SLS SQL/SPL syntax references for Alibaba Cloud Log Service.
FAQ
What Aliyun CLI version does alibabacloud-sls-query require?
alibabacloud-sls-query requires Aliyun CLI version 3.3.8 or newer. Run aliyun version to verify, then follow the bundled cli-installation-guide.md if the CLI is missing or outdated before executing get-logs-v2 queries.
Which SLS query modes does alibabacloud-sls-query support?
alibabacloud-sls-query supports four SLS query modes: index search for raw log filtering, SQL for aggregation and GROUP BY analytics, SQL scan as an unindexed fallback, and SPL for row-level pipeline processing. The skill reads get-index config first to pick the fastest valid mod
How do you install alibabacloud-sls-query for an AI agent?
Install alibabacloud-sls-query with npx skills add aliyun/alibabacloud-aiops-skills --skill alibabacloud-sls-query, confirm the skill folder exists, restart the agent, and configure Alibaba Cloud credentials via aliyun configure before running SLS queries.