
Openobserve Api
- 27 installs
- 946 repo stars
- Updated August 2, 2026
- fcakyon/claude-codex-settings
Helps with backend & apis tasks.
About
openobserve-api is a Claude Code skill for backend & apis. It helps solo builders move faster with AI-assisted coding.
- openobserve-api
- Backend & APIs
- AI-coding skill
Openobserve Api by the numbers
- 27 all-time installs (skills.sh)
- Ranked #3,389 of 4,348 Backend & APIs skills by installs in the Skillselion catalog
- Data as of Aug 3, 2026 (Skillselion catalog sync)
npx skills add https://github.com/fcakyon/claude-codex-settings --skill openobserve-apiAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 27 |
|---|---|
| repo stars | ★ 946 |
| Last updated | August 2, 2026 |
| Repository | fcakyon/claude-codex-settings ↗ |
What it does
Helps with backend & apis tasks.
Files
OpenObserve REST API Skill
Programmatic OpenObserve usage for AI agents. Talk to any OpenObserve instance (Cloud or self-hosted) using curl and the documented REST API. No CLI required — there is no first-party OpenObserve CLI.
Retrieval First
Your knowledge of OpenObserve API shapes may be outdated. Prefer retrieval over pre-training:
| Source | How to retrieve | Use for |
|---|---|---|
| Docs repo | `gh api repos/openobserve/openobserve-docs/contents/docs/reference/api/{path}.md -q .content \ | base64 -d` |
| Server source | `gh api repos/openobserve/openobserve/contents/src/handler/http/request/dashboards/mod.rs -q .content \ | base64 -d` |
| Panel schema | `gh api repos/openobserve/openobserve/contents/src/config/src/meta/dashboards/v8/mod.rs -q .content \ | base64 -d` |
When docs and server source disagree, trust the server source — handlers ship faster than docs.
1. Auth
HTTPS basic auth with email + password. There is no token endpoint.
# Method 1: curl -u shorthand
curl -u "you@example.com:PASSWORD" "https://eu1.openobserve.ai/api/<org>/streams"
# Method 2: explicit header
TOKEN=$(printf '%s' "you@example.com:PASSWORD" | base64)
curl -H "Authorization: Basic $TOKEN" "https://eu1.openobserve.ai/api/<org>/streams"Endpoints below assume BASE=https://<host>/api/<org> and AUTH="-u you@example.com:PASSWORD".
2. Search / Query — POST $BASE/_search
Optional query string ?type=logs|metrics|traces (default logs).
curl $AUTH -H 'Content-Type: application/json' \
"$BASE/_search?type=logs" \
-d '{
"query": {
"sql": "SELECT host_name, COUNT(*) AS n FROM \"my_stream\" GROUP BY host_name ORDER BY n DESC",
"start_time": 1777000000000000,
"end_time": 1777999999000000,
"from": 0,
"size": 100
},
"search_type": "ui"
}'- Timestamps are microseconds (Unix epoch × 1_000_000). Always set
start_time/end_time— missing them scans everything. search_type∈ui | dashboards | reports | alerts— affects rate limits and audit logs.- Pagination:
from(offset) +size(limit, max ~10000 per request). - Response:
{ took, hits[], total, from, size, scan_size }. - SQL flavor: DataFusion / Arrow SQL. Identifiers in double quotes (
"stream_name"), strings in single quotes ('value'). - Time-bucketed group by:
SELECT histogram(_timestamp, '5 minute') AS ts, COUNT(*) FROM "stream" GROUP BY ts ORDER BY ts. - Term aggregation:
SELECT k8s_namespace, COUNT(*) FROM "stream" GROUP BY k8s_namespace. - Full-text:
match_all('text'),str_match(field, 'text'). Default full-text fields:log, message, msg, content, data, json. - PromQL on metrics:
POST $BASE/prometheus/api/v1/query_range. - Trace context window:
GET $BASE/{stream}/_around?key=<ts_us>&size=N.
3. Streams — GET $BASE/streams
# List
curl $AUTH "$BASE/streams?fetchSchema=false&type=logs"
# Schema
curl $AUTH "$BASE/streams/<stream>/schema?type=logs"
# Update settings
curl $AUTH -X PUT -H 'Content-Type: application/json' "$BASE/streams/<stream>/settings" -d '{"partition_keys":["host_name"]}'
# Delete
curl $AUTH -X DELETE "$BASE/streams/<stream>?type=logs"Field types: Utf8 | Int64 | Float64 | Timestamp | Boolean. Timestamp field is always _timestamp (microseconds).
4. Dashboards — GET|POST|PUT|DELETE $BASE/dashboards
All take ?folder=<folder_id> (default default).
# List
curl $AUTH "$BASE/dashboards?folder=default"
# Get one (returns versioned wrapper {v1..v8, version, hash, updatedAt})
curl $AUTH "$BASE/dashboards/<dashboard_id>?folder=default"
# Create — body is the UNWRAPPED inner v8 object
curl $AUTH -X POST -H 'Content-Type: application/json' \
"$BASE/dashboards?folder=default" \
-d @dashboard-v8.json
# Update — REQUIRES the current hash for optimistic concurrency
HASH=$(curl -s $AUTH "$BASE/dashboards/<id>?folder=default" | jq -r .hash)
curl $AUTH -X PUT -H 'Content-Type: application/json' \
"$BASE/dashboards/<id>?folder=default&hash=$HASH" \
-d @updated-dashboard.json
# Delete
curl $AUTH -X DELETE "$BASE/dashboards/<id>?folder=default"
# Move between folders
curl $AUTH -X PUT -H 'Content-Type: application/json' \
"$BASE/folders/dashboards/<id>" \
-d '{"from":"default","to":"<target_folder_id>"}'Critical: PUT/POST body must be the unwrapped inner v8 object, not the full {v1..v8, version, hash} wrapper. The server returns the wrapper but expects you to send only the inner object back.
# Read, mutate, write — the correct pattern
RAW=$(curl -s $AUTH "$BASE/dashboards/<id>?folder=default")
HASH=$(echo "$RAW" | jq -r .hash)
echo "$RAW" | jq '.v8 | .title = "New Title"' \
| curl -s $AUTH -X PUT -H 'Content-Type: application/json' \
"$BASE/dashboards/<id>?folder=default&hash=$HASH" -d @-A 409 Conflict response means the hash is stale — refetch and retry.
Per-panel operations (v8 only — return new hash)
# Add panel
curl $AUTH -X POST -H 'Content-Type: application/json' \
"$BASE/dashboards/<id>/panels?folder=default&hash=$HASH" \
-d '{"panel": {...}, "tabId": "default"}'
# Update panel
curl $AUTH -X PUT -H 'Content-Type: application/json' \
"$BASE/dashboards/<id>/panels/<panel_id>?folder=default&hash=$HASH" \
-d '{...panel...}'
# Delete panel
curl $AUTH -X DELETE \
"$BASE/dashboards/<id>/panels/<panel_id>?folder=default&hash=$HASH&tabId=default"5. Panel JSON (v8)
The dashboard tree: dashboard.tabs[].panels[]. Each panel:
{
"id": "panel-1",
"type": "table",
"title": "Per-host stats",
"description": "",
"queryType": "sql",
"queries": [
{
"query": "SELECT host_name, COUNT(*) AS n FROM \"my_stream\" GROUP BY host_name ORDER BY n DESC",
"vrlFunctionQuery": "",
"customQuery": true,
"fields": {
"stream": "my_stream",
"stream_type": "logs",
"x": [
{ "label": "Host", "alias": "host_name", "column": "host_name", "color": null, "aggregationFunction": null }
],
"y": [
{
"label": "Count",
"alias": "n",
"column": "n",
"color": null,
"aggregationFunction": null,
"treatAsNonTimeseries": true
}
],
"z": [],
"breakdown": [],
"filter": { "filterType": "group", "logicalOperator": "AND", "conditions": [] }
},
"config": { "promql_legend": "", "layer_type": "scatter", "weight_fixed": 1, "limit": 0, "min": 0, "max": 100 }
}
],
"config": {
"show_legends": true,
"decimals": 2,
"unit": "currency",
"unit_custom": "USD"
},
"layout": { "x": 0, "y": 0, "w": 48, "h": 14, "i": 1 }
}Panel type values
metric (single big number) · table · bar · h-bar · stacked · h-stacked · line · area · area-stacked · scatter · pie · donut · heatmap · gauge · geomap · maps · sankey · html · markdown.
Layout grid
The grid is 96 columns wide (verified via inspection of returned panel layouts on April 2026 OpenObserve Cloud). Older docs mention 192 or 48 — when in doubt, GET an existing dashboard from the same org and copy the w values you see. Heights are unitless rows (h: 7 = small metric panel; h: 14 = standard table).
Useful config keys
| Key | Effect |
|---|---|
decimals | Number of decimal places for all numeric columns (0 = integers). |
unit | `numbers \ |
unit_custom | When unit=currency, ISO code like USD. |
show_legends | Boolean, charts only. |
legends_position | `right \ |
axis_border_show | Boolean. |
line_interpolation | `smooth \ |
connect_nulls | Boolean — line/area only. |
top_results | Cap series count for line/bar (e.g. 10). |
mark_line | `[{name, type:'avg'\ |
6. Critical pitfalls
6a. Re-aggregation when `customQuery: true` — the most common bug.
If your hand-written SQL already contains COUNT(*), SUM(...), AVG(...) etc., every entry in fields.y (and fields.x) must set `aggregationFunction: null`. Default 'sum' causes OpenObserve to wrap the already-aggregated column in another aggregation client-side, producing duplicate rows and wildly inflated numbers.
// WRONG — produces duplicate rows
"y": [{"column":"messages", "aggregationFunction":"count"}]
// RIGHT — SQL already did the aggregation
"y": [{"column":"messages", "aggregationFunction":null, "treatAsNonTimeseries":true}]6b. Multiple `fields.y` on Table panels — each Y entry can render as a separate series/row. For a Table that should display one row per group, put only one entry in fields.y (any one column); the renderer will then display all SQL columns as table columns.
6c. Metric panels with `customQuery: true` — must explicitly map the result column to fields.y:
"y": [{"label":"Value", "alias":"value", "column":"value", "aggregationFunction":"sum", "treatAsNonTimeseries":false}]The metric panel needs to know which column is the number to display.
6d. ROUND + wildcard timestamp expansion — OpenObserve's planner sometimes auto-injects _timestamp into queries that wrap SUM(col) in ROUND(...), producing Column "_timestamp" must appear in the GROUP BY clause errors. Workaround: drop ROUND() and use the panel's decimals config instead, or pre-cast: CAST(SUM(...) AS DOUBLE).
6e. Hash-based concurrency on PUT — every successful PUT changes the dashboard hash. If you mutate a dashboard from two scripts back-to-back, the second one needs to refetch. Always re-GET before each PUT to grab the current hash.
6f. `start_time`/`end_time` are microseconds — Date.now() * 1000, not milliseconds. Off-by-1000× returns no hits but no error.
7. Folders / alerts / ingestion
Folders (v2 API):
curl $AUTH "$BASE/folders/dashboards" # list
curl $AUTH -X POST "$BASE/folders/dashboards" -d '{"name":"my-folder"}'
curl $AUTH "$BASE/folders/dashboards/name/<folder_name>" # lookup by namefolder_type ∈ dashboards | alerts | reports.
Alerts:
curl $AUTH "$BASE/{stream}/alerts" # list per-stream
curl $AUTH -X POST "$BASE/{stream}/alerts" -d '{...}'
# templates and destinations are referenced by alert definitions:
curl $AUTH "$BASE/alerts/templates"
curl $AUTH "$BASE/alerts/destinations"Ingestion (POST your own data in):
# JSON
curl $AUTH -X POST "$BASE/<stream>/_json" -d '[{"event":"foo","level":"info"}]'
# Multi-line JSON (one per line)
curl $AUTH -X POST "$BASE/<stream>/_multi" --data-binary @file.ndjson
# Elasticsearch bulk
curl $AUTH -X POST "$BASE/_bulk" --data-binary @bulk.txt
# OTLP HTTP
curl $AUTH -X POST "$BASE/v1/logs" -d @otlp-logs.json
curl $AUTH -X POST "$BASE/v1/traces" -d @otlp-traces.json
curl $AUTH -X POST "$BASE/v1/metrics" -d @otlp-metrics.json
# Loki
curl $AUTH -X POST "$BASE/loki/api/v1/push" -d @loki.json
# Prometheus remote-write (binary protobuf)
curl $AUTH -X POST "$BASE/prometheus/api/v1/write" --data-binary @write.pb8. Common recipes
Get top hosts by message count (last 24h):
NOW=$(($(date +%s) * 1000000))
DAY=$((NOW - 86400 * 1000000))
curl $AUTH -H 'Content-Type: application/json' \
"$BASE/_search?type=logs" \
-d "{\"query\":{\"sql\":\"SELECT host_name, COUNT(*) AS n FROM \\\"my_stream\\\" GROUP BY host_name ORDER BY n DESC\",\"start_time\":$DAY,\"end_time\":$NOW,\"size\":50}}"Add a metric panel to an existing dashboard (single-shot, hash-aware):
DASH_ID=<dashboard_id>
HASH=$(curl -s $AUTH "$BASE/dashboards/$DASH_ID?folder=default" | jq -r .hash)
curl $AUTH -X POST -H 'Content-Type: application/json' \
"$BASE/dashboards/$DASH_ID/panels?folder=default&hash=$HASH" \
-d '{
"tabId": "default",
"panel": {
"id": "p-cost",
"type": "metric",
"title": "Total cost (USD)",
"queryType": "sql",
"queries": [{
"query": "SELECT SUM(CAST(cost_usd AS DOUBLE)) AS value FROM \"my_stream\"",
"customQuery": true,
"fields": {
"stream":"my_stream", "stream_type":"logs",
"x":[], "z":[], "breakdown":[],
"y":[{"label":"Value","alias":"value","column":"value","aggregationFunction":"sum","treatAsNonTimeseries":false}],
"filter":{"filterType":"group","logicalOperator":"AND","conditions":[]}
},
"config":{}
}],
"config": {"unit":"currency","unit_custom":"USD","decimals":2},
"layout": {"x":0,"y":0,"w":32,"h":7,"i":99}
}
}'Build a complete dashboard from scratch: GET an existing dashboard's panel JSON as a template (it's the safest way to learn the exact field shapes the server will accept), then mutate the tabs[0].panels array and PUT the unwrapped v8 body back. See the references/recipes/build-dashboard.sh script that ships with this skill for a working example.
9. SDKs / clients (no first-party CLI)
| Language | Repo | Status |
|---|---|---|
| Python | github.com/openobserve/openobserve-python-sdk | Active |
| Go | github.com/openobserve/openobserve-go-client | ZincObserve-era, partial |
| Helm chart | github.com/openobserve/openobserve-helm-chart | Active |
| OTel collector distro | github.com/openobserve/openobserve-otel-collector | Active |
For most agent tasks, plain curl against the REST API is the right tool — the SDKs add little value over an HTTP request and lag the server feature set.
References
- API docs (canonical): https://github.com/openobserve/openobserve-docs (path:
docs/reference/api/) - Server source: https://github.com/openobserve/openobserve (paths:
src/handler/http/request/,src/config/src/meta/dashboards/v8/mod.rs) - Cloud console: https://cloud.openobserve.ai (regions: us1, eu1, ap1)
- The
references/directory in this skill mirrors selected docs fromopenobserve-docsfor offline access.
Search around
Endpoint: GET /api/{organization}/{stream}/_around?key={timestamp}&size=10
Request
Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| stream | string | - | stream name |
| key | int64 | 0 | the _timestamp of the record what you want to search around |
| size | int64 | 0 | how many records do you want to response around the record, we will search the record forward & backward 5 minutes |
Response
{
"took": 155,
"hits": [
{
"_p": "F",
"_timestamp": 1674213225158000,
"kubernetes": {
"annotations": {
"kubernetes": {
"io/psp": "eks.privileged"
}
},
"container_hash": "dkr.ecr.us-west-2.amazonaws.com/ziox@sha256:3dbbb0dc1eab2d5a3b3e4a75fd87d194e8095c92d7b2b62e7cdbd07020f54589",
"container_image": "dkr.ecr.us-west-2.amazonaws.com/ziox:v0.0.3",
"container_name": "ziox",
"docker_id": "eb0983bdb9ff9360d227e6a0b268fe3b24a0868c2c2d725a1516c11e88bf5789",
"host": "ip.us-east-2.compute.internal",
"labels": {
"app": "ziox",
"controller-revision-hash": "ziox-ingester-579b7767cf",
"name": "ziox-ingester",
"role": "ingester",
"statefulset": {
"kubernetes": {
"io/pod-name": "ziox-ingester-0"
}
}
},
"namespace_name": "ziox",
"pod_id": "35a0421f-9203-4d73-9663-9ff0ce26d409",
"pod_name": "ziox-ingester-0"
},
"log": "[2023-01-20T11:13:45Z INFO actix_web::middleware::logger] 10.2.80.192 \"POST /api/demo/_bulk HTTP/1.1\" 200 68 \"-\" \"go-resty/2.7.0 (https://github.com/go-resty/resty)\" 0.001074",
"stream": "stderr"
}
],
"total": 10,
"from": 0,
"size": 0,
"scan_size": 28943
}Response description:
Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| took | int64 | - | unit: milliseconds, query execute time |
| from | int64 | 0 | value from query.from |
| size | int64 | 0 | value from query.size |
| scan_size | int64 | 0 | unit: MB, it response the data size scale when execute the query. |
| hits | array | - | records for query, each record is a log row what you ingested. |
Logs Ingestion - Bulk
Endpoint: POST /api/{organization}/_bulk
This will upload multiple records in batch with ndjson (newline delimited json). This API is compatible with Elasticsearch _bulk API.
Request
e.g. POST /api/myorg/_bulk
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus", "kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5", "kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal", "kubernetes.labels.app.kubernetes.io/component": "prometheus", "kubernetes.labels.app.kubernetes.io/instance": "k8s", "kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator", "kubernetes.labels.app.kubernetes.io/name": "prometheus", "kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus", "kubernetes.labels.app.kubernetes.io/version": "2.39.1", "kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c", "kubernetes.labels.operator.prometheus.io/name": "k8s", "kubernetes.labels.operator.prometheus.io/shard": "0", "kubernetes.labels.prometheus": "k8s", "kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1", "kubernetes.namespace_name": "monitoring", "kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64", "kubernetes.pod_name": "prometheus-k8s-1", "log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:108 level=warn component=k8s_client_runtime func=Warningf msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"", "stream": "stderr" }
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus", "kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5", "kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal", "kubernetes.labels.app.kubernetes.io/component": "prometheus", "kubernetes.labels.app.kubernetes.io/instance": "k8s", "kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator", "kubernetes.labels.app.kubernetes.io/name": "prometheus", "kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus", "kubernetes.labels.app.kubernetes.io/version": "2.39.1", "kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c", "kubernetes.labels.operator.prometheus.io/name": "k8s", "kubernetes.labels.operator.prometheus.io/shard": "0", "kubernetes.labels.prometheus": "k8s", "kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1", "kubernetes.namespace_name": "monitoring", "kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64", "kubernetes.pod_name": "prometheus-k8s-1", "log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:116 level=error component=k8s_client_runtime func=ErrorDepth msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"", "stream": "stderr" }First line is stream action
>
Second line is record data
Request action
Create record
{ "index" : { "_index" : "stream1" } } We only support create action.
The _index is stream name what you want to use.
Response
{
"code": 200,
"status": [
{
"name": "stream1",
"successful": 2,
"failed": 0
}
]
}Returns successful and failed count for each stream.
Restriction on number of fields/columns per record
Applicable to cloud version
Please note only records having 200 or less fields/columns will be considered for ingestion , records having more than 200 fields/columns will be discarded with failed status.
Applicable to open source version
One can configure ZO_COLS_PER_RECORD_LIMIT to set desired value for allowed number of fields/columns per record.
Timestamp
By default we add a field _timestamp for each record with the value of NOW in microseconds (unix epoch value).
we support use of two fields to override the default value.
- _timestamp
- @timestamp
2 data formats are supported for timestamp fields the value support two data type format:
- microseconds (unix epoch value)
- string value
- RFC 3339 and ISO 8601 date and time string such as
1996-12-19T16:39:57-08:00 - RFC 2822 date and time string such as
Tue, 1 Jul 2003 10:52:37 +0200
eg:
use microseconds
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.container_name": "prometheus", "_timestamp": "1674789786006000" }use string datetime
{ "index" : { "_index" : "stream1" } }
{ "kubernetes.container_name": "prometheus", "_timestamp": "2023-01-02T10:01:01Z" }Delete stream
OpenObserve provides multiple deletion strategies to manage your data lifecycle: immediate complete stream deletion, targeted time-range deletion with job tracking, and automatic retention-based cleanup.
Overview
The Delete Stream API allows you to:
- Delete an entire stream and all its data
- Delete data within a specific time period with job tracking
- Monitor deletion job progress across clusters
- Manage cached query results
All deletion operations are asynchronous and processed by the Compactor service.
Base URL
https://example.remote.dev/ Replace example.remote.dev with your actual OpenObserve instance URL.
Content type
All requests and responses use JSON format.
Content-Type: application/jsonEndpoints
Delete entire stream
Delete a complete stream and all associated data.
Request
Method: DELETE <br> Path: /api/{org_id}/streams/{stream_name}?type=logs&delete_all=true <br> Parameters:
| Name | Type | Location | Required | Description |
|---|---|---|---|---|
| org_id | string | path | Yes | Organization identifier |
| stream_name | string | path | Yes | Name of the stream to delete |
| type | string | query | Yes | Stream type: logs, metrics, or traces |
| delete_all | boolean | path | Yes | Delete all related resources like alerts and dashboards |
Request example
curl -X 'DELETE' \
'https://example.remote.dev/api/default/streams/pii_test?type=logs&delete_all=true' \
-H 'accept: application/json'Response
Status Code: 200 OK
{
"code": 200,
"message": "stream deleted"
}Response fields
| Field | Type | Description |
|---|---|---|
| code | integer | HTTP status code |
| message | string | Confirmation message |
Status codes
| Code | Meaning |
|---|---|
| 200 | Stream deleted successfully |
| 400 | Invalid parameters |
| 404 | Stream not found |
| 500 | Internal server error |
Behavior
Deletion is asynchronous and does not happen immediately:
1. When you call this API, the deletion request is marked in the system. 2. The API responds immediately, you do not wait for actual deletion. 3. A background service called Compactor checks for pending deletions every 10 minutes. 4. When Compactor runs, it starts deleting your stream. This can take anywhere from seconds to several minutes depending on how much data the stream contains. 5. In the worst-case scenario (if you request deletion just before Compactor runs), the entire process could take up to 30 minutes total. 6. You do not need to wait. The deletion happens in the background. You can check the stream status later to confirm it has been deleted.
!!! note "Notes"
- This operation cannot be undone.
- Data is deleted from both the
file_listtable and object store. - No job tracking is available for this endpoint
!!! note "Environment variables"
- You can change the
compactorrun interval:ZO_COMPACT_INTERVAL=600. Unit is second. default is10 minutes. - You can configure data life cycle to auto delete old data:
ZO_COMPACT_DATA_RETENTION_DAYS=30. The system will auto delete the data after30days. Note that the value must be greater than0.
Delete stream data by time range
Delete stream data within a specific time period with job tracking.
Request
Method: DELETE <br> Path: /api/{org_id}/streams/{stream_name}/data_by_time_range?start=<start_ts>&end=<end_ts>
Parameters
| Parameter | Type | Location | Description |
|---|---|---|---|
org_id | string | Path | Organization identifier |
stream_name | string | Path | Name of the stream |
start | long | path | Start timestamp in microseconds (UTC). Inclusive. |
end | long | path | End timestamp in microseconds (UTC). Inclusive. |
Request example
curl -X DELETE \
'https://example.remote.dev/api/default/streams/test_stream/data_by_time_range?start=1748736000000000&end=1751241600000000'Response
Status Code: 200 OK
{
"id": "30ernyKEEMznL8KIXEaZhmDYRR9"
}Response fields
| Field | Type | Description |
|---|---|---|
id | string | Unique job ID for tracking deletion progress |
Status codes
| Code | Meaning |
|---|---|
| 200 | Deletion job created successfully |
| 400 | Invalid parameters (For example, invalid timestamp format) |
| 404 | Stream not found |
Behavior
- Initiates a compaction delete job.
- Returns a job ID that can be used to track progress.
- Deletes data from:
file_listtable- Object store (for example, S3)
- Granularity:
- Logs: Data is deleted every hour.
- Traces: Data is deleted daily.
---
Get delete job status
Check the status of a time-range deletion job.
Request
Method: GET <br> Path: /api/{org_id}/streams/{stream_name}/data_by_time_range/status/{id}
Parameters
| Parameter | Type | Location | Description |
|---|---|---|---|
org_id | string | Path | Organization identifier |
stream_name | string | Path | Name of the stream |
id | string | Path | Job ID returned from deletion request |
Request example
curl -X GET \
'https://example.remote.dev/api/default/streams/test_stream/data_by_time_range/status/30ernyKEEMznL8KIXEaZhmDYRR9'Response: Completed
Status Code: 200 OK
{
"id": "30f080gLbU4i21VpY2O3YzwrKDH",
"status": "Completed",
"metadata": [
{
"cluster": "dev3",
"region": "us-test-3",
"id": "30f080gLbU4i21VpY2O3YzwrKDH",
"key": "default/logs/delete_d3/2025-07-27T04:00:00Z,2025-07-28T04:00:00Z",
"created_at": 1754003156467113,
"ended_at": 1754003356516415,
"status": "Completed"
},
{
"cluster": "dev4",
"region": "us-test-4",
"id": "30f080gLbU4i21VpY2O3YzwrKDH",
"key": "default/logs/delete_d3/2025-07-27T04:00:00Z,2025-07-28T04:00:00Z",
"created_at": 1754003156467113,
"ended_at": 1754003326523177,
"status": "Completed"
}
]
}Response: Pending
Status Code: 200 OK
{
"id": "30f080gLbU4i21VpY2O3YzwrKDH",
"status": "Pending",
"metadata": [
{
"cluster": "dev3",
"region": "us-test-3",
"id": "30f080gLbU4i21VpY2O3YzwrKDH",
"key": "default/logs/delete_d3/2025-07-27T04:00:00Z,2025-07-28T04:00:00Z",
"created_at": 1754003156467113,
"ended_at": 0,
"status": "Pending"
},
{
"cluster": "dev4",
"region": "us-test-4",
"id": "30f080gLbU4i21VpY2O3YzwrKDH",
"key": "default/logs/delete_d3/2025-07-27T04:00:00Z,2025-07-28T04:00:00Z",
"created_at": 1754003156467113,
"ended_at": 0,
"status": "Pending"
}
]
}Response: With Errors
Status Code: 200 OK
{
"id": "30fCWBSNWwTWnRJE0weFfDIc3zz",
"status": "Pending",
"metadata": [
{
"cluster": "dev4",
"region": "us-test-4",
"id": "30fCWBSNWwTWnRJE0weFfDIc3zz",
"key": "default/logs/delete_d4/2025-07-21T14:00:00Z,2025-07-22T00:00:00Z",
"created_at": 1754009269552227,
"ended_at": 1754009558553845,
"status": "Completed"
}
],
"errors": [
{
"cluster": "dev3",
"error": "Error getting delete job status from cluster node: Status { code: Internal, message: \"Database error: DbError# SeaORMError# job not found\", metadata: MetadataMap { headers: {\"content-type\": \"application/grpc\", \"date\": \"Fri, 01 Aug 2025 00:58:01 GMT\", \"content-length\": \"0\"} }, source: None }",
"region": "us-test-3"
}
]
}Response fields
| Field | Type | Description |
|---|---|---|
id | string | Job identifier |
status | string | Overall job status: Completed or Pending |
metadata | array | Array of per-cluster deletion details |
metadata[].cluster | string | Cluster identifier |
metadata[].region | string | Region/zone identifier |
metadata[].id | string | Job ID |
metadata[].key | string | Database key for the deletion operation |
metadata[].created_at | long | Job creation timestamp in microseconds |
metadata[].ended_at | long | Job completion timestamp in microseconds (0 if still pending) |
metadata[].status | string | Individual cluster deletion status |
errors | array | Array of errors from specific clusters (if any) |
errors[].cluster | string | Cluster where error occurred |
errors[].region | string | Region identifier |
errors[].error | string | Error message |
Status Codes
| Code | Meaning |
|---|---|
| 200 | Status retrieved successfully |
| 404 | Job ID not found |
Behavior
- Returns current status of deletion job
- Shows progress across all clusters in distributed setup
- Shows error details if any cluster encountered failures
- Status of
Pendingmeans deletion is still in progress - Status of
Completedmeans all clusters finished deletion
---
Delete cache results
Delete cached query results for a stream.
Request
Method: DELETE <br> Path: /api/{org_id}/streams/{stream_name}/cache/results?type=<stream_type>&ts=<timestamp>
Parameters
| Parameter | Type | Location | Description |
|---|---|---|---|
org_id | string | path | Organization identifier |
stream_name | string | Path | Stream name (use _all to delete cache for all streams) |
type | string | Query | Stream type: logs, metrics, or traces |
ts | long | Query | Timestamp threshold in microseconds. Deletes cache from start up to this timestamp. Retains cache from timestamp onwards. |
Request example
curl -X DELETE \
'https://example.remote.dev/api/default/streams/test_stream/_all/cache/results?type=logs&ts=1753849800000'Response
Status Code: 200 OK
{
"code": 200,
"message": "cache deleted"
}Response Fields
| Field | Type | Description |
|---|---|---|
code | integer | HTTP status code |
message | string | Confirmation message |
Status Codes
| Code | Meaning |
|---|---|
| 200 | Cache deleted successfully |
| 400 | Invalid parameters |
| 404 | Stream not found |
Behavior
- Accepts
ts(timestamp) query parameter in microseconds - Deletes cache from
cache_startup to the givents - Retains cache from
tsonwards
API Index
These APIs can be used to programmatically interact with OpenObserve.
All APIs must have an authorization header. Authorization header can be created using base64 encoded values of user id and password. For the sake of simplicity it is HTTP basic authentication mechanism.
Header creation mechanism:
Authorization: Basic base64("username:password")
e.g. Header:
Authorization: Basic YWRtaW46Q29tcGxleHBhc3MjMTIz
Make sure that you are sending the requests over HTTPS.
API List
1. Stream 1. List 1. Schema 1. Setting 1. Ingestion 1. Bulk 1. Json 1. Multi 1. Search 1. Function 1. User 1. Create 1. Delete 1. List 1. Metrics
Logs Ingestion - JSON
Endpoint: POST /api/{organization}/{stream}/_json
This will upload multiple records in batch with standard json format.
Request
e.g. POST /api/myorg/stream1/_json
[
{
"kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus",
"kubernetes.annotations.kubernetes.io/psp": "eks.privileged",
"kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38",
"kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1",
"kubernetes.container_name": "prometheus",
"kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5",
"kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal",
"kubernetes.labels.app.kubernetes.io/component": "prometheus",
"kubernetes.labels.app.kubernetes.io/instance": "k8s",
"kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator",
"kubernetes.labels.app.kubernetes.io/name": "prometheus",
"kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus",
"kubernetes.labels.app.kubernetes.io/version": "2.39.1",
"kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c",
"kubernetes.labels.operator.prometheus.io/name": "k8s",
"kubernetes.labels.operator.prometheus.io/shard": "0",
"kubernetes.labels.prometheus": "k8s",
"kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1",
"kubernetes.namespace_name": "monitoring",
"kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64",
"kubernetes.pod_name": "prometheus-k8s-1",
"log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:108 level=warn component=k8s_client_runtime func=Warningf msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"",
"stream": "stderr"
},
{
"kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus",
"kubernetes.annotations.kubernetes.io/psp": "eks.privileged",
"kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38",
"kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1",
"kubernetes.container_name": "prometheus",
"kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5",
"kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal",
"kubernetes.labels.app.kubernetes.io/component": "prometheus",
"kubernetes.labels.app.kubernetes.io/instance": "k8s",
"kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator",
"kubernetes.labels.app.kubernetes.io/name": "prometheus",
"kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus",
"kubernetes.labels.app.kubernetes.io/version": "2.39.1",
"kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c",
"kubernetes.labels.operator.prometheus.io/name": "k8s",
"kubernetes.labels.operator.prometheus.io/shard": "0",
"kubernetes.labels.prometheus": "k8s",
"kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1",
"kubernetes.namespace_name": "monitoring",
"kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64",
"kubernetes.pod_name": "prometheus-k8s-1",
"log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:116 level=error component=k8s_client_runtime func=ErrorDepth msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"",
"stream": "stderr"
}
]Each line is one record.
Response
{
"code": 200,
"status": [
{
"name": "stream1",
"successful": 2,
"failed": 0
}
]
}Returns successful and failed count for each stream.
Restriction on number of fields/columns per record
Applicable to cloud version
Please note only records having 200 or less fields/columns will be considered for ingestion , records having more than 200 fields/columns will be discarded with failed status.
Applicable to open source version
One can configure ZO_COLS_PER_RECORD_LIMIT to set desired value for allowed number of fields/columns per record.
Flattening of the JSON structure
OpenObserve flattens deep JSON logs. Below is an example log before and after being flattened.
Before
{
"actor": {
"ip": "[redacted]",
"id": 558875,
"parent" : {
"id": 45516,
"active": true
}
}
"response": {
"error_occured": false,
"status_code": 200
}
}After
{
"actor_ip": "[redacted]",
"actor_id": 558875,
"actor_parent_id": 45516,
"actor_parent_active": true,
"response_error_occured": false,
"response_status_code": 200
}Restriction on flattening depth
⚠️ For performance reasons, OpenObserve limits the depth at which the JSON structure gets flattened. Past that limit, the generated field will contain unparsed JSON as a string. The default depth is 3, but this limit can be configured via the ZO_INGEST_FLATTEN_LEVEL environment variable. ZO_INGEST_FLATTEN_LEVEL can either be 0, which disables the flattening limit, or any positive number, to change the depth at which the flattening stops.
Timestamp
By default we add a field _timestamp for each record with the value of NOW in microseconds (unix epoch value).
we support use of two fields to override the default value.
- _timestamp
- @timestamp
2 data formats are supported for timestamp fields the value support two data type format:
- microseconds (unix epoch value)
- string value
- RFC 3339 and ISO 8601 date and time string such as
1996-12-19T16:39:57-08:00 - RFC 2822 date and time string such as
Tue, 1 Jul 2003 10:52:37 +0200
eg:
use microseconds
[{
"kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1",
"kubernetes.container_name": "prometheus",
"_timestamp": "1674789786006000"
}]use string datetime
[{
"kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1",
"kubernetes.container_name": "prometheus",
"_timestamp": "2023-01-02T10:01:01Z"
}]
List streams
Endpoint: GET /api/{organization}/streams?fetchSchema=false&type={StreamType}
Request
- fetchSchema: true / false
fetchSchema set to true will response the schema for each stream or without schema.
- type: logs / metrics / traces
default is logs.
Response
{
"list": [
{
"name": "k8s",
"storage_type": "s3",
"stream_type": "logs",
"stats": {
"doc_time_min": 1673715046856933,
"doc_time_max": 1673849134852901,
"doc_num": 3300000,
"file_num": 16,
"storage_size": 3323.5,
"compressed_size": 11.42
},
"schema": [
{
"name": "_timestamp",
"type": "Int64"
},
{
"name": "kubernetes.annotations.kubernetes.io/psp",
"type": "Utf8"
},
],
"settings": {
"partition_keys": {},
"full_text_search_keys": ["log"]
}
}
]
}Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| name | string | - | stream name |
| storage_type | string | - | s3 / disk |
| stream_type | string | logs | logs / metrics / traces |
| stats | object | - | stats for the stream |
| stats.doc_time_min | int64 | 0 | the minimum timestamp of the record in the stream |
| stats.doc_time_max | int64 | 0 | the maximum timestamp of the record in the stream |
| stats.doc_num | int64 | 0 | the records num of the stream |
| stats.file_num | int64 | 0 | the files num in storage of the stream |
| stats.storage_size | int64 | 0 | ingestion data size of the original data |
| stats.compressed_size | int64 | 0 | stored size in storage after compression |
| schema | array | - | the schema of the stream, if fetchSchema set to false, has no this field |
| schema.name | string | - | field name |
| schema.type | string | - | field data type: Utf8 / Int64 / Float64 / Timestamp / Boolean |
| settings | object | - | settings of the stream |
| settings.partition_keys | object | - | custom partition keys for the stream. By default OpenObserve uses timestamp as the first level partition key |
| settings.full_text_search_keys | array[string] | - | full text search fields, default OpenObserve uses log, message, msg, content, data, json, if there is no those fields in your stream, will report error: you should set the full text search fields. |
Logs Ingestion - Loki
Endpoint: POST /api/{organization}/loki/api/v1/push
OpenObserve is compatible with the Grafana Loki push API. You can send logs using any Loki-compatible client (e.g. Promtail, Grafana Agent, Alloy) by pointing it at OpenObserve.
we useo2_stream_namelabel for custom stream name, default will push intodefaultstream.
Request
e.g. POST /api/myorg/loki/api/v1/push
Content-Type: application/json
{
"streams": [
{
"stream": {
"o2_stream_name": "custom_stream",
"kubernetes_namespace": "monitoring",
"kubernetes_pod_name": "prometheus-k8s-1",
"kubernetes_container_name": "prometheus"
},
"values": [
[
"1672149599212000000",
"ts=2022-12-27T14:09:59.212Z caller=klog.go:108 level=warn component=k8s_client_runtime msg=\"failed to list *v1.Pod\""
],
[
"1672149600000000000",
"ts=2022-12-27T14:10:00.000Z caller=klog.go:116 level=error component=k8s_client_runtime msg=\"Failed to watch *v1.Pod\""
]
]
}
]
}Fields
| Field | Type | Description |
|---|---|---|
streams | array | List of log streams to push. |
streams[].stream | object | Key-value label pairs that identify the stream. Labels are indexed and can be used for filtering. |
streams[].values | array | List of log entries. Each entry is a two-element array: [timestamp, line]. |
streams[].values[][0] | string | Unix timestamp in nanoseconds as a string. |
streams[].values[][1] | string | Log line content. |
Response
{}An empty JSON object {} with HTTP status 204 No Content indicates success, matching the standard Loki push API behavior.
Authentication
Pass your credentials using HTTP Basic Auth or via the Authorization header, the same as all other OpenObserve ingestion endpoints.
Authorization: Basic <base64(user:password)>Configuring Promtail
Point Promtail at OpenObserve by setting the Loki push URL in your promtail.yaml:
clients:
- url: https://<openobserve-host>/api/<organization>/loki/api/v1/push
basic_auth:
username: <user>
password: <password>Configuring Grafana Alloy
loki.write "openobserve" {
endpoint {
url = "https://<openobserve-host>/api/<organization>/loki/api/v1/push"
basic_auth {
username = "<user>"
password = "<password>"
}
}
}Logs Ingestion - Multi
Endpoint: POST /api/{organization}/{stream}/_multi
This will upload multiple records in batch with multiple json lines.
Request
e.g. POST /api/myorg/stream1/_multi
{ "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus", "kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5", "kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal", "kubernetes.labels.app.kubernetes.io/component": "prometheus", "kubernetes.labels.app.kubernetes.io/instance": "k8s", "kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator", "kubernetes.labels.app.kubernetes.io/name": "prometheus", "kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus", "kubernetes.labels.app.kubernetes.io/version": "2.39.1", "kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c", "kubernetes.labels.operator.prometheus.io/name": "k8s", "kubernetes.labels.operator.prometheus.io/shard": "0", "kubernetes.labels.prometheus": "k8s", "kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1", "kubernetes.namespace_name": "monitoring", "kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64", "kubernetes.pod_name": "prometheus-k8s-1", "log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:108 level=warn component=k8s_client_runtime func=Warningf msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"", "stream": "stderr" }
{ "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus", "kubernetes.docker_id": "563f8f40062cd0188c11f39e89d47e6eacddb5624a8a93b39f77ec53b5c38bf5", "kubernetes.host": "ip-10-2-50-35.us-east-2.compute.internal", "kubernetes.labels.app.kubernetes.io/component": "prometheus", "kubernetes.labels.app.kubernetes.io/instance": "k8s", "kubernetes.labels.app.kubernetes.io/managed-by": "prometheus-operator", "kubernetes.labels.app.kubernetes.io/name": "prometheus", "kubernetes.labels.app.kubernetes.io/part-of": "kube-prometheus", "kubernetes.labels.app.kubernetes.io/version": "2.39.1", "kubernetes.labels.controller-revision-hash": "prometheus-k8s-5857d9766c", "kubernetes.labels.operator.prometheus.io/name": "k8s", "kubernetes.labels.operator.prometheus.io/shard": "0", "kubernetes.labels.prometheus": "k8s", "kubernetes.labels.statefulset.kubernetes.io/pod-name": "prometheus-k8s-1", "kubernetes.namespace_name": "monitoring", "kubernetes.pod_id": "ebdc171d-c891-495f-b4d6-e24711b70e64", "kubernetes.pod_name": "prometheus-k8s-1", "log": "ts=2022-12-27T14:09:59.212Z caller=klog.go:116 level=error component=k8s_client_runtime func=ErrorDepth msg=\"pkg/mod/k8s.io/client-go@v0.25.1/tools/cache/reflector.go:169: Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User \\\"system:serviceaccount:monitoring:prometheus-k8s\\\" cannot list resource \\\"pods\\\" in API group \\\"\\\" at the cluster scope\"", "stream": "stderr" }Each line is one json record.
Response
{
"code": 200,
"status": [
{
"name": "stream1",
"successful": 2,
"failed": 0
}
]
}Returns successful and failed count for each stream.
Restriction on number of fields/columns per record
Applicable to cloud version
Please note only records having 200 or less fields/columns will be considered for ingestion , records having more than 200 fields/columns will be discarded with failed status.
Applicable to open source version
One can configure ZO_COLS_PER_RECORD_LIMIT to set desired value for allowed number of fields/columns per record.
Timestamp
By default we add a field _timestamp for each record with the value of NOW in microseconds (unix epoch value).
we support use of two fields to override the default value.
- _timestamp
- @timestamp
2 data formats are supported for timestamp fields the value support two data type format:
- microseconds (unix epoch value)
- string value
- RFC 3339 and ISO 8601 date and time string such as
1996-12-19T16:39:57-08:00 - RFC 2822 date and time string such as
Tue, 1 Jul 2003 10:52:37 +0200
eg:
use microseconds
{ "kubernetes.container_name": "prometheus", "_timestamp": "1674789786006000" }use string datetime
{ "kubernetes.container_name": "prometheus", "_timestamp": "2023-01-02T10:01:01Z" }Logs Ingestion - OTLP
Endpoint: POST /api/{organization}/v1/logs
OpenObserve supports the OpenTelemetry Protocol (OTLP) for log ingestion. You can send logs from any OpenTelemetry-compatible collector or SDK by pointing it at OpenObserve.
we use custom http headerstream-namefor speciafic stream name, default will push intodefaultstream.
Request
e.g. POST /api/myorg/v1/logs
Content-Type: application/json (JSON) or application/x-protobuf (Protobuf)
{
"resourceLogs": [
{
"resource": {
"attributes": [
{
"key": "service.name",
"value": { "stringValue": "my-service" }
},
{
"key": "service.version",
"value": { "stringValue": "1.2.3" }
},
{
"key": "host.name",
"value": { "stringValue": "ip-10-2-50-35.us-east-2.compute.internal" }
}
]
},
"scopeLogs": [
{
"scope": {
"name": "my-logger",
"version": "1.0.0"
},
"logRecords": [
{
"timeUnixNano": "1672149599212000000",
"observedTimeUnixNano": "1672149599212000000",
"severityNumber": 9,
"severityText": "INFO",
"body": {
"stringValue": "Request processed successfully"
},
"attributes": [
{
"key": "http.method",
"value": { "stringValue": "GET" }
},
{
"key": "http.status_code",
"value": { "intValue": "200" }
},
{
"key": "trace_id",
"value": { "stringValue": "4bf92f3577b34da6a3ce929d0e0e4736" }
}
],
"traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
"spanId": "00f067aa0ba902b7"
}
]
}
]
}
]
}Fields
| Field | Type | Description |
|---|---|---|
resourceLogs | array | List of resource log groups. |
resourceLogs[].resource | object | Resource describing the entity producing the logs (e.g. service, host). |
resourceLogs[].resource.attributes | array | Key-value pairs for resource-level metadata. |
resourceLogs[].scopeLogs | array | List of instrumentation scope log groups. |
scopeLogs[].scope | object | Instrumentation scope (library name and version). |
scopeLogs[].logRecords | array | List of individual log records. |
logRecords[].timeUnixNano | string | Log timestamp in nanoseconds since Unix epoch. |
logRecords[].observedTimeUnixNano | string | Time the log was observed by the collector, in nanoseconds. |
logRecords[].severityNumber | integer | Numeric severity level (1–24). See OTLP severity levels. |
logRecords[].severityText | string | Human-readable severity string (e.g. INFO, WARN, ERROR). |
logRecords[].body | object | Log message body. Typically a stringValue. |
logRecords[].attributes | array | Key-value pairs for log-level metadata. |
logRecords[].traceId | string | Trace ID associated with this log record (hex string). |
logRecords[].spanId | string | Span ID associated with this log record (hex string). |
Response
{
"partialSuccess": {}
}HTTP status 200 OK with an empty partialSuccess object indicates all records were accepted, matching the standard OTLP HTTP response format.
Authentication
Pass your credentials using HTTP Basic Auth or via the Authorization header:
Authorization: Basic <base64(user:password)>Configuring OpenTelemetry Collector
Configure the OTLP exporter in your OpenTelemetry Collector config.yaml to forward logs to OpenObserve:
exporters:
otlphttp/openobserve:
endpoint: https://<openobserve-host>/api/<organization>
headers:
Authorization: "Basic <base64(user:password)>"
stream-name: "custom_stream"
service:
pipelines:
logs:
receivers: [...]
processors: [...]
exporters: [otlphttp/openobserve]#!/bin/bash
# Reference recipe: create a small dashboard from scratch via the OpenObserve
# REST API. Replace the AUTH, HOST, ORG, and STREAM values for your env.
set -euo pipefail
HOST="${OO_HOST:-https://eu1.openobserve.ai}"
ORG="${OO_ORG:-your-org-id}"
AUTH="-u ${OO_EMAIL:?set OO_EMAIL}:${OO_PASSWORD:?set OO_PASSWORD}"
STREAM="${OO_STREAM:-claude_code}"
BASE="$HOST/api/$ORG"
PAYLOAD=$(cat <<JSON
{
"title": "API Generated Dashboard",
"description": "Created via REST API",
"version": 8,
"tabs": [{
"tabId": "default", "name": "Default",
"panels": [{
"id": "p1", "type": "metric", "title": "Total events",
"queryType": "sql",
"queries": [{
"query": "SELECT COUNT(*) AS value FROM \"$STREAM\"",
"customQuery": true,
"fields": {
"stream":"$STREAM","stream_type":"logs",
"x":[],"z":[],"breakdown":[],
"y":[{"label":"Value","alias":"value","column":"value","aggregationFunction":"sum","treatAsNonTimeseries":false}],
"filter":{"filterType":"group","logicalOperator":"AND","conditions":[]}
},
"config":{}
}],
"config": {"unit":"numbers","decimals":0},
"layout": {"x":0,"y":0,"w":96,"h":7,"i":1}
}]
}],
"variables": {"list": [], "showDynamicFilters": true},
"defaultDatetimeDuration": {"type":"relative","relativeTimePeriod":"30d"}
}
JSON
)
curl $AUTH -X POST -H 'Content-Type: application/json' \
"$BASE/dashboards?folder=default" -d "$PAYLOAD" | jq .
#!/bin/bash
# Reference recipe: search logs with a SQL query over a relative time window.
set -euo pipefail
HOST="${OO_HOST:-https://eu1.openobserve.ai}"
ORG="${OO_ORG:-your-org-id}"
AUTH="-u ${OO_EMAIL:?set OO_EMAIL}:${OO_PASSWORD:?set OO_PASSWORD}"
STREAM="${OO_STREAM:-claude_code}"
HOURS="${HOURS:-24}"
BASE="$HOST/api/$ORG"
NOW_US=$(($(date +%s) * 1000000))
START_US=$((NOW_US - HOURS * 3600 * 1000000))
curl -s $AUTH -H 'Content-Type: application/json' \
"$BASE/_search?type=logs" \
-d "{\"query\":{\"sql\":\"SELECT host_name, COUNT(*) AS n FROM \\\"$STREAM\\\" GROUP BY host_name ORDER BY n DESC\",\"start_time\":$START_US,\"end_time\":$NOW_US,\"size\":50}}" \
| jq .
#!/bin/bash
# Reference recipe: hash-aware mutation of a single dashboard panel.
set -euo pipefail
HOST="${OO_HOST:-https://eu1.openobserve.ai}"
ORG="${OO_ORG:-your-org-id}"
AUTH="-u ${OO_EMAIL:?set OO_EMAIL}:${OO_PASSWORD:?set OO_PASSWORD}"
DASH_ID="${1:?usage: update-panel.sh <dashboard_id> <panel_id>}"
PANEL_ID="${2:?usage: update-panel.sh <dashboard_id> <panel_id>}"
BASE="$HOST/api/$ORG"
# Fetch current dashboard to obtain hash
RAW=$(curl -s $AUTH "$BASE/dashboards/$DASH_ID?folder=default")
HASH=$(echo "$RAW" | jq -r .hash)
# Pull the existing panel, change just the title, and PUT it back
NEW_PANEL=$(echo "$RAW" | jq --arg pid "$PANEL_ID" '.v8.tabs[0].panels[] | select(.id == $pid) | .title = "Updated by API"')
curl $AUTH -X PUT -H 'Content-Type: application/json' \
"$BASE/dashboards/$DASH_ID/panels/$PANEL_ID?folder=default&hash=$HASH" \
-d "$NEW_PANEL" | jq .
Get schema for stream
Endpoint: GET /api/{organization}/streams/{stream}/schema?type={StreamType}
Request
- type: logs / metrics / traces
default is logs.
Response
{
"name": "k8s",
"storage_type": "s3",
"stream_type": "logs",
"stats": {
"doc_time_min": 1673715046856933,
"doc_time_max": 1673849134852901,
"doc_num": 3300000,
"file_num": 16,
"storage_size": 3323.5,
"compressed_size": 11.42
},
"schema": [
{
"name": "_timestamp",
"type": "Int64"
},
{
"name": "kubernetes.annotations.kubernetes.io/psp",
"type": "Utf8"
},
],
"settings": {
"partition_keys": {},
"full_text_search_keys": ["log"]
}
}Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| name | string | - | stream name |
| storage_type | string | - | s3 / disk |
| stream_type | string | logs | logs / metrics / traces |
| stats | object | - | stats for the stream |
| stats.doc_time_min | int64 | 0 | the minimum timestamp of the record in the stream |
| stats.doc_time_max | int64 | 0 | the maximum timestamp of the record in the stream |
| stats.doc_num | int64 | 0 | the records num of the stream |
| stats.file_num | int64 | 0 | the files num in storage of the stream |
| stats.storage_size | int64 | 0 | ingestion data size of the original data |
| stats.compressed_size | int64 | 0 | stored size in storage after compression |
| schema | array | - | the schema of the stream, if fetchSchema set to false, has no this field |
| schema.name | string | - | field name |
| schema.type | string | - | field data type: Utf8 / Int64 / Float64 / Timestamp / Boolean |
| settings | object | - | settings of the stream |
| settings.partition_keys | object | - | custom partition keys for the stream. By default OpenObserve uses timestamp as the first level partition key |
| settings.full_text_search_keys | array[string] | - | full text search fields, default OpenObserve uses log, message, msg, content, data, json, if there is no those fields in your stream, will report error: you should set the full text search fields. |
Search
Endpoint: POST /api/{organization}/_search
Request
{
"query": {
"sql": "SELECT * FROM {stream} WHERE [condition]",
"start_time": 1674789786006000,
"end_time": 1674789786006000,
"from": 0,
"size": 0
},
"search_type": "ui",
"timeout": 0
}Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| query | object | - | query params |
| query.sql | string | - | use SQL query data, and filter data by start_time and end_time, and default order by _timestamp, you can use order by override order, and fetch offset limit by form and size |
| query.start_time | int64 | 0 | unit: microseconds, filter data by time range, you need always provide this value |
| query.end_time | int64 | 0 | unit: microseconds, filter data by time range, you need always provide this value |
| query.from | int64 | 0 | offset in SQL |
| query.size | int64 | 0 | limit in SQL |
| search_type | string | - | default is empty, support: ui, dashboards, reports, alerts |
| timeout | int | 0 | default value based on ZO_QUERY_TIMEOUT=600 |
Response
{
"took": 155,
"hits": [
{
"_p": "F",
"_timestamp": 1674213225158000,
"kubernetes": {
"annotations": {
"kubernetes": {
"io/psp": "eks.privileged"
}
},
"container_hash": "dkr.ecr.us-west-2.amazonaws.com/ziox@sha256:3dbbb0dc1eab2d5a3b3e4a75fd87d194e8095c92d7b2b62e7cdbd07020f54589",
"container_image": "dkr.ecr.us-west-2.amazonaws.com/ziox:v0.0.3",
"container_name": "ziox",
"docker_id": "eb0983bdb9ff9360d227e6a0b268fe3b24a0868c2c2d725a1516c11e88bf5789",
"host": "ip.us-east-2.compute.internal",
"labels": {
"app": "ziox",
"controller-revision-hash": "ziox-ingester-579b7767cf",
"name": "ziox-ingester",
"role": "ingester",
"statefulset": {
"kubernetes": {
"io/pod-name": "ziox-ingester-0"
}
}
},
"namespace_name": "ziox",
"pod_id": "35a0421f-9203-4d73-9663-9ff0ce26d409",
"pod_name": "ziox-ingester-0"
},
"log": "[2023-01-20T11:13:45Z INFO actix_web::middleware::logger] 10.2.80.192 \"POST /api/demo/_bulk HTTP/1.1\" 200 68 \"-\" \"go-resty/2.7.0 (https://github.com/go-resty/resty)\" 0.001074",
"stream": "stderr"
}
],
"total": 27179431,
"from": 0,
"size": 1,
"scan_size": 28943
}Response description:
Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| took | int64 | - | unit: milliseconds, query execute time |
| from | int64 | 0 | value from query.from |
| size | int64 | 0 | value from query.size |
| scan_size | int64 | 0 | unit: MB, it response the data size scale when execute the query. |
| hits | array | - | records for query, each record is a log row what you ingested. |
SQL Syntax
Please refer to PostgreSQL for SQL Syntax.
Something need highlighted:
- We have a build-in time field,
_timestampyou can use it to do time range filter. - Field name can not start with
@. - Field name can use double quote or without quote.
- Field integer value without quote.
- Field string value must use single quote.
Limitation
- You should give a time range for each query or it will scan all data, it is a very expensive operate.
Examples
Here list some common examples, if you want more example please create a issue tell us, we will add it.
Query latest 10 record logs with histogram aggregation
{
"query": {
"sql": "SELECT * FROM {stream}",
"start_time": 1674789786006000,
"end_time": 1674789786006000,
"from": 0,
"size": 10
}
}Query latest 10 record logs
{
"query": {
"sql": "SELECT * FROM {stream}",
"start_time": 1674789786006000,
"end_time": 1674789786006000,
"from": 0,
"size": 10
}
}Query latest 10 record logs with filter
{
"query": {
"sql": "SELECT * FROM {stream} WHERE kubernetes.namespace_name='default' AND code=200 ",
"start_time": 1674789786006000,
"end_time": 1674789786006000,
"from": 0,
"size": 10
}
}Full text query
{
"query": {
"sql": "SELECT * FROM {stream} WHERE match_all('err') ",
"start_time": 1674789786006000,
"end_time": 1674789786006000,
"from": 0,
"size": 10
}
}Match on a filed (log)
{
"query": {
"sql": "SELECT * FROM {stream} WHERE str_match(log, 'err') ",
"start_time": 1674789786006000,
"end_time": 1674789786006000,
"from": 0,
"size": 10
}
}Histogram aggregation (full mode)
{
"query": {
"sql": "SELECT histogram(_timestamp, '5 minute') AS key, COUNT(*) AS num FROM {stream} GROUP BY key ORDER BY key LIMIT 10 OFFSET 1",
"start_time": 1674789786006000,
"end_time": 1674789786006000
}
}Term aggregation (full mode)
{
"query": {
"sql": "SELECT kubernetes.namespace_name AS namespace, COUNT(*) AS num FROM {stream} GROUP BY namespace ORDER BY namespace",
"start_time": 1674789786006000,
"end_time": 1674789786006000
}
}Use custom functions
{
"query": {
"sql": "SELECT *, my_func(log) as mykey FROM {stream}",
"start_time": 1674789786006000,
"end_time": 1674789786006000,
"from": 0,
"size": 10
}
}Set or Update Stream Settings
You can configure settings for a stream at creation time or update them later. Use the same endpoint with different HTTP methods depending on the operation.
Create Stream Settings
Use this operation to define stream settings when creating a stream.
Endpoint
POST /api/{org_id}/streams/{stream_name}/settingsRequest Body
Use the StreamSettings schema. All fields are optional.
{
"partition_keys": ["k8s_cluster", "k8s_namespace_name"],
"index_fields": ["k8s_pod_name", "k8s_container_name"],
"full_text_search_keys": ["body"],
"bloom_filter_fields": ["k8s_node_name"],
"data_retention": 30,
"flatten_level": 1,
"defined_schema_fields": [
"body",
"k8s_cluster",
"k8s_pod_name",
"k8s_app_component",
"log_file_path",
"service_name",
"service_version",
"severity"
],
"max_query_range": 30,
"store_original_data": true,
"approx_partition": false,
"extended_retention_days": [],
"index_original_data": false,
"index_all_values": false
}
Response
{
"code": 200
}Update Stream Settings
Use this operation to partially update an existing stream’s settings.
Endpoint
PUT /api/{org_id}/streams/{stream_name}/settingsRequest Body
Use the UpdateStreamSettings schema. Fields that support add, remove, or set use wrapper syntax.
{
"partition_keys": {
"set": ["k8s_cluster", "k8s_namespace_name"]
},
"full_text_search_keys": {
"set": ["body"]
},
"index_fields": {
"set": ["k8s_pod_name", "k8s_container_name"]
},
"bloom_filter_fields": {
"set": ["k8s_node_name"]
},
"data_retention": 30,
"flatten_level": 1,
"defined_schema_fields": {
"set": [
"body",
"k8s_cluster",
"k8s_pod_name",
"k8s_app_component",
"log_file_path",
"service_name",
"service_version",
"severity"
]
},
"max_query_range": 120,
"store_original_data": true,
"approx_partition": false,
"extended_retention_days": {
"add": []
},
"index_original_data": false,
"index_all_values": false
}
Response
{
"code": 200
}Field Description
StreamSettings Field Reference
| Field name | Description |
|---|---|
partition_keys | Fields used to create data partitions, shown as keyValue or Hash bucket in the UI. Improves read performance by skipping unrelated files. Does not make the field searchable. |
index_fields | Fields to create secondary indexes for exact-match filters. Improves query performance on field = value. |
full_text_search_keys | Fields tokenized for full-text search. Required for substring or match_all queries. Defaults to common log fields if not set. |
bloom_filter_fields | Fields with high-cardinality values to optimize rare value searches. Improves performance by skipping non-matching data blocks. |
data_retention | Number of days to retain data in the stream. Minimum is 3 days. Overrides the global retention setting. |
flatten_level | Maximum depth for flattening nested JSON objects into fields. Helps expose nested keys for querying. |
defined_schema_fields | Fields to retain in the user-defined schema. Others are excluded or stored as raw if store_original_data is true. |
max_query_range | Maximum time range in hours for a single query. Prevents resource-heavy long-range queries. |
store_original_data | Stores the full original log body if schema filtering is applied. Allows retrieval of dropped fields. |
approx_partition | Uses evenly divided time ranges for query execution. Helps distribute query load in skewed data. |
extended_retention_days | List of time ranges to retain data beyond data_retention. Must be applied before data expires. |
index_original_data | Enables full-text indexing on the raw log body. Allows search across fields not part of the schema. |
index_all_values | Indexes all fields for exact-match lookups. Increases ingestion and index size. Best for fixed schemas. |
Search around
Endpoint: GET /api/{organization}/{stream}/_values?fields={fields}&start_time={start_time}&end_time={end_time}&size=10&keyword=&no_count=false
Request
Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| stream | string | - | stream name |
| fields | string | - | the fields you want to get values, field1,field2 |
| size | int64 | 0 | how many values do you want to response, order by values num |
| start_time | int64 | 0 | Only list the values in the time range |
| end_time | int64 | 0 | Only list the values in the time range |
| keyword | string | - | search for the values |
| no_count | bool | false | set to true will not response count and order by the value |
Response
{
"took": 155,
"hits": [
{
"field": "field name",
"values": [
{
"zo_sql_key": "value1",
"zo_sql_num": 2070
}
]
}
],
"total": 10,
"from": 0,
"size": 0,
"scan_size": 28943
}Response description:
Description
| Field name | Data type | Default value | Description |
|---|---|---|---|
| took | int64 | - | unit: milliseconds, query execute time |
| from | int64 | 0 | value from query.from |
| size | int64 | 0 | value from query.size |
| scan_size | int64 | 0 | unit: MB, it response the data size scale when execute the query. |
| hits | array | - | records for query, each record is a log row what you ingested. |