Kibana Vega

Kibana Vega

Create and manage Kibana dashboards and Vega visualizations with ES|QL data sources.

Overview

Vega is a declarative visualization grammar for creating custom charts in Kibana. Combined with ES|QL queries, it enables highly customized visualizations beyond standard Kibana charts.

Important Version Requirement: This skill strictly supports ES|QL data sources and requires Serverless Kibana or version 9.4+ (SNAPSHOT). It will not work reliably on older versions or with older Lucene/KQL data source definitions.

Quick Start

Environment Configuration

Kibana connection is configured via environment variables. Run node scripts/kibana-vega.js test to verify the connection. If the test fails, suggest these setup options to the user, then stop. Do not try to explore further until a successful connection test.

Option 1: Elastic Cloud (recommended for production)

export KIBANA_CLOUD_ID="deployment-name:base64encodedcloudid"
export KIBANA_API_KEY="base64encodedapikey"

Option 2: Direct URL with API Key

export KIBANA_URL="https://your-kibana:5601"
export KIBANA_API_KEY="base64encodedapikey"

Option 3: Basic Authentication

export KIBANA_URL="https://your-kibana:5601"
export KIBANA_USERNAME="elastic"
export KIBANA_PASSWORD="changeme"

Option 4: Local Development with start-local

For local development and testing, use start-local to quickly spin up Elasticsearch and Kibana using Docker or Podman:

curl -fsSL https://elastic.co/start-local | sh

After installation completes, Elasticsearch runs at http://localhost:9200 and Kibana at http://localhost:5601. The script generates a random password for the elastic user, stored in the .env file inside the created elastic-start-local folder.

To configure the environment variables for this skill, source the .env file and export the connection settings:

source elastic-start-local/.env
export KIBANA_URL="$KB_LOCAL_URL"
export KIBANA_USERNAME="elastic"
export KIBANA_PASSWORD="$ES_LOCAL_PASSWORD"

Then run node scripts/kibana-vega.js test to verify the connection.

Optional: Skip TLS verification (development only)

export KIBANA_INSECURE="true"

Basic Workflow

# Test connection
node scripts/kibana-vega.js test

# Create visualization directly from stdin (no intermediate file needed)
echo '<json-spec>' | node scripts/kibana-vega.js visualizations create "My Chart" -

# Get visualization spec for review/modification
node scripts/kibana-vega.js visualizations get <vis-id>

# Update visualization from stdin
echo '<json-spec>' | node scripts/kibana-vega.js visualizations update <vis-id> -

# Create dashboard
node scripts/kibana-vega.js dashboards create "My Dashboard"

# Add visualization with grid position
node scripts/kibana-vega.js dashboards add-panel <dashboard-id> <vis-id> --x 0 --y 0 --w 24 --h 15

# Apply a complete layout from stdin
echo '<layout-json>' | node scripts/kibana-vega.js dashboards apply-layout <dashboard-id> -

Note: Use - as the file argument to read JSON from stdin. This enables direct spec creation without intermediate files.

Minimal Vega Spec with ES|QL

IMPORTANT: Always use proper JSON format (not HJSON with triple quotes) to avoid parse errors.

{
  "$schema": "https://vega.github.io/schema/vega-lite/v6.json",
  "title": "My Chart",
  "autosize": { "type": "fit", "contains": "padding" },

  "config": {
    "axis": { "domainColor": "#444", "tickColor": "#444" },
    "view": { "stroke": null }
  },

  "data": {
    "url": {
      "%type%": "esql",
      "query": "FROM logs-* | STATS count = COUNT() BY status | RENAME status AS category"
    }
  },

  "mark": { "type": "bar", "color": "#6092C0" },
  "encoding": {
    "x": { "field": "category", "type": "nominal" },
    "y": { "field": "count", "type": "quantitative" }
  }
}

ES|QL Data Source Options

| Property | Description | | --------------------------- | ------------------------------------------ | --------- | | %type%: "esql" | Required. Use ES | QL parser | | %context%: true | Apply dashboard filters | | %timefield%: "@timestamp" | Enable time range with ?_tstart/?_tend |

Examples

Stdin Examples

# Create visualization directly from JSON
echo '{"$schema":"https://vega.github.io/schema/vega-lite/v6.json",...}' | \
  node scripts/kibana-vega.js visualizations create "My Chart" -

# Update visualization
echo '{"$schema":...}' | node scripts/kibana-vega.js visualizations update <id> -

# Apply layout directly
echo '{"panels":[{"visualization":"<id>","x":0,"y":0,"w":24,"h":10}]}' | \
  node scripts/kibana-vega.js dashboards apply-layout <dash-id> -

Dashboard Layout Design

Grid System

Kibana dashboards use a 48-column grid:

Above the Fold (Critical)

Primary information must be visible without scrolling.

Height guidelines:

• h: 10 — Compact bar charts (≤7 items), fits above fold
• h: 12-13 — Standard charts, timelines
• h: 15+ — Detailed views, use below fold

Layout Pattern: Operational Dashboard

┌───────────────────────┬───────────────────────┐  y:0
│  Current State A      │  Current State B      │  h:10 (compact)
├───────────────────────┴───────────────────────┤  y:10
│         Primary Timeline                      │  h:12 (main trend)
├ ─ ─ ─ ─ ─ ─ ─ FOLD ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┤  y:22 (1080p fold)
│         Secondary Timeline                    │  h:12 (below fold OK)
├───────────────────────┬───────────────────────┤  y:34
│  Complementary 1      │  Complementary 2      │  h:10
└───────────────────────┴───────────────────────┘

Creating Layouts

Option 1: Add panels with positions

# Row 1: Two compact half-width charts (above fold)
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS1 --x 0 --y 0 --w 24 --h 10
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS2 --x 24 --y 0 --w 24 --h 10

# Row 2: Full-width timeline (above fold)
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS3 --x 0 --y 10 --w 48 --h 12

# Row 3: Below fold content
node scripts/kibana-vega.js dashboards add-panel $DASH $VIS4 --x 0 --y 22 --w 48 --h 12

Option 2: Apply layout file

Create layout.json:

{
  "title": "My Dashboard",
  "panels": [
    { "visualization": "<vis-id-1>", "x": 0, "y": 0, "w": 24, "h": 10 },
    { "visualization": "<vis-id-2>", "x": 24, "y": 0, "w": 24, "h": 10 },
    { "visualization": "<vis-id-3>", "x": 0, "y": 10, "w": 48, "h": 12 },
    { "visualization": "<vis-id-4>", "x": 0, "y": 22, "w": 48, "h": 12 }
  ]
}

Apply it:

node scripts/kibana-vega.js dashboards apply-layout <dashboard-id> layout.json

Design Checklist

1. Above the fold: Primary info in top ~22 height units (1080p) 2. Compact heights: Use h:10 for bar charts with ≤7 items 3. Prioritize: Most important info top-left 4. Group: Related charts side-by-side for comparison 5. Timelines: Full width (w:48), h:12 for compact 6. Below fold: Complementary/detailed panels OK to scroll

Guidelines

1. Use JSON, not HJSON triple-quotes — ''' multi-line strings cause parse errors in Kibana; use single-line queries with escaped quotes \" 2. Rename dotted fields — room.name breaks Vega (interpreted as nested path); use ES|QL RENAME room.name AS room 3. Don't set width/height — use autosize: { type: fit, contains: padding } 4. Set labelLimit on axes — horizontal bar chart labels truncate; use axis: { "labelLimit": 150 } 5. Sort bars by value — pre-sort in ES|QL with SORT field DESC and use sort: null in encoding (preserves data order); avoid sort: "-x" in layered specs (bar + text labels) as it causes "conflicting sort properties" warnings 6. Time axis: no rotated labels — use axis: { "labelAngle": 0, "tickCount": 8 }, let Vega auto-format dates 7. Descriptive titles replace axis titles — good title/subtitle makes axis titles redundant; use title: null on axes 8. Use color sparingly — color is a precious visual attribute; use a single default color (#6092C0) for bar charts where position already encodes value; reserve color encoding for categorical distinction (e.g., multiple lines in a time series) 9. Dark theme compatibility — always include config to avoid bright white borders:

   "config": {
     "axis": { "domainColor": "#444", "tickColor": "#444" },
     "view": { "stroke": null }
   }

CLI Commands

# Dashboards
node scripts/kibana-vega.js dashboards list [search]
node scripts/kibana-vega.js dashboards get <id>
node scripts/kibana-vega.js dashboards create <title>
node scripts/kibana-vega.js dashboards delete <id>
node scripts/kibana-vega.js dashboards add-panel <dash-id> <vis-id> [--x N] [--y N] [--w N] [--h N]
node scripts/kibana-vega.js dashboards apply-layout <dash-id> <file|->

# Visualizations (use - for stdin instead of file)
node scripts/kibana-vega.js visualizations list [vega]
node scripts/kibana-vega.js visualizations get <id>
node scripts/kibana-vega.js visualizations create <title> <file|->
node scripts/kibana-vega.js visualizations update <id> <file|->
node scripts/kibana-vega.js visualizations delete <id>

Full Documentation

• Dashboard Layout Reference — Grid system, layout patterns, design best

practices

• Vega-Lite Reference — Complete Vega-Lite grammar, chart patterns, best practices
• ES|QL in Vega Reference — ES|QL data source configuration, time filtering,

parameters

• Example Specs — Ready-to-use chart templates

Common Issues

Dashboard Layout Reference

This guide provides best practices for designing effective Kibana dashboard layouts.

Kibana Grid System

Kibana dashboards use a 48-column grid system:

• Full width: w: 48
• Half width: w: 24
• Third width: w: 16
• Quarter width: w: 12

Height guidelines (grid units):

• Compact: h: 8-10 — KPIs, small bar charts (≤7 items)
• Standard: h: 12-13 — Most charts, bar charts with labels
• Tall: h: 15-18 — Complex charts, detailed timelines

Above the Fold Design

Critical principle: The most important information must be visible without scrolling.

A typical screen at 1080p shows approximately h: 28-32 grid units above the fold (accounting for Kibana header, filters bar, and panel margins). At 1440p, this extends to h: 38-42.

Above the Fold Budget

Design Strategy

1. Primary panels above the fold: Current state, key metrics, main trend 2. Complementary panels below: Detailed breakdowns, secondary trends, historical data 3. Compact heights for bar charts: Use h: 10-12 instead of h: 15 for charts with ≤10 items

Dashboard Design Principles

1. Information Hierarchy

Place the most important information at the top-left where users look first:

┌─────────────────────────────────────────────────────┐
│  KPIs / Summary Metrics (top row)                   │
├────────────────────────┬────────────────────────────┤
│  Primary Chart         │  Secondary Chart           │
│  (main insight)        │  (supporting data)         │
├────────────────────────┴────────────────────────────┤
│  Timeline / Trend Chart (full width)                │
├────────────────────────┬────────────────────────────┤
│  Detail Chart 1        │  Detail Chart 2            │
└────────────────────────┴────────────────────────────┘

2. Chart Type Placement Guidelines

3. Common Layout Patterns

Operational Dashboard (Monitoring)

Best for: System health, real-time monitoring, alerts

┌──────────┬──────────┬──────────┬──────────┐
│  KPI 1   │  KPI 2   │  KPI 3   │  KPI 4   │  <- Status at a glance
├──────────┴──────────┴──────────┴──────────┤
│         Main Timeline (trends)            │  <- Primary metric over time
├───────────────────────┬───────────────────┤
│   Breakdown Chart 1   │  Breakdown Chart 2│  <- Drill-down by dimension
├───────────────────────┴───────────────────┤
│         Secondary Timeline                │  <- Supporting trends
└───────────────────────────────────────────┘

Grid coordinates:

• KPIs: {x:0, y:0, w:12, h:8}, {x:12, y:0, w:12, h:8}, {x:24, y:0, w:12, h:8}, {x:36, y:0, w:12, h:8}
• Main Timeline: {x:0, y:8, w:48, h:15}
• Breakdown 1: {x:0, y:23, w:24, h:15}
• Breakdown 2: {x:24, y:23, w:24, h:15}
• Secondary Timeline: {x:0, y:38, w:48, h:15}

Analytical Dashboard (Exploration)

Best for: Data analysis, comparisons, deep dives

┌───────────────────────┬───────────────────┐
│                       │   Filter/Summary  │
│   Primary Analysis    ├───────────────────┤
│   (large chart)       │   Top-N List      │
├───────────────────────┴───────────────────┤
│            Comparison Chart               │
├───────────────────────┬───────────────────┤
│   Dimension A         │   Dimension B     │
└───────────────────────────────────────────┘

Executive Dashboard (Overview)

Best for: High-level summaries, stakeholder reports

┌──────────┬──────────┬──────────┬──────────┐
│  KPI 1   │  KPI 2   │  KPI 3   │  KPI 4   │
├──────────┴──────────┼──────────┴──────────┤
│   Trend Chart 1     │    Trend Chart 2    │
├─────────────────────┴─────────────────────┤
│           Distribution / Breakdown        │
└───────────────────────────────────────────┘

Smart Home Dashboard Example

For IoT/smart home data with temperature, humidity, and device activity:

Compact Layout (Above the Fold)

Optimized for 1080p screens — all primary info visible without scrolling:

┌───────────────────────┬───────────────────────┐  y:0
│  Avg Temp by Room     │  Avg Humidity by Room │  h:10 (compact bars)
│  (7 rooms)            │  (6 rooms)            │
├───────────────────────┴───────────────────────┤  y:10
│         Temperature Timeline                  │  h:12 (primary trend)
├───────────────────────┴───────────────────────┤  y:22
│         Humidity Timeline                     │  h:12 (below fold on 1080p)
├───────────────────────┬───────────────────────┤  y:34
│  Device Activity      │  (future: alerts)     │  h:10 (complementary)
└───────────────────────┴───────────────────────┘

Above fold (y < 22-24): Bar charts + temperature timeline Below fold: Humidity timeline + device activity

Layout specification:

[
  { "id": "temp-by-room", "x": 0, "y": 0, "w": 24, "h": 10 },
  { "id": "humidity-by-room", "x": 24, "y": 0, "w": 24, "h": 10 },
  { "id": "temp-timeline", "x": 0, "y": 10, "w": 48, "h": 12 },
  { "id": "humidity-timeline", "x": 0, "y": 22, "w": 48, "h": 12 },
  { "id": "device-activity", "x": 0, "y": 34, "w": 24, "h": 10 }
]

Alternative: Three-Row Above Fold (1440p+)

For larger screens, fit more content above the fold:

[
  { "id": "temp-by-room", "x": 0, "y": 0, "w": 24, "h": 12 },
  { "id": "humidity-by-room", "x": 24, "y": 0, "w": 24, "h": 12 },
  { "id": "temp-timeline", "x": 0, "y": 12, "w": 48, "h": 13 },
  { "id": "humidity-timeline", "x": 0, "y": 25, "w": 48, "h": 13 },
  { "id": "device-activity", "x": 0, "y": 38, "w": 24, "h": 10 }
]

CLI Usage with Layout

Method 1: Individual Panels with Position

# Add panels with explicit grid positions
node scripts/kibana-vega.js dashboards add-panel <dashboard-id> <vis-id> --x 0 --y 0 --w 24 --h 15
node scripts/kibana-vega.js dashboards add-panel <dashboard-id> <vis-id> --x 24 --y 0 --w 24 --h 15

Method 2: Layout File

Create a layout file (dashboard-layout.json):

{
  "title": "Smart Home Operations",
  "panels": [
    { "visualization": "temp-by-room-id", "x": 0, "y": 0, "w": 24, "h": 15 },
    { "visualization": "humidity-by-room-id", "x": 24, "y": 0, "w": 24, "h": 15 },
    { "visualization": "temp-timeline-id", "x": 0, "y": 15, "w": 48, "h": 15 },
    { "visualization": "humidity-timeline-id", "x": 0, "y": 30, "w": 48, "h": 15 },
    { "visualization": "device-activity-id", "x": 0, "y": 45, "w": 24, "h": 15 }
  ]
}

Then apply:

node scripts/kibana-vega.js dashboards apply-layout <dashboard-id> dashboard-layout.json

Design Checklist

Before creating a dashboard, answer these questions:

1. Who is the audience? (Operators, analysts, executives) 2. What's the primary question? (Current status, trends, comparisons) 3. What actions should it enable? (Alerting, investigation, reporting)

Then follow this process:

1. Sketch the layout on paper or whiteboard first 2. Identify chart types for each data point 3. Prioritize - most important info top-left 4. Group related charts side-by-side for comparison 5. Use full width for timelines and trends 6. Keep KPIs small - they're glanceable summaries 7. Leave room for growth - dashboards evolve

Panel Size Quick Reference

Above the Fold Cheat Sheet

For 1080p (~30 units visible):

• 2 rows: h:10 + h:12 = 22 ✓ fits with margin
• 2 rows: h:12 + h:12 = 24 ✓ fits tight
• 2 rows: h:15 + h:15 = 30 ✗ second row cut off

For 1440p (~40 units visible):

• 3 rows: h:12 + h:12 + h:12 = 36 ✓ fits comfortably
• 3 rows: h:13 + h:13 + h:13 = 39 ✓ fits tight

Vega ES|QL Reference

Complete reference for using ES|QL queries in Kibana Vega visualizations.

Overview

Kibana's Vega plugin supports ES|QL as a data source through the %type%: "esql" URL configuration. This allows you to use Vega/Vega-Lite grammar with ES|QL's powerful piped query language.

CRITICAL: Always use proper JSON format for Vega specs. HJSON triple-quoted strings (''') cause parse errors in

Kibana's Vega plugin. Use single-line queries with escaped quotes instead.

Data URL Configuration

Basic ES|QL Query

{
  "data": {
    "url": {
      "%type%": "esql",
      "query": "FROM logs-* | STATS count=COUNT() BY status"
    }
  }
}

Full Configuration Options

| Property | Type | Required | Default | Description | | ----------------- | -------------------------------- | -------- | ------- | ----------------------------------------------------------------- | --------------- | | %type% | "esql" | Yes | - | Specifies the ES | QL query parser | | query | string | Yes | - | The ES | QL query string | | %context% | boolean | No | false | Apply dashboard filters to the query | | %timefield% | string | No | - | Field name for time-based filtering (enables ?_tstart/?_tend) | | dropNullColumns | boolean | No | true | Remove columns with all null values | | params | Array<Record<string, unknown>> | No | [] | Custom named parameters for the query |

Time Range Integration

Enabling Time Filtering

Set %timefield% to enable automatic time parameter injection:

{
  data: {
    url: {
      "%type%": "esql"
      "%timefield%": "@timestamp"
      query: "FROM logs-* | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend | STATS count=COUNT()"
    }
  }
}

Time Parameters

When %timefield% is set and your query contains these parameters, they are automatically populated:

Note: Parameters are case-insensitive (?_TSTART works too).

Example: Time Series

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json

  data: {
    url: {
      "%type%": "esql"
      "%timefield%": "@timestamp"
      query: '''
        FROM metrics-*
        | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
        | STATS avg_cpu = AVG(system.cpu.total.pct)
          BY bucket = DATE_TRUNC(5 minutes, @timestamp)
        | SORT bucket ASC
      '''
    }
  }

  mark: line

  encoding: {
    x: { field: bucket, type: temporal }
    y: { field: avg_cpu, type: quantitative }
  }
}

Dashboard Context (Filters)

Applying Dashboard Filters

Enable %context% to have dashboard-level filters automatically applied:

{
  data: {
    url: {
      "%type%": "esql"
      "%context%": true
      query: "FROM logs-* | STATS count=COUNT() BY host.name"
    }
  }
}

When a user adds filters in the dashboard (e.g., host.name: "server-01"), those filters are passed to the ES|QL query.

Combining Context and Time

{
  data: {
    url: {
      "%type%": "esql"
      "%context%": true
      "%timefield%": "@timestamp"
      query: '''
        FROM logs-*
        | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
        | STATS error_count = COUNT() BY service.name
        | WHERE error_count > 0
        | SORT error_count DESC
      '''
    }
  }
}

Custom Parameters

Using Named Parameters

Pass custom values to your ES|QL query:

{
  data: {
    url: {
      "%type%": "esql"
      query: "FROM logs-* | WHERE level = ?level | STATS count=COUNT()"
      params: [{ level: "ERROR" }]
    }
  }
}

Multiple Parameters

{
  data: {
    url: {
      "%type%": "esql"
      query: '''
        FROM logs-*
        | WHERE level = ?level AND service.name = ?service
        | STATS count=COUNT()
      '''
      params: [
        { level: "ERROR" }
        { service: "api-gateway" }
      ]
    }
  }
}

Combining with Time Parameters

{
  data: {
    url: {
      "%type%": "esql"
      "%timefield%": "@timestamp"
      query: '''
        FROM logs-*
        | WHERE @timestamp >= ?_tstart AND level = ?level
        | STATS count=COUNT()
      '''
      params: [{ level: "ERROR" }]
    }
  }
}

Response Transformation

ES|QL returns columnar data which is automatically transformed to row-based format for Vega.

ES|QL Response Format

{
  "columns": [
    { "name": "country", "type": "keyword" },
    { "name": "count", "type": "long" }
  ],
  "values": [
    ["US", 100],
    ["UK", 50]
  ]
}

Transformed Vega Data

[
  { "country": "US", "count": 100 },
  { "country": "UK", "count": 50 }
]

Handling Null Values

By default, dropNullColumns: true removes columns where all values are null. Set to false to preserve them:

{
  data: {
    url: {
      "%type%": "esql"
      query: "FROM logs-* | STATS count=COUNT(), errors=SUM(error) BY host.name"
      dropNullColumns: false
    }
  }
}

Multiple Data Sources

Named Data Sources

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json

  data: {
    name: "main_data"
    url: {
      "%type%": "esql"
      query: "FROM logs-* | STATS count=COUNT() BY status"
    }
  }

  // Additional data can be defined in layer or other sections
}

Layered Charts with Different Queries

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json

  layer: [
    {
      data: {
        url: {
          "%type%": "esql"
          "%timefield%": "@timestamp"
          query: "FROM logs-* | WHERE @timestamp >= ?_tstart | STATS requests=COUNT() BY bucket=DATE_TRUNC(1h, @timestamp)"
        }
      }
      mark: line
      encoding: {
        x: { field: bucket, type: temporal }
        y: { field: requests, type: quantitative }
      }
    }
    {
      data: {
        url: {
          "%type%": "esql"
          "%timefield%": "@timestamp"
          query: "FROM logs-* | WHERE @timestamp >= ?_tstart AND level == 'error' | STATS errors=COUNT() BY bucket=DATE_TRUNC(1h, @timestamp)"
        }
      }
      mark: { type: line, color: red }
      encoding: {
        x: { field: bucket, type: temporal }
        y: { field: errors, type: quantitative }
      }
    }
  ]
}

Kibana Configuration

Config Block

Use config.kibana for Kibana-specific settings:

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json

  config: {
    kibana: {
      hideWarnings: true
      type: "map"  // For map visualizations
      renderer: "svg"  // or "canvas"
    }
  }

  // ... rest of spec
}

Kibana Config Options

Best Practices

1. Don't Set Width/Height - Use Autosize

Kibana controls the visualization size through the dashboard panel. Let Kibana manage dimensions:

// Good - let Kibana control size
{
  $schema: https://vega.github.io/schema/vega-lite/v6.json

  autosize: {
    type: fit
    contains: padding
  }

  // ... rest of spec (no width/height)
}

// Bad - explicit dimensions conflict with Kibana
{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  width: 600
  height: 300
  // ...
}

2. Always Use Time Filtering for Time Series

// Good
{
  data: {
    url: {
      "%type%": "esql"
      "%timefield%": "@timestamp"
      query: "FROM logs-* | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend | ..."
    }
  }
}

// Bad - queries all data
{
  data: {
    url: {
      "%type%": "esql"
      query: "FROM logs-* | ..."
    }
  }
}

2. Use Context for Dashboard Integration

// When the chart should respond to dashboard filters
{
  data: {
    url: {
      "%type%": "esql"
      "%context%": true
      // ...
    }
  }
}

3. Limit Results

Always use LIMIT or aggregations to control result size:

{
  data: {
    url: {
      "%type%": "esql"
      query: "FROM logs-* | LIMIT 1000"  // Or use STATS for aggregation
    }
  }
}

4. Pre-sort Data for Time Series

{
  data: {
    url: {
      "%type%": "esql"
      query: '''
        FROM logs-*
        | STATS count=COUNT() BY bucket=DATE_TRUNC(1h, @timestamp)
        | SORT bucket ASC  // Important for line charts
      '''
    }
  }
}

5. Always Rename Dotted Fields

Dotted field names like service.name or room.name break Vega-Lite (they're interpreted as nested object paths). Always rename them:

{
  "data": {
    "url": {
      "%type%": "esql",
      "query": "FROM logs-* | STATS count=COUNT() BY service.name | RENAME service.name AS service"
    }
  }
}

6. Use Single-Line Queries (Avoid Triple Quotes)

HJSON triple-quoted strings (''') cause parse errors in Kibana. Use single-line queries with escaped quotes:

{
  "data": {
    "url": {
      "%type%": "esql",
      "query": "FROM logs-* | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend | STATS total = COUNT(), errors = COUNT(level == \"error\" OR NULL) BY service.name | RENAME service.name AS service | EVAL error_rate = errors / total * 100 | SORT error_rate DESC | LIMIT 20"
    }
  }
}

Comparison: ES|QL vs Elasticsearch DSL

ES|QL Approach

{
  data: {
    url: {
      "%type%": "esql"
      query: "FROM logs-* | STATS count=COUNT() BY status | SORT count DESC"
    }
  }
}

Elasticsearch DSL Approach (Traditional)

{
  data: {
    url: {
      index: "logs-*"
      body: {
        size: 0
        aggs: {
          by_status: {
            terms: { field: "status", order: { _count: "desc" } }
          }
        }
      }
    }
    format: { property: "aggregations.by_status.buckets" }
  }
}

ES|QL advantages:

• More readable syntax
• Easier to write and maintain
• Supports complex transformations inline
• Better for ad-hoc analysis

Vega-Lite Reference for Kibana

Complete reference for creating data visualizations using Vega-Lite in Kibana. This guide follows best practices from the UW Interactive Data Lab Visualization Curriculum.

Philosophy

At its core, a visualization maps data to visual properties. Vega-Lite embodies a grammar of graphics: you describe _what_ you want to visualize, not _how_ to draw it. This enables:

• Concise specifications
• Automatic inference of scales, axes, legends
• Composable multi-view displays
• Reproducible visualizations

---

Critical Pitfalls (Read First!)

⚠️ These issues cause silent failures. Your chart will render but show wrong/missing data.

1. Dot-Notation Field Names

Problem: Field names containing dots (e.g., room.name, host.ip, metric.value) are interpreted as nested object paths.

// Data from ES|QL: {"room.name": "Kitchen", "temp": 21}
// Vega-Lite looks for: {room: {name: "Kitchen"}}
// Result: "undefined" in labels, collapsed bars, broken legends

Solution: Use ES|QL RENAME to create simple field names:

FROM logs-*
| STATS count=COUNT() BY service.name
| RENAME service.name AS service

2. Don't Set Width/Height - Use Autosize

Problem: Kibana controls the panel size. Explicit dimensions cause conflicts.

// ❌ BAD - conflicts with Kibana panel sizing
{
  width: 600
  height: 300
  // ...
}

// ✅ GOOD - let Kibana control size
{
  autosize: {
    type: fit
    contains: padding
  }
  // ...
}

3. Horizontal Bar Chart Label Truncation

Y-axis labels get cut off on horizontal bar charts. Always set labelLimit:

"y": {"field": "category", "axis": {"labelLimit": 200}}

4. Legends vs Direct Labels

Legends force the reader's eye to jump back and forth. Label lines directly when possible:

{
  "layer": [
    { "mark": "line" },
    {
      "mark": { "type": "text", "align": "left", "dx": 5 },
      "transform": [
        {
          "window": [{ "op": "row_number", "as": "rank" }],
          "sort": [{ "field": "x", "order": "descending" }],
          "groupby": ["series"]
        },
        { "filter": "datum.rank === 1" }
      ],
      "encoding": { "text": { "field": "series" } }
    }
  ]
}

5. HJSON Triple-Quoted Strings Break Kibana

Problem: HJSON multi-line strings with ''' cause parse errors in Kibana's Vega plugin.

Error: End of input while parsing an object (missing '}')

Solution: Use proper JSON format with single-line queries and escaped quotes:

{
  "data": {
    "url": {
      "%type%": "esql",
      "query": "FROM logs-* | WHERE level == \"error\" | STATS count=COUNT() BY host"
    }
  }
}

6. Color Schemes Invisible on Dark Themes

Problem: Some color schemes with reverse: true render invisible on Kibana's dark theme.

// ❌ BAD - invisible bars on dark theme
"color": {
  "scale": { "scheme": "redyellowgreen", "reverse": true }
}

Solution: Use dark-theme-friendly schemes:

// ✅ GOOD - visible on both light and dark themes
"color": {
  "scale": { "scheme": "blues" }  // or: viridis, warmgreys, teals, purples
}

Safe color schemes: blues, greens, purples, teals, viridis, warmgreys, cividis

7. Sort Conflicts in Layered Specs

Problem: Using sort: "-x" on a shared encoding in a layered spec (e.g., bar + text value labels) causes:

Domains that should be unioned has conflicting sort properties. Sort will be set to true.

Vega-Lite tries to union the scale domains across layers and finds conflicting sort specifications.

Solution: Pre-sort data in ES|QL with SORT field DESC and use sort: null in encoding to preserve data order:

// ❌ BAD - causes "conflicting sort properties" warning in layered specs
"y": { "field": "category", "type": "nominal", "sort": "-x" }

// ✅ GOOD - pre-sort in ES|QL, use sort: null to preserve data order
// ES|QL: ... | SORT revenue DESC
"y": { "field": "category", "type": "nominal", "sort": null }

Note: sort: "-x" is fine in single-mark specs (no layers). The conflict only occurs in layer compositions where multiple marks share the same encoding axis.

8. Time Axis: No Rotated Labels

Problem: Rotated date labels create visual clutter and are hard to read.

// ❌ BAD - cluttered, hard to read
"axis": { "format": "%b %d %H:%M", "labelAngle": -45 }

Solution: Keep labels horizontal, let Vega auto-format, limit tick count:

// ✅ GOOD - clean horizontal labels, auto-formatted
"axis": { "labelAngle": 0, "tickCount": 8 }

Time axis best practices:

• Never rotate — use "labelAngle": 0
• Let Vega auto-format — omit format for intelligent date display
• Limit ticks — use "tickCount": 6-10 to prevent crowding
• Remove redundant title — use "title": null when axis is self-explanatory
• Compact y-axis titles — use "°C" or "%" instead of "Temperature (°C)"

// Optimal time series encoding
"encoding": {
  "x": {
    "field": "timestamp",
    "type": "temporal",
    "title": null,
    "axis": { "labelAngle": 0, "tickCount": 8 }
  },
  "y": {
    "field": "value",
    "type": "quantitative",
    "title": "°C",
    "scale": { "zero": false }
  }
}

---

Specification Structure

A Vega-Lite specification is a JSON/HJSON object:

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json

  // Let Kibana control sizing
  autosize: { type: fit, contains: padding }

  data: { ... }
  mark: "..."
  encoding: { ... }
}

Core Components

| Component | Description | | ----------- | --------------------------------------------------- | ----------------------------- | | data | Input data (ES | QL query, inline values, URL) | | mark | Geometric shape (bar, line, point, area, etc.) | | encoding | Mapping of data fields to visual channels | | transform | Data transformations (filter, aggregate, calculate) | | config | Styling defaults |

---

Data Types

Understanding data types is fundamental to choosing appropriate visual encodings.

Type Selection Guidelines

• Nominal: Use when equality comparison matters (A = B?)
• Ordinal: Use when rank order matters (A < B?)
• Quantitative: Use when magnitude/distance matters (A - B = ?)
• Temporal: Use for time-based data with calendar semantics

---

Encoding Channels

Channels map data fields to visual properties.

Position Channels

"encoding": {
  "x": {"field": "date", "type": "temporal"},
  "y": {"field": "value", "type": "quantitative"},
  "x2": {"field": "end_date"},
  "y2": {"field": "high_value"}
}

Mark Property Channels

"encoding": {
  "color": {"field": "category", "type": "nominal"},
  "size": {"field": "population", "type": "quantitative"},
  "shape": {"field": "region", "type": "nominal"},
  "opacity": {"field": "confidence", "type": "quantitative"}
}

Text & Tooltip Channels

"encoding": {
  "text": {"field": "label"},
  "tooltip": [
    {"field": "name", "title": "Country"},
    {"field": "value", "title": "GDP", "format": ",.0f"}
  ]
}

Facet Channels

"encoding": {
  "row": {"field": "region", "type": "nominal"},
  "column": {"field": "year", "type": "ordinal"}
}

---

Mark Types

Basic Marks

Composite Marks

Mark Properties

"mark": {
  "type": "bar",
  "color": "#4c78a8",
  "opacity": 0.8,
  "cornerRadius": 2,
  "strokeWidth": 0
}

---

Scales

Scales map data values to visual values.

Scale Types

Scale Configuration

"encoding": {
  "x": {
    "field": "value",
    "type": "quantitative",
    "scale": {
      "domain": [0, 100],
      "zero": true,
      "nice": true
    }
  }
}

Color Scales

"encoding": {
  "color": {
    "field": "temperature",
    "type": "quantitative",
    "scale": {
      "scheme": "viridis",
      "domain": [-10, 40]
    }
  }
}

Recommended Color Schemes:

---

Transforms

Data transformations within the spec.

Filter

"transform": [
  {"filter": "datum.year == 2020"},
  {"filter": {"field": "country", "oneOf": ["USA", "China", "India"]}}
]

Calculate

"transform": [
  {"calculate": "datum.revenue - datum.cost", "as": "profit"},
  {"calculate": "datum.value * 100 / datum.total", "as": "percentage"}
]

Aggregate

"transform": [
  {
    "aggregate": [
      {"op": "mean", "field": "temperature", "as": "avg_temp"},
      {"op": "count", "as": "n"}
    ],
    "groupby": ["month", "location"]
  }
]

Aggregation Operations: count, sum, mean, median, min, max, stdev, variance, q1, q3, distinct

Bin

"encoding": {
  "x": {
    "bin": true,
    "field": "temperature"
  },
  "y": {"aggregate": "count"}
}

Time Unit

"encoding": {
  "x": {
    "timeUnit": "yearmonth",
    "field": "date"
  }
}

Time Units: year, quarter, month, week, day, hours, minutes, yearmonth, yearmonthdate, hoursminutes

Window

"transform": [
  {
    "window": [
      {"op": "row_number", "as": "rank"},
      {"op": "sum", "field": "value", "as": "cumulative"}
    ],
    "sort": [{"field": "value", "order": "descending"}]
  }
]

Fold (Unpivot)

"transform": [
  {"fold": ["temp_min", "temp_max"], "as": ["measure", "value"]}
]

Regression

"transform": [
  {"regression": "y", "on": "x", "method": "linear"}
]

---

Multi-View Composition

Layer

Superimpose multiple marks on shared axes.

{
  "layer": [
    {
      "mark": { "type": "area", "opacity": 0.3 }
    },
    {
      "mark": { "type": "line", "color": "black" }
    }
  ],
  "encoding": {
    "x": { "field": "date", "type": "temporal" },
    "y": { "field": "value", "type": "quantitative" }
  }
}

Horizontal Concatenation (hconcat)

{
  "hconcat": [
    {"mark": "bar", "encoding": {...}},
    {"mark": "line", "encoding": {...}}
  ]
}

Vertical Concatenation (vconcat)

{
  "vconcat": [
    {"mark": "bar", "encoding": {...}},
    {"mark": "line", "encoding": {...}}
  ]
}

Facet (Small Multiples)

{
  "mark": "bar",
  "encoding": {
    "x": { "field": "value", "type": "quantitative" },
    "y": { "field": "category", "type": "nominal" },
    "column": { "field": "region", "type": "nominal" }
  }
}

Resolve

Control how scales/axes/legends are shared or independent.

{
  "layer": [...],
  "resolve": {
    "scale": {"y": "independent"},
    "axis": {"y": "independent"}
  }
}

---

Common Chart Patterns for Kibana

All examples use ES|QL data sources and proper Kibana sizing.

Horizontal Bar Chart with Value Labels

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  title: { text: "Sales by Region", anchor: "start" }
  autosize: { type: fit, contains: padding }

  data: {
    url: {
      "%type%": "esql"
      "%context%": true
      query: '''
        FROM sales-*
        | STATS sales = SUM(amount) BY region
        | SORT sales DESC
        | LIMIT 10
      '''
    }
  }

  layer: [
    { mark: { type: bar, cornerRadiusEnd: 3 } }
    {
      mark: { type: text, align: left, dx: 5, fontSize: 11 }
      encoding: { text: { field: sales, format: "," } }
    }
  ]

  // Use sort: null with layered specs; data is pre-sorted by ES|QL SORT
  encoding: {
    y: {
      field: region
      type: nominal
      sort: null
      title: null
      axis: { labelLimit: 150 }
    }
    x: { field: sales, type: quantitative, title: "Sales ($)" }
    color: { value: "#4c78a8" }
  }
}

Time Series with Area and Line

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  title: {
    text: "Request Rate"
    subtitle: "Requests per minute"
    anchor: start
  }
  autosize: { type: fit, contains: padding }

  data: {
    url: {
      "%type%": "esql"
      "%context%": true
      "%timefield%": "@timestamp"
      query: '''
        FROM logs-*
        | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
        | STATS requests = COUNT() BY bucket = DATE_TRUNC(1 minute, @timestamp)
        | SORT bucket ASC
      '''
    }
  }

  layer: [
    { mark: { type: area, opacity: 0.2, color: "#4c78a8" } }
    { mark: { type: line, color: "#4c78a8", strokeWidth: 2 } }
  ]

  encoding: {
    x: {
      field: bucket
      type: temporal
      title: null
      axis: { labelAngle: 0, tickCount: 8 }
    }
    y: {
      field: requests
      type: quantitative
      title: "Requests"
    }
  }
}

Multi-Line Chart with Direct Labels

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  title: { text: "Service Performance", anchor: start }
  autosize: { type: fit, contains: padding }

  data: {
    url: {
      "%type%": "esql"
      "%context%": true
      "%timefield%": "@timestamp"
      query: '''
        FROM metrics-*
        | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
        | STATS avg_latency = AVG(latency) BY bucket = DATE_TRUNC(5 minutes, @timestamp), service
        | SORT bucket ASC
      '''
    }
  }

  layer: [
    {
      mark: { type: line, strokeWidth: 2 }
    }
    {
      transform: [
        {
          window: [{ op: "row_number", as: "rank" }]
          sort: [{ field: "bucket", order: "descending" }]
          groupby: ["service"]
        }
        { filter: "datum.rank === 1" }
      ]
      mark: { type: text, align: left, dx: 8, fontSize: 12, fontWeight: bold }
      encoding: { text: { field: service } }
    }
  ]

  encoding: {
    x: {
      field: bucket
      type: temporal
      title: null
      axis: { labelAngle: 0, tickCount: 8 }
    }
    y: { field: avg_latency, type: quantitative, title: "ms" }
    color: { field: service, type: nominal, legend: null }
  }
}

Heatmap

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  title: { text: "Activity by Day and Hour", anchor: start }
  autosize: { type: fit, contains: padding }

  data: {
    url: {
      "%type%": "esql"
      "%context%": true
      "%timefield%": "@timestamp"
      query: '''
        FROM logs-*
        | WHERE @timestamp >= ?_tstart AND @timestamp <= ?_tend
        | EVAL hour = DATE_EXTRACT("HOUR_OF_DAY", @timestamp)
        | EVAL day = DATE_FORMAT("EEE", @timestamp)
        | STATS activity = COUNT() BY hour, day
      '''
    }
  }

  mark: { type: rect, cornerRadius: 2 }

  encoding: {
    x: { field: hour, type: ordinal, title: "Hour" }
    y: { field: day, type: nominal, title: null }
    color: {
      field: activity
      type: quantitative
      scale: { scheme: blues }
      title: "Events"
    }
  }
}

Grouped Bar Chart

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  autosize: { type: fit, contains: padding }

  data: {
    url: {
      "%type%": "esql"
      query: '''
        FROM sales-*
        | STATS revenue = SUM(amount) BY category, quarter
        | SORT category, quarter
      '''
    }
  }

  mark: { type: bar, cornerRadius: 2 }

  encoding: {
    x: { field: category, type: nominal, title: null }
    y: { field: revenue, type: quantitative, title: "Revenue" }
    xOffset: { field: quarter, type: nominal }
    color: {
      field: quarter
      type: nominal
      title: "Quarter"
      scale: { range: ["#4c78a8", "#72b7b2"] }
    }
  }
}

Scatter Plot with Trend Line

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  autosize: { type: fit, contains: padding }

  data: {
    url: {
      "%type%": "esql"
      query: '''
        FROM metrics-*
        | STATS cpu = AVG(cpu_percent), memory = AVG(memory_percent) BY host
      '''
    }
  }

  layer: [
    {
      mark: { type: point, filled: true, size: 60, opacity: 0.7 }
    }
    {
      mark: { type: line, color: "firebrick", strokeWidth: 2 }
      transform: [{ regression: "memory", on: "cpu" }]
    }
  ]

  encoding: {
    x: { field: cpu, type: quantitative, title: "CPU %" }
    y: { field: memory, type: quantitative, title: "Memory %" }
  }
}

---

Best Practices

1. Never Use Pie or Donut Charts

Humans cannot accurately compare arc lengths or angles. Always use sorted bar charts instead.

// ❌ AVOID - pie/donut charts
{ mark: { type: arc, innerRadius: 50 } }

// ✅ USE - sorted horizontal bar chart (pre-sort in ES|QL, use sort: null for layered specs)
{
  mark: bar
  encoding: {
    y: { field: category, sort: null }
    x: { field: value }
  }
}

2. Use Color to Encode Data, Not Decorate

• Single series = single color (don't add rainbow gradients)
• Reserve color for encoding meaningful data dimensions
• Use sequential schemes for quantitative data
• Use categorical schemes only for nominal data (≤10 categories)

3. Sort by Value, Not Alphabetically

Pre-sort data in ES|QL (SORT value DESC) and use sort: null to preserve that order. This is required for layered specs (bar + text labels) to avoid "conflicting sort properties" warnings. For single-mark specs, sort: "-x" also works.

// ✅ PREFERRED - works in all specs (single-mark and layered)
// ES|QL: ... | SORT revenue DESC
"encoding": {
  "y": {"field": "category", "sort": null}
}

// ⚠️ OK for single-mark only - causes warnings in layered specs
"encoding": {
  "y": {"field": "category", "sort": "-x"}
}

4. Annotate Values Directly on Bars

Use sort: null on the categorical axis (not sort: "-x") since this is a layered spec. Pre-sort data via ES|QL.

{
  layer: [
    { mark: bar }
    {
      mark: { type: text, align: left, dx: 5, fontSize: 11 }
      encoding: { text: { field: value, format: "," } }
    }
  ]
  // Data pre-sorted by ES|QL: ... | SORT value DESC
  encoding: {
    y: { field: category, sort: null }
    x: { field: value }
  }
}

5. Direct Label Instead of Legends

Place labels directly on data points:

{
  layer: [
    {
      mark: { type: line, strokeWidth: 2 }
      encoding: { color: { field: series, legend: null } }
    }
    {
      mark: { type: text, align: left, dx: 8, fontWeight: bold }
      transform: [
        {
          window: [{ op: "row_number", as: "rank" }]
          sort: [{ field: "x", order: "descending" }]
          groupby: ["series"]
        }
        { filter: "datum.rank === 1" }
      ]
      encoding: {
        text: { field: series }
        color: { field: series, legend: null }
      }
    }
  ]
}

6. Add Reference Lines for Context

{
  layer: [
    { mark: bar, encoding: {...} }
    {
      mark: { type: rule, strokeDash: [4, 4], color: "#999" }
      encoding: { y: { datum: 20 } }
    }
    {
      mark: { type: text, align: left, dx: 5, color: "#666" }
      encoding: {
        y: { datum: 20 }
        text: { value: "Target: 20" }
      }
    }
  ]
}

7. Descriptive Titles Replace Axis Titles

A good title/subtitle makes axis titles redundant. Remove them to reduce clutter.

// ❌ REDUNDANT - title and axis titles say the same thing
{
  "title": "Temperature Over Time",
  "encoding": {
    "x": { "field": "time", "title": "Time" },
    "y": { "field": "temp", "title": "Temperature (°C)" }
  }
}

// ✅ CLEAN - descriptive title, no axis titles needed
{
  "title": {
    "text": "Temperature Over Time",
    "subtitle": "Hourly readings, December 2025"
  },
  "encoding": {
    "x": { "field": "time", "title": null },
    "y": { "field": "temp", "title": null }
  }
}

When to keep axis titles:

• Units aren't obvious (use compact: "°C", "%", "ms")
• Multiple y-axes with different scales
• Scientific/technical charts where precision matters

"title": {
  "text": "Room Climate Comparison",
  "subtitle": "Temperature and humidity by room",
  "anchor": "start"
}

8. Use Small Multiples Over Complexity

Instead of overloading one chart, use faceting:

{
  mark: line
  encoding: {
    column: { field: region, type: nominal }
    x: { field: date, type: temporal }
    y: { field: value, type: quantitative }
  }
}

---

Kibana-Specific Configuration

Config Block for Kibana

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json

  config: {
    kibana: {
      hideWarnings: true
      renderer: "svg"  // or "canvas"
    }
    view: { stroke: null }
    axis: { labelFontSize: 12, titleFontSize: 14 }
  }

  // ... rest of spec
}

Kibana Config Options

---

Professional Theming

Light Theme

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  background: white

  config: {
    title: {
      color: "#171717"
      subtitleColor: "#737373"
      fontSize: 16
      subtitleFontSize: 12
      anchor: start
    }
    axis: {
      labelColor: "#525252"
      titleColor: "#525252"
      gridColor: "#e5e5e5"
      domainColor: "#d4d4d4"
      tickColor: "#d4d4d4"
    }
    legend: {
      labelColor: "#525252"
      titleColor: "#525252"
    }
    view: { stroke: null }
  }
}

Dark Theme

{
  $schema: https://vega.github.io/schema/vega-lite/v6.json
  background: "#0a0a0a"

  config: {
    title: {
      color: "#e5e5e5"
      subtitleColor: "#a3a3a3"
      fontSize: 16
      subtitleFontSize: 12
      anchor: start
    }
    axis: {
      labelColor: "#a3a3a3"
      titleColor: "#a3a3a3"
      gridColor: "#262626"
      domainColor: "#404040"
      tickColor: "#404040"
    }
    legend: {
      labelColor: "#a3a3a3"
      titleColor: "#a3a3a3"
    }
    view: { stroke: null }
  }
}

---

Quick Reference Checklist

Before finalizing any chart, verify:

• [ ] Descriptive title/subtitle — makes axis titles unnecessary
• [ ] Remove redundant axis titles — use title: null when chart title is clear
• [ ] No width/height set — use autosize: { type: fit, contains: padding }
• [ ] Simple field names — use RENAME in ES|QL for dotted fields
• [ ] Sort bars by value — pre-sort in ES|QL, use sort: null in layered specs (not sort: "-x")
• [ ] Time axis horizontal — labelAngle: 0, tickCount: 8, auto-format
• [ ] Value labels on bars for precise reading
• [ ] Direct labels on lines instead of legends (or legend at right)
• [ ] Compact units if needed — use "°C" not "Temperature (°C)"
• [ ] Reference lines for thresholds/targets with labels
• [ ] Consistent theming — same colors mean same things

References

• Vega-Lite Documentation
• Vega-Lite Examples
• UW Visualization Curriculum