Statsd

StatsD

Choose the right metric type, name it with dot-delimited hierarchy, tag dimensions instead of encoding them in names. StatsD is fire-and-forget: UDP means zero latency impact on your application, but wrong metric types or bad naming corrupt your data silently.

References

Metric Types

Wire format: <metric_name>:<value>|<type>[|@<sample_rate>][|#<tags>]

Decision Matrix

Wrong metric type = wrong math at the server. A gauge used as a counter loses data between flushes. A counter used as a gauge produces meaningless rates.

Counter (|c)

Measures rate of events over time. Server sums all values during flush interval, resets to 0 after flush, reports both raw count and per-second rate.

• Use for: request counts, error counts, event occurrences (cache hits, logins)
• Sample rate correction: value multiplied by 1/rate
• Supports sampling (|@<rate>)

Gauge (|g)

Instantaneous value at a point in time. Server stores last value received, retains between flushes (sticky).

• Use for: queue depth, active connections, memory/CPU usage, thread pool size
• Signed values (+N, -N) modify current value incrementally
• Cannot set to a negative number directly — set to 0 first, then decrement
• Do not sample gauges — server cannot correct for sampling on point-in-time values

Timer (|ms)

Duration of an operation in milliseconds. Server computes per flush interval: count, mean, upper (max), lower (min), sum, stddev, median, configurable percentiles (p90, p95, p99).

• Use for: HTTP request latency, DB query duration, function execution time
• Supports sampling (|@<rate>)

Histogram (|h)

Distribution of values over time. Identical to timer in most implementations. DogStatsD treats histograms as the native distribution type.

• Use for: request payload sizes, response body sizes, batch sizes
• Conceptually: timers measure duration, histograms measure arbitrary distributions

Set (|s)

Count of unique values per flush interval. Server tracks distinct values, reports cardinality at flush, resets.

• Use for: unique users, unique IPs, unique error codes per interval
• Do not sample sets — sampling breaks uniqueness tracking

Distribution (|d) — DogStatsD Only

Global distribution across all hosts. Raw values sent to Datadog servers (not aggregated locally). Use when you need accurate fleet-wide percentiles.

See ${CLAUDE_SKILL_DIR}/references/dogstatsd.md for distributions vs histograms comparison and protocol version details.

Naming

Format: <namespace>.<subsystem>.<target>.<metric>.<unit>

Example: myapp.api.request.duration.ms, myapp.cache.hit.count.total

Naming Rules

• Always namespace by service name — myapp.api.requests not just requests
• Use dot-delimited hierarchy
• Include the unit: .ms, .bytes, .total, .items
• Dimensions go in tags, not metric names (when tags are available)
• Use lowercase everywhere — some backends are case-sensitive
• Use underscores within path segments: http_request not httpRequest
• No dashes — they break Graphite navigation

See ${CLAUDE_SKILL_DIR}/references/naming.md for Graphite namespace mapping, character rules table, and naming anti-patterns.

Tags (DogStatsD)

Format: metric.name:1|c|#key1:value1,key2:value2 — comma-separated, no spaces.

Tag Rules

• Use tags for dimensions you will filter or group by — not metric names
• Keep cardinality bounded — each unique tag combination creates a separate time series
• No spaces in tag values — use underscores: region:us_east

Unified Service Tagging

Set these as global/constant tags on the client — attach to every metric automatically:

Tag Cardinality

Rule of thumb: if a tag can have >1000 distinct values, do not use it. Use logs or traces for high-cardinality data.

Aggregation and Flush

The flush cycle determines metric resolution. Default: 10 seconds.

• Counters reset to 0 after flush; gauges are sticky (retain last value)
• If no counter values received during flush: behavior depends on deleteCounters

config (default: send 0)

• Enable client-side aggregation for high-throughput applications (Go v5.0+,

Java v3.0+, .NET v7.0+) — pre-aggregates before sending to Agent

See ${CLAUDE_SKILL_DIR}/references/aggregation.md for flush mechanics, Graphite downsampling rules, DogStatsD aggregation details, and pre-aggregated timestamps.

Client Patterns

Initialization

• One client instance per application — do not create per-request
• Set namespace prefix — auto-prepends to all metric names
• Set global/constant tags — env, service, version set once
• Close/flush on shutdown — buffered metrics lost otherwise

Buffering

Enable client-side buffering — packs multiple metrics into single UDP packets. Reduces syscall overhead in hot paths. Most modern DogStatsD clients buffer by default. Call flush() before shutdown.

Sampling

Client randomly decides whether to send each metric based on sample rate. Datagram includes |@<rate> so server corrects the count.

Never sample gauges or sets — server cannot correct for these types.

See ${CLAUDE_SKILL_DIR}/references/client-patterns.md for high-throughput tuning steps, error handling, and Kubernetes deployment patterns.

Backends

See ${CLAUDE_SKILL_DIR}/references/backends.md for statsd_exporter configuration, Telegraf setup, and migration guides.

Application

When writing StatsD instrumentation:

• Choose the metric type based on what the value represents, not convenience.
• Apply naming conventions silently — don't narrate each rule.
• If an existing codebase contradicts a convention, follow the codebase

pattern and flag the divergence once.

• Always configure client-side buffering for production use.

When reviewing StatsD instrumentation:

• Check metric type correctness first — most common and most damaging mistake.
• Verify tag cardinality is bounded.
• Cite the specific issue and show the fix inline.

Integration

The coding skill governs workflow; this skill governs StatsD instrumentation choices.

Aggregation and Flush

StatsD is an aggregation daemon. Understanding flush intervals and aggregation rules is essential — misconfigured aggregation silently corrupts metric data, especially after downsampling.

The Flush Cycle

   App sends metrics (UDP)
          |
          v
   StatsD server buffers
          |
          v  (every flush interval, default 10s)
   Aggregate per metric type
          |
          v
   Forward to backend (Graphite, Datadog, etc.)
          |
          v
   Reset counters, sets; retain gauges

Flush Interval

Default: 10 seconds.

The flush interval determines: 1. Resolution — the finest granularity of your metric data 2. Backend alignment — must match the backend's storage schema 3. Network overhead — shorter intervals = more flushes = more traffic

Rule: Flush interval must be >= the backend's highest-resolution retention period. If Graphite stores 10-second data, flushing every 5 seconds means only the last value per 10-second window survives.

Aggregation Rules by Type

Counters at Flush

The server sends two values:

• count: Sum of all values received (corrected for sample rate)
• rate: count / flush_interval (per-second rate)

If no values received during a flush, behavior depends on deleteCounters config (default: send 0).

Gauges at Flush

The server sends the last value received. If no update occurred during the flush interval, it resends the previous value (sticky).

Exception: deleteGauges: true sends nothing if no update occurred. Use this for gauges that should disappear when a source stops reporting.

Timers at Flush

The server computes and sends multiple derived metrics:

stats.timers.<name>.count          # number of values received
stats.timers.<name>.mean           # average
stats.timers.<name>.upper          # maximum
stats.timers.<name>.lower          # minimum
stats.timers.<name>.sum            # sum of all values
stats.timers.<name>.stddev         # standard deviation
stats.timers.<name>.median         # 50th percentile
stats.timers.<name>.mean_90        # mean of values in 90th percentile
stats.timers.<name>.upper_90       # 90th percentile value
stats.timers.<name>.sum_90         # sum of values in 90th percentile

Percentile thresholds are configurable via percentThreshold.

Graphite Downsampling

Graphite stores data at multiple resolutions. As data ages, it is downsampled (rolled up) from high to low resolution. The aggregation method used during downsampling determines whether your data is correct.

Storage Schema Example

[stats]
pattern = ^stats.*
retentions = 10s:6h,1min:6d,10min:1800d

This means:

• 6 hours of 10-second resolution
• 6 days of 1-minute resolution
• ~5 years of 10-minute resolution

Storage Aggregation Rules

Different metric suffixes require different downsampling methods:

[min]
pattern = \.lower$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = \.upper(_\d+)?$
xFilesFactor = 0.1
aggregationMethod = max

[sum]
pattern = \.sum$
xFilesFactor = 0
aggregationMethod = sum

[count]
pattern = \.count$
xFilesFactor = 0
aggregationMethod = sum

[count_legacy]
pattern = ^stats_counts.*
xFilesFactor = 0
aggregationMethod = sum

[default_average]
pattern = .*
xFilesFactor = 0.3
aggregationMethod = average

Why This Matters

Consider a counter reporting count = 10 every 10 seconds. At 1-minute downsampling:

Consider a timer reporting upper = 500ms:

If your downsampling is wrong, you won't notice until you look at graphs for data older than your highest-resolution retention.

xFilesFactor

The minimum fraction of data points that must be non-null for a downsampled value to be stored (vs. stored as null).

• 0.0 — store a value even if only one data point exists
• 0.3 — require 30% of data points to be non-null
• 1.0 — require all data points to be non-null

For counts and sums: use 0 (every event matters). For averages: use 0.1-0.3 (a single sample is likely unrepresentative).

DogStatsD Aggregation

DogStatsD follows the same 10-second flush interval but aggregates differently depending on the metric type:

Key difference from plain StatsD: DogStatsD COUNTs are stored as RATE in Datadog. To see raw counts, apply cumulative_sum() or integral() functions in dashboards.

Client-Side Aggregation

Modern DogStatsD clients (Go v5.0+, Java v3.0+, .NET v7.0+) can aggregate metrics before sending to the Agent, reducing network traffic and Agent CPU load.

What gets aggregated client-side:

• Counters: summed
• Gauges: last value kept
• Sets: unique values tracked

What is NOT aggregated client-side:

• Histograms and distributions: raw values forwarded

Enable client-side aggregation for high-throughput applications that emit thousands of metrics per second.

Timestamps and No-Aggregation

DogStatsD v1.3+ supports sending pre-aggregated metrics with explicit timestamps. When a timestamp is present, the Agent forwards the value without aggregation.

page.views:150|c|#env:prod|T1656581400

Use case: When your application already performs its own aggregation (e.g., collecting metrics in-memory and flushing periodically), send pre-aggregated values with timestamps to avoid double-aggregation.

Backends and Integrations

StatsD is a protocol, not a destination. Metrics flow from the StatsD daemon to one or more backends for storage, visualization, and alerting. Each backend has different requirements for metric naming, types, and aggregation.

Architecture Overview

Application  --UDP-->  StatsD Daemon  --flush-->  Backend
                       (aggregation)              (storage + query)

Common topologies:

# Classic: StatsD -> Graphite
App -> statsd (Node.js) -> Graphite (Carbon + Whisper)

# Datadog: App -> DogStatsD Agent -> Datadog API
App -> datadog-agent (DogStatsD) -> Datadog cloud

# Prometheus: StatsD -> Exporter -> Prometheus
App -> statsd_exporter -> Prometheus (scrape)

# Telegraf: StatsD -> Telegraf -> InfluxDB/other
App -> Telegraf (StatsD input) -> InfluxDB / Prometheus / etc.

Graphite

The original StatsD backend. Graphite stores time-series data in Whisper files with configurable retention and downsampling.

Key Considerations

1. Dots become folders. myapp.api.request.count creates the path myapp/api/request/count.wsp. Choose names that form a navigable hierarchy.

2. Retention must align with flush interval. If Graphite's highest-resolution retention is 10 seconds, StatsD must flush at least every 10 seconds. Faster flushes cause data loss (only last value per interval survives).

3. Aggregation method must match metric type. Misconfigured downsampling silently corrupts data. See aggregation.md for correct configuration.

4. No native tags. All dimensions must be encoded in the metric name. This is the primary limitation of Graphite-backed StatsD.

Storage Schema Example

[stats]
pattern = ^stats.*
retentions = 10s:6h,1min:6d,10min:1800d

Prometheus via statsd_exporter

The statsd_exporter translates StatsD push metrics into Prometheus pull metrics. It is a drop-in replacement for a StatsD server.

Deployment Pattern

Recommended: Run as a sidecar alongside the application pod.

+-------------+    +----------+    +------------+
| Application +--->| Exporter |<---+ Prometheus |
+-------------+    +----------+    +------------+

Transitional: Run alongside existing StatsD with relay mode.

+-------------+    +----------+    +--------+
| Application +--->| Exporter +--->| StatsD |
+-------------+    +----------+    +--------+
                        ^
                   +----+-------+
                   | Prometheus |
                   +------------+

Metric Type Mapping

Timer conversion: StatsD timers report in milliseconds; Prometheus expects seconds. The exporter converts automatically.

Mapping Rules

Configure metric name translation via YAML:

mappings:
- match: "myapp.api.*.request.duration"
  name: "http_request_duration_seconds"
  labels:
    endpoint: "$1"
  observer_type: histogram
  histogram_options:
    buckets: [0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10]

- match: "myapp.api.*.request.count"
  name: "http_requests_total"
  labels:
    endpoint: "$1"

Glob matching: * matches one dot-separated segment. Use for extracting labels from metric names.

Regex matching: Available for complex patterns but significantly slower. Glob rules are evaluated first regardless of order.

Tag Format Support

The exporter parses multiple tag formats:

Do not mix tag formats on the same metric.

Unmapped Metrics

Metrics that don't match any mapping rule:

• Non-alphanumeric characters (including dots) become underscores
• No labels are added
• Type is inferred from StatsD type

To drop unmapped metrics:

mappings:
- match: "."
  match_type: regex
  action: drop
  name: "dropped"

Telegraf StatsD Input

Telegraf can act as a StatsD server, receiving metrics and forwarding them to any Telegraf output (InfluxDB, Prometheus, Datadog, etc.).

Key Configuration

[[inputs.statsd]]
  protocol = "udp"
  service_address = ":8125"
  percentiles = [50.0, 90.0, 99.0, 99.9, 100.0]
  metric_separator = "_"

  # Enable DogStatsD extensions (tags, events, service checks)
  datadog_extensions = false

  # Enable DogStatsD distribution metrics
  datadog_distributions = false

Metric Output

Telegraf transforms StatsD metrics into its own data model:

• Counters — fields: value
• Gauges — fields: value
• Sets — fields: value (count of uniques)
• Timers/Histograms — fields: lower, upper, mean, median,

stddev, sum, count, percentile_<P>

All metrics get the tag metric_type=<gauge|counter|set|timing|histogram>.

Template Patterns

Telegraf can transform StatsD bucket names into tagged metrics:

templates = [
    "cpu.* measurement.measurement.region",
    "mem.* measurement.measurement.host",
]

Transforms:

cpu.load.us-west:100|g  =>  cpu_load,region=us-west value=100
mem.cached.host01:256|g =>  mem_cached,host=host01 value=256

DogStatsD Compatibility

Enable datadog_extensions = true to parse:

• DogStatsD tags (|#key:val)
• Events (_e{...})
• Service checks (_sc|...)
• Distribution metrics (with datadog_distributions = true)

Backend Selection Guide

Migration Considerations

StatsD -> Prometheus

1. Deploy statsd_exporter as sidecar 2. Configure mapping rules for clean Prometheus metric names 3. Use relay mode to keep existing StatsD backend during transition 4. Gradually switch dashboards and alerts to Prometheus queries 5. Remove relay once transition is complete

Plain StatsD -> DogStatsD

1. Replace StatsD server with Datadog Agent 2. Add tags to metric calls (instead of encoding dimensions in names) 3. Switch from |ms timers to |h histograms or |d distributions 4. Set up unified service tagging (env, service, version) 5. Update dashboards to use tag-based queries

Client Library Patterns

StatsD clients send UDP datagrams from your application to the local StatsD/DogStatsD agent. Client configuration has a direct impact on metric accuracy and application performance.

Client Setup

Connection

StatsD listens on UDP port 8125 by default. DogStatsD also supports Unix Domain Sockets (UDS) for lower overhead.

# Standard StatsD
host: 127.0.0.1, port: 8125, protocol: UDP

# DogStatsD with UDS (lower overhead, recommended for containers)
socket_path: /var/run/datadog/dsd.socket

Initialization Best Practices

1. Create one client instance per application. Do not create a new client per request — connection setup and buffering state would be lost. 2. Set a namespace prefix. Automatically prepends to all metric names: namespace: "myapp" turns request.count into myapp.request.count. 3. Set global/constant tags. Tags that apply to every metric (env, service, version) should be set once at initialization. 4. Close/flush on shutdown. Buffered metrics are lost if the client is not flushed before process exit.

Language Examples

Go:

client, err := statsd.New("127.0.0.1:8125",
    statsd.WithNamespace("myapp."),
    statsd.WithTags([]string{"env:prod", "service:api", "version:2.1"}),
)
defer client.Close()

Python:

from datadog import initialize, statsd

initialize(
    statsd_host="127.0.0.1",
    statsd_port=8125,
    statsd_namespace="myapp",
    statsd_constant_tags=["env:prod", "service:api", "version:2.1"],
)

Java:

StatsDClient client = new NonBlockingStatsDClientBuilder()
    .prefix("myapp")
    .hostname("localhost")
    .port(8125)
    .constantTags("env:prod", "service:api", "version:2.1")
    .build();

Ruby:

statsd = Datadog::Statsd.new(
  'localhost', 8125,
  namespace: 'myapp',
  tags: ['env:prod', 'service:api', 'version:2.1']
)

Buffering

By default, some clients send one UDP packet per metric call. This creates excessive syscall overhead in hot paths.

Enable Buffering

Buffering packs multiple metrics into a single UDP packet, separated by newlines. Most modern DogStatsD clients buffer by default.

Key settings:

• Max packet size: 1432 bytes (UDP, safe for Ethernet MTU) or

8192 bytes (UDS)

• Flush interval: Automatic flush every ~100-300ms (client-dependent)
• Manual flush: Call flush() before shutdown or at critical points

Go — buffers by default, no configuration needed.

Python:

# v0.43.0+: buffering enabled by default
dsd = DogStatsd(host="127.0.0.1", port=8125, disable_buffering=False)

# Pre-v0.43.0: use context manager
with DogStatsd() as dsd:
    dsd.gauge('metric_1', 123)
    dsd.gauge('metric_2', 456)
# Flushes on context exit

Java:

StatsDClient client = new NonBlockingStatsDClientBuilder()
    .hostname("127.0.0.1")
    .port(8125)
    .maxPacketSizeBytes(1500)  // Buffer up to 1500 bytes per packet
    .build();

Sampling

Sampling reduces UDP traffic at the cost of statistical accuracy. The client randomly decides whether to send each metric based on the sample rate.

How It Works

client.increment("requests", rate=0.1)

• Client rolls a random number: if < 0.1, send the metric
• The datagram includes |@0.1 so the server knows to multiply by 10
• ~90% of calls produce zero network traffic

Sample Rate Corrections by Type

When to Sample

Do not sample gauges or sets. The server cannot correct for sampling on these types — you'll get randomly missing data points.

High-Throughput Tuning

When sending thousands of metrics per second, the default configuration may cause packet drops. Symptoms:

• High Agent CPU usage
• datadog.dogstatsd.client.packets_dropped increasing
• Missing data points in dashboards

Mitigation Strategies (in order)

1. Enable client-side buffering — packs metrics into fewer packets.

2. Enable client-side aggregation — client pre-aggregates counters, gauges, and sets before sending. Available in Go v5.0+, Java v3.0+, .NET v7.0+.

   // Go: enabled by default in v5.0+
   client, _ := statsd.New("127.0.0.1:8125")

   // Java: enabled by default in v3.0+
   StatsDClient client = new NonBlockingStatsDClientBuilder()
       .enableAggregation(true)
       .build();

3. Use UDS instead of UDP — Unix Domain Sockets have lower overhead than UDP (no IP/UDP header processing, no kernel buffer limits).

4. Increase kernel buffer sizes (Linux, for UDP):

   sysctl -w net.core.rmem_max=26214400

Set dogstatsd_so_rcvbuf: 26214400 in Agent config to match.

5. Sample high-volume metrics — reduce rate to 0.5 or 0.1 for counters and timers in hot paths.

6. Enable Agent pipeline autoadjust — Agent uses multiple cores for metric processing:

   dogstatsd_pipeline_autoadjust: true

7. Increase client queue size — prevents drops when Agent is temporarily slow:

   .queueSize(8192)  // default: 4096

Error Handling

StatsD uses UDP, which is fire-and-forget. The client cannot know if the Agent received the metric. This is by design — metric emission should never block or crash the application.

Client Error Patterns

Do:

• Log client initialization failures at startup
• Monitor datadog.dogstatsd.client.packets_dropped telemetry
• Set up an error handler for internal client errors (Java, .NET)

Do not:

• Wrap every metric call in try/catch
• Retry failed sends (UDP has no concept of retry)
• Block the application if StatsD is unavailable

Graceful Degradation

If the StatsD agent is down:

• UDP sends silently fail (packets dropped by OS)
• Application continues unaffected
• Metrics are lost for the downtime period
• No recovery of lost data (StatsD is not durable)

This is the tradeoff: zero latency impact in exchange for at-most-once delivery.

Kubernetes Deployment

Finding the Agent Host

Use the downward API to expose the node IP:

env:
  - name: DD_AGENT_HOST
    valueFrom:
      fieldRef:
        fieldPath: status.hostIP

The application connects to $DD_AGENT_HOST:8125.

UDS in Kubernetes

Mount the DogStatsD socket as a volume:

volumes:
  - name: dsdsocket
    hostPath:
      path: /var/run/datadog/
volumeMounts:
  - name: dsdsocket
    mountPath: /var/run/datadog

The application connects to /var/run/datadog/dsd.socket.

UDS provides better performance and avoids hostPort networking complexities.

DogStatsD Extensions

DogStatsD is Datadog's extension of the StatsD protocol. It adds tags, histograms, distributions, service checks, and events. Any compliant StatsD client works with DogStatsD for basic metrics, but the extensions require Datadog client libraries.

Protocol Format

<METRIC_NAME>:<VALUE>|<TYPE>|@<SAMPLE_RATE>|#<TAG_KEY>:<TAG_VALUE>,<TAG2>

Tags

Tags are the primary advantage of DogStatsD over plain StatsD. They allow multi-dimensional queries without encoding dimensions in the metric name.

Format

metric.name:1|c|#key1:value1,key2:value2,bare_tag

• Key-value tags: env:production, method:GET
• Bare tags (no value): deprecated, canary
• Comma-separated, no spaces

Unified Service Tagging

Datadog recommends three global tags on every metric:

Set these as constant_tags / global_tags on the client instance so they attach to every metric automatically.

Tag Best Practices

1. Use tags for queryable dimensions. If you will filter or group by a value in dashboards, it should be a tag. 2. Keep cardinality bounded. Each unique tag combination creates a separate time series. Unbounded tags (user IDs, request IDs) explode custom metric counts and backend costs. 3. Prefer lowercase. Tags are case-sensitive in Datadog. Method:GET and method:GET are different tags. 4. No spaces in tag values. Use underscores: region:us_east.

Histograms (|h)

DogStatsD histograms compute aggregates locally in the Agent, then flush summary statistics to Datadog.

Produced metrics per histogram:

Configure aggregates in datadog.yaml:

• histogram_aggregates: max, median, avg, count (default)
• histogram_percentiles: 0.95 (default)

Distributions (|d)

Distributions send raw values to Datadog for server-side aggregation. This provides globally accurate percentiles across all hosts.

request.duration:42|d|#service:api,env:prod

Produced metrics: sum, count, avg, min, max, p50, p75, p90, p95, p99.

When to Use Distribution vs. Histogram

Use Distribution when:

• You need accurate p99 across 100+ hosts
• Per-host percentiles would be misleading (e.g., load-balanced traffic)
• You want to configure percentile thresholds without Agent restart

Use Histogram when:

• Per-host aggregation is sufficient
• You want lower network overhead
• You need compatibility with plain StatsD backends

Events

DogStatsD can send events (not just metrics) to the Datadog event stream.

Format:

_e{<TITLE_LENGTH>,<TEXT_LENGTH>}:<TITLE>|<TEXT>|d:<TIMESTAMP>|h:<HOSTNAME>|p:<PRIORITY>|t:<ALERT_TYPE>|#<TAGS>

Parameters:

Example:

_e{14,21}:Deploy started|v2.1.0 to prod env|t:info|#service:api,env:prod

When to use events:

• Deployment markers
• Configuration changes
• Significant application events (not per-request)
• Events you want to overlay on dashboards

Service Checks

Report health status of a service or dependency.

Format:

_sc|<NAME>|<STATUS>|d:<TIMESTAMP>|h:<HOSTNAME>|#<TAGS>|m:<MESSAGE>

Status codes:

Example:

_sc|redis.connection|0|#env:prod|m:Connection healthy
_sc|redis.connection|2|#env:prod|m:Connection timed out after 10s

When to use service checks:

• Database connectivity status
• External API reachability
• Health check results
• Circuit breaker state

DogStatsD Protocol Versions

Value packing (v1.1): Send multiple values for the same metric in a single datagram, reducing packet count:

request.duration:42:38:55:41|d|#service:api

Timestamps (v1.3): Pre-aggregated metrics can include a Unix timestamp to skip Agent-side aggregation:

page.views:150|c|#env:prod|T1656581400

Metric Types

StatsD supports several metric types, each with distinct aggregation semantics. Choosing the wrong type produces silently incorrect data.

Wire Format

All metrics use UTF-8 text over UDP:

<metric_name>:<value>|<type>[|@<sample_rate>][|#<tags>]

Multiple metrics can share a single UDP packet, separated by newlines. Keep total payload under the network MTU (Fast Ethernet: 1432 bytes, Gigabit/jumbo: 8932 bytes, Internet: ~512 bytes).

Counter (|c)

What it measures: Rate of events over time.

Wire format: metric.name:<value>|c[|@<sample_rate>]

Server behavior:

• Sums all values received during the flush interval
• Resets to 0 after each flush
• Reports both the raw count and the per-second rate
• Sample rate correction: value multiplied by 1/rate

When to use:

• Request counts
• Error counts
• Event occurrences (cache hits, logins, purchases)
• Any "how many times did X happen?"

Examples:

requests.total:1|c                    # increment by 1
requests.total:5|c                    # increment by 5
requests.total:1|c|@0.1              # sampled: server multiplies by 10
page.views:1|c|#page:/home,method:GET # DogStatsD tagged counter

Aggregation at flush:

• stats.counters.<name>.count = sum of all received values (rate-corrected)
• stats.counters.<name>.rate = count / flush_interval

Gauge (|g)

What it measures: Instantaneous value at a point in time.

Wire format: metric.name:<value>|g

Server behavior:

• Stores the last value received
• Retains value between flushes (persists until next update)
• Signed values (+N, -N) modify the current value incrementally

When to use:

• Queue depth
• Active connections
• Memory/CPU usage
• Thread pool size
• Any "what is the current value of X?"

Examples:

queue.depth:42|g                # set to 42
temperature.celsius:21.5|g      # set to 21.5
queue.depth:+5|g                # increment current value by 5
queue.depth:-3|g                # decrement current value by 3

Important: You cannot set a gauge to a negative number directly. Set to zero first, then decrement:

gauge.value:0|g
gauge.value:-10|g

Sampling: Do not sample gauges. The server cannot correct for sampling on point-in-time values.

Timer (|ms)

What it measures: Duration of an operation in milliseconds.

Wire format: metric.name:<value>|ms[|@<sample_rate>]

Server behavior: Computes statistical aggregates per flush interval:

• count — number of timing values received
• mean / mean_<pct> — average (overall and per-percentile)
• upper / upper_<pct> — maximum value (overall and per-percentile)
• lower — minimum value
• sum / sum_<pct> — total (overall and per-percentile)
• stddev — standard deviation
• median — 50th percentile
• Configurable percentile thresholds (e.g., p90, p95, p99)

When to use:

• HTTP request latency
• Database query duration
• External API call time
• Function execution time
• Any "how long did X take?"

Examples:

api.request.duration:320|ms             # 320ms request
db.query.time:45|ms                     # 45ms query
api.request.duration:120|ms|@0.5        # sampled at 50%
render.time:85|ms|#template:homepage    # DogStatsD tagged

Histogram (|h)

What it measures: Distribution of values over time.

Wire format: metric.name:<value>|h[|@<sample_rate>]

Server behavior: Identical to timer in most implementations. DogStatsD treats histograms as the native distribution type, producing:

• .count — number of values received
• .avg — average value
• .median — 50th percentile
• .max — maximum value
• .95percentile — 95th percentile (configurable)

When to use:

• Request payload sizes
• Response body sizes
• Batch sizes
• Any distribution measurement that isn't strictly a duration

Difference from timer: Conceptually, timers measure duration; histograms measure arbitrary distributions. Most StatsD servers treat them identically. DogStatsD uses |h as the canonical histogram type and treats |ms as a histogram that happens to record durations.

Set (|s)

What it measures: Count of unique values per flush interval.

Wire format: metric.name:<value>|s

Server behavior:

• Tracks unique values in a set data structure
• Reports the cardinality (count of distinct values) at each flush
• Resets the set after each flush

When to use:

• Unique users per interval
• Unique IP addresses
• Unique error codes
• Any "how many distinct X occurred?"

Examples:

users.unique:user123|s           # track user123
users.unique:user456|s           # track user456
users.unique:user123|s           # duplicate, ignored
# At flush: count = 2

Sampling: Do not sample sets. Sampling breaks uniqueness tracking.

Distribution (|d) — DogStatsD Only

What it measures: Global distribution across all hosts.

Wire format: metric.name:<value>|d[|@<sample_rate>][|#<tags>]

Server behavior:

• Raw values sent to Datadog servers (not aggregated locally)
• Computes percentiles globally across all reporting hosts
• Produces: sum, count, avg, min, max, p50, p75, p90, p95, p99

When to use:

• Latency when you need global percentiles (not per-host)
• Request sizes across a fleet
• Any metric where per-host aggregation would lose meaning

Difference from histogram: Histograms aggregate locally per agent, then send summary statistics. Distributions send raw data points for global aggregation. Distributions are more accurate for fleet-wide percentiles but cost more in data transfer and Datadog custom metric billing.

Decision Matrix

Multi-Metric Packets

Pack multiple metrics into a single UDP datagram separated by \n:

requests.total:1|c\nresponse.time:42|ms\nqueue.depth:10|g

This reduces syscall overhead. Most client libraries handle this automatically when buffering is enabled.

Naming Conventions

Metric names are the primary axis of organization in StatsD. In Graphite, dots become folder separators. In Datadog, dots become hierarchical groupings. Poor naming makes metrics undiscoverable and dashboards unmaintainable.

Hierarchy Structure

<namespace>.<subsystem>.<target>.<metric>.<unit>

Examples:

myapp.api.request.duration.ms
myapp.api.request.count.total
myapp.db.query.duration.ms
myapp.cache.hit.count.total
myapp.queue.depth.items
myapp.worker.job.processed.total

Namespace

Top-level identifier, typically the service or application name. Prevents metric collisions across services sharing a StatsD server.

payment-service.checkout.amount.usd     # good: namespaced
checkout.amount.usd                      # bad: collides with other services

Subsystem

Functional area within the service: api, db, cache, queue, worker, auth, email.

Target

The specific thing being measured: request, query, hit, miss, job, connection.

Metric + Unit

What is being measured and its unit: duration.ms, count.total, size.bytes, depth.items, rate.per_second.

Character Rules

Consistent case: Use lowercase everywhere. Some backends are case-sensitive; mixing cases creates distinct metric paths.

Word separation: Use underscores within path segments: http_request not httpRequest or http-request.

Graphite Namespace Mapping

StatsD automatically namespaces metrics under stats.<type>:

The modern namespace (enabled with legacyNamespace: false) is cleaner and groups counters under stats.counters.* instead of splitting them.

Tags vs. Metric Name Encoding

With Tags (DogStatsD, InfluxDB, Telegraf)

Use tags for dimensions. Keep metric names stable:

http.request.duration:42|ms|#method:GET,status:200,endpoint:/api/users
http.request.duration:15|ms|#method:POST,status:201,endpoint:/api/users

One metric name, many tag combinations. Dashboards can filter and group by any tag dimension.

Without Tags (Plain StatsD + Graphite)

Encode dimensions in the metric name:

http.request.get.200.api_users.duration:42|ms
http.request.post.201.api_users.duration:15|ms

Problem: Every new dimension creates new metric paths. This explodes Graphite file count and makes dashboards rigid.

Recommendation: Use DogStatsD tags or Telegraf InfluxDB-style tags when possible. Fall back to name encoding only for plain StatsD + Graphite setups.

Tag Cardinality

Tags with unbounded values create unbounded metric series, which consumes backend resources (disk, memory, query time, billing).

Rule of thumb: If a tag can have more than ~1000 distinct values, do not use it. Use logs or traces for high-cardinality data.

Naming Anti-Patterns

Checklist for New Metrics

• [ ] Namespaced by service name
• [ ] Dot-delimited hierarchy with subsystem
• [ ] Lowercase with underscores for word separation
• [ ] Unit suffix included (.ms, .bytes, .total, .items)
• [ ] Dimensions expressed as tags, not metric name segments
• [ ] Tag cardinality is bounded (< 1000 distinct values per tag)
• [ ] Name is discoverable: someone searching for "request latency"

would find myapp.api.request.duration.ms