Dt Obs Predictive Analytics

Predictive Analytics Skill

Forecast resource saturation, detect trends, analyze anomalies, and characterize signal behavior using DQL and Dynatrace analyzer tools.

Analysis Disciplines

---

Choosing the Right Detection Tool

The single most important decision: are you asking "did this metric change?" or "is this metric currently wrong?"

Decision rule in plain language

• Use `timeseries-novelty-detection` when the question contains "changed", "shifted", "spiked", "dropped", "started", "when did", or "did anything unusual happen". The tool answers whether a change occurred and when. It requires no predefined threshold.
• Use an anomaly detector (adaptive, seasonal, or static) when the question is about ongoing or current state relative to an expected range: "which are highest", "who is violating", "what is above X". These tools count violation samples inside a sliding window — they confirm how long something has been bad, not whether the signal changed.

Pitfall: Running adaptive-anomaly-detector on a broad fleet to answer "which service changed load?" typically flags every service that has any variation, producing low-signal results. Use timeseries-novelty-detection first to identify entities where the load character genuinely shifted, then use the anomaly detectors to measure the severity of those specific signals.

When to Use This Skill

• Capacity: "Which hosts will hit 90% CPU in the next 30 days?"
• Forecast: "Forecast service request volume for the next 7 days"
• Trend: "Is memory usage growing across our Kubernetes nodes?"
• Anomaly: "Which services have unusual error rates right now?"
• Baseline: "How does today's traffic compare to last week?"
• Signal profile: "Is this metric seasonal or trending before I set up alerting?"

---

Important Constraints

Dynatrace Forecast Analyzer supports univariate forecasting only — predicting one metric based on its own historical values. Multivariate forecasting (using multiple metrics as inputs) requires external tools (Python, R, Azure AutoML).

Tooling Rule: Run analyses using Dynatrace tools: timeseries-forecast, adaptive-anomaly-detector, seasonal-baseline-anomaly-detector, static-threshold-analyzer, and timeseries-novelty-detection. Use execute-dql for DQL queries.

Result Analysis Rule: Always analyse and summarise results directly from the raw tool output. Derive all numbers, trends, and conclusions inline.

---

Result Presentation Format

Always present forecast results as a structured table:

Always follow the table with a Key Findings section (3–5 bullet points, ranked by priority).

---

Core DQL Techniques

DQL has no native forecast function. For forward-looking forecasts, use timeseries-forecast (see references/forecasting-analyzer.md).

Key DQL Rules

1. timeseries returns arrays — one value per time slot per entity 2. arrayLast(arr) = most recent value; arrayFirst(arr) = oldest 3. Growth = (arrayLast - arrayFirst) / number_of_intervals 4. Always filter isNotNull(field) before sorting to avoid null ordering issues 5. Use toLong() when dividing Long fields to avoid type errors 6. Use dt.smartscape.* not deprecated dt.entity.* in DQL display fields; use dt.smartscape.* in by:{} grouping clauses for entity-level queries

---

Standard Query Patterns

Moving Average Trend

timeseries cpu = avg(dt.host.cpu.usage), from: now()-24h, interval: 1h, by: {dt.smartscape.host}
| fieldsAdd moving_avg = arrayMovingAvg(cpu, 4)
| fieldsAdd current = arrayLast(cpu)
| fieldsAdd trend = arrayLast(cpu) - arrayFirst(cpu)
| filter isNotNull(current)
| sort trend desc
| limit 20
| fields dt.smartscape.host, current, trend, moving_avg

Saturation Risk Classification

timeseries cpu = avg(dt.host.cpu.usage), from: now()-7d, interval: 1h, by: {dt.smartscape.host}
| fieldsAdd p95 = arrayPercentile(cpu, 95)
| fieldsAdd saturation_risk = if(p95 > 85, "HIGH", else: if(p95 > 70, "MEDIUM", else: "LOW"))
| filter isNotNull(p95)
| sort p95 desc
| fields dt.smartscape.host, p95, saturation_risk

Days to Saturation Forecast

timeseries cpu = avg(dt.host.cpu.usage), from: now()-30d, interval: 1d, by: {dt.smartscape.host}
| fieldsAdd current = arrayLast(cpu)
| fieldsAdd daily_growth = (arrayLast(cpu) - arrayFirst(cpu)) / 30
| filter isNotNull(current)
| fieldsAdd days_to_saturation = if(daily_growth > 0, toLong((90 - current) / daily_growth), else: 9999)
| sort days_to_saturation asc
| limit 20
| fields dt.smartscape.host, current, daily_growth, days_to_saturation

Anomaly Scoring

timeseries cpu = avg(dt.host.cpu.usage), from: now()-24h, interval: 1h, by: {dt.smartscape.host}
| fieldsAdd baseline_avg = arrayAvg(cpu)
| fieldsAdd current = arrayLast(cpu)
| fieldsAdd anomaly_score = if(isNotNull(current) and isNotNull(baseline_avg), abs(current - baseline_avg), else: 0)
| sort anomaly_score desc
| limit 20
| fields dt.smartscape.host, current, baseline_avg, anomaly_score

Metric Discovery

Before forecasting, discover available metrics by keyword:

metrics from: now() - 1h
| filter contains(metric.key, "cpu")
| summarize count(), by: {metric.key}
| sort `count()` desc

---

Reference Guides

• `references/forecasting-analyzer.md` — timeseries-forecast tool:

data requirements, parameter reference, interval selection, horizon limits, common pitfalls

• `references/capacity-forecasting.md` — CPU/memory/disk/K8s saturation

forecasts; multi-resource risk scoring; days-to-saturation DQL patterns

• `references/anomaly-scoring.md` — adaptive-anomaly-detector, seasonal-baseline-anomaly-detector,

static-threshold-analyzer; DQL deviation scoring

• `references/novelty-detection.md` — timeseries-novelty-detection tool: spike, drop, step change,

trend onset, and variability change detection; all novelty types; parameter reference; worked examples

• `references/trend-detection.md` — timeseries-novelty-detection for trend onset and change points;

week-over-week joins; growth rate and acceleration detection

Related Skills

• dt-dql-essentials — DQL syntax, timeseries command rules, array function reference
• dt-obs-hosts — Host and process metrics catalog
• dt-obs-services — Service RED metrics for service-level trend analysis
• dt-obs-problems — Davis AI problem history for anomaly correlation

Anomaly Scoring and Detection

Detect anomalies using Dynatrace analyzer tools. Use the DQL patterns below for transparent, tunable scoring when you need explicit deviation numbers alongside the analyzer output.

---

Dynatrace Anomaly Detection Tools

When to use this reference vs. novelty detection

>

The three anomaly detector tools (adaptive, seasonal, static) answer "is this metric currently violating an expected range?" They operate on a sliding window of recent samples and fire when enough consecutive samples breach a learned or fixed threshold. Use them to rank entities by ongoing severity or to confirm a sustained problem.

>

Use timeseries-novelty-detection (see references/novelty-detection.md) when the question is "did this metric change?" — no threshold required, answers whether the signal's character shifted and when. Running an anomaly detector fleet-wide to find "what changed" typically flags every service with any variation; novelty detection is the correct tool for that question.

All tools return a raisedAlerts array per entity; an empty output array means no anomalies were detected in the analysis window.

Adaptive Anomaly Detection

Learns a dynamic threshold from the signal's own distribution. Good default for most host and service metrics.

Tool: adaptive-anomaly-detector

• timeSeriesData: timeseries avg(dt.host.cpu.usage), by:{dt.entity.host}, interval:1m | limit 20
• numberOfSignalFluctuations: 1.0 — multiplier for threshold width; increase to reduce sensitivity
• alertCondition: ABOVE
• violatingSamples: 3 — consecutive violations required before alerting
• slidingWindow: 5
• generalParameters.timeframe.startTime: now-24h

Seasonal Baseline Anomaly Detection

Accounts for daily/weekly patterns. tolerance controls sensitivity — lower value = more alerts (valid range 0.1–10.0; default 4.0).

Tool: seasonal-baseline-anomaly-detector

• timeSeriesData: timeseries sum(dt.service.request.count), by:{dt.entity.service}, interval:5m | limit 10
• tolerance: 3.0
• violatingSamples: 3
• slidingWindow: 5
• alertCondition: OUTSIDE
• generalParameters.timeframe.startTime: now-7d

Static Threshold Detection

Use when the acceptable limit is known and stable (e.g., CPU > 90%). threshold must be in the base unit of the queried metric (response time metrics use nanoseconds — set 5 seconds as 5000000000, not 5000).

Tool: static-threshold-analyzer

• timeSeriesData: timeseries avg(dt.host.cpu.usage), by:{dt.entity.host}, interval:1m | limit 20
• threshold: 90
• alertCondition: ABOVE
• violatingSamples: 3
• slidingWindow: 5
• generalParameters.timeframe.startTime: now-24h

Novelty Detection

Detects previously unseen patterns, including step changes, trend onsets, and isolated spikes. Use analysisNoveltyType to focus on specific event types.

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries sum(dt.service.request.failure_count), by:{dt.entity.service}, interval:5m | limit 10
• detectionMode: ALL
• minNoveltyScore: 0.45 (0.0–1.0; higher = fewer but more confident detections)
• significanceLevel: HIGH
• analysisNoveltyType: ["CHANGE_IN_VALUES", "SPIKE", "TREND_IN_VALUES"]
• generalParameters.timeframe.startTime: now-7d

Available novelty types: CHANGE_IN_VALUES, CHANGE_IN_VARIABILITY, CHANGE_IN_MISSING_VALUES, GAP_WITH_MISSING_VALUES, TREND_IN_VALUES, SPIKE.

---

Anomaly Types

---

Result Presentation

---

Best Practices

• Compare anomaly candidates against the same time window in the prior

day/week before escalating

• Consider seasonality — a spike at Monday 9 AM may be expected
• Correlate Davis AI findings with DQL scores to prioritize what matters
• dt.service.request.response_time is in nanoseconds — divide by

1000000 for ms before scoring

• For static-threshold-analyzer, always set threshold in the metric's

base unit (nanoseconds for response time, percent for CPU)

• violatingSamples must not exceed slidingWindow

Capacity Forecasting

Forecast resource saturation for hosts, Kubernetes workloads, and billing metrics. Use timeseries-forecast for forward-looking projections; use the DQL patterns below for immediate risk classification.

---

Dynatrace Forecast Analyzer

Use timeseries-forecast for all capacity forecasts. See references/forecasting-analyzer.md for the full parameter reference, feasibility decision tree, and horizon sizing rules.

Key rule: For 30-day forecasts, use interval:1d and forecastHorizon:30. forecastHorizon is in steps (intervals), not absolute days.

Host CPU — 30-Day Saturation Forecast

Training: 120 days. Interval: 1d. Horizon: 30 daily steps.

Tool: timeseries-forecast

• query: timeseries avg(dt.host.cpu.usage), by:{dt.entity.host}, interval:1d | limit 500
• forecastHorizon: 30
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-60d

Host Memory — 30-Day Saturation Forecast

Tool: timeseries-forecast

• query: timeseries avg(dt.host.memory.usage), by:{dt.entity.host}, interval:1d | limit 500
• forecastHorizon: 30
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-120d

Kubernetes Container CPU — Namespace Forecast

Tool: timeseries-forecast

• query: timeseries avg(dt.kubernetes.container.cpu_usage), by:{k8s.namespace.name, k8s.workload.name}, interval:1d | limit 500
• forecastHorizon: 30
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-60d

Kubernetes PVC Storage Saturation

Tool: timeseries-forecast

• query: timeseries avg(dt.kubernetes.persistentvolumeclaim.used), by:{k8s.namespace.name}, interval:1d | limit 500
• forecastHorizon: 30
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-60d

---

Result Presentation

---

Best Practices

• Use at least 60 days of history for 30-day forecasts (2:1 rule)
• Use interval:1d for 30-day forecasts; interval:1h for 7-day
• Saturation threshold is typically 90%; adjust per SLA
• Always filter isNotNull(field) before sort
• Use if(daily_growth > 0, ..., else: 9999) — negative growth = no saturation risk
• Entities with data gaps in the most recent third of the training window will

fail forecasting; filter them out or shorten the training window

Dynatrace Forecast Analyzer

The timeseries-forecast tool generates time series forecasts for any numeric metric queryable from Grail. It uses Davis AI statistical models to produce point forecasts and confidence bands.

⚠️ Univariate only: Dynatrace Forecast Analyzer currently supports only univariate forecasting. See Section 1 for details and multivariate alternatives.

🚫 Result Analysis Rule: After running a forecast or DQL query, always analyse and summarise the results directly from the raw tool output.

📊 Result Presentation Rule: Present forecast results as a structured Markdown table:

After the table, add a Key Findings section (3–5 bullet points) ranked by priority.

---

Tool Reference: timeseries-forecast

Parameters

Output Fields

The result timeseries contains these fields per record:

• `<metric_expression>` — historical values (nulls in the forecast period)
• `dt.davis.forecast:point` — point forecast (nulls in the historical period)
• `dt.davis.forecast:lower` — lower confidence bound
• `dt.davis.forecast:upper` — upper confidence bound

The forecast begins at the last non-null historical value. Read the point forecast array from the first non-null position onward.

Common Failure: Missing Values in Recent History

The analyzer requires at least 14 non-null values in the last third of the training window. If the series has too many gaps near the end, the analysis status will be FAILED with a warning:

"Too many values are missing from the most recent history."

Fix: filter entities with sufficient continuous data, or shorten the training window to exclude inactive periods.

Interval and Horizon Sizing

forecastHorizon is counted in steps (intervals), not in absolute time: forecast time = forecastHorizon × interval.

Do not omit interval: from the query — without it, the tool auto-selects an interval that may differ from what you expect, making the horizon ambiguous.

---

1. Forecast and Prediction

Core Principles

Univariate vs. Multivariate Forecasting

Univariate Forecasting: Predicts a single metric based on its own historical values.

• Best for: Stable patterns with strong autocorrelation (e.g., daily traffic patterns)
• Status with Dynatrace: ✅ Supported via timeseries-forecast

Multivariate Forecasting: Predicts a target metric using multiple related metrics as features.

• Best for: Complex systems where multiple factors drive the target metric
• Status with Dynatrace: ⏳ Not Supported by Dynatrace Forecast Analyzer. Use external tools (Python, R, Azure AutoML) or approximate with multiple univariate forecasts.

Statistical Foundation

Time series forecasting works best when:

• Stationarity: Statistical properties (mean, variance) remain roughly constant
• Seasonality: Regular, repeating patterns exist (hourly, daily, weekly)
• Trend: Long-term direction (upward, downward, or flat)
• Autocorrelation: Current values relate to past values

Critical Data Requirements

Minimum Historical Data

• Hard minimum: 20 non-null data points
• Recency requirement: At least 14 non-null values must fall in the last third of the training window
• Practical rule: Ensure continuous data collection with no more than 5% gaps

The 2:1 Rule (Data Science Best Practice)

Training history must be at least 2× the forecast horizon:

Data Quality Rules

1. No more than 5% missing data points within the training window 2. Sufficient variance: CV (std dev / mean) > 5%; flat metrics cannot be forecast 3. Complete seasonal cycles: Include ≥ 2 full cycles (e.g., 2 weeks for weekly patterns) 4. Clean outliers: Incidents and one-off spikes distort models; consider excluding them

Metric Discovery

Before forecasting, use execute-dql to confirm which metrics are available:

fetch metric.series
| filter contains(metric.key, "cpu")
| dedup metric.key
| limit 100

Change "cpu" to any keyword (e.g., "request", "memory", "kubernetes") to explore other domains.

Query Patterns for Forecast Input

Always pair a DQL query with timeseries-forecast. The query must return a timeseries result; limit controls how many entities are analyzed.

Host CPU (univariate, by host):

timeseries avg(dt.host.cpu.usage), by:{dt.smartscape.host}, interval:1d | limit 50

Service request volume (univariate, by service):

timeseries sum(dt.service.request.count), by:{dt.entity.service}, interval:1h | limit 50

Single entity forecast (filter first, then forecast):

timeseries sum(dt.service.request.count), by:{dt.entity.service}, interval:1h
| filter dt.entity.service == "SERVICE-XXXXXXXX"

Do's and Don'ts

✅ DO

• Specify interval: explicitly so the horizon count is predictable
• Use at least 3× training history relative to the forecast horizon
• Include ≥ 2 complete seasonal cycles in training data
• Retrain regularly — patterns change after deployments and events
• Set forecastOffset: 1 to validate the last point against forecast

❌ DON'T

• Forecast with fewer than 20 data points
• Omit interval: from the query (leads to ambiguous horizon sizing)
• Set forecastHorizon > 600 (hard limit)
• Forecast beyond 2–3× the training window (uncertainty explodes)
• Use a metric with CV < 5% (flat signals produce meaningless forecasts)
• Mix pre/post-deployment data in the training window

---

2. Top Use Cases

Capacity Planning (30 days)

Tool: timeseries-forecast

• query: timeseries avg(dt.host.cpu.usage), by:{dt.entity.host}, interval:1d | limit 500
• forecastHorizon: 30 (30 daily steps = 30 days)
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-60d

Repeat for memory (dt.host.memory.usage) and disk (dt.host.disk.used.percent). See references/capacity-forecasting.md for multi-resource patterns.

Traffic Forecasting (7 days)

Tool: timeseries-forecast

• query: timeseries sum(dt.service.request.count), by:{dt.entity.service}, interval:1h | limit 500
• forecastHorizon: 168 (168 hourly steps = 7 days)
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-14d

SLA / Response Time Forecast (24 hours)

Tool: timeseries-forecast

• query: timeseries avg(dt.service.request.response_time), by:{dt.entity.service}, interval:1h | limit 500
• forecastHorizon: 24 (24 hourly steps = 24 hours)
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-7d

Note: dt.service.request.response_time is in microseconds — divide by 1 000 for milliseconds in DQL before displaying.

Anomaly Baseline Forecasting

Generate a dynamic baseline by running a short-horizon forecast, then alert when actuals exceed dt.davis.forecast:upper:

Tool: timeseries-forecast

• query: timeseries avg(dt.host.cpu.usage), by:{dt.entity.host}, interval:1h | limit 500
• forecastHorizon: 24
• forecastOffset: 1
• generalParameters.timeframe.startTime: now-2d

Retrain daily. Alert when actual > dt.davis.forecast:upper.

---

3. Feasibility Decision Tree

Q: Should we forecast this metric?
├─ Do we have ≥ 20 historical data points?
│  ├─ No  → Wait and collect more data
│  └─ Yes → Continue
├─ Does the last third of the window have ≥ 14 non-null values?
│  ├─ No  → Shorten the training window or filter to active entities
│  └─ Yes → Continue
├─ Does the metric have meaningful variance? (CV > 5%)
│  ├─ No  → Skip forecasting; use static thresholds instead
│  └─ Yes → Continue
├─ Is training history ≥ 2× the forecast horizon?
│  ├─ No  → Reduce forecastHorizon or extend startTime
│  └─ Yes → Proceed
└─ Single or multiple metrics?
   ├─ Single → Univariate forecast with timeseries-forecast ✅
   └─ Multiple → External tools (multivariate not supported) ⏳

---

4. Monitoring Forecast Accuracy

Track these metrics post-deployment:

Retrain after significant business events (launches, incidents, major config changes).

---

5. Common Pitfalls

Novelty Detection

Use timeseries-novelty-detection to check whether a time series showed a significant change in the character of its behavior during a given period. Unlike threshold-based detectors, novelty detection does not require a known acceptable limit — it learns what "normal" looks like and flags anything that is structurally new: spikes, drops, step changes, shifts in variability, or the onset of a sustained trend.

---

When to Use

Use this tool as the first check when you suspect something changed but you do not yet know what kind of change it was. Run it with analysisNoveltyType: ["SPIKE", "CHANGE_IN_VALUES", "TREND_IN_VALUES"] and let the results tell you the category before you narrow in.

Do not use anomaly detectors (`adaptive-anomaly-detector`, `seasonal-baseline-anomaly-detector`, `static-threshold-analyzer`) to answer "did this metric change?" questions. Those tools count how long a metric stays outside a learned or fixed threshold — they confirm ongoing severity, not the presence of a change. On a broad fleet query they will flag every service that has any variation, producing low-signal results. Novelty detection is the correct first tool whenever the question contains "changed", "shifted", "spiked", "dropped", "started", "when did", or "did anything unusual happen".

---

Tool Parameters

Tool: timeseries-novelty-detection

Novelty Types

Combine multiple types in one call: ["SPIKE", "CHANGE_IN_VALUES", "TREND_IN_VALUES"].

---

Worked Examples

Check for any novel behavior (general anomaly sweep)

Use as a first-pass detector across a set of entities when you suspect something changed but do not know the shape:

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries avg(dt.host.cpu.usage), by:{dt.entity.host}, interval:5m | limit 20
• detectionMode: ALL
• analysisNoveltyType: ["SPIKE", "CHANGE_IN_VALUES", "TREND_IN_VALUES", "CHANGE_IN_VARIABILITY"]
• minNoveltyScore: 0.45
• significanceLevel: HIGH
• generalParameters.timeframe.startTime: now-24h

Spike and drop detection (isolated events)

Check whether a service's error rate showed any sudden spikes or drops in a recent incident window:

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries sum(dt.service.request.failure_count), by:{dt.entity.service}, interval:1m | limit 10
• detectionMode: ALL
• analysisNoveltyType: ["SPIKE"]
• minNoveltyScore: 0.5
• significanceLevel: MEDIUM
• filterSpikes: false
• generalParameters.timeframe.startTime: now-6h

Step change after a deployment

Confirm whether a deployment caused a permanent baseline shift (not just a transient spike):

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries avg(dt.service.request.response_time), by:{dt.entity.service}, interval:5m | limit 10
• detectionMode: INCREASE
• analysisNoveltyType: ["CHANGE_IN_VALUES"]
• minNoveltyScore: 0.55
• significanceLevel: HIGH
• filterSpikes: true
• generalParameters.timeframe.startTime: now-48h

Trend onset detection

Identify when a gradual upward or downward trend began:

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries avg(dt.host.memory.usage.percent), by:{dt.entity.host}, interval:1h | limit 15
• detectionMode: INCREASE
• analysisNoveltyType: ["TREND_IN_VALUES"]
• minNoveltyScore: 0.45
• significanceLevel: HIGH
• filterSpikes: true
• generalParameters.timeframe.startTime: now-30d

Variability change (signal becoming erratic)

Detect when a metric that was previously stable starts fluctuating:

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries avg(dt.service.request.response_time), by:{dt.entity.service}, interval:5m | limit 10
• detectionMode: ALL
• analysisNoveltyType: ["CHANGE_IN_VARIABILITY"]
• minNoveltyScore: 0.45
• significanceLevel: MEDIUM
• generalParameters.timeframe.startTime: now-14d

---

Interpreting Results

The tool returns a raisedAlerts array per entity. Each alert includes:

• The novelty type that was detected
• A novelty score (0.0–1.0) — higher means more confident
• The time range where the novelty was detected

An empty output array (or no raisedAlerts entries) means the signal showed no significant novel behavior in the analysis window.

---

Result Presentation

---

Best Practices

• Set filterSpikes: true when looking for structural changes (step changes,

trends) — spikes inflate variance and can obscure change-point detection

• Set filterSpikes: false when spikes are the signal of interest
• Use INCREASE / DECREASE mode to reduce noise when the anomaly direction

is known from context (e.g., "CPU went up after deploy")

• Combine with adaptive-anomaly-detector or seasonal-baseline-anomaly-detector

for cross-validation: novelty detection finds when the character changed; threshold detectors confirm how much it exceeds acceptable limits

• dt.service.request.response_time is in nanoseconds — results are returned

in that unit; divide by 1000000 when presenting ms values

• Keep | limit N in the DQL query low (10–20) for faster results; increase

only when a full fleet sweep is required

Trend Detection

Detect growth trends, week-over-week changes, and acceleration patterns in service and infrastructure metrics. Use timeseries-novelty-detection for structural break and trend onset detection; use the DQL patterns for growth rate analysis and week-over-week comparisons.

---

Dynatrace Trend Analyzers

Change Point Detection

Detects when a metric's baseline permanently shifted (e.g., after a deployment). Use this before declaring a trend to confirm it is a true structural change vs. noise.

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries sum(dt.service.request.count), by:{dt.entity.service}, interval:1h | limit 10
• detectionMode: ALL
• analysisNoveltyType: ["CHANGE_IN_VALUES", "CHANGE_IN_VARIABILITY"]
• minNoveltyScore: 0.45
• significanceLevel: HIGH
• generalParameters.timeframe.startTime: now-30d

For host CPU upward step changes:

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries avg(dt.host.cpu.usage), by:{dt.entity.host}, interval:1h | limit 10
• detectionMode: INCREASE
• analysisNoveltyType: ["CHANGE_IN_VALUES"]
• minNoveltyScore: 0.45
• significanceLevel: MEDIUM
• generalParameters.timeframe.startTime: now-30d

Trend Onset Detection

Detects when a metric starts moving in a sustained new direction (growing or declining trend beginning):

Tool: timeseries-novelty-detection

• timeSeriesData: timeseries avg(dt.service.request.response_time), by:{dt.entity.service}, interval:5m | limit 10
• detectionMode: ALL
• analysisNoveltyType: ["TREND_IN_VALUES"]
• minNoveltyScore: 0.45
• significanceLevel: HIGH
• filterSpikes: true
• generalParameters.timeframe.startTime: now-14d

---

DQL Trend Detection Patterns

Use these with execute-dql for quantified growth rates.

Week-over-Week Request Volume Comparison

timeseries this_week = sum(dt.service.request.count),
  from: now()-7d, interval: 1h, by: {dt.smartscape.service}
| join [
    timeseries last_week = sum(dt.service.request.count),
      from: now()-14d, to: now()-7d, interval: 1h,
      by: {dt.smartscape.service}
  ], on: {left[dt.smartscape.service] == right[dt.smartscape.service]}
| fieldsAdd this_total = arraySum(this_week)
| fieldsAdd last_total = arraySum(right.last_week)
| filter isNotNull(this_total) and isNotNull(last_total) and last_total > 0
| fieldsAdd wow_change_pct = round((this_total - last_total) / last_total * 100, decimals: 1)
| sort wow_change_pct desc
| limit 20
| fields dt.smartscape.service, this_total, last_total, wow_change_pct

Growth Rate Detection (30-Day Trend)

timeseries cpu = avg(dt.host.cpu.usage), from: now()-30d, interval: 1d,
  by: {dt.smartscape.host}
| fieldsAdd current = arrayLast(cpu)
| fieldsAdd baseline = arrayFirst(cpu)
| filter isNotNull(current) and isNotNull(baseline)
| fieldsAdd daily_growth = (current - baseline) / 30
| fieldsAdd total_growth_pct = round((current - baseline) / baseline * 100, decimals: 1)
| sort daily_growth desc
| limit 20
| fields dt.smartscape.host, baseline, current, daily_growth, total_growth_pct

Acceleration Detection

Detect whether the rate of change is itself increasing:

timeseries cpu = avg(dt.host.cpu.usage), from: now()-14d, interval: 1d,
  by: {dt.smartscape.host}
| fieldsAdd diffs = arrayDiff(cpu)
| fieldsAdd avg_acceleration = arrayAvg(diffs)
| fieldsAdd current = arrayLast(cpu)
| filter isNotNull(current) and avg_acceleration > 0
| sort avg_acceleration desc
| limit 20
| fields dt.smartscape.host, current, avg_acceleration

Service Error Rate Trend

timeseries {
  errors = sum(dt.service.request.failure_count),
  total = sum(dt.service.request.count)
}, from: now()-7d, interval: 1h, by: {dt.smartscape.service}
| fieldsAdd error_rate_now = arrayLast(errors) / arrayLast(total) * 100
| fieldsAdd error_rate_7d_avg = arrayAvg(errors) / arrayAvg(total) * 100
| filter isNotNull(error_rate_now) and error_rate_7d_avg > 0
| fieldsAdd rate_change = error_rate_now - error_rate_7d_avg
| sort rate_change desc
| limit 20
| fields dt.smartscape.service, error_rate_now, error_rate_7d_avg, rate_change

Service Response Time Trend (Week-over-Week)

timeseries rt_now = avg(dt.service.request.response_time),
  from: now()-7d, interval: 1h, by: {dt.smartscape.service}
| join [
    timeseries rt_prev = avg(dt.service.request.response_time),
      from: now()-14d, to: now()-7d, interval: 1h,
      by: {dt.smartscape.service}
  ], on: {left[dt.smartscape.service] == right[dt.smartscape.service]}
| fieldsAdd avg_now = arrayAvg(rt_now) / 1000
| fieldsAdd avg_prev = arrayAvg(right.rt_prev) / 1000
| filter isNotNull(avg_now) and avg_prev > 0
| fieldsAdd wow_change_pct = round((avg_now - avg_prev) / avg_prev * 100, decimals: 1)
| sort wow_change_pct desc
| limit 20
| fields dt.smartscape.service, avg_now, avg_prev, wow_change_pct

---

Trend Result Presentation

---

Best Practices

• Use timeseries-novelty-detection first to confirm a trend is structural,

not just noise; filterSpikes: true removes transient spikes before trend analysis

• Use CHANGE_IN_VALUES for step changes (permanent baseline shifts);

use TREND_IN_VALUES for gradual directional movement

• Week-over-week joins require identical interval: values on both sides

so join keys align; right-side fields are prefixed with right.

• dt.service.request.response_time is in nanoseconds — divide by

1000000 for ms

• A positive acceleration (arrayAvg(arrayDiff(arr)) > 0) means the metric

is speeding up — worth flagging even if the current value is acceptable