Ai Ml Timeseries

Time Series Forecasting — Modern Patterns & Production Best Practices

Modern Best Practices (January 2026):

• Treat time as a first-class axis: temporal splits, rolling backtests, and point-in-time correctness.
• Default to strong baselines (naive/seasonal naive) before complex models.
• Prevent leakage: feature windows and aggregations must use only information available at prediction time.
• Evaluate by horizon and segment; a single aggregate metric hides failures.
• Prefer probabilistic forecasts when decisions are risk-sensitive (quantiles/intervals); evaluate calibration (coverage) and use pinball/CRPS.
• For many related series, consider global + hierarchical approaches (shared models + reconciliation); validate across levels and key segments.
• Treat time zones/DST as first-class; validate timestamp alignment before feature generation.
• Define retraining cadence and degraded modes (fallback model, last-known-good forecast).

This skill provides operational, copy-paste-ready workflows for forecasting with recent advances: TS-specific EDA, temporal validation, lag/rolling features, model selection, multi-step forecasting, backtesting, generative AI (Chronos, TimesFM), and production deployment with drift monitoring.

It focuses on hands-on forecasting execution, not theory.

---

When to Use This Skill

Claude should invoke this skill when the user asks for hands-on time series forecasting, e.g.:

• "Build a time series model for X."
• "Create lag features / rolling windows."
• "Help design a forecasting backtest."
• "Pick the right forecasting model for my data."
• "Fix leakage in forecasting."
• "Evaluate multi-horizon forecasts."
• "Use LLMs or generative models for TS."
• "Set up monitoring for a forecast system."
• "Implement LightGBM for time series."
• "Use transformer models (TimesFM, Chronos) for forecasting."
• "Apply temporal classification/survival modelling for event prediction."

If the user is asking about general ML modelling, deployment, or infrastructure, prefer:

• ai-ml-data-science - General data science workflows, EDA, feature engineering, evaluation
• ai-mlops - Model deployment, monitoring, drift detection, retraining automation

If the user is asking about LLM/RAG/search, prefer:

• ai-llm - LLM fine-tuning, prompting, evaluation
• ai-rag - RAG pipeline design and optimization

---

Quick Reference

---

Decision Tree: Choosing Time Series Approach

User needs time series forecasting for: [Data Type]
    ├─ Strong Seasonality?
    │   ├─ Simple patterns? → LightGBM with seasonal features
    │   ├─ Complex patterns? → LightGBM + Prophet comparison
    │   └─ Multiple seasonalities? → Prophet or TBATS
    │
    ├─ Long-term Dependencies (>50 steps)?
    │   ├─ Transformers (TimesFM, Chronos) → Best for complex patterns
    │   └─ RNNs/LSTMs → Good for sequential dependencies
    │
    ├─ Event Forecasting (binary outcomes)?
    │   └─ Temporal classification / survival modelling → validate with time-based splits
    │
    ├─ Intermittent/Sparse Data (many zeros)?
    │   ├─ Croston/SBA → Classical intermittent methods
    │   └─ LightGBM with zero-inflation features → Modern approach
    │
    ├─ Multiple Covariates?
    │   ├─ LightGBM → Best with many features
    │   └─ TFT/DeepAR → If deep learning needed
    │
    └─ Explainability Required (healthcare, finance)?
        ├─ LightGBM → SHAP values, feature importance
        └─ Linear models → Most interpretable

---

Core Concepts (Vendor-Agnostic)

• Time axis: splits, features, and labels must respect time ordering and availability.
• Non-stationarity: seasonality, trend, and regime shifts are normal; monitor and retrain intentionally.
• Evaluation: rolling/expanding backtests; report horizon-wise and segment-wise performance.
• Operationalization: define retraining cadence, fallback models, and data freshness contracts.
• Data governance: treat time series as potentially sensitive; enforce access control, retention, and PII scrubbing in logs.

Implementation Practices (Tooling Examples)

• Build features with explicit time windows; store cutoff timestamps with each training run.
• Backtest with a standardized harness (rolling/expanding windows, horizon-wise metrics).
• Log production forecasts with metadata (model version, horizon, data cut) to enable debugging.
• Implement fallbacks (baseline model, last-known-good, “insufficient data” handling) for outages and anomalies.

Do / Avoid

Do

• Do start with naive/seasonal naive baselines and compare against learned models (Forecasting: Principles and Practice: https://otexts.com/fpp3/).
• Do backtest with rolling windows and preserve point-in-time correctness.
• Do monitor for data pipeline changes (missing timestamps, level shifts, calendar changes).
• Do align metrics/loss to the decision: asymmetric costs, service levels, and probabilistic targets (quantiles/intervals) when needed.

Avoid

• Avoid random splits for forecasting problems.
• Avoid features that use future information (future aggregates, leakage via target encoding).
• Avoid optimizing only aggregate metrics; always inspect horizon-wise errors and worst segments.
• Avoid MAPE when the target can be 0 or near-0; prefer MASE/WAPE/sMAPE and horizon-wise reporting.

Navigation: Core Patterns

Time Series EDA & Data Preparation

• [TS EDA Best Practices](references/ts-eda-best-practices.md)
• Frequency detection, missing timestamps, decomposition
• Outlier detection, level shifts, seasonality analysis
• Granularity selection and stability checks

Feature Engineering

• [Lag & Rolling Patterns](references/lag-rolling-patterns.md)
• Lag features (lag_1, lag_7, lag_28 for daily data)
• Rolling windows (mean, std, min, max, EWM)
• Avoiding leakage, seasonal lags, datetime features

Model Selection

• [Model Selection Guide](references/model-selection-guide.md)
• Decision rules: Strong seasonality → LightGBM, Long-term → Transformers
• Benchmark comparison: LightGBM vs Prophet vs Transformers vs RNNs
• Explainability considerations for mission-critical domains

• [LightGBM TS Patterns](references/lightgbm-ts-patterns.md) (feature-based forecasting best practices)
• Why LightGBM excels: performance + efficiency + explainability
• Feature engineering for tree-based models
• Hyperparameter tuning for time series

Forecasting Strategies

• [Multi-Step Forecasting Patterns](references/multistep-forecasting-patterns.md)
• Direct strategy (separate models per horizon)
• Recursive strategy (feed predictions back)
• Seq2Seq strategy (Transformers, RNNs for long horizons)

• [Intermittent Demand Patterns](references/intermittent-demand-patterns.md)
• Croston, SBA, ADIDA for sparse data
• LightGBM with zero-inflation features (modern approach)
• Two-stage hurdle models, hierarchical Bayesian

Validation & Evaluation

• [Backtesting Patterns](references/backtesting-patterns.md)
• Rolling window backtest, expanding window
• Temporal train/validation split (no IID splits!)
• Horizon-wise metrics, segment-level evaluation

Generative & Advanced Models

• [TS-LLM Patterns](references/ts-llm-patterns.md)
• Chronos, TimesFM, Lag-Llama (Transformer models)
• Event forecasting patterns (temporal classification, survival modelling)
• Tokenization, discretization, trajectory sampling

Production Deployment

• [Production Deployment Patterns](references/production-deployment-patterns.md)
• Feature pipelines (same code for train/serve)
• Retraining strategies (time-based, drift-triggered)
• Monitoring (error drift, feature drift, volume drift)
• Fallback strategies, streaming ingestion, data governance

Advanced Forecasting

• [Anomaly Detection Patterns](references/anomaly-detection-patterns.md)
• Statistical, ML, and deep learning anomaly detectors for time series
• Threshold tuning, alert fatigue reduction, seasonal adjustment

• [Hierarchical Forecasting](references/hierarchical-forecasting.md)
• Bottom-up, top-down, and reconciliation methods
• Cross-level coherence, grouped series, MinT/WLS approaches

• [Probabilistic Forecasting](references/probabilistic-forecasting.md)
• Quantile regression, conformal prediction, prediction intervals
• Calibration metrics (CRPS, pinball loss, coverage), decision-making under uncertainty

---

Navigation: Templates (Copy-Paste Ready)

Data Preparation

• [TS EDA Template](assets/timeseries/template-ts-eda.md) - Reproducible structure for time series analysis
• [Resample & Fill Template](assets/timeseries/template-resample-fill.md) - Handle missing timestamps and resampling

Feature Templates

• [Lag & Rolling Features](assets/timeseries/template-lag-rolling.md) - Create temporal features for ML models
• [Calendar Features](assets/timeseries/template-calendar-features.md) - Business calendars, holidays, events

Model Templates

• [Forecast Model Template](assets/timeseries/template-forecast-model.md) - End-to-end forecasting pipeline (LightGBM, transformers, RNNs)
• [Multi-Step Strategy](assets/timeseries/template-multistep-strategy.md) - Direct, recursive, and seq2seq approaches

Evaluation Templates

• [Backtest Template](assets/timeseries/template-backtest.md) - Rolling window validation setup
• [TS Metrics Template](assets/timeseries/template-ts-metrics.md) - MAPE, MAE, RMSE, MASE, pinball loss

Advanced Templates

• [TS-LLM Template](assets/timeseries/template-ts-llm.md) - Time series foundation model patterns and experimental approaches

---

Related Skills

For adjacent topics, reference these skills:

• [ai-ml-data-science](../ai-ml-data-science/SKILL.md) - EDA workflows, feature engineering patterns, model evaluation, SQLMesh transformations
• [ai-mlops](../ai-mlops/SKILL.md) - Production deployment, monitoring, retraining pipelines
• [ai-llm](../ai-llm/SKILL.md) - Fine-tuning approaches applicable to time series LLMs (Chronos, TimesFM)
• [ai-prompt-engineering](../ai-prompt-engineering/SKILL.md) - Prompt design patterns for time series LLMs
• [data-sql-optimization](../data-sql-optimization/SKILL.md) - SQL optimization for time series data storage and retrieval

---

External Resources

See data/sources.json for curated web resources including:

• Classical methods (statsmodels, Prophet, ARIMA)
• Deep learning frameworks (PyTorch Forecasting, GluonTS, Darts, NeuralProphet)
• Transformer models (TimesFM, Chronos, Lag-Llama, Informer, Autoformer)
• Anomaly detection tools (PyOD, STUMPY, Isolation Forest)
• Feature engineering libraries (tsfresh, TSFuse, Featuretools)
• Production deployment (Kats, MLflow, sktime)
• Benchmarks and datasets (M5 Competition, Monash Time Series, UCI)

---

Usage Notes

For Claude:

• Activate this skill for hands-on forecasting tasks, feature engineering, backtesting, or production setup
• Start with Quick Reference and Decision Tree for fast guidance
• Drill into references/ for detailed implementation patterns
• Use assets/ for copy-paste ready code
• Always check for temporal leakage (future data in training)
• Start with strong baselines; choose model family based on horizon, covariates, and latency/cost constraints
• Emphasize explainability for healthcare/finance domains
• Monitor for data distribution shifts in production

Key Principle: Time series forecasting is about temporal structure, not IID assumptions. Use temporal validation, avoid future leakage, and choose models based on horizon length and data characteristics.

Fact-Checking

• Use web search/web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.
• Prefer primary sources; report source links and dates for volatile information.
• If web access is unavailable, state the limitation and mark guidance as unverified.

Backtesting Template

Fully defined backtest configuration for forecasting models.

---

1. Backtest Window

backtest: start: "<date>" end: "<date>" horizon: <N_days> frequency: "<D/H/W>"

---

2. Window Type

window_type: "<rolling|expanding>"

---

3. Backtest Steps

1. Create initial train/val split 2. Train model on train window 3. Predict next horizon 4. Slide window 5. Repeat N times 6. Aggregate metrics

---

4. Metrics

metrics: "mae" "rmse" "mape" "smape"

---

5. Output Format

{ "date": "<prediction_date>", "y_true": <value>, "y_pred": <value> }

---

6. Checklist

• [ ] Temporal order respected
• [ ] Multiple windows tested
• [ ] Horizon-specific error analyzed

Calendar Feature Template

Standardized calendar & event features for time series.

---

1. Calendar Features

calendar: day_of_week: true day_of_month: true week_of_year: true month: true quarter: true is_weekend: true

---

2. Holiday/Event Features

events: holiday_calendar: "<country_or_custom>" special_days:

• "black_friday"
• "cyber_monday"
• "end_of_quarter"

---

3. Weather (Optional)

weather: include: true lag_hours: 24 variables:

• temperature
• precipitation
• humidity

---

4. Checklist

• [ ] Timezone aligned
• [ ] Event features region-specific
• [ ] Weather features lagged (no leakage)

Forecast Model Template

Define forecasting model configuration and training logic.

---

1. Model Overview

model_type: "<sarima|prophet|xgboost|lightgbm|lstm|tft|nbeats>" version: <vX.Y> description: "<short text>"

---

2. Training Config

training: target: "<column>" features: "<list_of_features>" train_start: "<date>" train_end: "<date>"

---

3. Hyperparameters

params: learning_rate: <value> max_depth: <value> num_leaves: <value> seasonal_period: <value>

(Use appropriate params for model family.)

---

4. Validation Strategy

validation: split_type: "temporal" horizon_days: <H> backtest_windows: <N>

---

5. Output

outputs: model_artifact: "<path>" metrics_report: "<path>" feature_importance: true/false

---

6. Checklist

• [ ] Baseline compared
• [ ] No leakage
• [ ] Validation horizon correct
• [ ] Metrics stable

Lag & Rolling Feature Template

Define lag and window-based features for forecasting.

---

1. Lag Features

lags: 1 7 14 28

(Optional) hourly_lags: 1 24 48

---

2. Rolling Windows

rolling: windows:

• 7
• 14
• 30

metrics:

• mean
• std
• min
• max
• sum

---

3. Target Leakage Checks

• [ ] Lags only reference past timestamps
• [ ] Rolling windows computed on historical data
• [ ] Seasonal lags added when needed

Multi-Step Forecasting Strategy Template

Define how to generate forecasts across multiple future steps.

---

1. Strategy Type

Choose one:

strategy: "<direct|recursive|seq2seq>"

---

2. Configurations

Direct

direct: horizons:

• 1
• 7
• 28

Recursive

recursive: max_horizon: <value> refit: false

Seq2Seq (NN)

seq2seq: encoder_length: <value> decoder_length: <value>

---

3. Checklist

• [ ] Strategy aligned with horizon length
• [ ] Error propagation checked
• [ ] Covariates aligned with forecast range

Resample & Fill Template

Standardized resampling and missing value handling for time series.

---

1. Resampling Rule

resample_frequency: <D/H/W/M> aggregation_method: <sum|mean|max|min|count|custom>

---

2. Missing Value Strategy

missing_values: forward_fill: true interpolation: "linear" # linear | time | spline seasonal_interpolation: false drop_threshold: <percent>

Notes

• Use seasonal interpolation for data with strong periodicity
• Drop periods with > threshold gaps

---

3. Quality Checks

• [ ] Missingness removed
• [ ] No new anomalies introduced
• [ ] Rolling window alignment validated

Time Series EDA Template

A reproducible structure for analyzing any time series dataset.

---

1. Series Summary

series_name: <name> frequency: <D/H/W/M> start_date: <date> end_date: <date> num_points: <int>

---

2. Timestamp Validation

• [ ] Sorted
• [ ] No duplicates
• [ ] Frequency consistent
• [ ] Missing timestamps identified

missing_timestamps: <list_or_count> duplicate_timestamps: <list_or_count>

---

3. Trend & Seasonality

Document:

• Observed trend (up/down/flat)
• Seasonal cycles (daily/weekly/yearly)
• Strength of seasonality
• Change points

---

4. Outliers

outlier_method: <zscore|iqr|rolling> outliers_detected: <count> outlier_dates: <sample_list>

---

5. Visualizations Checklist

• [ ] Raw line chart
• [ ] Rolling mean/variance
• [ ] Seasonal decomposition
• [ ] ACF/PACF plots

---

6. EDA Findings

Summaries:

• Data quality issues
• Missingness behavior
• Stationarity impressions

TS-LLM Template (Chronos, Time-LLM, Generative TS)

A template for building LLM-based forecasting pipelines.

---

1. Tokenization / Discretization

discretization: method: "quantize" # quantize | bucketize | round num_bins: 500 normalize: true

---

2. Input Structure

inputs: past_values: <list> exogenous_features: <optional> context_length: 128

---

3. Generation Settings

generation: max_horizon: <N> num_samples: 20 temperature: 0.8 top_p: 0.9

---

4. Output Reconstruction

reconstruction: method: "dequantize"

---

5. Scenario Simulation

Calculate P10/P50/P90:

quantiles: [0.1, 0.5, 0.9]

---

6. Checklist

• [ ] Values discretized
• [ ] No leakage
• [ ] Multiple trajectories aggregated
• [ ] Evaluated vs baseline

Time Series Metrics Template

Defines evaluation metrics for point and probabilistic forecasts.

---

1. Point Forecast Metrics

point_metrics: mae: true rmse: true mape: true smape: true mase: false

---

2. Probabilistic Metrics

prob_metrics: pinball_loss: true crps: true quantiles: [0.1, 0.5, 0.9]

---

3. Business Metrics (Custom)

business_metrics: stockout_cost: false overforecast_cost: false

---

4. Output Format

{ "metric": "<name>", "value": <float> }

---

5. Checklist

• [ ] Metrics aligned with objective
• [ ] Horizon-specific results included
• [ ] Slice by product/region if applicable

Time Series Anomaly Detection Patterns

Operational guide for detecting anomalies in time series data. Covers point, contextual, and collective anomaly types. Methods span statistical baselines through deep learning. Focus on production alerting, threshold tuning, and false positive management.

Freshness anchor: January 2026 — scikit-learn 1.5+, PyOD 1.1+, STUMPY 1.13+, Prophet 1.1+

---

Decision Tree: Choosing a Detection Method

START
│
├─ Anomaly type?
│   ├─ Point (single unexpected value)
│   │   ├─ Univariate?
│   │   │   ├─ YES → Z-score / IQR baseline, then Isolation Forest
│   │   │   └─ NO  → Isolation Forest or LOF on multivariate
│   │   └─ Stationary series?
│   │       ├─ YES → Statistical methods sufficient
│   │       └─ NO  → Decompose first (STL), detect on residuals
│   │
│   ├─ Contextual (normal value in wrong context)
│   │   └─ Time-of-day / seasonality matters?
│   │       ├─ YES → Prophet / STL decomposition → residual detection
│   │       └─ NO  → Rolling window z-score with adaptive threshold
│   │
│   └─ Collective (sequence of points form anomaly)
│       ├─ Known pattern length?
│       │   ├─ YES → Matrix Profile (STUMPY) with fixed window
│       │   └─ NO  → Autoencoder on sliding windows
│       └─ Subsequence anomaly?
│           └─ Matrix Profile → Discord discovery
│
├─ Labeled anomaly data available?
│   ├─ YES (>100 labeled anomalies) → Supervised (XGBoost on features)
│   ├─ PARTIAL (<100 labels) → Semi-supervised (tune threshold on labels)
│   └─ NO → Unsupervised (statistical or isolation-based)
│
└─ Latency requirement?
    ├─ Real-time (< 1 sec) → Z-score, EWM, simple thresholds
    ├─ Near real-time (< 1 min) → Isolation Forest, LOF (pre-trained)
    └─ Batch (hourly+) → Autoencoder, Matrix Profile, full pipeline

---

Quick Reference: Methods Comparison

---

Operational Patterns

Pattern 1: Statistical Baseline (Start Here)

• Use when: First pass on any time series, establishing baseline
• Implementation:

import numpy as np
import pandas as pd

def zscore_detector(series, window=168, threshold=3.0):
    """Rolling z-score anomaly detection.

    Args:
        series: pd.Series with datetime index
        window: rolling window size (168 = 1 week hourly)
        threshold: z-score threshold
    """
    rolling_mean = series.rolling(window=window, min_periods=window//2).mean()
    rolling_std = series.rolling(window=window, min_periods=window//2).std()
    zscore = (series - rolling_mean) / (rolling_std + 1e-8)
    anomalies = zscore.abs() > threshold
    return anomalies, zscore

def iqr_detector(series, window=168, k=1.5):
    """Rolling IQR anomaly detection."""
    q1 = series.rolling(window).quantile(0.25)
    q3 = series.rolling(window).quantile(0.75)
    iqr = q3 - q1
    lower = q1 - k * iqr
    upper = q3 + k * iqr
    anomalies = (series < lower) | (series > upper)
    return anomalies

• Threshold tuning guide:

Pattern 2: STL Decomposition + Residual Detection

• Use when: Series has clear seasonality and trend
• Implementation:

from statsmodels.tsa.seasonal import STL

def stl_anomaly_detector(series, period=24, threshold=3.0):
    """Decompose, then detect anomalies in residuals."""
    stl = STL(series, period=period, robust=True)
    result = stl.fit()

    residuals = result.resid
    resid_mean = residuals.mean()
    resid_std = residuals.std()

    zscore = (residuals - resid_mean) / (resid_std + 1e-8)
    anomalies = zscore.abs() > threshold

    return anomalies, result

# For multiple seasonality: use MSTL
from statsmodels.tsa.seasonal import MSTL
mstl = MSTL(series, periods=[24, 168])  # daily + weekly

• Key rule: Always use robust=True to prevent anomalies from distorting the decomposition

Pattern 3: Isolation Forest for Multivariate

• Use when: Multiple correlated features, unknown anomaly structure
• Implementation:

from sklearn.ensemble import IsolationForest

def build_features(df, target_col, lags=[1, 2, 3, 24], windows=[6, 24]):
    """Build time series features for anomaly detection."""
    features = pd.DataFrame(index=df.index)
    for lag in lags:
        features[f'lag_{lag}'] = df[target_col].shift(lag)
    for w in windows:
        features[f'rolling_mean_{w}'] = df[target_col].rolling(w).mean()
        features[f'rolling_std_{w}'] = df[target_col].rolling(w).std()
    features['hour'] = df.index.hour
    features['dayofweek'] = df.index.dayofweek
    return features.dropna()

features = build_features(df, 'value')

iforest = IsolationForest(
    n_estimators=300,
    contamination=0.01,    # expected anomaly rate
    max_samples='auto',
    random_state=42,
    n_jobs=-1,
)
iforest.fit(features)

scores = iforest.decision_function(features)  # lower = more anomalous
anomalies = iforest.predict(features) == -1

• Contamination tuning: Start at 0.01 (1%), adjust based on domain knowledge
• Feature engineering is critical — raw values alone are insufficient

Pattern 4: Matrix Profile for Subsequence Anomalies

• Use when: Looking for unusual patterns (not just unusual values)
• Implementation:

import stumpy

# Compute matrix profile
window_size = 24  # pattern length to search for
mp = stumpy.stump(series.values, m=window_size)

# Discord (most unusual subsequence)
discord_idx = mp[:, 0].argmax()
discord_distance = mp[discord_idx, 0]

# Top-k anomalous subsequences
k = 10
top_k_idx = np.argsort(mp[:, 0])[-k:][::-1]

# Threshold: discords with distance > mean + 3*std of all distances
threshold = mp[:, 0].mean() + 3 * mp[:, 0].std()
anomalous_subsequences = np.where(mp[:, 0] > threshold)[0]

• Window size selection:
• Known period (e.g., daily cycle = 24 hourly points) → use period
• Unknown → try multiple window sizes, look for consistent discords

Pattern 5: Autoencoder for Complex Patterns

• Use when: High-dimensional, non-linear patterns, sufficient training data
• Implementation:

import torch
import torch.nn as nn

class TSAutoencoder(nn.Module):
    def __init__(self, input_dim, latent_dim=16):
        super().__init__()
        self.encoder = nn.Sequential(
            nn.Linear(input_dim, 64), nn.ReLU(),
            nn.Linear(64, 32), nn.ReLU(),
            nn.Linear(32, latent_dim),
        )
        self.decoder = nn.Sequential(
            nn.Linear(latent_dim, 32), nn.ReLU(),
            nn.Linear(32, 64), nn.ReLU(),
            nn.Linear(64, input_dim),
        )

    def forward(self, x):
        z = self.encoder(x)
        return self.decoder(z)

# Train on normal data only
# Anomaly = high reconstruction error
recon = model(X_test)
errors = ((X_test - recon) ** 2).mean(dim=1)
threshold = errors.quantile(0.99)  # calibrate on validation set
anomalies = errors > threshold

• Training rule: Train ONLY on clean/normal data — if anomalies are in training set, autoencoder learns to reconstruct them

---

Production Alerting Configuration

Alert Fatigue Management

Threshold Calibration Workflow

1. Deploy detector with logging only (no alerts) — 2 weeks
2. Review flagged anomalies with domain expert
3. Calculate precision at current threshold
4. Adjust threshold to target precision > 70%
5. Enable alerts, track weekly precision
6. Re-calibrate monthly or after distribution shifts

---

Anti-Patterns

---

Validation Checklist

• [ ] Statistical baseline (z-score/IQR) established first
• [ ] Seasonality handled (decomposition or feature engineering)
• [ ] Contamination rate estimated from domain knowledge
• [ ] Threshold calibrated on labeled validation data (if available)
• [ ] Alert cooldown and severity tiers configured
• [ ] False positive rate tracked in production
• [ ] Detector retrained on schedule (monthly minimum)
• [ ] Feedback loop from operators to anomaly labels
• [ ] Multiple detection methods compared before production
• [ ] Edge cases tested: missing data, zero-variance periods, holidays

---

Cross-References

• ai-ml-timeseries/references/probabilistic-forecasting.md — prediction intervals as anomaly bounds
• ai-ml-timeseries/references/hierarchical-forecasting.md — detecting anomalies across hierarchy levels
• ai-mlops/references/automated-retraining-patterns.md — drift detection triggering retraining
• ai-mlops/references/experiment-tracking-patterns.md — logging detector performance metrics

Backtesting Patterns for Forecasting

Reliable, repeatable frameworks for evaluating forecasting models.

---

1. Avoid Random Splits

Forecasting requires temporal integrity → never sample randomly.

---

2. Valid Backtest Structures

Pattern A: Holdout Window

• Train on early data
• Test on final N days/weeks

Pattern B: Rolling Window Backtest

Example windows:

• Train: t0 → t100
• Validate: t101 → t120
• Slide forward: t20 → t120, validate t121 → t140

Pattern C: Expanding Window

• Train expands each iteration
• Test on next fixed horizon

---

3. Multi-Horizon Evaluation

For horizon H (e.g., 1–30 days):

• Compute error for each horizon separately
• Plot error curve
• Identify weaknesses at long horizons

Checklist

• [ ] Horizon-specific metrics computed
• [ ] Error curves plotted

---

4. Metrics for Forecasting

Point Forecast Metrics

• MAE
• RMSE
• MAPE
• sMAPE

Probabilistic Metrics

• Pinball loss
• CRPS

Business Metrics

• Stockouts
• Overforecast cost

---

5. Backtest Execution Workflow

1. Freeze target horizon 2. Select window scheme 3. Train model per window 4. Record metrics 5. Aggregate across all windows 6. Document results

---

6. Backtesting Checklist

• [ ] Temporal ordering preserved
• [ ] Enough windows to test variability
• [ ] Metrics stable
• [ ] Baseline included

Hierarchical Forecasting

Operational guide for forecasting hierarchically organized time series. Covers reconciliation methods, implementation with Python libraries, and patterns for product, geographic, and temporal hierarchies. Focus on coherent forecasts that add up correctly across levels.

Freshness anchor: January 2026 — hierarchicalforecast 0.6+, scikit-hts 0.7+, statsforecast 1.7+

---

Decision Tree: Choosing a Reconciliation Approach

START
│
├─ Hierarchy depth?
│   ├─ 2 levels (e.g., total / products)
│   │   └─ Bottom-up usually sufficient
│   ├─ 3+ levels
│   │   └─ Continue to method selection
│   └─ Grouped (cross-product, e.g., product x region)
│       └─ Use MinT or ERM — simpler methods break on groups
│
├─ Forecast quality varies by level?
│   ├─ Bottom level is best → Bottom-up
│   ├─ Top level is best → Top-down (proportions)
│   ├─ Middle level is best → Middle-out
│   └─ Mixed / unknown → MinT (optimal reconciliation)
│
├─ Need probabilistic forecasts?
│   ├─ YES → MinT with bootstrap or normality assumption
│   └─ NO  → Any reconciliation method
│
├─ Series count?
│   ├─ < 100 series → MinT (full covariance feasible)
│   ├─ 100–10,000 → MinT with shrinkage or diagonal covariance
│   └─ > 10,000 → Bottom-up or top-down (covariance too large)
│
└─ Temporal aggregation needed (daily → weekly → monthly)?
    └─ Temporal reconciliation (FoReco / thief)

---

Quick Reference: Reconciliation Methods

---

Operational Patterns

Pattern 1: Hierarchy Definition

• Use when: Setting up any hierarchical forecast
• Implementation:

# Example: Product hierarchy
# Total → Category → Subcategory → SKU
#
# Summing matrix S maps bottom level to all levels:
# [Total]         = [1 1 1 1 1 1] @ [SKU1..SKU6]
# [Category A]    = [1 1 1 0 0 0]
# [Category B]    = [0 0 0 1 1 1]
# [SubCat A1]     = [1 1 0 0 0 0]
# [SubCat A2]     = [0 0 1 0 0 0]
# [SubCat B1]     = [0 0 0 1 1 0]
# [SubCat B2]     = [0 0 0 0 0 1]
# [SKU 1..6]      = I (identity)

import numpy as np
import pandas as pd

# Define hierarchy tags
hierarchy_df = pd.DataFrame({
    'sku': ['SKU1', 'SKU2', 'SKU3', 'SKU4', 'SKU5', 'SKU6'],
    'subcategory': ['A1', 'A1', 'A2', 'B1', 'B1', 'B2'],
    'category': ['A', 'A', 'A', 'B', 'B', 'B'],
})

Pattern 2: Bottom-Up with statsforecast + hierarchicalforecast

• Use when: Reliable bottom-level data, simple hierarchy

from statsforecast import StatsForecast
from statsforecast.models import AutoETS, AutoARIMA
from hierarchicalforecast.core import HierarchicalReconciliation
from hierarchicalforecast.methods import BottomUp, MinTrace

# Prepare data in long format with hierarchy columns
# Required columns: unique_id, ds (date), y (value)

# Step 1: Generate base forecasts at all levels
sf = StatsForecast(
    models=[AutoETS(season_length=12)],
    freq='M',
    n_jobs=-1,
)
base_forecasts = sf.forecast(h=12)

# Step 2: Reconcile
reconciler = HierarchicalReconciliation(
    reconcilers=[
        BottomUp(),
        MinTrace(method='mint_shrink'),
    ]
)
reconciled = reconciler.reconcile(
    Y_hat_df=base_forecasts,
    Y_df=train_df,
    S=summing_matrix,
    tags=hierarchy_tags,
)

Pattern 3: MinT Optimal Reconciliation

• Use when: Want statistically optimal coherent forecasts
• Implementation:

from hierarchicalforecast.methods import MinTrace

# Method options for covariance estimation:
# 'ols'          — identity covariance (equal weights)
# 'wls_struct'   — structural scaling (by level)
# 'wls_var'      — variance scaling (from residuals)
# 'mint_shrink'  — shrinkage estimator (recommended default)
# 'mint_cov'     — full sample covariance (small hierarchies only)

reconciler = HierarchicalReconciliation(
    reconcilers=[
        MinTrace(method='mint_shrink'),  # best general choice
    ]
)

# For grouped time series (product x region):
# Use the same approach but define grouped summing matrix

• Covariance method selection:

Pattern 4: Grouped Hierarchies (Product x Region)

• Use when: Multiple hierarchy dimensions that cross (not just nested)

# Grouped hierarchy example:
# Total → {Product A, Product B} × {Region East, Region West}
#
# This is NOT a tree — it's a cross-product.
# Each combination must be forecasted.

# Define tags for grouped hierarchy
tags = {
    'total': train_df['unique_id'].unique().tolist(),
    'product': ['Product_A', 'Product_B'],
    'region': ['East', 'West'],
    'product_region': ['A_East', 'A_West', 'B_East', 'B_West'],
}

# hierarchicalforecast handles grouped structures natively
# MinTrace with shrinkage is recommended for grouped hierarchies

Pattern 5: Temporal Aggregation

• Use when: Need coherent forecasts across time granularities (daily, weekly, monthly)
• Concept: Forecasts at different temporal granularities should be consistent

# Temporal hierarchy: daily → weekly → monthly → quarterly
# Use temporal reconciliation to ensure:
#   sum(daily forecasts in week) = weekly forecast
#   sum(weekly in month) = monthly forecast

# Implementation with FoReco (R) or custom:
def temporal_bottom_up(daily_forecasts, freq_map):
    """Aggregate daily forecasts to higher frequencies."""
    weekly = daily_forecasts.resample('W').sum()
    monthly = daily_forecasts.resample('M').sum()
    quarterly = daily_forecasts.resample('Q').sum()
    return {'D': daily_forecasts, 'W': weekly, 'M': monthly, 'Q': quarterly}

# For optimal temporal reconciliation:
# Forecast at each temporal level independently
# Then reconcile using MinT across temporal aggregation matrix

Pattern 6: Evaluation Across Hierarchy Levels

• Use when: Always — evaluate at each level, not just aggregate

from hierarchicalforecast.evaluation import HierarchicalEvaluation

def evaluate_hierarchy(actual, forecasted, hierarchy_tags):
    """Evaluate forecast accuracy at each hierarchy level."""
    results = {}
    for level, series_ids in hierarchy_tags.items():
        level_actual = actual[actual['unique_id'].isin(series_ids)]
        level_forecast = forecasted[forecasted['unique_id'].isin(series_ids)]

        # MASE (scale-independent, preferred for hierarchical)
        mase = compute_mase(level_actual, level_forecast)
        # RMSSE
        rmsse = compute_rmsse(level_actual, level_forecast)

        results[level] = {'MASE': mase, 'RMSSE': rmsse}

    return pd.DataFrame(results).T

# Key metrics for hierarchical evaluation:
# - MASE: scale-free, comparable across levels
# - RMSSE: used in M5 competition
# - Coherence error: sum(bottom) - top (should be ~0 after reconciliation)

---

Coherence Verification

def check_coherence(forecasts, summing_matrix, tolerance=1e-6):
    """Verify that reconciled forecasts are coherent."""
    bottom_level = forecasts.iloc[-summing_matrix.shape[1]:]
    reconstructed = summing_matrix @ bottom_level.values

    all_levels = forecasts.values
    max_error = np.abs(all_levels - reconstructed).max()

    is_coherent = max_error < tolerance
    return is_coherent, max_error

# Run after every reconciliation
coherent, error = check_coherence(reconciled, S)
assert coherent, f"Incoherent forecasts: max error = {error}"

---

Anti-Patterns

---

Validation Checklist

• [ ] Hierarchy structure defined and summing matrix verified
• [ ] Base forecasts generated at all levels independently
• [ ] Reconciliation method chosen based on hierarchy size and quality
• [ ] Coherence verified post-reconciliation (sum check)
• [ ] Accuracy evaluated at every hierarchy level (not just top)
• [ ] Proportions or covariances updated with recent data
• [ ] Grouped hierarchies handled with cross-product structure
• [ ] New series / discontinued series handled in hierarchy updates
• [ ] Temporal coherence checked if multiple frequencies used
• [ ] Reconciliation improves (or at least doesn't degrade) vs base forecasts

---

Cross-References

• ai-ml-timeseries/references/probabilistic-forecasting.md — coherent prediction intervals across hierarchy
• ai-ml-timeseries/references/anomaly-detection-patterns.md — detecting anomalies at different hierarchy levels
• ai-mlops/references/automated-retraining-patterns.md — scheduling hierarchy reconciliation in pipelines
• ai-mlops/references/experiment-tracking-patterns.md — logging per-level accuracy metrics

Intermittent Demand Forecasting Patterns

Operational patterns for forecasting sparse, intermittent, or erratic demand with many zeros.

---

Overview

Intermittent demand occurs when:

• Data has many zeros (>50% zero values)
• Demand is sporadic or lumpy
• Traditional forecasting methods fail due to sparsity

Common in: retail, spare parts, slow-moving inventory, industrial equipment.

---

Pattern 1: Classical Intermittent Methods

Croston's Method

When to Use:

• Regular intervals between non-zero demands
• Approximately constant demand size

How It Works: 1. Forecast demand size separately from demand intervals 2. Combine forecasts: forecast = size / interval

import numpy as np

def croston_forecast(demand, alpha=0.1):
    """
    Croston's intermittent demand forecasting

    Args:
        demand: Array of demand values (with zeros)
        alpha: Smoothing parameter (0-1)
    """
    non_zero_idx = np.where(demand > 0)[0]

    # Initialize
    size_forecast = demand[non_zero_idx[0]]
    interval_forecast = non_zero_idx[1] - non_zero_idx[0]

    forecasts = []

    for i in range(1, len(non_zero_idx)):
        # Update size forecast
        size_forecast = alpha * demand[non_zero_idx[i]] + (1 - alpha) * size_forecast

        # Update interval forecast
        interval = non_zero_idx[i] - non_zero_idx[i-1]
        interval_forecast = alpha * interval + (1 - alpha) * interval_forecast

        # Forecast = size / interval
        forecast = size_forecast / interval_forecast
        forecasts.append(forecast)

    return forecasts

Syntetos-Boylan Approximation (SBA)

Improvement over Croston:

• Addresses Croston's bias
• Better for highly intermittent demand

def sba_forecast(demand, alpha=0.1):
    """SBA reduces Croston's bias"""
    croston_fc = croston_forecast(demand, alpha)

    # Bias correction factor
    correction = 1 - (alpha / 2)

    return [fc * correction for fc in croston_fc]

ADIDA (Aggregate-Disaggregate Intermittent Demand Approach)

When to Use:

• Need balance between Croston and naive forecasts
• Want simpler approach

def adida_forecast(demand, window=4):
    """
    Aggregate demand over window, then disaggregate
    """
    # Aggregate
    aggregated = [sum(demand[i:i+window]) for i in range(0, len(demand), window)]

    # Forecast aggregated demand
    agg_forecast = aggregated[-1]  # Naive or SES

    # Disaggregate
    return agg_forecast / window

---

Pattern 2: Modern ML Approaches

LightGBM with Zero-Inflation Features

Best Performer for Intermittent Demand (2024-2025)

import lightgbm as lgb
import pandas as pd

def create_intermittent_features(df, target_col='demand'):
    """
    Feature engineering for intermittent demand
    """
    # Standard lag features
    for lag in [1, 7, 28]:
        df[f'lag_{lag}'] = df[target_col].shift(lag)

    # Zero-inflation specific features
    df['zero_count_7d'] = (df[target_col] == 0).rolling(7).sum()
    df['zero_count_28d'] = (df[target_col] == 0).rolling(28).sum()
    df['nonzero_count_7d'] = (df[target_col] > 0).rolling(7).sum()
    df['nonzero_count_28d'] = (df[target_col] > 0).rolling(28).sum()

    # Recency of last non-zero demand
    nonzero_idx = df[df[target_col] > 0].index
    df['days_since_last_demand'] = 0
    for i in df.index:
        recent_nonzero = nonzero_idx[nonzero_idx < i]
        if len(recent_nonzero) > 0:
            df.loc[i, 'days_since_last_demand'] = i - recent_nonzero[-1]

    # Average non-zero demand
    df['avg_nonzero_7d'] = df[target_col].replace(0, np.nan).rolling(7).mean()
    df['avg_nonzero_28d'] = df[target_col].replace(0, np.nan).rolling(28).mean()

    # Intermittency coefficient (Croston-inspired)
    df['intermittency_ratio'] = df['zero_count_28d'] / 28

    return df

# Train LightGBM
params = {
    'objective': 'regression',
    'metric': 'mae',
    'boosting_type': 'gbdt',
    'num_leaves': 31,
    'learning_rate': 0.05,
}

train_data = lgb.Dataset(X_train, y_train)
model = lgb.train(params, train_data, num_boost_round=100)

Why LightGBM Works:

• Handles sparse data naturally
• Captures non-linear patterns in zero occurrence
• Fast training and inference
• Explainable with SHAP values

---

Pattern 3: Two-Stage Modeling

Hurdle Model (Probability + Magnitude)

Stage 1: Predict probability of non-zero demand Stage 2: Predict magnitude if non-zero

from sklearn.linear_model import LogisticRegression
from lightgbm import LGBMRegressor

# Stage 1: Binary classifier (will there be demand?)
binary_target = (y_train > 0).astype(int)
classifier = LogisticRegression()
classifier.fit(X_train, binary_target)

# Stage 2: Regressor for non-zero demand
non_zero_mask = y_train > 0
regressor = LGBMRegressor()
regressor.fit(X_train[non_zero_mask], y_train[non_zero_mask])

# Prediction
prob_nonzero = classifier.predict_proba(X_test)[:, 1]
magnitude = regressor.predict(X_test)
final_forecast = prob_nonzero * magnitude

Benefits:

• Models different processes separately
• Improves accuracy for highly intermittent data
• Interpretable (why forecast is zero)

---

Pattern 4: Hierarchical Bayesian Models

When to Use:

• Multiple related intermittent series
• Want probabilistic forecasts
• Have domain knowledge for priors

import pymc as pm

with pm.Model() as hierarchical_model:
    # Global hyperparameters
    mu_global = pm.Normal('mu_global', mu=10, sigma=10)
    sigma_global = pm.HalfNormal('sigma_global', sigma=5)

    # Series-specific parameters
    mu_series = pm.Normal('mu_series', mu=mu_global, sigma=sigma_global, shape=n_series)

    # Zero-inflation parameter
    p_zero = pm.Beta('p_zero', alpha=2, beta=2, shape=n_series)

    # Likelihood
    for i in range(n_series):
        # Zero-inflated Poisson
        pm.ZeroInflatedPoisson(
            f'demand_{i}',
            psi=p_zero[i],
            mu=mu_series[i],
            observed=demand_data[i]
        )

    # Sample posterior
    trace = pm.sample(1000, tune=1000)

---

Pattern 5: Evaluation Metrics for Intermittent Demand

Standard Metrics Often Mislead

Problem: MAPE undefined when actual = 0

Better Metrics:

1. WAPE (Weighted Absolute Percentage Error)

def wape(y_true, y_pred):
    return np.abs(y_true - y_pred).sum() / y_true.sum()

2. MAE-over-Volume

def mae_over_volume(y_true, y_pred):
    return np.abs(y_true - y_pred).mean() / y_true.mean()

3. MASE (Mean Absolute Scaled Error)

def mase(y_true, y_pred, y_train):
    mae = np.abs(y_true - y_pred).mean()
    naive_mae = np.abs(np.diff(y_train)).mean()
    return mae / naive_mae

4. Zero-forecast Accuracy

def zero_forecast_accuracy(y_true, y_pred, threshold=0.5):
    """How well do we predict zeros?"""
    true_zeros = (y_true == 0)
    pred_zeros = (y_pred < threshold)
    return (true_zeros == pred_zeros).mean()

---

Pattern 6: Forecasting Very Sparse Data (<10% non-zero)

Challenges

• Not enough non-zero samples
• Classical methods fail
• Need robust baselines

Approach

1. Start with simple baseline:

# Baseline: Average non-zero demand * historical frequency
non_zero_avg = demand[demand > 0].mean()
non_zero_freq = (demand > 0).sum() / len(demand)
baseline_forecast = non_zero_avg * non_zero_freq

2. Try Croston/SBA first:

croston_fc = croston_forecast(demand, alpha=0.1)
sba_fc = sba_forecast(demand, alpha=0.1)

3. If data allows, use LightGBM with engineered features:

df = create_intermittent_features(df)
model = lgb.train(params, lgb.Dataset(X_train, y_train))

4. Fall back to aggregation:

# Aggregate to weekly/monthly if daily is too sparse
weekly_demand = demand.resample('W').sum()
weekly_forecast = forecast_weekly(weekly_demand)
daily_forecast = weekly_forecast / 7  # Disaggregate

---

Decision Tree: Choosing Intermittent Demand Method

Intermittent Demand Forecasting:
    ├─ Zero frequency > 70% (very sparse)?
    │   ├─ Yes → Croston/SBA or aggregate to higher level
    │   └─ No → LightGBM with zero-inflation features
    │
    ├─ Multiple related series?
    │   ├─ Yes → Hierarchical Bayesian model
    │   └─ No → Single-series approach
    │
    ├─ Need probabilistic forecasts?
    │   ├─ Yes → Hurdle model or Bayesian
    │   └─ No → LightGBM or Croston
    │
    └─ Computational constraints?
        ├─ High → Croston/SBA (fast, simple)
        └─ Low → LightGBM (best performance)

---

Checklist: Intermittent Demand Forecasting

Data Analysis

• [ ] Computed zero frequency (% zeros)
• [ ] Identified demand pattern (lumpy, erratic, intermittent)
• [ ] Checked for sufficient non-zero samples
• [ ] Analyzed intervals between non-zero demands

Method Selection

• [ ] Tried simple baseline (avg non-zero × frequency)
• [ ] Tested Croston/SBA for regular patterns
• [ ] Tested LightGBM with zero-inflation features
• [ ] Considered two-stage hurdle model
• [ ] Evaluated hierarchical model if multiple series

Feature Engineering (for ML)

• [ ] Created zero-count features
• [ ] Added days-since-last-demand
• [ ] Computed average non-zero demand
• [ ] Added intermittency ratio
• [ ] Included seasonal/calendar features

Evaluation

• [ ] Used appropriate metrics (WAPE, MAE-over-volume, MASE)
• [ ] Avoided MAPE (undefined for zeros)
• [ ] Tracked zero-forecast accuracy
• [ ] Compared against baselines
• [ ] Evaluated by demand category (very sparse vs moderately sparse)

Production

• [ ] Implemented fallback for very sparse items
• [ ] Set threshold for zero forecasts
• [ ] Monitored forecast accuracy by sparsity level
• [ ] Documented aggregation strategy if needed

---

References

See also:

• Model Selection Guide - Choosing forecasting models
• LightGBM TS Patterns - Tree-based forecasting best practices
• Lag & Rolling Patterns - Feature engineering for ML

Lag & Rolling Feature Engineering Patterns

Concrete patterns to generate effective temporal features for time series forecasting using ML or deep models.

---

1. Lag Feature Patterns

Daily Data

• lag_1
• lag_7
• lag_14
• lag_28

Hourly Data

• lag_1
• lag_24
• lag_48

Checklist

• [ ] All lags based strictly on past timestamps
• [ ] No future leakage
• [ ] Seasonal lags included

---

2. Rolling Window Features

Rolling windows

• mean, std, min, max, median
• rolling_sum
• rolling_count
• exponentially weighted mean

Window sizes

• Daily: 7, 14, 30
• Hourly: 24, 48, 72

---

3. Calendar & Event Features

• Day of week
• Week of year
• Month, quarter
• Holiday flag
• End-of-quarter flag
• Weather lag features (no future weather!)

Checklist

• [ ] Holidays localized to region
• [ ] Event flags validated
• [ ] Weather features lagged

---

4. Categorical & Static Features

Useful for:

• Product hierarchy
• Region
• Store type
• Item category

Keep static features separate from temporal ones.

---

5. Feature Parity for Forecast Horizons

For multi-step prediction, ensure:

• [ ] Covariates aligned with each forecast horizon
• [ ] Future-known covariates correctly applied (holidays)
• [ ] Future-unknown covariates excluded or forecasted separately

---

6. Lag/Rolling Feature Checklist

• [ ] No future leakage
• [ ] Windows computed on historical data only
• [ ] Seasonal structure captured
• [ ] Covariates aligned with forecast horizon

LightGBM for Time Series Forecasting - Best Practices

Operational patterns for using LightGBM in time series forecasting (2024-2025 best practices).

---

Why LightGBM for Time Series

Key Advantages:

• Majority of M5 Competition winners used LightGBM
• Handles missing data well
• Fast training and prediction speed
• Works efficiently with large datasets
• Supports custom loss functions
• Competes with and outperforms XGBoost, AdaBoost, CatBoost

When to Use LightGBM:

• Lots of data available (high frequency, decent volume)
• Want to include many external features (weather, holidays, events)
• Need fast training and prediction
• Require computational efficiency
• Have strong seasonality patterns
• Multiple covariates involved

---

Critical Limitation

LightGBM doesn't natively understand time - you must manually create time-based features to teach it temporal awareness.

---

Essential Feature Engineering

1. Lag Features

Represent past values of the series:

# Daily data
df['lag_1'] = df['target'].shift(1)
df['lag_7'] = df['target'].shift(7)
df['lag_28'] = df['target'].shift(28)

# Hourly data
df['lag_24'] = df['target'].shift(24)
df['lag_48'] = df['target'].shift(48)

2. Rolling Statistics

Moving averages and windows:

# Rolling means
df['rolling_mean_7'] = df['target'].rolling(window=7).mean()
df['rolling_mean_30'] = df['target'].rolling(window=30).mean()

# Rolling std
df['rolling_std_7'] = df['target'].rolling(window=7).std()

# Exponentially weighted means
df['ewm_7'] = df['target'].ewm(span=7).mean()

3. Prophet-Derived Features

Extract features from Prophet model:

# Train Prophet model
prophet_model.fit(train_data)

# Extract features
predictions = prophet_model.predict(df)
df['prophet_pred'] = predictions['yhat']
df['prophet_lower'] = predictions['yhat_lower']
df['prophet_upper'] = predictions['yhat_upper']
df['prophet_daily_seasonality'] = predictions['daily']
df['prophet_weekly_seasonality'] = predictions['weekly']
df['prophet_trend'] = predictions['trend']

4. Calendar Features

df['dayofweek'] = df['date'].dt.dayofweek
df['day'] = df['date'].dt.day
df['month'] = df['date'].dt.month
df['quarter'] = df['date'].dt.quarter
df['year'] = df['date'].dt.year
df['weekofyear'] = df['date'].dt.weekofyear

# Cyclical encoding
df['month_sin'] = np.sin(2 * np.pi * df['month']/12)
df['month_cos'] = np.cos(2 * np.pi * df['month']/12)

5. External Variables

Weather data can improve performance by 42% (MAE reduction):

# Weather features
df['temperature']
df['precipitation']
df['wind_speed']

# Holiday indicators
df['is_holiday']
df['is_weekend']

# Event flags
df['black_friday']
df['end_of_quarter']

---

Hyperparameter Optimization

Approach

Use Grid Search + Repeated K-Fold Cross Validation with manual tuning.

Key Parameters

params = {
    'objective': 'regression',
    'metric': 'mae',  # or 'rmse'
    'boosting_type': 'gbdt',
    'num_leaves': 31,
    'learning_rate': 0.05,
    'feature_fraction': 0.9,
    'bagging_fraction': 0.8,
    'bagging_freq': 5,
    'verbose': 0
}

Grid Search Example

from sklearn.model_selection import GridSearchCV
import lightgbm as lgb

param_grid = {
    'num_leaves': [15, 31, 63],
    'learning_rate': [0.01, 0.05, 0.1],
    'n_estimators': [100, 500, 1000],
    'max_depth': [-1, 5, 10]
}

model = lgb.LGBMRegressor()
grid_search = GridSearchCV(
    estimator=model,
    param_grid=param_grid,
    cv=5,
    scoring='neg_mean_absolute_error',
    n_jobs=-1
)

---

Multi-Step Forecasting Strategy

Challenge

LightGBM does not support multi-output models - you need to predict one step at a time.

Solutions

1. Direct Strategy: Train one model per horizon step:

# Train model for h=1
model_h1.fit(X_train, y_train_h1)

# Train model for h=7
model_h7.fit(X_train, y_train_h7)

# Each model predicts its specific horizon

2. Recursive Strategy: Predict one step, feed it back, predict next step:

predictions = []
for h in range(1, horizon+1):
    pred = model.predict(X_current)
    predictions.append(pred)
    # Update features with new prediction
    X_current = update_features(X_current, pred)

---

Handling Seasonality

Short-Term Seasonality

LightGBM handles well with lag features (daily, weekly).

Long-Term Seasonality

Does not handle as well as traditional models (SARIMA, Prophet).

Solution: Combine approaches:

• Use Prophet for trend and long-term seasonality
• Use LightGBM to model residuals and short-term patterns

---

Production Best Practices

1. Feature Consistency

Ensure training and serving use identical feature engineering:

class FeatureEngineer:
    def fit(self, train_data):
        # Store parameters for transform
        self.rolling_params = {...}
        return self

    def transform(self, data):
        # Apply same transformations
        return engineered_data

# Use in both training and serving
engineer.fit(train_data)
train_features = engineer.transform(train_data)
serve_features = engineer.transform(new_data)

2. Avoid Data Leakage

# WRONG - uses future data
df['rolling_mean'] = df['target'].rolling(window=7).mean()

# CORRECT - only uses past data
df['rolling_mean'] = df['target'].shift(1).rolling(window=7).mean()

3. Temporal Cross-Validation

from sklearn.model_selection import TimeSeriesSplit

tscv = TimeSeriesSplit(n_splits=5)
for train_index, test_index in tscv.split(X):
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    # Train and evaluate

---

Hybrid Approaches (2024-2025)

LazyProphet Pattern

Combines Prophet feature extraction with LightGBM modeling:

Steps: 1. Train Prophet model 2. Extract Prophet features (trend, seasonality, predictions) 3. Add Prophet features to LightGBM feature set 4. Train LightGBM on combined features

Benefits:

• Prophet captures long-term patterns
• LightGBM captures complex interactions
• Often outperforms either model alone

---

Evaluation Checklist

• [ ] Temporal split (no random shuffle)
• [ ] No data leakage in features
• [ ] Rolling window validation performed
• [ ] Baseline comparison (naive, seasonal naive)
• [ ] Metrics: MAE, RMSE, MAPE, MASE
• [ ] Horizon-wise error analysis
• [ ] External variable impact measured
• [ ] Feature importance reviewed
• [ ] Model explainability documented

---

Common Pitfalls

1. Forgetting Time Direction

Features must only use past data:

# WRONG
df['future_mean'] = df['target'].rolling(window=7, center=True).mean()

# CORRECT
df['lag_mean'] = df['target'].shift(1).rolling(window=7).mean()

2. Not Encoding Cyclical Features

Month, day of week should be cyclical:

# BETTER than raw month=1,2,3...12
df['month_sin'] = np.sin(2 * np.pi * df['month']/12)
df['month_cos'] = np.cos(2 * np.pi * df['month']/12)

3. Ignoring Feature Scaling

While LightGBM doesn't require scaling, some external features benefit:

from sklearn.preprocessing import StandardScaler

# Scale weather features
scaler = StandardScaler()
df[['temp', 'humidity', 'pressure']] = scaler.fit_transform(
    df[['temp', 'humidity', 'pressure']]
)

---

Performance Benchmarks (2024)

M5 Competition Results:

• Majority of winners used LightGBM
• Typical MAE improvement: 15-30% over baselines
• With weather data: 42% MAE reduction

Computational Efficiency:

• 2-5x faster than XGBoost
• Scales well to millions of rows
• Memory efficient

---

References

• M5 Competition Winners
• LightGBM Time Series Guide
• LazyProphet Approach

Model Selection Guide for Forecasting

Operational decision rules for selecting forecasting models based on data patterns and constraints.

---

1. First Principles

Always

• Start with baseline forecasts
• Document naive performance
• Choose model by data behavior, not hype

---

2. Baseline Models (Required)

• Naive (y[t] = y[t-1])
• Seasonal naive (y[t] = y[t-7])
• Moving average

Checklist

• [ ] Baseline implemented
• [ ] Candidate model must outperform baseline

---

3. Model Family Decision Rules

A. Classical Statistical Models

Use when:

• Strong linear trend
• Regular seasonality
• Few covariates
• Daily/hourly data

Models:

• ARIMA / SARIMA
• ETS
• TBATS
• Prophet

---

B. Machine Learning Models

Use when:

• Many covariates
• Complex interactions
• Intermittent demand
• Non-linear patterns

Models:

• XGBoost
• LightGBM
• Random Forest
• Gradient-boosted trees

---

C. Deep Learning Models

Use when:

• Large dataset
• Multiple related series
• Long horizon forecasting
• Complex sequences

Models:

• LSTM / GRU
• DeepAR
• N-BEATS
• Temporal Fusion Transformer (TFT)

---

D. Generative Models (LLM-based TS)

Use when:

• Need scenario simulation
• Multi-modal distributions
• Irregular patterns
• Very long horizons

Models:

• Chronos
• Time-LLM
• Diffusion-based TS models

---

4. Model Constraints

Consider

• Training time
• Inference latency
• Explainability requirements
• Hardware constraints
• Availability of covariates

---

5. Model Selection Checklist

• [ ] Baseline compared
• [ ] Model matched to data patterns
• [ ] Hardware constraints respected
• [ ] Horizon requirements satisfied
• [ ] Evaluation metrics defined ahead of time

Multi-Step Forecasting Patterns

Operational patterns for forecasting multiple time steps ahead.

---

Overview

Multi-step forecasting predicts multiple future time points. Three main strategies exist:

1. Direct Strategy - Train separate models for each horizon 2. Recursive Strategy - Predict one step, feed back as input 3. Sequence-to-Sequence - Generate entire horizon at once

---

Pattern 1: Direct Strategy

When to Use

• Short horizons (1-7 steps)
• Need independent predictions per horizon
• Have sufficient data per horizon

Implementation

Train one model per horizon step:

# Example: Direct strategy for 7-day forecast
models = {}
for h in range(1, 8):
    X_train = create_features(df, lag_window=28)
    y_train = df['target'].shift(-h)  # Target at h steps ahead

    models[h] = LGBMRegressor().fit(X_train, y_train)

Pros

• No error propagation
• Different features per horizon
• Easier to interpret

Cons

• Multiple models to maintain
• More training time
• Features must be aligned carefully

---

Pattern 2: Recursive Strategy

When to Use

• Medium horizons (1-30 steps)
• Sequential dependencies matter
• Prefer single model simplicity

Implementation

# Recursive forecasting
model = LGBMRegressor().fit(X_train, y_train)

predictions = []
current_features = X_test[0].copy()

for h in range(1, forecast_horizon + 1):
    pred = model.predict([current_features])[0]
    predictions.append(pred)

    # Update features with prediction
    current_features = update_features(current_features, pred)

Pros

• Single model
• Captures sequential dependencies
• Memory efficient

Cons

• Error compounds over horizon
• Slower inference (sequential)
• Harder to parallelize

---

Pattern 3: Sequence-to-Sequence (Deep Learning)

When to Use

• Long horizons (30-180 steps)
• Complex temporal patterns
• Have sufficient data for DL

Frameworks

• Transformers: TimesFM, Chronos (for long dependencies)
• RNNs/LSTMs: Good for sequential data
• TFT: Temporal Fusion Transformers
• N-BEATS: Neural Basis Expansion
• DeepAR: Probabilistic forecasting

Implementation Example

from pytorch_forecasting import TemporalFusionTransformer

model = TemporalFusionTransformer(
    max_prediction_length=30,  # Forecast horizon
    max_encoder_length=60,     # Historical window
    # ... other config
)

predictions = model.predict(data)  # Full horizon at once

Pros

• Outputs full horizon
• Captures complex patterns
• Probabilistic forecasts

Cons

• Needs more data
• Harder to debug
• Computationally expensive

---

Choosing the Right Strategy

Additional Considerations

Data characteristics:

• High noise → Direct (less error propagation)
• Strong autocorrelation → Recursive or Seq2Seq
• Multiple seasonalities → Seq2Seq or Prophet

Computational constraints:

• Limited resources → Direct or Recursive
• Need real-time predictions → Direct (parallelizable)
• Batch predictions → Any strategy

Explainability:

• Need interpretability → Direct with LightGBM
• Black box acceptable → Seq2Seq deep learning

---

Hybrid Approaches

Direct-Recursive Hybrid

# Use direct for near-term (1-7), recursive for long-term
short_term_models = {h: train_direct_model(h) for h in range(1, 8)}
long_term_model = train_recursive_model()

# Combine predictions
predictions = []
predictions.extend([short_term_models[h].predict(X) for h in range(1, 8)])
predictions.extend(recursive_forecast(long_term_model, X, horizon=23))

Ensemble Approach

# Combine multiple strategies
direct_preds = direct_forecast(X, horizon)
recursive_preds = recursive_forecast(X, horizon)
seq2seq_preds = seq2seq_forecast(X, horizon)

# Weighted average
final_preds = 0.4 * direct_preds + 0.3 * recursive_preds + 0.3 * seq2seq_preds

---

Error Analysis by Strategy

Horizon-Specific Metrics

Track performance at each forecast step:

# Evaluate each horizon separately
for h in range(1, horizon + 1):
    y_true_h = actuals[:, h-1]
    y_pred_h = predictions[:, h-1]

    mae_h = mean_absolute_error(y_true_h, y_pred_h)
    print(f"Horizon {h}: MAE = {mae_h:.2f}")

Error Propagation Monitoring

For recursive strategy, monitor cumulative error:

# Track error growth over horizon
errors = []
for h in range(1, horizon + 1):
    error_h = np.abs(actuals[:, h-1] - predictions[:, h-1]).mean()
    errors.append(error_h)

# Plot error growth
plt.plot(range(1, horizon + 1), errors)
plt.xlabel("Forecast Horizon")
plt.ylabel("MAE")
plt.title("Error Propagation Over Horizon")

---

Checklist: Multi-Step Strategy Implementation

Planning

• [ ] Forecast horizon defined (H-step)
• [ ] Strategy chosen based on horizon length
• [ ] Data characteristics analyzed
• [ ] Computational constraints documented

Direct Strategy

• [ ] Separate models trained for each horizon
• [ ] Features aligned correctly for each target
• [ ] Models saved with horizon identifier
• [ ] Parallel prediction implemented

Recursive Strategy

• [ ] Feature update logic implemented
• [ ] Error propagation monitored
• [ ] Stopping criteria defined
• [ ] Fallback for divergence cases

Seq2Seq Strategy

• [ ] Encoder/decoder architecture chosen
• [ ] Historical window size optimized
• [ ] Attention mechanisms configured
• [ ] Probabilistic outputs enabled (if needed)

Evaluation

• [ ] Horizon-wise metrics computed
• [ ] Error growth analyzed
• [ ] Baseline comparison done
• [ ] Segment-level performance checked

---

References

See also:

• Backtesting Patterns - Temporal validation strategies
• Model Selection Guide - Choosing forecasting models
• TS-LLM Patterns - Deep learning approaches

Probabilistic Forecasting

Operational guide for generating prediction intervals, quantile forecasts, and distributional predictions. Covers quantile regression, conformal prediction, calibration assessment, and decision support using uncertainty. Focus on producing reliable uncertainty estimates, not just point forecasts.

Freshness anchor: January 2026 — MAPIE 0.9+, LightGBM 4.x, statsforecast 1.7+, scikit-learn 1.5+

---

Decision Tree: Choosing a Probabilistic Method

START
│
├─ Need distribution-free guarantees?
│   ├─ YES → Conformal Prediction (MAPIE)
│   └─ NO  → Continue
│
├─ Model type?
│   ├─ LightGBM / XGBoost / tree-based
│   │   ├─ Need specific quantiles → Quantile regression (native)
│   │   └─ Need full distribution → Conformal on top of point model
│   │
│   ├─ Linear / GLM
│   │   ├─ Known distribution → Distributional (Normal, Poisson, NegBin)
│   │   └─ Unknown → Quantile regression (QuantReg)
│   │
│   ├─ Neural network (temporal fusion transformer, etc.)
│   │   └─ Distributional output head or quantile loss
│   │
│   └─ Statistical (ARIMA, ETS, Prophet)
│       └─ Built-in prediction intervals (use them, then calibrate)
│
├─ What uncertainty do you need?
│   ├─ Symmetric intervals (80%, 95%) → Conformal or normal approx
│   ├─ Asymmetric intervals → Quantile regression
│   └─ Full predictive distribution → Distributional model or ensemble
│
└─ How many training samples?
    ├─ < 500 → Conformal (works with small calibration sets)
    ├─ 500–50k → Quantile regression
    └─ > 50k → Any method

---

Quick Reference: Methods Comparison

---

Operational Patterns

Pattern 1: Conformal Prediction with MAPIE

• Use when: Need guaranteed coverage with any base model
• Implementation:

from mapie.regression import MapieRegressor
from mapie.time_series import MapieTimeSeriesRegressor
from sklearn.ensemble import GradientBoostingRegressor

# Standard conformal (exchangeable data)
base_model = GradientBoostingRegressor(n_estimators=300)
mapie = MapieRegressor(
    estimator=base_model,
    method='plus',         # jackknife+ (recommended)
    cv=5,                  # cross-conformal
)
mapie.fit(X_train, y_train)

y_pred, y_intervals = mapie.predict(X_test, alpha=[0.05, 0.20])
# y_intervals shape: (n_samples, 2, n_alphas)
# alpha=0.05 → 95% interval; alpha=0.20 → 80% interval

# Time series conformal (accounts for temporal dependence)
mapie_ts = MapieTimeSeriesRegressor(
    estimator=base_model,
    method='enbpi',        # ensemble batch prediction intervals
    cv='prefit',
)
mapie_ts.fit(X_train, y_train)
y_pred_ts, y_intervals_ts = mapie_ts.predict(
    X_test, alpha=0.05, ensemble=True, optimize_beta=True
)

• Key property: Conformal guarantees P(y in interval) >= 1 - alpha regardless of model quality
• Gotcha: Guarantee is marginal (average coverage), not conditional (per-instance)

Pattern 2: LightGBM Quantile Regression

• Use when: Need specific quantiles, tabular data, fast training
• Implementation:

import lightgbm as lgb

def train_quantile_model(X_train, y_train, quantile, params=None):
    """Train a single quantile model."""
    default_params = {
        'objective': 'quantile',
        'alpha': quantile,
        'metric': 'quantile',
        'n_estimators': 500,
        'learning_rate': 0.05,
        'num_leaves': 63,
        'verbosity': -1,
    }
    if params:
        default_params.update(params)

    model = lgb.LGBMRegressor(**default_params)
    model.fit(X_train, y_train)
    return model

# Train multiple quantiles
quantiles = [0.025, 0.10, 0.25, 0.50, 0.75, 0.90, 0.975]
models = {q: train_quantile_model(X_train, y_train, q) for q in quantiles}

# Predict
predictions = {q: m.predict(X_test) for q, m in models.items()}

# 95% interval: [0.025, 0.975]
# 80% interval: [0.10, 0.90]
# Point forecast: 0.50 (median)

• Quantile crossing fix: Sort predicted quantiles per instance

import numpy as np

def fix_crossing(pred_dict, quantiles):
    """Ensure quantile predictions don't cross."""
    matrix = np.column_stack([pred_dict[q] for q in sorted(quantiles)])
    matrix_sorted = np.sort(matrix, axis=1)  # enforce monotonicity
    return {q: matrix_sorted[:, i] for i, q in enumerate(sorted(quantiles))}

Pattern 3: Distributional Forecasting

• Use when: Know the data-generating distribution, need full predictive distribution
• Distribution selection:

# Example: Negative Binomial for sales count data
import statsmodels.api as sm

# GLM approach
model = sm.GLM(
    y_train,
    sm.add_constant(X_train),
    family=sm.families.NegativeBinomial(alpha=1.0),
)
result = model.fit()

# Generate prediction intervals from fitted distribution
from scipy.stats import nbinom
mu_pred = result.predict(sm.add_constant(X_test))
alpha = result.scale
# Convert to scipy parameterization and compute intervals

Pattern 4: Calibration Assessment

• Use when: Always — every probabilistic forecast must be calibrated

def assess_calibration(y_true, lower, upper, nominal_coverage=0.95):
    """Check if prediction intervals achieve stated coverage."""
    covered = ((y_true >= lower) & (y_true <= upper)).mean()

    width = (upper - lower).mean()

    return {
        'nominal_coverage': nominal_coverage,
        'actual_coverage': covered,
        'miscalibration': abs(covered - nominal_coverage),
        'mean_interval_width': width,
    }

# Multi-level calibration (reliability diagram)
def calibration_curve(y_true, quantile_preds, quantiles):
    """PIT histogram / reliability diagram."""
    results = []
    for q in quantiles:
        below = (y_true <= quantile_preds[q]).mean()
        results.append({'quantile': q, 'observed_fraction': below})
    return pd.DataFrame(results)

# Perfect calibration: observed_fraction ≈ quantile at all levels

• Calibration targets:

Pattern 5: Scoring Rules

• Use when: Comparing probabilistic forecasts (not just point accuracy)

def pinball_loss(y_true, y_pred, quantile):
    """Quantile loss (pinball loss) — lower is better."""
    errors = y_true - y_pred
    return np.where(errors >= 0, quantile * errors, (quantile - 1) * errors).mean()

def winkler_score(y_true, lower, upper, alpha=0.05):
    """Winkler interval score — penalizes width + miscoverage."""
    width = upper - lower
    penalty_lower = (2 / alpha) * np.maximum(lower - y_true, 0)
    penalty_upper = (2 / alpha) * np.maximum(y_true - upper, 0)
    return (width + penalty_lower + penalty_upper).mean()

def crps_empirical(y_true, ensemble_preds):
    """CRPS from ensemble predictions — measures full distribution quality."""
    n_ensemble = ensemble_preds.shape[1]
    n_samples = len(y_true)

    crps_values = []
    for i in range(n_samples):
        fc = np.sort(ensemble_preds[i])
        obs = y_true[i]
        term1 = np.mean(np.abs(fc - obs))
        term2 = np.mean(np.abs(fc[:, None] - fc[None, :])) / 2
        crps_values.append(term1 - term2)
    return np.mean(crps_values)

# Metric selection:
# - Pinball loss → evaluating specific quantiles
# - Winkler score → evaluating prediction intervals
# - CRPS → evaluating full predictive distribution

Pattern 6: Decision Support with Prediction Intervals

• Use when: Translating uncertainty into actionable decisions

# Inventory planning with quantile forecasts
def compute_safety_stock(demand_forecast, upper_quantile, lead_time_days):
    """Safety stock from prediction intervals."""
    # Upper quantile represents service level
    # e.g., 0.95 quantile → 95% service level
    expected_demand = demand_forecast['q50'] * lead_time_days
    safety_stock = (upper_quantile - demand_forecast['q50']) * np.sqrt(lead_time_days)
    reorder_point = expected_demand + safety_stock
    return {
        'expected_demand': expected_demand,
        'safety_stock': safety_stock,
        'reorder_point': reorder_point,
    }

# Risk-aware decision thresholds
# - Conservative (risk-averse): use 95th percentile
# - Balanced: use 80th percentile
# - Aggressive (cost-minimizing): use median (50th percentile)

---

Anti-Patterns

---

Validation Checklist

• [ ] Probabilistic method chosen based on model type and data properties
• [ ] Coverage assessed at multiple levels (80%, 90%, 95%)
• [ ] Calibration error < 5% at each level
• [ ] Interval sharpness (width) reported alongside coverage
• [ ] Scoring rule used for model comparison (CRPS or Winkler)
• [ ] Quantile crossing handled (sorting or monotonic constraint)
• [ ] Decision framework maps uncertainty to business actions
• [ ] Conformal guarantee assumptions verified (exchangeability or time-series variant)
• [ ] Recalibration scheduled after each model retrain
• [ ] PIT histogram checked for uniformity

---

Cross-References

• ai-ml-timeseries/references/hierarchical-forecasting.md — coherent probabilistic forecasts across levels
• ai-ml-timeseries/references/anomaly-detection-patterns.md — prediction intervals as anomaly bounds
• ai-ml-data-science/references/hyperparameter-optimization.md — tuning quantile models
• ai-mlops/references/experiment-tracking-patterns.md — logging interval metrics and calibration

Production Time Series Deployment Patterns

Operational patterns for deploying, monitoring, and maintaining time series forecasting systems in production.

---

Overview

Production time series systems require:

• Automated feature pipelines
• Scheduled retraining
• Drift monitoring
• Fallback strategies
• Data quality checks
• Streaming ingestion handling

---

Pattern 1: Feature Pipeline Architecture

Key Principles

• Same code for training and serving
• Idempotent operations
• Historical replay capability
• Version control for features

Implementation

# feature_pipeline.py
from datetime import datetime, timedelta
import pandas as pd

class TimeSeriesFeaturePipeline:
    def __init__(self, config):
        self.config = config
        self.feature_version = config['feature_version']

    def create_features(self, df, end_date=None):
        """
        Create features for both training and serving
        Same code path for consistency
        """
        if end_date is None:
            end_date = datetime.now()

        # Feature engineering (identical for train/serve)
        df = self._add_lag_features(df, end_date)
        df = self._add_rolling_features(df, end_date)
        df = self._add_calendar_features(df, end_date)

        # Version stamp
        df['feature_version'] = self.feature_version

        return df

    def _add_lag_features(self, df, end_date):
        """Ensure no future leakage"""
        df = df[df.index <= end_date].copy()
        for lag in [1, 7, 28]:
            df[f'lag_{lag}'] = df['target'].shift(lag)
        return df

    def _add_rolling_features(self, df, end_date):
        """Temporal aggregations"""
        df = df[df.index <= end_date].copy()
        for window in [7, 14, 28]:
            df[f'rolling_mean_{window}'] = df['target'].rolling(window).mean()
            df[f'rolling_std_{window}'] = df['target'].rolling(window).std()
        return df

    def _add_calendar_features(self, df, end_date):
        """Calendar features"""
        df['day_of_week'] = df.index.dayofweek
        df['month'] = df.index.month
        df['is_weekend'] = df.index.dayofweek.isin([5, 6]).astype(int)
        return df

Scheduling

# airflow_dag.py or cron schedule
# Daily feature pipeline
schedule: "0 1 * * *"  # 1 AM daily

tasks:
  - name: extract_raw_data
    source: database
    table: time_series_raw

  - name: transform_features
    code: feature_pipeline.create_features()
    output: feature_store

  - name: validate_features
    checks:
      - no_nulls_in_lag_features
      - feature_version_matches
      - date_range_complete

---

Pattern 2: Model Retraining Strategy

Time-Based Retraining

class ModelRetrainingScheduler:
    def __init__(self, schedule='weekly'):
        self.schedule = schedule  # 'daily', 'weekly', 'monthly'

    def should_retrain(self, last_train_date):
        """Determine if retraining is needed"""
        today = datetime.now().date()

        if self.schedule == 'daily':
            return (today - last_train_date).days >= 1
        elif self.schedule == 'weekly':
            return (today - last_train_date).days >= 7
        elif self.schedule == 'monthly':
            return (today - last_train_date).days >= 30

        return False

    def retrain_model(self, data, config):
        """Execute retraining"""
        # Load historical data
        train_data = data[data['date'] <= datetime.now() - timedelta(days=7)]

        # Train model
        model = self._train(train_data, config)

        # Validate on recent data
        val_metrics = self._validate(model, data)

        if val_metrics['mae'] < config['max_acceptable_mae']:
            self._save_model(model)
            return True
        else:
            print("Retraining failed validation, keeping old model")
            return False

Trigger-Based Retraining (Drift Detection)

class DriftTriggeredRetraining:
    def __init__(self, drift_threshold=0.15):
        self.drift_threshold = drift_threshold

    def check_drift_and_retrain(self, model, recent_data):
        """
        Monitor forecast error drift
        Retrain if performance degrades
        """
        # Compute recent forecast errors
        recent_mae = self._compute_recent_mae(model, recent_data)
        baseline_mae = model.metadata['training_mae']

        # Check drift
        drift = (recent_mae - baseline_mae) / baseline_mae

        if drift > self.drift_threshold:
            print(f"Drift detected: {drift:.2%}. Retraining...")
            return self._retrain(recent_data)

        return False

---

Pattern 3: Monitoring & Drift Detection

Multi-Level Monitoring

class ForecastMonitoring:
    def __init__(self):
        self.metrics = []

    def monitor(self, forecasts, actuals, features):
        """
        Track multiple drift signals
        """
        # 1. Forecast error drift
        error_drift = self._monitor_error_drift(forecasts, actuals)

        # 2. Feature drift (distribution shift)
        feature_drift = self._monitor_feature_drift(features)

        # 3. Volume drift (data pattern change)
        volume_drift = self._monitor_volume_drift(actuals)

        # 4. Horizon-specific drift
        horizon_drift = self._monitor_horizon_drift(forecasts, actuals)

        # Aggregate metrics
        self.metrics.append({
            'timestamp': datetime.now(),
            'error_drift': error_drift,
            'feature_drift': feature_drift,
            'volume_drift': volume_drift,
            'horizon_drift': horizon_drift
        })

        return self._should_alert(self.metrics[-1])

    def _monitor_error_drift(self, forecasts, actuals):
        """MAE drift over time"""
        recent_mae = np.abs(forecasts - actuals).mean()
        historical_mae = self._get_historical_mae()
        return (recent_mae - historical_mae) / historical_mae

    def _monitor_feature_drift(self, features):
        """Feature distribution drift (PSI or KS)"""
        from scipy.stats import ks_2samp

        drift_scores = {}
        for col in features.columns:
            historical_dist = self._get_historical_dist(col)
            recent_dist = features[col].values

            # Kolmogorov-Smirnov test
            ks_stat, p_value = ks_2samp(historical_dist, recent_dist)
            drift_scores[col] = ks_stat

        return max(drift_scores.values())

    def _monitor_volume_drift(self, actuals):
        """Detect sudden volume changes"""
        recent_mean = actuals[-30:].mean()
        historical_mean = actuals[-180:-30].mean()
        return abs(recent_mean - historical_mean) / historical_mean

    def _monitor_horizon_drift(self, forecasts, actuals):
        """Track error by forecast horizon"""
        horizon_errors = {}
        for h in range(forecasts.shape[1]):
            mae_h = np.abs(forecasts[:, h] - actuals[:, h]).mean()
            horizon_errors[h+1] = mae_h

        return horizon_errors

Alert Thresholds

MONITORING_THRESHOLDS = {
    'error_drift': 0.15,      # 15% MAE increase triggers alert
    'feature_drift': 0.10,    # 10% KS statistic
    'volume_drift': 0.20,     # 20% volume change
    'horizon_drift': {
        1: 0.10,  # Near-term: 10% acceptable
        7: 0.15,  # Week-ahead: 15%
        30: 0.25  # Month-ahead: 25%
    }
}

---

Pattern 4: Fallback Strategies

Graceful Degradation

class ForecastingWithFallback:
    def __init__(self, primary_model, fallback_strategy='seasonal_naive'):
        self.primary_model = primary_model
        self.fallback_strategy = fallback_strategy

    def predict(self, X, historical_data):
        """
        Try primary model, fall back if fails
        """
        try:
            forecast = self.primary_model.predict(X)

            # Sanity checks
            if self._is_valid_forecast(forecast):
                return forecast, 'primary'
            else:
                return self._fallback(historical_data), 'fallback_sanity_check'

        except Exception as e:
            print(f"Primary model failed: {e}")
            return self._fallback(historical_data), 'fallback_exception'

    def _is_valid_forecast(self, forecast):
        """Sanity checks on forecast"""
        # No NaNs
        if np.isnan(forecast).any():
            return False

        # No extreme values
        if (forecast < 0).any() or (forecast > 1e6).any():
            return False

        # No flat forecasts (all same value)
        if forecast.std() < 1e-6:
            return False

        return True

    def _fallback(self, historical_data):
        """Fallback forecast strategies"""
        if self.fallback_strategy == 'last_known':
            return historical_data[-1]

        elif self.fallback_strategy == 'seasonal_naive':
            # Last year same period
            return historical_data[-365:]

        elif self.fallback_strategy == 'moving_average':
            return historical_data[-30:].mean()

        else:
            # Ultimate fallback: median
            return historical_data.median()

---

Pattern 5: Streaming Ingestion & Backfill

Real-Time Data Ingestion

class StreamingTimeSeriesIngestion:
    def __init__(self, kafka_topic, feature_pipeline):
        self.kafka_topic = kafka_topic
        self.feature_pipeline = feature_pipeline
        self.buffer = []

    def consume_stream(self):
        """
        Consume real-time events
        Handle late arrivals and out-of-order data
        """
        from kafka import KafkaConsumer

        consumer = KafkaConsumer(
            self.kafka_topic,
            bootstrap_servers=['localhost:9092'],
            auto_offset_reset='latest'
        )

        for message in consumer:
            event = self._parse_event(message.value)

            # Handle late arrivals
            if self._is_late_arrival(event):
                self._backfill(event)
            else:
                self._process_event(event)

    def _is_late_arrival(self, event):
        """Check if event timestamp is in the past"""
        event_time = event['timestamp']
        processing_time = datetime.now()
        return (processing_time - event_time).total_seconds() > 3600  # 1 hour late

    def _backfill(self, event):
        """Handle late data with idempotent upsert"""
        # Upsert into feature store (overwrites if exists)
        self.feature_store.upsert(
            timestamp=event['timestamp'],
            features=self.feature_pipeline.create_features([event])
        )

    def _process_event(self, event):
        """Process on-time event"""
        self.buffer.append(event)

        # Micro-batch every 100 events
        if len(self.buffer) >= 100:
            self._flush_buffer()

    def _flush_buffer(self):
        """Write buffered events to feature store"""
        features = self.feature_pipeline.create_features(self.buffer)
        self.feature_store.write(features)
        self.buffer = []

Historical Backfill

class HistoricalBackfill:
    def __init__(self, feature_pipeline, start_date, end_date):
        self.feature_pipeline = feature_pipeline
        self.start_date = start_date
        self.end_date = end_date

    def run_backfill(self, chunk_size='1D'):
        """
        Backfill historical data with windowing
        """
        date_range = pd.date_range(self.start_date, self.end_date, freq=chunk_size)

        for chunk_start in date_range:
            chunk_end = chunk_start + pd.Timedelta(chunk_size)

            # Extract raw data for chunk
            raw_data = self._extract_raw_data(chunk_start, chunk_end)

            # Create features (same code as production)
            features = self.feature_pipeline.create_features(raw_data)

            # Validate features
            if self._validate_features(features):
                self._write_features(features)
            else:
                print(f"Backfill validation failed for {chunk_start}")

        print("Backfill complete")

    def _validate_features(self, features):
        """Ensure backfilled features match schema"""
        required_cols = self.feature_pipeline.get_feature_names()
        return all(col in features.columns for col in required_cols)

---

Pattern 6: Data Residency & Governance

Multi-Tenant Isolation

class MultiTenantForecastingPipeline:
    def __init__(self, tenant_id):
        self.tenant_id = tenant_id

    def get_data(self):
        """Ensure tenant isolation"""
        query = f"""
        SELECT * FROM time_series_data
        WHERE tenant_id = '{self.tenant_id}'
        AND date >= CURRENT_DATE - INTERVAL '2 years'
        """
        return self._execute_query(query)

    def save_forecast(self, forecast):
        """Tag with tenant for isolation"""
        forecast['tenant_id'] = self.tenant_id
        self._write_to_db(forecast, table='forecasts')

PII Handling

class PIIHandling:
    def anonymize_features(self, df):
        """Remove or hash PII before model training"""
        pii_cols = ['customer_id', 'email', 'phone']

        for col in pii_cols:
            if col in df.columns:
                df[col] = df[col].apply(self._hash_pii)

        return df

    def _hash_pii(self, value):
        """One-way hash for PII"""
        import hashlib
        return hashlib.sha256(str(value).encode()).hexdigest()

Audit Trail & Provenance

class ForecastProvenance:
    def log_forecast(self, forecast, metadata):
        """Track full lineage of forecast"""
        provenance = {
            'forecast_id': uuid.uuid4(),
            'timestamp': datetime.now(),
            'model_version': metadata['model_version'],
            'feature_version': metadata['feature_version'],
            'data_source': metadata['data_source'],
            'data_date_range': metadata['data_date_range'],
            'retraining_trigger': metadata['retraining_trigger'],  # 'scheduled' or 'drift'
            'forecast': forecast
        }

        # Store in audit log
        self._write_to_audit_log(provenance)

---

Checklist: Production-Ready Time Series System

Feature Pipeline

• [ ] Same code for training and serving
• [ ] Idempotent operations (safe to re-run)
• [ ] Version control for feature definitions
• [ ] Historical replay capability
• [ ] Scheduled execution (daily/hourly)
• [ ] Data quality validation

Model Deployment

• [ ] Retraining schedule defined (time-based or drift-based)
• [ ] Model versioning implemented
• [ ] Rollback capability for failed deployments
• [ ] A/B testing for new model versions
• [ ] Prediction caching for efficiency

Monitoring

• [ ] Forecast error tracking (MAE, MAPE, WAPE)
• [ ] Feature drift detection (PSI, KS)
• [ ] Volume drift monitoring
• [ ] Horizon-specific error tracking
• [ ] Alerts configured with thresholds
• [ ] Dashboard for real-time monitoring

Fallback & Resilience

• [ ] Fallback strategy defined (seasonal naive, last known, etc.)
• [ ] Sanity checks on forecasts
• [ ] Circuit breaker for model failures
• [ ] Fallback usage tracked and alerted

Data Ingestion

• [ ] Streaming ingestion implemented (Kafka/Kinesis)
• [ ] Late arrival handling (backfill)
• [ ] Out-of-order data handling
• [ ] Idempotent upserts for duplicates
• [ ] Gap filling with business rules

Governance

• [ ] Tenant isolation (multi-tenant systems)
• [ ] PII handling documented
• [ ] Data residency compliance (GDPR, CCPA)
• [ ] Audit trail for forecasts
• [ ] Provenance tracking (data → features → forecast)

---

References

See also:

• Backtesting Patterns - Validation strategies
• Model Selection Guide - Choosing models for production
• TS EDA Best Practices - Data quality checks

Time Series EDA Best Practices

A structured, operational workflow for analyzing univariate and multivariate time series.

---

1. Timestamp Integrity

Required checks:

• [ ] Confirm frequency (daily, hourly, weekly, etc.)
• [ ] Identify missing timestamps
• [ ] Remove or merge duplicates
• [ ] Align timezone information
• [ ] Validate monotonic ordering

Commands (example):

ts = ts.sort_index() ts.asfreq('D') ts.index.is_monotonic_increasing

---

2. Visualization Patterns

Plots:

• Line plot (raw series)
• Rolling mean/variance
• Seasonal decomposition plot
• Weekly/yearly seasonal plot
• Autocorrelation (ACF) and partial autocorrelation (PACF)

Checklist

• [ ] Trend observed?
• [ ] Seasonality present?
• [ ] Variance stable or increasing?

---

3. Trend & Seasonality Analysis

Determine:

• Direction (up/down)
• Stability (consistent/inconsistent)
• Strength (e.g., seasonal_strength metric)
• Season length (7, 24, 365, etc.)

---

4. Outlier Detection

Techniques:

• Z-score
• IQR thresholds
• Sudden spikes/drops based on rolling windows

Checklist

• [ ] Outliers flagged
• [ ] Outlier handling strategy chosen (cap/remove/flag)

---

5. Missing Value Strategies

Approaches:

• Forward fill
• Interpolation
• Seasonal interpolation
• Leave missing (if models support it)

Choose based on:

• Frequency
• Impact on downstream training

---

6. Volume/Granularity Normalization

Aggregate or resample when:

• Granularity doesn’t match business need
• Noise is too high at lower granularity

---

7. TS EDA Final Deliverables

• [ ] Frequency validated
• [ ] Seasonal/trend patterns documented
• [ ] Outlier plan defined
• [ ] Missing value rules defined
• [ ] Data dictionary for temporal features

LLM-Based Forecasting & Generative TS Patterns

Operational patterns for using foundation models (Chronos-2, TimesFM 2.5, Lag-Llama) and generative TS systems.

Modern Best Practices (January 2026):

• Chronos-2 (Oct 2025): Universal forecasting with multivariate/covariate support — best accuracy on benchmarks
• Chronos-Bolt (Nov 2024): 250x faster inference, 20x memory efficient — production-ready univariate
• TimesFM 2.5: XReg covariate support for regression tasks
• Zero-shot foundation models often match or outperform trained-from-scratch Transformers

---

Model Comparison (January 2026)

Benchmark Performance (GIFT-Eval, Chronos Benchmark II):

• Chronos-2 > TimesFM-2.5 > TiRex on most benchmarks
• Zero-shot TSFMs often match trained Transformers with no tuning

---

1. When to Use TS-LLM

Use for:

• Long horizon forecasts
• Multi-modal distributions
• Simulation of scenarios
• Extremely irregular or non-linear patterns
• Zero or limited domain features

---

2. Tokenization & Value Discretization

LLM TS models require discrete tokens.

Options

• Quantize numeric values → buckets
• Scale → round → tokenize
• Diffusion-style embeddings

Checklist

• [ ] Values discretized consistently
• [ ] Resolution high enough for accuracy
• [ ] No leakage from future values

---

3. TS-LLM Inference Workflow

Steps

1. Provide past N values 2. Model auto-regressively generates next H tokens 3. Convert tokens back to continuous values 4. Optionally run multiple samples for probabilistic outputs

---

4. Scenario Simulation Pattern

Use multiple model samples:

• 20–100 stochastic trajectories
• Compute percentiles (P10/P50/P90)
• Use for risk-aware planning

---

5. Hybrid TS Pattern (Classical + TS-LLM)

Combine:

• Statistical + ML forecast
• LLM for long-horizon adjustment
• Weighted or rule-based fusion

Example: final_forecast = 0.7 ml_model + 0.3 llm_model

---

6. Evaluation Pattern for TS-LLMs

Evaluate:

• MASE
• Pinball loss
• Horizon-by-horizon accuracy
• Drift tolerance
• Scenario coverage

---

7. TS-LLM Checklist

• [ ] Discretization validated
• [ ] Long horizon stable
• [ ] No leakage
• [ ] Backtested against baselines
• [ ] Scenario coverage documented