
Siem Logging
- 74 installs
- 426 repo stars
- Updated December 11, 2025
- ancoleman/ai-design-components
Siem-logging is a Claude skill that configures SIEM systems (Elastic, Sentinel, Wazuh, Splunk) for threat detection, log aggregation and compliance auditing.
About
This skill configures security information and event management (SIEM) systems for threat detection, log aggregation and compliance. Developers use it when implementing centralized security logging, writing detection rules, or meeting audit requirements. It covers platform selection, SIGMA and platform-specific detection rules, alert tuning and retention policies.
- SIEM platform selection (Elastic, Sentinel, Wazuh, Splunk)
- SIGMA universal detection rules compiled to any SIEM
- Log retention for GDPR, HIPAA, PCI DSS and SOC 2
Siem Logging by the numbers
- 74 all-time installs (skills.sh)
- Ranked #1,153 of 2,203 Security skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
siem-logging capabilities & compatibility
- Capabilities
- siem configuration · threat detection · log aggregation · detection rules · compliance audit
- Works with
- elasticsearch · splunk · azure · aws · gcp · kubernetes
- Use cases
- security audit · data analysis
- Pricing
- Free
What siem-logging says it does
Configure security information and event management (SIEM) systems for threat detection, log aggregation, and compliance.
SIGMA provides a universal detection rule format that compiles to any SIEM query language
npx skills add https://github.com/ancoleman/ai-design-components --skill siem-loggingAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 74 |
|---|---|
| repo stars | ★ 426 |
| Last updated | December 11, 2025 |
| Repository | ancoleman/ai-design-components ↗ |
What it does
Configure SIEM platforms for threat detection, log aggregation and compliance across cloud and on-prem infrastructure.
Who is it for?
Implementing centralized security logging and detection rules for compliance.
Skip if: Small apps without security-monitoring or audit-trail requirements.
When should I use this skill?
Implementing centralized security logging, writing detection rules, or meeting audit requirements.
What you get
A SIEM deployment with detection rules, tuned alerts and compliant log retention.
- SIEM platform selection
- SIGMA and platform-specific detection rules
- Compliance-aligned retention policy
By the numbers
- Four SIEM platforms compared (Elastic, Sentinel, Wazuh, Splunk)
- Brute-force rule at 10+ failed logins in 10 minutes
Files
SIEM Logging
Purpose
Configure comprehensive security logging infrastructure using SIEM platforms (Elastic SIEM, Microsoft Sentinel, Wazuh, Splunk) to detect threats, investigate incidents, and maintain compliance audit trails. This skill covers platform selection, log aggregation architecture, detection rule development (SIGMA format and platform-specific), alert tuning, and retention policies for regulatory compliance (GDPR, HIPAA, PCI DSS, SOC 2).
When to Use This Skill
Use this skill when:
- Implementing centralized security event monitoring across infrastructure
- Writing threat detection rules for authentication failures, privilege escalation, data exfiltration
- Designing log aggregation for multi-cloud environments (AWS, Azure, GCP, Kubernetes)
- Meeting compliance requirements for log retention and audit trails
- Tuning security alerts to reduce false positives and alert fatigue
- Calculating costs for high-volume security logging (TB/day scale)
- Integrating security logging with incident response workflows
SIEM Platform Selection
Quick Decision Framework
Choose SIEM platform based on:
Budget Considerations:
- Unlimited budget → Splunk Enterprise Security (enterprise features, proven scale)
- Moderate budget ($50k-$500k/year) → Microsoft Sentinel or Elastic SIEM (cloud-native, flexible)
- Tight budget (<$50k/year) → Wazuh (free, open-source XDR/SIEM)
Infrastructure Context:
- Heavy Azure investment → Microsoft Sentinel (native integration, built-in SOAR)
- Heavy AWS investment → AWS Security Lake + OpenSearch (AWS-native)
- Multi-cloud or on-premise → Elastic SIEM or Wazuh (platform-agnostic)
Data Volume:
- >1 TB/day → Splunk or Elastic Cloud (proven at scale)
- 100 GB - 1 TB/day → Microsoft Sentinel or Elastic SIEM
- <100 GB/day → Wazuh or Sentinel 50 GB tier
Team Expertise:
- Elasticsearch experience → Elastic SIEM (familiar tooling)
- Microsoft/Azure expertise → Microsoft Sentinel (Azure ecosystem)
- Generalists or limited resources → Wazuh (easiest learning curve)
Platform Comparison Summary
| Platform | Cost | Deployment | Best For |
|---|---|---|---|
| Elastic SIEM | $$$ | Cloud/Self-Hosted | Multi-cloud, customization needs, DevOps teams |
| Microsoft Sentinel | $$$ | Cloud (Azure) | Azure-heavy orgs, built-in SOAR, cloud-first |
| Wazuh | Free | Self-Hosted | Cost-conscious, SMBs, compliance requirements |
| Splunk ES | $$$$$ | Cloud/On-Prem | Large enterprises, massive scale, unlimited budget |
For detailed feature comparison, see references/platform-comparison.md.
Detection Rules
Universal Format: SIGMA Rules
SIGMA provides a universal detection rule format that compiles to any SIEM query language (Elastic EQL, Splunk SPL, Microsoft KQL).
SIGMA Rule Structure:
title: Multiple Failed Login Attempts from Single Source
id: 8a9e3c7f-4b2d-4e8a-9f1c-2d5e6f7a8b9c
status: stable
description: Detects potential brute force attacks (10+ failed logins in 10 minutes)
author: Security Team
date: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1110/
tags:
- attack.credential_access
- attack.t1110
logsource:
category: authentication
product: linux
detection:
selection:
event.type: authentication
event.outcome: failure
timeframe: 10m
condition: selection | count() by source.ip > 10
level: highCompile SIGMA to Platform-Specific:
# Install SIGMA compiler
pip install sigma-cli
# Compile to Elastic EQL
sigmac -t es-eql sigma_rule.yml
# Compile to Splunk SPL
sigmac -t splunk sigma_rule.yml
# Compile to Microsoft KQL
sigmac -t kusto sigma_rule.ymlPlatform-Specific Detection Formats
Elastic EQL (Event Query Language):
sequence by user.name with maxspan=5m
[process where process.name == "powershell.exe" and
process.args : ("Invoke-WebRequest", "iwr", "wget")]
[process where process.parent.name == "powershell.exe"]Microsoft Sentinel KQL:
SigninLogs
| where TimeGenerated > ago(1h)
| where ResultType != 0 // Failed login
| summarize FailedAttempts=count() by UserPrincipalName, IPAddress
| where FailedAttempts >= 10Splunk SPL:
index=web_logs sourcetype=access_combined
| rex field=uri "(?<sql_keywords>union|select|insert|update|delete)"
| where isnotnull(sql_keywords)
| stats count by src_ip, uri
| where count > 5For comprehensive detection rule examples, see:
examples/sigma-rules/- Universal SIGMA detection rulesexamples/elastic-eql/- Elastic-specific queriesexamples/microsoft-kql/- Microsoft Sentinel queriesexamples/splunk-spl/- Splunk searchesreferences/detection-rules-guide.md- Complete guide
Log Aggregation Architecture
Centralized Architecture
Single SIEM instance for all logs. Use when:
- Single region deployment
- Small to medium volumes (<1 TB/day)
- Single cloud provider or on-premise
- Limited security team (1-10 analysts)
Architecture:
Application Servers → Log Shippers (Filebeat/Fluentd)
↓
Log Aggregator (Logstash/Fluentd)
↓
SIEM Platform (Elasticsearch/Splunk/Sentinel)
↓
Security Analysts (Dashboard/Alerts)Distributed Architecture (Multi-Region)
Regional SIEM instances with global aggregation. Use when:
- Multi-region global deployments
- Data residency requirements (GDPR, sovereignty)
- High volumes (>1 TB/day per region)
- Low-latency requirements for regional analysis
Architecture:
Global SIEM (Correlation, Threat Intelligence)
↓
Regional SIEM (US-East) | Regional SIEM (EU-West) | Regional SIEM (APAC)
↓ ↓ ↓
Local Logs Local Logs Local LogsCloud-Native Architecture
Leverage managed cloud services. Use when:
- Cloud-first organization (AWS/Azure/GCP)
- Want to avoid managing infrastructure
- Elastic workloads with variable log volumes
- Budget for cloud service costs
AWS Example:
CloudTrail + VPC Flow Logs + GuardDuty
↓
AWS Security Lake (S3 Data Lake)
↓
OpenSearch (Analysis) | Athena (SQL Queries)For deployment examples, see:
examples/architectures/elk-stack-docker-compose.ymlexamples/architectures/fluentd-kubernetes-daemonset.yamlexamples/architectures/aws-security-lake-terraform/examples/architectures/wazuh-docker-compose.ymlreferences/cloud-native-logging.md
Log Aggregation Tools
Fluentd (Cloud-Native): CNCF project for Kubernetes and multi-cloud environments. Use for containerized applications.
Logstash (Elastic Stack): Native Elasticsearch integration. Use for advanced parsing (grok patterns) and data enrichment.
For complete configuration examples, see examples/logstash-pipelines/ and references/cloud-native-logging.md.
Log Retention and Compliance
Compliance Requirements
| Framework | Minimum Retention | Hot Storage | Warm Storage | Cold Storage |
|---|---|---|---|---|
| GDPR | 30-90 days | 7 days | 30 days | 60 days |
| HIPAA | 6 years | 30 days | 180 days | 6 years |
| PCI DSS | 1 year | 90 days | 180 days | 1 year |
| SOC 2 | 1 year | 30 days | 90 days | 1 year |
Storage Tiering Strategy
Hot Tier (SSD, Real-Time):
- Last 7-30 days
- Real-time indexing and fast queries
- Most expensive ($0.10/GB/month)
Warm Tier (HDD, Recent):
- 30-90 days
- Read-only indices, occasional searches
- Moderate cost ($0.05/GB/month)
Cold Tier (S3/Blob, Archive):
- 90 days to retention limit
- Searchable snapshots, rare queries
- Cheapest ($0.01/GB/month)
Example Cost Optimization:
500 GB/day log volume, 1-year retention
Hot (30 days): 15 TB @ $0.10/GB = $1,500/month
Warm (60 days): 30 TB @ $0.05/GB = $1,500/month
Cold (275 days): 137.5 TB @ $0.01/GB = $1,375/month
Total: $4,375/month = $52,500/year
vs. Hot-only: $18,250/month = $219,000/year
Savings: 76% ($166,500/year)For detailed retention policies and cost optimization, see:
references/log-retention-policies.mdreferences/cost-optimization.mdscripts/cost-calculator.py
What to Log (Security Events)
Critical Events (MUST LOG):
- Authentication: Login attempts, MFA, password changes, privilege escalation
- Authorization: Permission changes, role modifications, access denials
- Data Access: Sensitive database/file access, API calls, exports
- Network: Connections, firewall denials, VPN, DNS queries
- System: Service changes, configuration modifications, software installations
Severity Levels: Failed auth (3+): HIGH alert | Privilege escalation: CRITICAL alert | Data export: HIGH alert | Config change: MEDIUM (no alert)
Alert Tuning and Noise Reduction
Alert Lifecycle
1. Detection Rule Created - Conservative thresholds, deploy to production 2. Baseline Period (2-4 weeks) - Collect alert data, tag true/false positives 3. Tuning Phase - Add whitelisting, adjust thresholds, refine correlation 4. Continuous Improvement - Weekly metrics review, monthly effectiveness review
Noise Reduction Techniques
Whitelisting (Known-Safe Patterns):
# Example: Allow scanner IPs
- rule_id: brute_force_detection
whitelist:
- source_ip: "10.0.0.100" # Security scanner
- user_agent: "Nagios" # Monitoring systemThreshold Tuning:
# Before: Too sensitive (500 alerts/day, 5% true positive rate)
- rule: failed_login_attempts
threshold: 3 attempts in 5 minutes
# After: Tuned (50 alerts/day, 40% true positive rate)
- rule: failed_login_attempts
threshold: 10 attempts in 10 minutesMulti-Event Correlation:
# Instead of: Single event alert
- alert_on: "Failed authentication"
# Use: Correlated pattern
- alert_on:
- "Failed authentication (5+ times)"
- AND "From new IP address"
- AND "Successful authentication follows"
- WITHIN: 30 minutesTarget Alert Metrics
| Metric | Target |
|---|---|
| Total Alerts/Day | <100 |
| True Positive Rate | >30% |
| Mean Time to Investigate | <15 min |
| False Positive Rate | <50% |
| Critical Alerts/Day | <10 |
For comprehensive alert tuning strategies, see references/alert-tuning-strategies.md.
Quick Start
Deploy Wazuh: git clone https://github.com/wazuh/wazuh-docker.git && cd wazuh-docker/single-node && docker-compose up -d (see examples/architectures/wazuh-docker-compose.yml)
Create SIGMA Rule: See examples/sigma-rules/brute-force-detection.yml for SSH brute force detection template
Elastic Cloud: Sign up at cloud.elastic.co, create Security tier deployment, install Elastic Agent on endpoints
Integration with Related Skills
observability skill:
- Route security logs to SIEM, performance logs to observability platform
- Shared log aggregation infrastructure (Fluentd/Logstash)
- Different analysis purposes (security vs. performance)
incident-management skill:
- SIEM alerts trigger incident response workflows
- Integration with PagerDuty, Opsgenie, ServiceNow
- Automated incident creation for critical security events
security-hardening skill:
- SIEM monitors security configurations and compliance
- Detect configuration drift from CIS benchmarks
- Alert on security policy violations
building-ci-pipelines skill:
- Log CI/CD security events (deployments, secrets access)
- GitHub Actions/GitLab CI integration with SIEM
- Supply chain security monitoring
secret-management skill:
- Audit all secrets access operations
- HashiCorp Vault/AWS Secrets Manager logs to SIEM
- Detect unauthorized secrets access attempts
Reference Documentation
Detailed Guides
references/platform-comparison.md- Comprehensive SIEM platform feature comparisonreferences/detection-rules-guide.md- Detection rule formats (SIGMA, EQL, KQL, SPL)references/log-retention-policies.md- Compliance requirements and retention strategiesreferences/cloud-native-logging.md- AWS, Azure, GCP, Kubernetes logging setupreferences/alert-tuning-strategies.md- False positive reduction and alert optimizationreferences/cost-optimization.md- Storage tiering and cost management
Working Examples
examples/sigma-rules/- Universal SIGMA detection rules (10+ examples)examples/elastic-eql/- Elastic Event Query Language queriesexamples/microsoft-kql/- Microsoft Sentinel Kusto queriesexamples/splunk-spl/- Splunk Search Processing Languageexamples/architectures/- Complete deployment examples (Docker, Kubernetes, Terraform)examples/logstash-pipelines/- Logstash pipeline configurations
Utility Scripts
scripts/sigma-to-elastic.sh- Convert SIGMA rules to Elastic EQLscripts/cost-calculator.py- Estimate SIEM costs based on volume and retention
Official Documentation
- Elastic SIEM: https://www.elastic.co/security
- Microsoft Sentinel: https://azure.microsoft.com/en-us/products/microsoft-sentinel
- Wazuh: https://wazuh.com/
- Splunk Enterprise Security: https://www.splunk.com/en_us/products/enterprise-security.html
- SIGMA Rules Repository: https://github.com/SigmaHQ/sigma
- MITRE ATT&CK Framework: https://attack.mitre.org/
version: '3.8'
# ELK Stack (Elasticsearch, Logstash, Kibana) Docker Compose Deployment
# Production-ready SIEM architecture for centralized log aggregation and analysis
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
hostname: elasticsearch
restart: always
ports:
- "9200:9200" # HTTP API
- "9300:9300" # Node communication
environment:
# Cluster Configuration
- node.name=elasticsearch-node-1
- cluster.name=elk-siem-cluster
- discovery.type=single-node
# Memory Settings (adjust based on available resources)
- "ES_JAVA_OPTS=-Xms2g -Xmx2g"
- bootstrap.memory_lock=true
# Security Configuration (for production, use proper certificates)
- xpack.security.enabled=true
- xpack.security.enrollment.enabled=true
- ELASTIC_PASSWORD=SecureElasticPassword123!
# Disable SSL for internal communication (enable in production)
- xpack.security.http.ssl.enabled=false
- xpack.security.transport.ssl.enabled=false
# License (basic features are free)
- xpack.license.self_generated.type=basic
ulimits:
# Required for Elasticsearch performance
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- elasticsearch_data:/usr/share/elasticsearch/data
- elasticsearch_logs:/usr/share/elasticsearch/logs
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:9200/_cluster/health || exit 1"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
networks:
- elk
logstash:
image: docker.elastic.co/logstash/logstash:8.11.0
hostname: logstash
restart: always
ports:
- "5000:5000/tcp" # TCP input for log shippers
- "5000:5000/udp" # UDP input for syslog
- "5044:5044" # Beats input (Filebeat, Metricbeat)
- "9600:9600" # Monitoring API
environment:
# Elasticsearch output configuration
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
- ELASTICSEARCH_USERNAME=elastic
- ELASTICSEARCH_PASSWORD=SecureElasticPassword123!
# Pipeline Configuration
- "LS_JAVA_OPTS=-Xmx1g -Xms1g"
- pipeline.workers=2
- pipeline.batch.size=125
- pipeline.batch.delay=50
# Monitoring
- xpack.monitoring.enabled=true
- xpack.monitoring.elasticsearch.hosts=http://elasticsearch:9200
volumes:
# Pipeline configuration files
- ./logstash-config/pipelines.yml:/usr/share/logstash/config/pipelines.yml:ro
- ./logstash-config/pipeline/:/usr/share/logstash/pipeline/:ro
# Patterns and templates
- logstash_data:/usr/share/logstash/data
depends_on:
elasticsearch:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:9600/_node/stats || exit 1"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
networks:
- elk
kibana:
image: docker.elastic.co/kibana/kibana:8.11.0
hostname: kibana
restart: always
ports:
- "5601:5601" # Web interface
environment:
# Elasticsearch connection
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
- ELASTICSEARCH_USERNAME=elastic
- ELASTICSEARCH_PASSWORD=SecureElasticPassword123!
# Kibana Configuration
- SERVER_NAME=kibana-siem
- SERVER_HOST=0.0.0.0
# Security settings
- xpack.security.enabled=true
- xpack.encryptedSavedObjects.encryptionKey=min-32-char-encryption-key-here-change-in-production
# SIEM features enabled
- xpack.securitySolution.enabled=true
# Monitoring
- monitoring.ui.container.elasticsearch.enabled=true
volumes:
- kibana_data:/usr/share/kibana/data
depends_on:
elasticsearch:
condition: service_healthy
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:5601/api/status || exit 1"]
interval: 30s
timeout: 10s
retries: 5
start_period: 90s
networks:
- elk
# Filebeat - Log shipper for collecting logs from files
filebeat:
image: docker.elastic.co/beats/filebeat:8.11.0
hostname: filebeat
restart: always
user: root
command: filebeat -e -strict.perms=false
environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
- ELASTICSEARCH_USERNAME=elastic
- ELASTICSEARCH_PASSWORD=SecureElasticPassword123!
- KIBANA_HOST=http://kibana:5601
volumes:
# Filebeat configuration
- ./filebeat-config/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
# Collect Docker logs from all containers
- /var/lib/docker/containers:/var/lib/docker/containers:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
# System logs (optional)
- /var/log:/var/log:ro
# Filebeat data
- filebeat_data:/usr/share/filebeat/data
depends_on:
elasticsearch:
condition: service_healthy
kibana:
condition: service_healthy
networks:
- elk
volumes:
# Elasticsearch volumes
elasticsearch_data:
driver: local
elasticsearch_logs:
driver: local
# Logstash volumes
logstash_data:
driver: local
# Kibana volumes
kibana_data:
driver: local
# Filebeat volumes
filebeat_data:
driver: local
networks:
elk:
driver: bridge
# Resource Requirements (Recommended for Production):
# - Elasticsearch: 4GB RAM minimum, 8GB recommended, 16GB+ for high volume
# - Logstash: 2GB RAM minimum, 4GB recommended
# - Kibana: 2GB RAM minimum
#
# Total System: 8GB RAM minimum, 16GB+ recommended for production
#
# Storage Requirements:
# - Plan for 1.5-2x raw log size (with indexing overhead)
# - Example: 100GB/day logs = 150-200GB/day storage needed
# - Implement index lifecycle management (ILM) for retention
#
# Deployment Instructions:
# 1. Create config directories:
# mkdir -p logstash-config/pipeline filebeat-config
#
# 2. Create basic Logstash pipeline (logstash-config/pipeline/main.conf):
# input {
# beats { port => 5044 }
# tcp { port => 5000 codec => json }
# }
# filter {
# # Add your parsing logic here
# }
# output {
# elasticsearch {
# hosts => ["http://elasticsearch:9200"]
# user => "elastic"
# password => "SecureElasticPassword123!"
# index => "logs-%{+YYYY.MM.dd}"
# }
# }
#
# 3. Create Filebeat config (filebeat-config/filebeat.yml):
# filebeat.inputs:
# - type: container
# paths:
# - '/var/lib/docker/containers/*/*.log'
# output.logstash:
# hosts: ["logstash:5044"]
#
# 4. Start the stack:
# docker-compose up -d
#
# 5. Access Kibana at http://localhost:5601
# Username: elastic
# Password: SecureElasticPassword123!
#
# Security Notes:
# - Change all default passwords before production deployment
# - Enable SSL/TLS for Elasticsearch and Kibana in production
# - Use proper certificate management (Let's Encrypt, internal CA)
# - Configure firewall rules to restrict access to ports
# - Enable Elasticsearch authentication and role-based access control (RBAC)
# - Regularly update images to latest security patches
---
# Fluentd DaemonSet for Kubernetes Log Collection
# Deploys Fluentd on every node to collect container and system logs
# Forwards logs to Elasticsearch for SIEM analysis
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluentd
namespace: kube-system
labels:
app: fluentd
component: log-collector
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: fluentd
labels:
app: fluentd
component: log-collector
rules:
# Allow reading pod logs
- apiGroups: [""]
resources:
- pods
- pods/log
- namespaces
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: fluentd
labels:
app: fluentd
component: log-collector
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: fluentd
subjects:
- kind: ServiceAccount
name: fluentd
namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: fluentd-config
namespace: kube-system
labels:
app: fluentd
component: log-collector
data:
# Main Fluentd configuration
fluent.conf: |
# Input: Collect container logs from Kubernetes
<source>
@type tail
@id in_tail_container_logs
path /var/log/containers/*.log
pos_file /var/log/fluentd-containers.log.pos
tag kubernetes.*
read_from_head true
<parse>
# Parse Docker JSON log format
@type json
time_format %Y-%m-%dT%H:%M:%S.%NZ
time_key time
keep_time_key true
</parse>
</source>
# Input: Collect systemd journal logs
<source>
@type systemd
@id in_systemd
path /var/log/journal
matches [{ "_SYSTEMD_UNIT": "kubelet.service" }, { "_SYSTEMD_UNIT": "docker.service" }]
read_from_head true
tag systemd
<storage>
@type local
persistent true
path /var/log/fluentd-journald-cursor.json
</storage>
<entry>
fields_strip_underscores true
fields_lowercase true
</entry>
</source>
# Filter: Extract Kubernetes metadata (pod name, namespace, labels)
<filter kubernetes.**>
@type kubernetes_metadata
@id filter_kube_metadata
kubernetes_url "#{ENV['FLUENT_FILTER_KUBERNETES_URL'] || 'https://' + ENV.fetch('KUBERNETES_SERVICE_HOST') + ':' + ENV.fetch('KUBERNETES_SERVICE_PORT') + '/api'}"
verify_ssl "#{ENV['KUBERNETES_VERIFY_SSL'] || true}"
ca_file "#{ENV['KUBERNETES_CA_FILE']}"
skip_labels "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_LABELS'] || 'false'}"
skip_container_metadata "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_CONTAINER_METADATA'] || 'false'}"
skip_master_url "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_MASTER_URL'] || 'false'}"
skip_namespace_metadata "#{ENV['FLUENT_KUBERNETES_METADATA_SKIP_NAMESPACE_METADATA'] || 'false'}"
</filter>
# Filter: Enrich with security-relevant fields
<filter kubernetes.**>
@type record_transformer
@id filter_security_enrichment
enable_ruby true
<record>
# Add cluster identifier
cluster_name "#{ENV['CLUSTER_NAME'] || 'kubernetes-cluster'}"
# Add environment tag
environment "#{ENV['ENVIRONMENT'] || 'production'}"
# Extract security-relevant fields
security_context ${record.dig("kubernetes", "pod_security_context")}
# Add timestamp in ISO8601 format
timestamp ${time.iso8601}
# Flag potential security events
is_privileged ${record.dig("kubernetes", "container_security_context", "privileged") || false}
is_host_network ${record.dig("kubernetes", "host_network") || false}
is_host_pid ${record.dig("kubernetes", "host_pid") || false}
</record>
</filter>
# Filter: Parse application logs (JSON format)
<filter kubernetes.**>
@type parser
@id filter_parser
key_name log
reserve_data true
remove_key_name_field false
<parse>
@type multi_format
# Try JSON parsing first
<pattern>
format json
time_key timestamp
keep_time_key true
</pattern>
# Fallback to plain text
<pattern>
format none
</pattern>
</parse>
</filter>
# Filter: Detect security events in logs
<filter kubernetes.**>
@type grep
@id filter_security_events
<regexp>
key log
pattern /(authentication|authorization|permission denied|access denied|failed login|privilege escalation|unauthorized)/i
</regexp>
<and>
<regexp>
key log
pattern /.+/
</regexp>
</and>
</filter>
# Filter: Rate limiting to prevent log storms
<filter kubernetes.**>
@type throttle
@id filter_throttle
group_key kubernetes.namespace_name
group_bucket_period_s 60
group_bucket_limit 10000
group_reset_rate_s 600
</filter>
# Output: Forward to Elasticsearch
<match kubernetes.**>
@type elasticsearch
@id out_es
@log_level info
# Elasticsearch connection
host "#{ENV['FLUENT_ELASTICSEARCH_HOST'] || 'elasticsearch.kube-system.svc.cluster.local'}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT'] || '9200'}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
# Authentication
user "#{ENV['FLUENT_ELASTICSEARCH_USER'] || 'elastic'}"
password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD'] || 'changeme'}"
# Index naming: logs-kubernetes-YYYY.MM.DD
logstash_format true
logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_INDEX_PREFIX'] || 'logs-kubernetes'}"
logstash_dateformat %Y.%m.%d
include_timestamp true
# Index lifecycle management
type_name _doc
# Performance tuning
flush_interval 10s
flush_thread_count 2
# Retry configuration
request_timeout 30s
reload_connections false
reconnect_on_error true
reload_on_failure true
# Buffer configuration
<buffer>
@type file
path /var/log/fluentd-buffers/kubernetes.system.buffer
flush_mode interval
retry_type exponential_backoff
flush_interval 10s
retry_forever false
retry_max_interval 30s
chunk_limit_size 5M
queue_limit_length 32
overflow_action block
</buffer>
</match>
# Output: Forward systemd logs to Elasticsearch
<match systemd>
@type elasticsearch
@id out_es_systemd
@log_level info
host "#{ENV['FLUENT_ELASTICSEARCH_HOST'] || 'elasticsearch.kube-system.svc.cluster.local'}"
port "#{ENV['FLUENT_ELASTICSEARCH_PORT'] || '9200'}"
scheme "#{ENV['FLUENT_ELASTICSEARCH_SCHEME'] || 'http'}"
user "#{ENV['FLUENT_ELASTICSEARCH_USER'] || 'elastic'}"
password "#{ENV['FLUENT_ELASTICSEARCH_PASSWORD'] || 'changeme'}"
logstash_format true
logstash_prefix "#{ENV['FLUENT_ELASTICSEARCH_INDEX_PREFIX'] || 'logs-systemd'}"
logstash_dateformat %Y.%m.%d
include_timestamp true
type_name _doc
<buffer>
@type file
path /var/log/fluentd-buffers/systemd.buffer
flush_mode interval
retry_type exponential_backoff
flush_interval 10s
retry_max_interval 30s
chunk_limit_size 5M
queue_limit_length 32
</buffer>
</match>
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd
namespace: kube-system
labels:
app: fluentd
component: log-collector
version: v1
spec:
selector:
matchLabels:
app: fluentd
component: log-collector
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
app: fluentd
component: log-collector
version: v1
annotations:
# Force pod restart on config change
checksum/config: "{{ include (print $.Template.BasePath \"/configmap.yaml\") . | sha256sum }}"
spec:
serviceAccountName: fluentd
priorityClassName: system-node-critical
tolerations:
# Run on all nodes including master
- key: node-role.kubernetes.io/master
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
effect: NoSchedule
# Tolerate all taints to ensure logs from all nodes
- operator: Exists
effect: NoSchedule
- operator: Exists
effect: NoExecute
containers:
- name: fluentd
image: fluent/fluentd-kubernetes-daemonset:v1.16-debian-elasticsearch8-1
imagePullPolicy: IfNotPresent
# Environment variables for configuration
env:
# Elasticsearch connection
- name: FLUENT_ELASTICSEARCH_HOST
value: "elasticsearch.kube-system.svc.cluster.local"
- name: FLUENT_ELASTICSEARCH_PORT
value: "9200"
- name: FLUENT_ELASTICSEARCH_SCHEME
value: "http"
- name: FLUENT_ELASTICSEARCH_USER
value: "elastic"
- name: FLUENT_ELASTICSEARCH_PASSWORD
valueFrom:
secretKeyRef:
name: elasticsearch-credentials
key: password
optional: true
# Index configuration
- name: FLUENT_ELASTICSEARCH_INDEX_PREFIX
value: "logs-kubernetes"
# Cluster identification
- name: CLUSTER_NAME
value: "production-cluster"
- name: ENVIRONMENT
value: "production"
# Kubernetes API configuration
- name: FLUENT_FILTER_KUBERNETES_URL
value: "https://kubernetes.default.svc:443"
- name: KUBERNETES_VERIFY_SSL
value: "true"
- name: KUBERNETES_CA_FILE
value: "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
# Performance tuning
- name: FLUENT_CONTAINER_TAIL_PARSER_TYPE
value: "json"
- name: FLUENT_CONTAINER_TAIL_EXCLUDE_PATH
value: '["/var/log/containers/fluentd-*.log"]'
# Resource limits (adjust based on log volume)
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 256Mi
# Volume mounts
volumeMounts:
# Fluentd configuration
- name: config
mountPath: /fluentd/etc/fluent.conf
subPath: fluent.conf
readOnly: true
# Container logs directory
- name: varlog
mountPath: /var/log
readOnly: true
# Docker container logs
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
# Systemd journal logs
- name: systemd
mountPath: /var/log/journal
readOnly: true
# Buffer storage (persistent)
- name: fluentd-buffer
mountPath: /var/log/fluentd-buffers
# Liveness probe
livenessProbe:
httpGet:
path: /metrics
port: 24231
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
# Readiness probe
readinessProbe:
httpGet:
path: /metrics
port: 24231
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# Volumes
volumes:
# Fluentd configuration from ConfigMap
- name: config
configMap:
name: fluentd-config
defaultMode: 0644
# Host paths for log collection
- name: varlog
hostPath:
path: /var/log
type: Directory
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
type: DirectoryOrCreate
- name: systemd
hostPath:
path: /var/log/journal
type: DirectoryOrCreate
# Persistent buffer storage
- name: fluentd-buffer
hostPath:
path: /var/lib/fluentd-buffer
type: DirectoryOrCreate
---
# Secret for Elasticsearch credentials (create manually)
# kubectl create secret generic elasticsearch-credentials \
# --from-literal=password='SecureElasticPassword123!' \
# -n kube-system
apiVersion: v1
kind: Secret
metadata:
name: elasticsearch-credentials
namespace: kube-system
labels:
app: fluentd
type: Opaque
data:
# Base64 encoded password: SecureElasticPassword123!
# Change this in production: echo -n 'YourPassword' | base64
password: U2VjdXJlRWxhc3RpY1Bhc3N3b3JkMTIzIQ==
---
# Deployment Instructions:
#
# 1. Prerequisites:
# - Kubernetes cluster (1.20+)
# - Elasticsearch deployed (in-cluster or external)
# - kubectl configured with cluster access
#
# 2. Update Elasticsearch credentials:
# kubectl create secret generic elasticsearch-credentials \
# --from-literal=password='YourSecurePassword' \
# -n kube-system --dry-run=client -o yaml | kubectl apply -f -
#
# 3. Customize environment variables:
# Edit the DaemonSet spec to update:
# - FLUENT_ELASTICSEARCH_HOST (if Elasticsearch is external)
# - CLUSTER_NAME (your cluster identifier)
# - ENVIRONMENT (production, staging, development)
#
# 4. Deploy Fluentd:
# kubectl apply -f fluentd-kubernetes-daemonset.yaml
#
# 5. Verify deployment:
# kubectl get daemonset -n kube-system fluentd
# kubectl get pods -n kube-system -l app=fluentd
#
# 6. Check logs:
# kubectl logs -n kube-system -l app=fluentd --tail=100
#
# 7. Verify data in Elasticsearch:
# curl -u elastic:password http://elasticsearch:9200/_cat/indices?v
# # Should see indices like: logs-kubernetes-2024.12.05
#
# Resource Planning:
# - CPU: 100m request, 500m limit per node (adjust based on log volume)
# - Memory: 256Mi request, 512Mi limit per node
# - Buffer Storage: 10-20GB per node for high-volume clusters
# - Network: Plan for 1-10 MB/s outbound traffic to Elasticsearch per node
#
# Security Considerations:
# - Use TLS for Elasticsearch connection in production (update scheme to https)
# - Store credentials in Kubernetes Secrets (not in ConfigMap)
# - Enable RBAC (ClusterRole provides minimal required permissions)
# - Consider network policies to restrict Fluentd to Elasticsearch communication
# - Regularly rotate Elasticsearch credentials
# - Use pod security policies/admission controllers to enforce security context
#
# Performance Tuning:
# - Adjust flush_interval based on log volume and latency requirements
# - Increase buffer sizes for high-volume clusters
# - Use SSD for buffer storage on nodes
# - Monitor Fluentd memory usage and adjust limits if needed
# - Consider using multiple Fluentd deployments for different log types
#
# Monitoring:
# - Fluentd exposes metrics on port 24231 (/metrics endpoint)
# - Monitor buffer queue length, retry counts, and flush times
# - Set up alerts for Fluentd pod crashes or high memory usage
# - Track log ingestion rate and compare with expected volume
#
# Troubleshooting:
# - Check pod logs: kubectl logs -n kube-system <fluentd-pod> --tail=100
# - Verify RBAC permissions: kubectl auth can-i get pods --as=system:serviceaccount:kube-system:fluentd
# - Test Elasticsearch connectivity from pod:
# kubectl exec -n kube-system <fluentd-pod> -- curl -u elastic:password http://elasticsearch:9200/_cluster/health
# - Check buffer directory: kubectl exec -n kube-system <fluentd-pod> -- ls -lh /var/log/fluentd-buffers
version: '3.8'
# Wazuh Single-Node Docker Compose Deployment
# Production-ready SIEM with manager, indexer, and dashboard
services:
wazuh-manager:
image: wazuh/wazuh-manager:4.7.0
hostname: wazuh-manager
restart: always
ports:
- "1514:1514" # Agent connection (TCP)
- "1515:1515" # Agent registration
- "514:514/udp" # Syslog
- "55000:55000" # API
environment:
- INDEXER_URL=https://wazuh-indexer:9200
- INDEXER_USERNAME=admin
- INDEXER_PASSWORD=SecurePassword123!
- FILEBEAT_SSL_VERIFICATION_MODE=none
- API_USERNAME=wazuh-wui
- API_PASSWORD=SecurePassword123!
volumes:
- wazuh_api_configuration:/var/ossec/api/configuration
- wazuh_etc:/var/ossec/etc
- wazuh_logs:/var/ossec/logs
- wazuh_queue:/var/ossec/queue
- wazuh_ruleset:/var/ossec/ruleset
- wazuh_var_multigroups:/var/ossec/var/multigroups
- wazuh_integrations:/var/ossec/integrations
- wazuh_active_response:/var/ossec/active-response/bin
- wazuh_agentless:/var/ossec/agentless
- wazuh_wodles:/var/ossec/wodles
- filebeat_etc:/etc/filebeat
- filebeat_var:/var/lib/filebeat
wazuh-indexer:
image: wazuh/wazuh-indexer:4.7.0
hostname: wazuh-indexer
restart: always
ports:
- "9200:9200"
environment:
- "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
- "bootstrap.memory_lock=true"
- "discovery.type=single-node"
- "DISABLE_SECURITY_PLUGIN=false"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- wazuh-indexer-data:/var/lib/wazuh-indexer
wazuh-dashboard:
image: wazuh/wazuh-dashboard:4.7.0
hostname: wazuh-dashboard
restart: always
ports:
- "443:5601"
environment:
- INDEXER_USERNAME=admin
- INDEXER_PASSWORD=SecurePassword123!
- WAZUH_API_URL=https://wazuh-manager
- API_USERNAME=wazuh-wui
- API_PASSWORD=SecurePassword123!
- DASHBOARD_USERNAME=kibanaserver
- DASHBOARD_PASSWORD=SecurePassword123!
depends_on:
- wazuh-indexer
- wazuh-manager
links:
- wazuh-indexer:wazuh-indexer
- wazuh-manager:wazuh-manager
volumes:
wazuh_api_configuration:
wazuh_etc:
wazuh_logs:
wazuh_queue:
wazuh_ruleset:
wazuh_var_multigroups:
wazuh_integrations:
wazuh_active_response:
wazuh_agentless:
wazuh_wodles:
filebeat_etc:
filebeat_var:
wazuh-indexer-data:
networks:
default:
driver: bridge
---
# Brute Force Attack Detection Rule (Splunk SPL Format)
# Platform: Splunk Enterprise Security
# Category: Credential Access
# MITRE ATT&CK: T1110 - Brute Force
name: Multiple Failed Authentication Attempts - Brute Force Detection
severity: high
description: |
Detects potential brute force authentication attacks by identifying multiple failed
login attempts from a single source IP address within a short time window. This rule
correlates failed authentication events to identify credential stuffing, password
spraying, and traditional brute force attacks.
mitre_attack:
tactic: Credential Access
technique: T1110 - Brute Force
sub_technique:
- T1110.001 - Password Guessing
- T1110.003 - Password Spraying
- T1110.004 - Credential Stuffing
splunk_search: |
index=auth sourcetype=linux_secure OR sourcetype=windows:security OR sourcetype=aws:cloudtrail
(action=failure OR EventCode=4625 OR errorCode=Failed)
| bucket _time span=10m
| stats count as failed_attempts
dc(user) as unique_users
values(user) as targeted_users
earliest(_time) as first_attempt
latest(_time) as last_attempt
by src_ip, _time
| where failed_attempts >= 10
| eval duration=last_attempt-first_attempt
| eval attack_pattern=case(
unique_users > 10, "Password Spraying",
unique_users <= 3, "Brute Force",
1=1, "Credential Stuffing"
)
| table _time, src_ip, failed_attempts, unique_users, targeted_users, duration, attack_pattern
| sort -failed_attempts
# Alternative search for cloud environments (Azure AD, AWS Cognito, Okta)
cloud_search: |
index=azure_ad OR index=okta OR index=aws_cognito
eventName=signin OR eventType=user.session.start
result=failure OR outcome=FAILURE
| bucket _time span=10m
| stats count as failed_attempts
dc(userPrincipalName) as unique_users
values(userPrincipalName) as targeted_users
values(appDisplayName) as targeted_apps
by src_ip, _time
| where failed_attempts >= 10
detection_logic:
threshold: 10 failed authentication attempts
time_window: 10 minutes
aggregation_field: source IP address
correlation:
- Multiple failed attempts from same source
- Optionally followed by successful authentication (indicates breach)
- Pattern analysis (password spraying vs. targeted brute force)
tuning_recommendations:
reduce_false_positives:
- Whitelist legitimate security scanners (Nessus, Qualys, vulnerability scanners)
- Exclude monitoring system IPs (Nagios, Zabbix, health checks)
- Increase threshold to 15-20 for high-traffic environments
- Add user whitelist for service accounts with expected failures
- Filter out internal IP ranges for password self-service portals
whitelist_example: |
| search NOT src_ip IN ("10.0.0.100", "192.168.1.50")
| search NOT user IN ("monitoring-svc", "health-check")
| search NOT src_ip="scanner.company.com"
threshold_tuning:
conservative: 5 attempts in 5 minutes (high alert volume)
balanced: 10 attempts in 10 minutes (recommended default)
relaxed: 20 attempts in 15 minutes (low alert volume, may miss attacks)
alert_configuration:
trigger_condition: Search completes with results
throttling: Suppress duplicate alerts from same src_ip for 1 hour
priority: high
notable_event_fields:
- src_ip (source IP address)
- failed_attempts (total count)
- unique_users (number of distinct users targeted)
- targeted_users (list of usernames)
- attack_pattern (classification)
- duration (time span of attack)
response_actions:
automated:
- Block source IP at firewall (if exceeds 50 attempts)
- Add IP to threat intelligence watchlist
- Trigger MFA step-up for targeted user accounts
manual_investigation:
- Verify if successful authentication followed failed attempts
- Check geolocation of source IP (unexpected country = higher risk)
- Review targeted user accounts for privilege level
- Correlate with other security events (data exfiltration, lateral movement)
- Check if source IP has history of malicious activity
false_positives:
common_causes:
- Misconfigured monitoring systems repeatedly checking authentication
- Legitimate users forgetting passwords (especially after password policy changes)
- Automated deployment scripts with incorrect credentials
- Load balancers or health checks with authentication probes
- Security vulnerability scanners (Nessus, Qualys, OpenVAS)
- Password manager browser extensions with incorrect saved credentials
mitigation:
- Build whitelist of known-safe IPs over 2-week baseline period
- Exclude service accounts from detection (create separate rule)
- Increase threshold during password policy changes
- Document and whitelist security scanner IP ranges
baseline_period: |
Run rule in "log-only" mode for 2-4 weeks before enabling alerting.
Analyze false positive patterns and build whitelist:
index=auth sourcetype=linux_secure action=failure
| stats count by src_ip, user
| where count > 100
| table src_ip, user, count
| outputlookup baseline_auth_failures.csv
metrics:
target_alert_volume: 5-15 alerts per day
target_true_positive_rate: 30-50%
mean_time_to_investigate: 10-15 minutes
recommended_review_frequency: Weekly tuning for first month, then monthly
references:
- https://attack.mitre.org/techniques/T1110/
- https://docs.splunk.com/Documentation/ES/latest/Admin/Correlationsearches
- https://www.owasp.org/index.php/Blocking_Brute_Force_Attacks
- https://www.cisecurity.org/controls/account-monitoring-and-control
author: Security Team
date_created: 2025-12-05
last_modified: 2025-12-05
version: 1.0
---
# Privilege Escalation Detection Rule (Splunk SPL Format)
# Platform: Splunk Enterprise Security
# Category: Privilege Escalation
# MITRE ATT&CK: T1548 - Abuse Elevation Control Mechanism
name: Suspicious Privilege Escalation - Sudo and UAC Bypass Detection
severity: critical
description: |
Detects unauthorized privilege escalation attempts through sudo abuse, UAC bypass,
service account elevation, and exploitation of SUID/SGID binaries. This rule identifies
privilege escalation by monitoring sudo execution from non-privileged users, unusual
process spawning patterns, and suspicious parent-child process relationships.
mitre_attack:
tactic: Privilege Escalation
technique: T1548 - Abuse Elevation Control Mechanism
sub_technique:
- T1548.003 - Sudo and Sudo Caching
- T1548.002 - Bypass User Account Control (Windows)
- T1548.001 - Setuid and Setgid
splunk_search: |
index=linux OR index=windows sourcetype=linux_secure OR sourcetype=WinEventLog:Security
(process_name=sudo OR process_name=su OR EventCode=4672 OR EventCode=4673)
| eval suspicious_user=case(
user IN ("www-data", "nobody", "nginx", "apache", "postgres", "mysql", "tomcat", "jenkins"), "Service Account",
user IN ("guest", "anonymous"), "Low Privilege User",
1=1, "Standard User"
)
| where suspicious_user IN ("Service Account", "Low Privilege User")
| stats count as escalation_attempts
values(process_name) as processes
values(parent_process) as parent_processes
values(command_line) as commands
earliest(_time) as first_seen
latest(_time) as last_seen
by user, src_host, suspicious_user
| eval duration=last_seen-first_seen
| where escalation_attempts >= 1
| eval risk_score=case(
suspicious_user="Service Account" AND match(commands, "(?i)(bash|sh|python|perl|nc)"), 90,
suspicious_user="Service Account", 70,
suspicious_user="Low Privilege User", 60,
1=1, 40
)
| table _time, user, src_host, suspicious_user, escalation_attempts, processes, commands, risk_score
| sort -risk_score
# Linux-specific SUID/SGID abuse detection
linux_suid_search: |
index=linux sourcetype=linux_audit OR sourcetype=syslog
(syscall=execve OR type=EXECVE)
| rex field=exe "(?<binary_name>[^/]+)$"
| lookup suid_binaries.csv binary_name OUTPUT is_suid, is_legitimate
| where is_suid=1 AND is_legitimate=0
| stats count by user, binary_name, command_line, parent_process
| where user NOT IN ("root", "admin")
# Windows UAC bypass detection
windows_uac_search: |
index=windows sourcetype=WinEventLog:Security
(EventCode=4688 OR EventCode=4673)
| eval uac_bypass_indicator=case(
match(process_path, "(?i)(eventvwr.exe|fodhelper.exe|computerdefaults.exe)") AND
match(parent_process, "(?i)cmd.exe|powershell.exe"), "UAC Bypass via Trusted Binary",
match(command_line, "(?i)(bypassuac|uacme|eventvwr)"), "UAC Bypass Tool Detected",
EventCode=4673 AND privilege_list="SeDebugPrivilege", "Debug Privilege Enabled",
1=1, "None"
)
| where uac_bypass_indicator != "None"
| table _time, user, computer_name, process_name, command_line, uac_bypass_indicator
# Sudo caching exploitation detection
sudo_caching_search: |
index=linux sourcetype=linux_secure
process_name=sudo
| transaction user maxspan=15m
| where eventcount > 5
| stats count as rapid_sudo_uses
values(command) as commands
dc(command) as unique_commands
by user, src_host
| where rapid_sudo_uses > 5 AND unique_commands < 3
detection_logic:
indicators:
- Sudo execution by service accounts (www-data, nobody, nginx, apache)
- Non-root users executing SUID/SGID binaries
- UAC bypass patterns (eventvwr.exe, fodhelper.exe abuse)
- Rapid sudo caching exploitation (multiple sudo calls within 15 minutes)
- Shell spawning from web servers or database processes
- Privilege token manipulation (SeDebugPrivilege, SeImpersonatePrivilege)
correlation_rules:
- Privilege escalation followed by lateral movement (within 30 minutes)
- Service account privilege escalation + outbound network connection
- Escalation + file access to sensitive directories (/etc/shadow, C:\Windows\System32\config)
tuning_recommendations:
reduce_false_positives:
- Whitelist legitimate administrative automation (Ansible, Puppet, Chef)
- Exclude configuration management service accounts with documented sudo access
- Filter deployment scripts running during maintenance windows
- Create baseline of expected sudo usage per service account
- Whitelist containerized environments where service account elevation is normal
whitelist_example: |
# Exclude legitimate automation tools
| search NOT (user="ansible-svc" AND parent_process="ansible-playbook")
| search NOT (user="puppet" AND command_line="*puppet agent*")
| search NOT src_host IN ("jenkins-master", "gitlab-runner-*")
# Exclude scheduled maintenance windows
| search NOT (date_hour>=2 AND date_hour<=4 AND user="deploy-svc")
baseline_configuration: |
# Build 30-day baseline of normal sudo usage
index=linux sourcetype=linux_secure process_name=sudo
earliest=-30d latest=now
| stats count avg(count) as avg_usage stdev(count) as std_dev
by user, src_host
| eval threshold=avg_usage + (2 * std_dev)
| outputlookup sudo_baseline.csv
threshold_tuning:
strict: Any sudo by service account = alert (high volume, zero tolerance)
balanced: Sudo by service account with shell spawning = alert (recommended)
relaxed: Multiple escalations within 1 hour = alert (may miss single-event attacks)
alert_configuration:
trigger_condition: Search returns results with risk_score >= 60
throttling: Suppress duplicate alerts per user/host pair for 4 hours
priority: critical (risk_score >= 80), high (risk_score 60-79)
notable_event_fields:
- user (account performing escalation)
- src_host (source system)
- suspicious_user (classification: Service Account, Low Privilege User)
- escalation_attempts (count)
- processes (sudo, su, UAC bypass tools)
- commands (full command line)
- risk_score (calculated severity 0-100)
response_actions:
automated:
- Disable compromised service account (if risk_score >= 90)
- Kill suspicious processes (sudo shells from www-data, nobody)
- Isolate affected host from network (if lateral movement detected)
- Create forensic snapshot of affected system
- Trigger incident response workflow
manual_investigation:
- Verify if escalation resulted from legitimate change management
- Check user's recent activity (commands, file access, network connections)
- Review sudo logs for full command history
- Identify initial access vector (web exploit, SSH compromise, insider threat)
- Assess blast radius (what did escalated user access?)
- Check for persistence mechanisms (cron jobs, SSH keys, backdoor accounts)
forensic_queries:
post_escalation_activity: |
# What did the user do after privilege escalation?
index=linux sourcetype=linux_audit user="$compromised_user$"
earliest=$escalation_time$ latest=$escalation_time$+1h
| stats count by type, exe, command_line
| sort -count
lateral_movement_check: |
# Did attacker move laterally after escalation?
index=network OR index=linux
(src_ip=$affected_host_ip$ OR src_host=$affected_host$)
earliest=$escalation_time$ latest=$escalation_time$+4h
| search (dest_port=22 OR dest_port=3389 OR dest_port=5985)
| stats count by dest_ip, dest_port, user
false_positives:
common_causes:
- Legitimate configuration management tools (Ansible, Puppet, Chef, SaltStack)
- Automated deployment pipelines (Jenkins, GitLab CI, GitHub Actions)
- Containerized applications running as service accounts with sudo access
- Scheduled maintenance scripts requiring elevated privileges
- Database backup scripts running as postgres/mysql users
- Web application frameworks with legitimate sudo for system tasks
mitigation:
- Document and whitelist approved automation service accounts
- Schedule expected maintenance windows and suppress alerts during those times
- Create separate detection rule for containerized environments
- Implement "expected sudo" lookup table based on 30-day baseline
- Require change management tickets for new sudo permissions
baseline_approach: |
Step 1: Run detection in "log-only" mode for 30 days
Step 2: Identify recurring patterns (daily at 2 AM, user "backup-svc", command "/usr/bin/mysqldump")
Step 3: Classify as legitimate or suspicious based on business context
Step 4: Build whitelist for legitimate patterns
Step 5: Enable alerting with whitelist applied
metrics:
target_alert_volume: 2-8 alerts per day
target_true_positive_rate: 50-70% (privilege escalation is rare but critical)
mean_time_to_investigate: 20-30 minutes (requires context gathering)
mean_time_to_contain: 45 minutes (for confirmed compromise)
recommended_review_frequency: Weekly review of all escalation events
escalation_severity_matrix:
critical_indicators:
- Service account (www-data, nobody) spawning interactive shell
- Sudo from guest or anonymous accounts
- UAC bypass tool execution detected
- Escalation followed by /etc/shadow or SAM database access
- Escalation + lateral movement within 30 minutes
high_indicators:
- Service account using sudo for non-standard commands
- Multiple sudo attempts from same user within 15 minutes (caching exploitation)
- Non-root user executing unknown SUID binaries
medium_indicators:
- Standard user using sudo (may be legitimate admin activity)
- Sudo during non-business hours (requires context)
monitoring_recommendations:
create_suid_baseline: |
# Generate list of legitimate SUID binaries (run on hardened baseline system)
find / -type f \( -perm -4000 -o -perm -2000 \) -ls 2>/dev/null \
| awk '{print $11}' \
| sed 's/.*\///' \
| sort -u > legitimate_suid_binaries.txt
# Upload to Splunk lookup table
| inputlookup suid_binaries.csv
| append [| makeresults | eval binary_name="legitimate_binary", is_suid=1, is_legitimate=1]
integration_with_edr:
- Correlate with EDR process telemetry (CrowdStrike, SentinelOne, Carbon Black)
- Enrich with parent process chain (full process tree)
- Add hash-based reputation checks (VirusTotal, threat intelligence)
references:
- https://attack.mitre.org/techniques/T1548/
- https://attack.mitre.org/techniques/T1548/003/ (Sudo and Sudo Caching)
- https://gtfobins.github.io/ (SUID binary exploitation techniques)
- https://book.hacktricks.xyz/linux-hardening/privilege-escalation (Linux privilege escalation)
- https://www.fuzzysecurity.com/tutorials/16.html (Windows privilege escalation)
- https://docs.splunk.com/Documentation/ES/latest/Admin/Privilegeescalation
compliance_mappings:
pci_dss:
- Requirement 10.2.2 (All actions taken by root or administrative privileges)
- Requirement 10.2.5 (Use of identification and authentication mechanisms)
nist_csf:
- DE.AE-2 (Detected events are analyzed)
- DE.CM-1 (Network monitored to detect anomalous activity)
cis_controls:
- Control 4.3 (Configure privilege access management)
- Control 6.8 (Define and maintain role-based access control)
author: Security Team
date_created: 2025-12-05
last_modified: 2025-12-05
version: 1.0
title: Multiple Failed Login Attempts from Single Source
id: 8a9e3c7f-4b2d-4e8a-9f1c-2d5e6f7a8b9c
status: stable
description: Detects potential brute force attacks (10+ failed logins in 10 minutes from same source IP)
author: Security Team
date: 2025/12/03
modified: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1110/
tags:
- attack.credential_access
- attack.t1110
- attack.t1110.001
logsource:
category: authentication
product: linux
detection:
selection:
event.type: authentication
event.outcome: failure
timeframe: 10m
condition: selection | count() by source.ip > 10
falsepositives:
- Misconfigured monitoring systems
- Legitimate password reset attempts
- Security scanners (Nessus, Qualys)
level: high
title: Large Data Upload to External Domain
id: 9c0d1e2f-3a4b-5c6d-7e8f-9a0b1c2d3e4f
status: stable
description: Detects uploads >100MB to non-whitelisted external domains, indicating potential data exfiltration
author: Security Team
date: 2025/12/03
modified: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1041/
tags:
- attack.exfiltration
- attack.t1041
logsource:
category: network_traffic
product: proxy
detection:
selection:
http.request.method: POST
http.request.bytes: '>100000000'
filter_whitelist:
destination.domain:
- 'backup.company.com'
- 's3.amazonaws.com'
- 'blob.core.windows.net'
- 'storage.googleapis.com'
condition: selection and not filter_whitelist
falsepositives:
- Large file uploads to legitimate cloud storage
- Backup operations
- Software deployment to cloud platforms
level: critical
title: RDP Connection from Non-Admin Workstation
id: 1e2f3a4b-5c6d-7e8f-9a0b-1c2d3e4f5a6b
status: stable
description: Detects RDP connections from workstations (not jump servers), indicating potential lateral movement
author: Security Team
date: 2025/12/03
modified: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1021/001/
tags:
- attack.lateral_movement
- attack.t1021.001
logsource:
category: network_traffic
product: firewall
detection:
selection:
destination.port: 3389
network.protocol: tcp
filter_jumpservers:
source.ip:
- '10.0.1.100' # Jump Server 1
- '10.0.1.101' # Jump Server 2
condition: selection and not filter_jumpservers
falsepositives:
- Administrators connecting from workstations (should use jump servers)
- Remote support sessions (TeamViewer, LogMeIn)
level: medium
title: Suspicious Sudo Execution by Service Account
id: 7b8c9d0e-1f2a-3b4c-5d6e-7f8a9b0c1d2e
status: stable
description: Detects sudo execution by non-privileged users (web servers, service accounts) indicating potential privilege escalation
author: Security Team
date: 2025/12/03
modified: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1548/003/
tags:
- attack.privilege_escalation
- attack.t1548.003
logsource:
category: process_creation
product: linux
detection:
selection:
process.name: sudo
user.name:
- 'www-data'
- 'nobody'
- 'nginx'
- 'apache'
- 'postgres'
- 'mysql'
condition: selection
falsepositives:
- Legitimate administrative scripts running as service accounts
- Configuration management tools (Ansible, Puppet)
level: high
skill: "siem-logging"
version: "1.0"
domain: "security"
base_outputs:
# Core SIEM configuration files
- path: "siem/config/log-sources.yaml"
must_contain: ["log sources", "data ingestion", "parsing rules"]
- path: "siem/config/retention-policy.yaml"
must_contain: ["retention periods", "storage tiers", "compliance requirements"]
# Detection rules (SIGMA universal format)
- path: "siem/detection-rules/sigma/authentication-failures.yml"
must_contain: ["logsource", "detection", "level", "MITRE ATT&CK"]
- path: "siem/detection-rules/sigma/privilege-escalation.yml"
must_contain: ["logsource", "detection", "level", "MITRE ATT&CK"]
# Alert configuration
- path: "siem/alerts/alert-rules.yaml"
must_contain: ["severity levels", "notification channels", "thresholds"]
- path: "siem/alerts/whitelist.yaml"
must_contain: ["known-safe patterns", "scanner IPs", "monitoring systems"]
conditional_outputs:
maturity:
starter:
# Simple setup for small teams
- path: "siem/config/wazuh-config.yml"
must_contain: ["ossec.conf", "agent configuration", "log collection"]
- path: "siem/detection-rules/sigma/basic-threats.yml"
must_contain: ["failed authentication", "brute force", "unauthorized access"]
- path: "siem/docker-compose.yml"
must_contain: ["wazuh", "elasticsearch", "kibana"]
intermediate:
# Multi-cloud log aggregation
- path: "siem/log-aggregation/fluentd-config.yaml"
must_contain: ["source", "filter", "match", "elasticsearch"]
- path: "siem/detection-rules/platform-specific/elastic-eql/"
must_contain: ["sequence queries", "correlation rules"]
- path: "siem/config/storage-tiering.yaml"
must_contain: ["hot tier", "warm tier", "cold tier", "ILM policy"]
- path: "siem/cost-analysis/volume-estimate.yaml"
must_contain: ["daily volume", "retention", "monthly cost"]
advanced:
# Enterprise SIEM with distributed architecture
- path: "siem/architecture/multi-region-setup.yaml"
must_contain: ["regional instances", "global correlation", "data residency"]
- path: "siem/detection-rules/advanced/correlation-rules.yml"
must_contain: ["multi-event correlation", "threat intelligence", "anomaly detection"]
- path: "siem/automation/soar-playbooks/"
must_contain: ["automated response", "incident enrichment", "threat hunting"]
- path: "siem/compliance/audit-reports.yaml"
must_contain: ["compliance frameworks", "audit trails", "evidence collection"]
- path: "siem/optimization/alert-tuning-metrics.yaml"
must_contain: ["true positive rate", "false positive rate", "MTTI", "performance KPIs"]
cloud_provider:
aws:
- path: "siem/cloud-integration/aws-security-lake.tf"
must_contain: ["CloudTrail", "VPC Flow Logs", "GuardDuty", "S3 data lake"]
- path: "siem/log-sources/aws-config.yaml"
must_contain: ["CloudWatch Logs", "EventBridge", "Security Hub"]
- path: "siem/detection-rules/aws-specific/iam-abuse.yml"
must_contain: ["IAM policy changes", "root account usage", "privilege escalation"]
gcp:
- path: "siem/cloud-integration/gcp-chronicle.yaml"
must_contain: ["Cloud Logging", "Security Command Center", "Chronicle SIEM"]
- path: "siem/log-sources/gcp-config.yaml"
must_contain: ["Cloud Audit Logs", "VPC Flow Logs", "GKE logs"]
- path: "siem/detection-rules/gcp-specific/gce-security.yml"
must_contain: ["GCE metadata", "service account abuse", "firewall changes"]
azure:
- path: "siem/cloud-integration/microsoft-sentinel.tf"
must_contain: ["Log Analytics workspace", "Microsoft Sentinel", "data connectors"]
- path: "siem/log-sources/azure-config.yaml"
must_contain: ["Azure Activity Logs", "NSG Flow Logs", "Entra ID logs"]
- path: "siem/detection-rules/azure-specific/entra-id-threats.kql"
must_contain: ["SigninLogs", "AuditLogs", "risky sign-ins", "conditional access"]
scaffolding:
# Directory structure for SIEM implementation
- "siem/"
- "siem/config/"
- "siem/detection-rules/"
- "siem/detection-rules/sigma/"
- "siem/detection-rules/platform-specific/"
- "siem/alerts/"
- "siem/log-aggregation/"
- "siem/log-sources/"
- "siem/compliance/"
- "siem/automation/"
- "siem/dashboards/"
- "siem/reports/"
- "logging/"
- "logging/shippers/"
- "logging/parsers/"
- "detection/"
- "detection/rules/"
- "detection/correlation/"
metadata:
primary_blueprints: ["security", "observability"]
contributes_to:
- "SIEM integration"
- "Security logging infrastructure"
- "Threat detection and response"
- "Compliance audit trails"
- "Log aggregation pipelines"
- "Security monitoring dashboards"
Alert Tuning Strategies
Alert Lifecycle
1. Detection Rule Created - Conservative thresholds, deploy to production 2. Baseline Period (2-4 weeks) - Collect alert data, tag true/false positives 3. Tuning Phase - Add whitelisting, adjust thresholds, refine correlation 4. Continuous Improvement - Weekly metrics review, monthly effectiveness review
Noise Reduction Techniques
Whitelisting
# Example: Allow scanner IPs
- rule_id: brute_force_detection
whitelist:
- source_ip: "10.0.0.100" # Security scanner
- user_agent: "Nagios" # Monitoring systemThreshold Tuning
# Before: Too sensitive (500 alerts/day, 5% true positive rate)
- rule: failed_login_attempts
threshold: 3 attempts in 5 minutes
# After: Tuned (50 alerts/day, 40% true positive rate)
- rule: failed_login_attempts
threshold: 10 attempts in 10 minutesMulti-Event Correlation
# Instead of: Single event alert
- alert_on: "Failed authentication"
# Use: Correlated pattern
- alert_on:
- "Failed authentication (5+ times)"
- AND "From new IP address"
- AND "Successful authentication follows"
- WITHIN: 30 minutesTarget Alert Metrics
| Metric | Target | Current | Status |
|---|---|---|---|
| Total Alerts/Day | <100 | 87 | ✅ |
| True Positive Rate | >30% | 42% | ✅ |
| Mean Time to Investigate | <15 min | 12 min | ✅ |
| False Positive Rate | <50% | 58% | ⚠️ Needs tuning |
| Critical Alerts/Day | <10 | 6 | ✅ |
Summary
- Baseline first - Collect 2-4 weeks of data before tuning
- Whitelist known-safe patterns - Reduce noise from legitimate activity
- Adjust thresholds - Balance sensitivity vs. alert fatigue
- Use correlation - Multi-event patterns reduce false positives
- Track metrics - Monitor true positive rate, alert volume, investigation time
Cloud-Native Logging
Table of Contents
AWS Logging
Key Services
- CloudTrail: API activity logs
- VPC Flow Logs: Network traffic logs
- CloudWatch Logs: Application and system logs
- GuardDuty: Threat detection
- Security Hub: Aggregated security findings
AWS Security Lake Setup
# Enable AWS Security Lake
aws securitylake create-data-lake \
--region us-east-1 \
--meta-store-manager-role-arn arn:aws:iam::123456789012:role/SecurityLakeRole
# Add CloudTrail as source
aws securitylake create-aws-log-source \
--sources '[{"sourceName":"CLOUD_TRAIL_MGMT","sourceVersion":"2.0"}]'
# Add VPC Flow Logs
aws securitylake create-aws-log-source \
--sources '[{"sourceName":"VPC_FLOW","sourceVersion":"2.0"}]'Azure Logging
Key Services
- Azure Monitor: Centralized monitoring
- Azure Activity Logs: Subscription-level events
- Azure AD Logs: Authentication and authorization
- Azure Security Center: Security recommendations
- Microsoft Sentinel: Cloud-native SIEM
Sentinel Setup
# Create Log Analytics workspace
az monitor log-analytics workspace create \
--resource-group security-rg \
--workspace-name security-sentinel \
--location eastus
# Enable Microsoft Sentinel
az sentinel workspace create \
--resource-group security-rg \
--workspace-name security-sentinel
# Connect Azure AD
az sentinel data-connector create \
--resource-group security-rg \
--workspace-name security-sentinel \
--name AzureActiveDirectory \
--kind AzureActiveDirectoryKubernetes Logging
Fluentd DaemonSet
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: fluentd
namespace: kube-logging
spec:
selector:
matchLabels:
app: fluentd
template:
metadata:
labels:
app: fluentd
spec:
serviceAccountName: fluentd
containers:
- name: fluentd
image: fluent/fluentd-kubernetes-daemonset:v1-debian-elasticsearch
env:
- name: FLUENT_ELASTICSEARCH_HOST
value: "elasticsearch.kube-logging.svc.cluster.local"
- name: FLUENT_ELASTICSEARCH_PORT
value: "9200"
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containersSummary
- AWS: Use Security Lake for centralized security data lake
- Azure: Use Microsoft Sentinel for cloud-native SIEM
- GCP: Use Cloud Logging and Chronicle for security monitoring
- Kubernetes: Use Fluentd/Fluent Bit for container log aggregation
Cost Optimization
Storage Tiering
Cost Comparison (1 TB storage, 1 year)
| Storage Tier | AWS Cost | Azure Cost | Use Case |
|---|---|---|---|
| Hot (Elasticsearch) | ~$1,200/year | ~$1,100/year | Real-time analysis |
| Warm (S3 Standard) | ~$276/year | ~$230/year | Occasional queries |
| Cold (S3 Glacier) | ~$48/year | ~$24/year | Compliance archive |
Tiering Strategy
Real-time logs (7 days) → Hot (Elasticsearch)
Recent logs (30 days) → Warm (S3 Standard)
Archive (1 year) → Cold (Glacier Deep Archive)
Delete after 1 year (GDPR compliance)Data Volume Reduction
Log Sampling
# Fluentd: Keep 10% of info-level logs
<filter application.info>
@type sampling
interval 10
sample_rate 1
</filter>Selective Field Retention
# Logstash: Remove sensitive/unnecessary fields
filter {
mutate {
remove_field => ["headers", "cookies", "query_params"]
}
}Example Cost Optimization
500 GB/day log volume, 1-year retention
Hot (30 days): 15 TB @ $0.10/GB = $1,500/month
Warm (60 days): 30 TB @ $0.05/GB = $1,500/month
Cold (275 days): 137.5 TB @ $0.01/GB = $1,375/month
Total: $4,375/month = $52,500/year
vs. Hot-only: $18,250/month = $219,000/year
Savings: 76% ($166,500/year)Summary
- Use tiering - Save 70-80% with hot/warm/cold strategy
- Sample non-critical logs - Reduce volume 50-90%
- Strip unnecessary fields - Reduce storage 20-30%
- Implement ILM - Automate lifecycle management
Detection Rules Guide
Table of Contents
- SIGMA Rule Format
- Elastic EQL
- Microsoft Sentinel KQL
- Splunk SPL
- Rule Development Workflow
- MITRE ATT&CK Mapping
- Testing and Validation
SIGMA Rule Format
What is SIGMA?
SIGMA is an open-source, generic signature format for SIEM systems. Write detection logic once, compile to any SIEM query language (Elastic EQL, Splunk SPL, Microsoft KQL, etc.).
SIGMA Rule Structure
title: [Detection Rule Title]
id: [UUID]
status: [stable|test|experimental]
description: [Detailed description of what this detects]
author: [Author name or team]
date: [YYYY/MM/DD]
modified: [YYYY/MM/DD]
references:
- [URL to documentation, threat research]
tags:
- attack.[tactic]
- attack.[technique_id]
logsource:
category: [process_creation|network_traffic|authentication]
product: [windows|linux|aws|azure]
detection:
selection:
[field_name]: [value or pattern]
condition: [selection logic]
falsepositives:
- [Known false positive scenario]
level: [low|medium|high|critical]SIGMA Rule Examples
Brute Force Detection
title: Multiple Failed Login Attempts from Single Source
id: 8a9e3c7f-4b2d-4e8a-9f1c-2d5e6f7a8b9c
status: stable
description: Detects potential brute force attacks (10+ failed logins in 10 minutes)
author: Security Team
date: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1110/
tags:
- attack.credential_access
- attack.t1110
logsource:
category: authentication
product: linux
detection:
selection:
event.type: authentication
event.outcome: failure
timeframe: 10m
condition: selection | count() by source.ip > 10
falsepositives:
- Misconfigured monitoring systems
- Password reset attempts
level: highPrivilege Escalation (sudo)
title: Suspicious Sudo Execution
id: 7b8c9d0e-1f2a-3b4c-5d6e-7f8a9b0c1d2e
status: stable
description: Detects sudo execution by non-privileged users (web servers, service accounts)
author: Security Team
date: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1548/003/
tags:
- attack.privilege_escalation
- attack.t1548.003
logsource:
category: process_creation
product: linux
detection:
selection:
process.name: sudo
user.name:
- 'www-data'
- 'nobody'
- 'nginx'
- 'apache'
condition: selection
falsepositives:
- Legitimate administrative scripts running as service accounts
level: highData Exfiltration
title: Large Data Upload to External Domain
id: 9c0d1e2f-3a4b-5c6d-7e8f-9a0b1c2d3e4f
status: stable
description: Detects uploads >100MB to non-whitelisted external domains
author: Security Team
date: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1041/
tags:
- attack.exfiltration
- attack.t1041
logsource:
category: network_traffic
product: proxy
detection:
selection:
http.request.method: POST
http.request.bytes: '>100000000'
filter_whitelist:
destination.domain:
- 'backup.company.com'
- 's3.amazonaws.com'
- 'blob.core.windows.net'
condition: selection and not filter_whitelist
falsepositives:
- Large file uploads to legitimate cloud storage
- Backup operations
level: criticalMimikatz Detection
title: Mimikatz Execution Detected
id: 0d1e2f3a-4b5c-6d7e-8f9a-0b1c2d3e4f5a
status: stable
description: Detects Mimikatz credential dumping tool execution
author: Security Team
date: 2025/12/03
references:
- https://attack.mitre.org/techniques/T1003/001/
tags:
- attack.credential_access
- attack.t1003.001
logsource:
category: process_creation
product: windows
detection:
selection:
CommandLine|contains:
- 'sekurlsa::logonpasswords'
- 'sekurlsa::tickets'
- 'lsadump::sam'
- 'kerberos::golden'
condition: selection
falsepositives:
- Security testing (authorized penetration tests)
level: criticalCompiling SIGMA Rules
# Install SIGMA compiler
pip install sigma-cli
# Compile to Elastic EQL
sigmac -t es-eql brute-force-detection.yml
# Compile to Splunk SPL
sigmac -t splunk brute-force-detection.yml
# Compile to Microsoft KQL
sigmac -t kusto brute-force-detection.yml
# Compile to all formats
sigmac -t es-eql,splunk,kusto brute-force-detection.yml---
Elastic EQL
Event Query Language Overview
EQL (Event Query Language) is Elastic's native query language for sequence-based detection and complex event correlation.
Advantages
- Native to Elastic SIEM
- Sequence-based detection (multi-event correlation)
- Time-based analysis with
maxspan - MITRE ATT&CK mapping built-in
- Efficient query execution
Basic EQL Query
process where event.type == "start" and
process.name == "powershell.exe" and
process.args : ("Invoke-WebRequest", "iwr", "wget", "curl")Sequence Detection
sequence by user.name with maxspan=5m
[process where event.type == "start" and process.name == "powershell.exe"]
[library where dll.name == "kernel32.dll" and
dll.code_signature.subject_name != "Microsoft Corporation"]
[network where process.name == "powershell.exe"]Explanation: 1. PowerShell starts 2. Loads non-Microsoft signed DLL (suspicious) 3. Makes network connection (potential C2) 4. All within 5 minutes, by same user
Advanced Examples
Credential Dumping (Registry Access):
registry where registry.path : (
"HKLM\\SAM\\SAM\\Domains\\Account\\Users\\*",
"HKLM\\SECURITY\\Policy\\Secrets\\*"
) and not process.executable : (
"C:\\Windows\\System32\\lsass.exe",
"C:\\Windows\\System32\\svchost.exe"
)Ransomware Behavior:
sequence by host.id with maxspan=1h
[file where event.action == "modification" and file.extension : ("docx", "xlsx", "pdf")]
[file where event.action == "modification" and file.extension : ("encrypted", "locked", "crypted")]
[file where event.action == "creation" and file.name : ("*README*", "*DECRYPT*", "*RANSOM*")]Lateral Movement (RDP Chain):
sequence by user.name with maxspan=10m
[authentication where event.outcome == "success" and source.ip == "10.0.1.100"]
[process where process.name == "mstsc.exe"]
[authentication where event.outcome == "success" and source.ip != "10.0.1.100"]---
Microsoft Sentinel KQL
Kusto Query Language Overview
KQL (Kusto Query Language) is Microsoft's query language for Azure Monitor, Log Analytics, and Sentinel.
Advantages
- Native to Azure ecosystem
- Rich aggregation and summarization functions
- Powerful join and union operators
- Time-series analysis
- Integration with Azure Logic Apps for automation
Basic KQL Query
SigninLogs
| where TimeGenerated > ago(1h)
| where ResultType != 0 // Failed login
| where UserPrincipalName == "admin@company.com"
| project TimeGenerated, UserPrincipalName, IPAddress, LocationAdvanced Examples
Failed Logins from Multiple Countries:
let threshold = 3;
SigninLogs
| where TimeGenerated > ago(1h)
| where ResultType != 0 // Failed login
| summarize FailedCountries=dcount(LocationDetails.countryOrRegion),
FailedAttempts=count() by UserPrincipalName
| where FailedCountries >= threshold
| project UserPrincipalName, FailedCountries, FailedAttemptsSuspicious Azure Resource Deletion:
AzureActivity
| where TimeGenerated > ago(24h)
| where OperationNameValue endswith "DELETE"
| where ActivityStatusValue == "Success"
| where ResourceProvider in ("Microsoft.Compute", "Microsoft.Storage", "Microsoft.Sql")
| summarize DeletedResources=count() by Caller, ResourceProvider
| where DeletedResources > 5Office 365 Email Forwarding Rule Created:
OfficeActivity
| where TimeGenerated > ago(7d)
| where Operation == "New-InboxRule"
| where Parameters has "ForwardTo" or Parameters has "RedirectTo"
| extend ForwardingAddress = extract("(ForwardTo|RedirectTo):([^;]+)", 2, Parameters)
| project TimeGenerated, UserId, ClientIP, ForwardingAddressAnomalous Sign-in Pattern:
let historical_logins = SigninLogs
| where TimeGenerated between (ago(30d) .. ago(1d))
| where ResultType == 0
| summarize hist_countries = make_set(LocationDetails.countryOrRegion) by UserPrincipalName;
SigninLogs
| where TimeGenerated > ago(1h)
| where ResultType == 0
| extend current_country = LocationDetails.countryOrRegion
| join kind=inner historical_logins on UserPrincipalName
| where not(current_country in (hist_countries))
| project TimeGenerated, UserPrincipalName, current_country, IPAddress---
Splunk SPL
Search Processing Language Overview
SPL (Search Processing Language) is Splunk's query language for searching, filtering, and transforming security events.
Advantages
- Powerful data transformation (eval, rex, stats)
- Pipeline-based query structure
- Rich statistical functions
- Extensive field extraction
- Mature ecosystem with thousands of searches
Basic SPL Query
index=windows_logs sourcetype=WinEventLog:Security EventCode=4625
| stats count by src_ip, user
| where count > 5Advanced Examples
SQL Injection Detection:
index=web_logs sourcetype=access_combined
| rex field=uri "(?<sql_keywords>union|select|insert|update|delete|drop|exec|script)"
| where isnotnull(sql_keywords)
| stats count by src_ip, uri, sql_keywords
| where count > 5Unusual Process Parent-Child Relationship:
index=windows_logs sourcetype=WinEventLog:Security EventCode=4688
| eval parent_child=parent_process_name + "->" + new_process_name
| search parent_child IN (
"winword.exe->cmd.exe",
"excel.exe->powershell.exe",
"outlook.exe->wscript.exe"
)
| stats count by ComputerName, user, parent_childAWS CloudTrail Anomalous API Activity:
index=aws_cloudtrail
| stats count by userName, eventName
| eventstats avg(count) as avg_count, stdev(count) as stdev_count by eventName
| eval threshold = avg_count + (2 * stdev_count)
| where count > threshold
| table userName, eventName, count, avg_count, thresholdSuccessful Login After Multiple Failures:
index=authentication
| transaction user maxspan=30m
| search failed_attempts>5 AND success_attempts>0
| table _time, user, src_ip, failed_attempts, success_attempts---
Rule Development Workflow
1. Threat Research
- Identify attack technique (MITRE ATT&CK framework)
- Research indicators of compromise (IOCs)
- Understand attacker TTPs (Tactics, Techniques, Procedures)
- Review public threat intelligence reports
2. Log Analysis
- Identify log sources containing relevant events
- Analyze log structure and fields
- Determine detection logic and thresholds
- Identify potential false positive scenarios
3. Rule Development
- Write SIGMA rule (universal format) OR platform-specific query
- Include MITRE ATT&CK mapping
- Document false positives
- Set appropriate severity level
4. Testing
- Test rule against historical data (2-4 weeks)
- Validate true positives (attack scenarios)
- Identify false positives
- Adjust thresholds and whitelisting
5. Deployment
- Deploy rule to production SIEM
- Monitor alert volume and quality
- Tune rule based on operational feedback
- Document any exceptions or whitelists
6. Maintenance
- Review rule effectiveness monthly
- Update based on new threat intelligence
- Adjust thresholds as environment changes
- Archive or deprecate ineffective rules
---
MITRE ATT&CK Mapping
Top 10 Techniques to Detect
| Technique ID | Technique Name | Detection Method |
|---|---|---|
| T1078 | Valid Accounts | Unusual login patterns, impossible travel |
| T1059.001 | PowerShell | Malicious PowerShell commands, script downloads |
| T1003 | Credential Dumping | Mimikatz signatures, registry access |
| T1021.001 | Remote Desktop | RDP from unusual sources, lateral movement |
| T1071 | Application Layer Protocol | C2 communication via HTTP/DNS |
| T1105 | Ingress Tool Transfer | File downloads from internet |
| T1486 | Data Encrypted for Impact | Ransomware file encryption patterns |
| T1087 | Account Discovery | Enumeration commands (net user, whoami) |
| T1098 | Account Manipulation | Privilege escalation, role changes |
| T1562.001 | Disable Security Tools | Stopping AV, SIEM agents, logging |
Mapping SIGMA Rules to MITRE ATT&CK
tags:
- attack.credential_access # Tactic
- attack.t1110 # Technique (Brute Force)
- attack.t1110.001 # Sub-technique (Password Guessing)Tactics (14 total):
- Initial Access, Execution, Persistence, Privilege Escalation
- Defense Evasion, Credential Access, Discovery, Lateral Movement
- Collection, Command and Control, Exfiltration, Impact
---
Testing and Validation
Unit Testing Detection Rules
Test Data Creation:
# Simulate brute force attack (for testing)
for i in {1..15}; do
ssh user@target-host -p 22 2>&1 | logger -t sshd
doneRule Validation:
# Elastic: Test EQL query
POST /_eql/search
{
"query": "process where process.name == 'powershell.exe'",
"size": 10
}
# Splunk: Test SPL query
index=test_data sourcetype=authentication
| stats count by src_ip
| where count > 10
# Sentinel: Test KQL query
SigninLogs
| where TimeGenerated > ago(1h)
| take 10False Positive Analysis
Common False Positive Scenarios:
1. Monitoring Systems - Health checks, scanners triggering rate-limit alerts 2. Automated Processes - Scripts, CI/CD pipelines triggering unusual process alerts 3. Admin Operations - Legitimate bulk operations (user creation, config changes) 4. Known Tools - Security tools (Nessus, Qualys) triggering vulnerability scan alerts
Mitigation Strategies:
- Whitelisting - Exclude known-safe IPs, users, processes
- Threshold Tuning - Adjust count/frequency thresholds
- Context Enrichment - Add business context (service accounts, admin users)
- Time-Based Rules - Suppress alerts during maintenance windows
Continuous Improvement
Metrics to Track:
- True Positive Rate - % of alerts that are actual threats (target: >30%)
- False Positive Rate - % of alerts that are benign (target: <50%)
- Mean Time to Investigate - Average time to triage alert (target: <15 min)
- Alert Volume - Total alerts per day (target: <100)
- Coverage - % of MITRE ATT&CK techniques detected (target: >70%)
Monthly Review:
- Analyze alert metrics
- Review incidents detected by each rule
- Identify rules needing tuning or deprecation
- Update rules based on new threat intelligence
- Document improvements and lessons learned
---
Summary
Use SIGMA rules when:
- Need to support multiple SIEM platforms
- Want vendor-agnostic detection logic
- Building reusable detection rule library
- Contributing to open-source security community
Use Elastic EQL when:
- Using Elastic SIEM exclusively
- Need sequence-based detection (multi-event correlation)
- Require time-based analysis with
maxspan - Want native Elastic performance optimizations
Use Microsoft Sentinel KQL when:
- Using Microsoft Sentinel/Azure
- Need rich aggregation and statistical analysis
- Want integration with Azure Logic Apps
- Require time-series analysis
Use Splunk SPL when:
- Using Splunk Enterprise Security
- Need powerful data transformation (eval, rex)
- Require complex field extraction
- Want pipeline-based query structure
Log Retention Policies
Table of Contents
- Compliance Framework Requirements
- Storage Tiering Strategy
- Retention Policy Examples
- Index Lifecycle Management (ILM)
- Data Minimization Strategies
- GDPR-Specific Requirements
- Summary
Compliance Framework Requirements
| Framework | Minimum Retention | Key Requirements | Penalties for Non-Compliance |
|---|---|---|---|
| GDPR | 30-90 days (varies) | Right to be forgotten, data minimization, encryption | Up to €20M or 4% of revenue |
| HIPAA | 6 years | Encryption at rest and in transit, access controls, audit trails | Up to $1.5M/year |
| PCI DSS | 1 year | Log integrity, tamper-proof storage, quarterly reviews | Up to $100k/month fines |
| SOC 2 | 1 year | Access controls, encryption, audit trail | Loss of certification |
| NIST 800-53 | Varies by control | Comprehensive logging, 90-day retention minimum | Federal contract loss |
Storage Tiering Strategy
Hot Tier (Real-Time Analysis)
Duration: Last 7-30 days Storage: SSD, high-IOPS Cost: $0.10-0.15/GB/month Use Cases:
- Real-time threat detection
- Active incident investigation
- Dashboard queries
- Frequent searches
Warm Tier (Recent History)
Duration: 30-90 days Storage: HDD, lower IOPS Cost: $0.03-0.05/GB/month Use Cases:
- Occasional threat hunting
- Weekly/monthly reports
- Historical trend analysis
- Non-urgent investigations
Cold Tier (Compliance Archive)
Duration: 90 days to retention limit Storage: S3 Glacier, Azure Cool Blob Cost: $0.004-0.01/GB/month Use Cases:
- Compliance audit requirements
- Legal discovery
- Rare searches (acceptable latency)
- Long-term retention
Retention Policy Examples
Startup (Moderate Budget)
Data Volume: 50 GB/day
Retention: 90 days (SOC 2 Type II)
Hot Tier (7 days): 350 GB @ $0.10 = $35/month
Warm Tier (30 days): 1.5 TB @ $0.05 = $75/month
Cold Tier (53 days): 2.65 TB @ $0.01 = $26.50/month
Total Cost: $136.50/month ($1,638/year)Mid-Market (Healthcare, HIPAA)
Data Volume: 200 GB/day
Retention: 6 years (HIPAA)
Hot Tier (30 days): 6 TB @ $0.10 = $600/month
Warm Tier (90 days): 18 TB @ $0.05 = $900/month
Cold Tier (6 years): 425 TB @ $0.01 = $4,250/month
Total Cost: $5,750/month ($69,000/year)Enterprise (Financial Services, PCI DSS)
Data Volume: 1 TB/day
Retention: 1 year (PCI DSS)
Hot Tier (30 days): 30 TB @ $0.10 = $3,000/month
Warm Tier (90 days): 90 TB @ $0.05 = $4,500/month
Cold Tier (245 days): 245 TB @ $0.01 = $2,450/month
Total Cost: $9,950/month ($119,400/year)Index Lifecycle Management (ILM)
Elastic ILM Policy
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover": {
"max_size": "50GB",
"max_age": "7d"
},
"set_priority": {
"priority": 100
}
}
},
"warm": {
"min_age": "7d",
"actions": {
"shrink": {
"number_of_shards": 1
},
"forcemerge": {
"max_num_segments": 1
},
"set_priority": {
"priority": 50
}
}
},
"cold": {
"min_age": "90d",
"actions": {
"searchable_snapshot": {
"snapshot_repository": "security-snapshots"
},
"set_priority": {
"priority": 0
}
}
},
"delete": {
"min_age": "365d",
"actions": {
"delete": {}
}
}
}
}
}Microsoft Sentinel Retention
# Azure CLI: Configure 90-day interactive retention, 1-year total
az monitor log-analytics workspace update \
--resource-group security-rg \
--workspace-name security-sentinel \
--retention-time 365 \
--total-retention-time 365Data Minimization Strategies
Log Sampling
When to Use: High-volume non-critical logs (info/debug level)
# Fluentd sampling (keep 10%)
<filter application.info>
@type sampling
@id sample_info_logs
interval 10
sample_rate 1
</filter>Selective Field Retention
Example: Strip PII from logs before storage
# Logstash: Remove sensitive fields
filter {
mutate {
remove_field => ["credit_card", "ssn", "password"]
}
# Mask email addresses
mutate {
gsub => [
"message", "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b", "[EMAIL_REDACTED]"
]
}
}Aggregation Before Storage
Example: Store aggregated metrics, not raw logs
# Store: "200 failed logins from 10.0.0.1 in last hour"
# Instead of: 200 individual log entriesGDPR-Specific Requirements
Right to Be Forgotten
# Elasticsearch: Delete user data
POST /security-logs/_delete_by_query
{
"query": {
"term": {
"user.email": "user@example.com"
}
}
}Data Anonymization
# Python: Anonymize IP addresses
import hashlib
def anonymize_ip(ip_address, salt="secret"):
return hashlib.sha256(f"{ip_address}{salt}".encode()).hexdigest()[:16]
# 192.168.1.100 → 7a8f9d3c2b1e4f6aConsent Management
- Log user consent for data collection
- Track data processing purposes
- Enable audit trails for data access
- Implement data export capabilities
Summary
Critical Retention Requirements:
- GDPR: 30-90 days (minimize data, enable deletion)
- HIPAA: 6 years (healthcare, encryption required)
- PCI DSS: 1 year (payment data, quarterly reviews)
- SOC 2: 1 year (varies by security control)
Cost Optimization:
- Use hot/warm/cold tiering (saves 70-80%)
- Sample non-critical logs (reduces volume 50-90%)
- Strip PII before storage (compliance + cost savings)
- Aggregate before storing (reduces volume 80-95%)
Implementation:
- Use Index Lifecycle Management (Elastic, Splunk)
- Configure Azure Sentinel retention policies
- Implement S3 lifecycle rules for AWS Security Lake
- Automate data deletion after retention period
SIEM Platform Comparison
Table of Contents
- Overview
- Elastic SIEM
- Microsoft Sentinel
- Wazuh
- Splunk Enterprise Security
- AWS Security Lake
- Feature Comparison Matrix
- Selection Decision Tree
- Cost Comparison
Overview
This guide provides comprehensive comparison of leading SIEM platforms in 2025, covering features, pricing, deployment models, and use cases to help select the right platform for your organization.
Elastic SIEM
Overview
- Vendor: Elastic
- Deployment: Cloud (Elastic Cloud) or Self-Hosted
- Foundation: Open-source Elasticsearch
- License: Elastic License (commercial) or Apache 2.0 (open-source core)
Strengths
Open-Source Foundation:
- Built on Elasticsearch (open-source core)
- Start with ELK stack (free), add commercial features later
- Full control and customization
- Extensive plugin ecosystem
Unified XDR Platform:
- Extends beyond SIEM to endpoint detection (EDR)
- Cloud security posture management (CSPM)
- Unified detection across SIEM, endpoint, and cloud
AI-Powered Detection:
- Elastic AI SOC Engine (EASE) for automated alert correlation
- Machine learning for anomaly detection
- Behavioral analytics
Detection-as-Code:
- Extensive community-contributed detection rules
- SIGMA rule support
- Event Query Language (EQL) for sequence-based detection
- MITRE ATT&CK mapping built-in
Multi-Cloud Support:
- Works across AWS, Azure, GCP, on-premise
- No vendor lock-in
- Flexible deployment options
Weaknesses
- Steeper learning curve than Sentinel
- Requires Elasticsearch expertise for advanced configurations
- Self-hosted option requires infrastructure management
- Less mature SOAR capabilities compared to Sentinel
When to Use
- DevOps/engineering teams comfortable with Elasticsearch
- Multi-cloud or hybrid environments
- Need for customization and extensibility
- Want option to start open-source, add commercial features later
- Medium to large data volumes (100 GB - 10 TB/day)
Pricing
Elastic Cloud (Managed):
- Standard: ~$95/month (16 GB RAM, 128 GB storage)
- Hot tier: ~$0.11/GB/month
- Warm tier: ~$0.03/GB/month
- Cold tier: ~$0.01/GB/month
Self-Hosted (Open-Source):
- Free for ELK stack (Elasticsearch, Logstash, Kibana)
- Commercial features (SIEM, ML, alerting): License required
- Infrastructure costs only
Example Deployment
# Elastic Cloud (Managed)
# Sign up at cloud.elastic.co
# Create deployment (Security tier)
# Self-Hosted (Docker Compose)
version: '3.8'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
environment:
- discovery.type=single-node
- xpack.security.enabled=true
ports:
- "9200:9200"
volumes:
- esdata:/usr/share/elasticsearch/data
kibana:
image: docker.elastic.co/kibana/kibana:8.11.0
environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
ports:
- "5601:5601"
volumes:
esdata:---
Microsoft Sentinel
Overview
- Vendor: Microsoft
- Deployment: Cloud-only (Azure)
- Foundation: Azure Monitor, Log Analytics
- License: Pay-as-you-go (per GB ingested)
Strengths
Cloud-Native:
- Built on Azure, infinite scale
- No infrastructure management
- Auto-scaling to any volume
Azure Ecosystem Integration:
- Seamless integration with Microsoft 365, Azure AD, Defender, Intune
- Native Azure resource monitoring
- Azure Security Center integration
Built-in SOAR:
- Azure Logic Apps for automation (included)
- No separate SOAR product needed
- Pre-built playbooks for common responses
AI-Driven Threat Intelligence:
- Microsoft Threat Intelligence integration
- Behavioral analytics
- ML-based anomaly detection
- Sentinel Graph for attack path visualization
Ease of Use:
- User-friendly interface
- Pre-built dashboards and workbooks
- Guided onboarding
Cost-Effective for SMBs:
- 50 GB commitment tier introduced (2024)
- Pay-as-you-go pricing
- No upfront costs
Weaknesses
- Azure-locked (limited multi-cloud support)
- Less customization than Elastic/Splunk
- Requires Azure expertise
- Data egress costs for multi-cloud scenarios
When to Use
- Heavy Azure/Microsoft 365 investment
- Want cloud-native, managed SIEM (no infrastructure)
- Need built-in SOAR capabilities
- Small to large organizations (scalable pricing)
- Existing Microsoft ecosystem (AD, Office 365, Defender)
Pricing
Pay-As-You-Go:
- $2.80/GB for first 5 TB/day
- Volume discounts at higher tiers
- 50 GB commitment tier: ~$140/month minimum
Commitment Tiers:
- 50 GB/day: ~$140/day (~$4,200/month)
- 100 GB/day: ~$260/day (~$7,800/month)
- 200 GB/day: ~$500/day (~$15,000/month)
Example Deployment
# Azure CLI
az monitor log-analytics workspace create \
--resource-group security-rg \
--workspace-name security-sentinel \
--location eastus
az sentinel workspace create \
--resource-group security-rg \
--workspace-name security-sentinel
# Connect data sources
az sentinel data-connector create \
--resource-group security-rg \
--workspace-name security-sentinel \
--name AzureActiveDirectory \
--kind AzureActiveDirectory---
Wazuh
Overview
- Vendor: Wazuh (Open-Source)
- Deployment: Self-Hosted (on-premise or cloud VMs)
- Foundation: OpenSearch (fork of Elasticsearch)
- License: GPL v2 (free and open-source)
Strengths
Free and Open-Source:
- Zero licensing costs
- Active community (10,000+ GitHub stars)
- Regular updates and security patches
XDR Capabilities:
- Unified threat prevention, detection, and response
- Endpoint security (file integrity monitoring, rootkit detection)
- Vulnerability detection
- Security configuration assessment
Multi-Platform Support:
- Linux, Windows, macOS, Docker, Kubernetes, cloud VMs
- Unified agent across platforms
- Centralized management
Built-in Compliance Features:
- PCI DSS compliance modules
- HIPAA, GDPR, SOC 2 compliance templates
- CIS benchmarks
- Automated compliance reporting
Easy to Deploy:
- Docker Compose for quick start
- Kubernetes Helm charts
- Ansible playbooks for automation
Active Development:
- Regular releases (quarterly)
- Responsive to security landscape
- Community-contributed rules
Weaknesses
- Fewer enterprise features than commercial SIEMs
- Requires self-hosting and maintenance
- Smaller ecosystem than Elastic/Splunk
- Less mature AI/ML capabilities
- Limited advanced correlation features
When to Use
- Tight budget (startups, SMBs, non-profits)
- Want full control and customization
- Need compliance features (PCI DSS, HIPAA, GDPR, SOC 2)
- Comfortable managing infrastructure
- Small to medium data volumes (<500 GB/day)
Pricing
Software Cost: Free (GPL v2 license)
Infrastructure Costs:
- Cloud VMs: ~$200-$1,000/month (depending on scale)
- On-premise: Hardware costs only
- Storage: S3/Azure Blob/GCS for log archive
Example Deployment
# Docker Compose
git clone https://github.com/wazuh/wazuh-docker.git
cd wazuh-docker/single-node
# Generate certificates
docker-compose -f generate-indexer-certs.yml run --rm generator
# Start Wazuh stack
docker-compose up -d
# Access dashboard: https://localhost
# Default credentials: admin / SecretPassword (change immediately)---
Splunk Enterprise Security
Overview
- Vendor: Splunk
- Deployment: On-Premise or Splunk Cloud
- Foundation: Proprietary Splunk platform
- License: Commercial (per GB ingested/day)
Strengths
Market Leader:
- Used by Fortune 100 companies
- Proven at enterprise scale
- Decades of development
Massive Scalability:
- Proven at petabyte scale
- Handles TB/day ingestion rates
- Distributed search and indexing
Customization:
- Extensive customization options
- SPL (Search Processing Language) - powerful query language
- Splunkbase (1,000+ apps and integrations)
Ecosystem:
- Thousands of pre-built integrations
- Vendor partnerships
- Extensive training and certification programs
Enterprise Support:
- 24/7 support
- Professional services
- Dedicated account teams
- SLAs and uptime guarantees
Advanced Analytics:
- Machine learning toolkit
- ITSI (IT Service Intelligence) integration
- User behavior analytics (UBA)
Weaknesses
- Very expensive: Licensing costs can be prohibitive ($150k-$1M+/year)
- Complex to deploy and manage
- Requires dedicated Splunk team
- Separate SOAR product (Splunk SOAR) adds significant cost
- Steep learning curve
When to Use
- Large enterprises with unlimited budget
- Need for massive scale (TB/day+)
- Require enterprise support and SLAs
- Existing Splunk investment
- Complex use cases requiring deep customization
Pricing
On-Premise:
- ~$1,800/GB/day ingestion license
- Example: 100 GB/day = ~$180,000/year
Splunk Cloud:
- ~$2,300/GB/day (includes infrastructure)
- Example: 100 GB/day = ~$230,000/year
Enterprise Security Add-On:
- Additional ~20-30% on top of core license
---
AWS Security Lake
Overview
- Vendor: Amazon Web Services
- Deployment: Cloud-only (AWS)
- Foundation: S3, OpenSearch, Athena
- License: Pay-as-you-go (AWS services)
Strengths
AWS-Native:
- Seamless integration with AWS services
- CloudTrail, VPC Flow Logs, GuardDuty, Security Hub
- Native IAM integration
Centralized Data Lake:
- OCSF (Open Cybersecurity Schema Framework) format
- Single repository for security data from multiple sources
- S3-based storage (cheap, scalable)
Flexible Analysis:
- OpenSearch for SIEM-like analysis
- Athena for SQL queries
- SageMaker for ML analysis
- Third-party SIEM integration
Cost-Effective:
- Pay only for S3 storage and query compute
- No per-GB ingestion fees
- Data tiering (S3 Standard → Glacier)
Weaknesses
- AWS-locked (limited multi-cloud support)
- Newer platform (less mature than competitors)
- Requires AWS expertise
- SIEM features less advanced than dedicated platforms
- Manual setup and configuration
When to Use
- AWS-heavy organizations (90%+ AWS infrastructure)
- Want centralized security data lake
- Need flexible analysis options (OpenSearch, Athena, custom)
- Budget-conscious (avoid per-GB ingestion fees)
Pricing
Security Lake:
- Free service (pay for underlying resources)
Storage (S3):
- S3 Standard: ~$0.023/GB/month
- S3 Glacier: ~$0.004/GB/month
OpenSearch:
- ~$0.15/hour per node (t3.medium.search)
- ~$3,600/month for 3-node cluster
Athena Queries:
- $5/TB scanned
---
Feature Comparison Matrix
| Feature | Elastic SIEM | Microsoft Sentinel | Wazuh | Splunk ES | AWS Security Lake |
|---|---|---|---|---|---|
| Cost (GB/day) | $$$ | $$$ | Free | $$$$$ | $$ |
| Deployment | Cloud/Self-Hosted | Cloud | Self-Hosted | Cloud/On-Prem | Cloud |
| Scalability | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ |
| Ease of Use | ⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐ |
| Customization | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
| AI/ML | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
| Built-in SOAR | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐ |
| Multi-Cloud | ⭐⭐⭐⭐⭐ | ⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐ |
| Community | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
| Learning Curve | Medium-High | Medium | Low-Medium | High | Medium-High |
| Open-Source | Partial | No | Yes | No | Partial |
| Detection Rules | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
| Compliance | ⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ |
---
Selection Decision Tree
START: Which SIEM platform should I use?
Q1: What is your budget?
├─ Unlimited ($500k+/year) → Q2
├─ Moderate ($50k-$500k/year) → Q3
└─ Tight (<$50k/year) → Wazuh (Free, Open-Source)
Q2: Do you have existing vendor ecosystem?
├─ Heavy Azure investment → Microsoft Sentinel
├─ Heavy AWS investment → AWS Security Lake + OpenSearch
└─ Multi-cloud or on-premise → Splunk Enterprise Security
Q3: What is your data volume?
├─ >1 TB/day → Elastic Cloud or Splunk Cloud
├─ 100 GB - 1 TB/day → Microsoft Sentinel or Elastic SIEM
└─ <100 GB/day → Wazuh or Sentinel (50 GB tier)
Q4: What is your team's expertise?
├─ Elasticsearch experts → Elastic SIEM
├─ Microsoft/Azure experts → Microsoft Sentinel
├─ Splunk experience → Splunk
└─ Generalists → Wazuh (easiest learning curve)
Q5: What deployment model do you prefer?
├─ Cloud-native (no infrastructure) → Microsoft Sentinel or Elastic Cloud
├─ Self-hosted (full control) → Wazuh or Elastic (self-hosted)
└─ Hybrid (flexible) → Elastic SIEM---
Cost Comparison
Scenario: 200 GB/day ingestion, 1-year retention
| Platform | Monthly Cost | Annual Cost | Notes |
|---|---|---|---|
| Elastic Cloud | ~$6,600 | ~$79,200 | Hot: 30d, Warm: 90d, Cold: 245d |
| Microsoft Sentinel | ~$16,800 | ~$201,600 | 200 GB commitment tier |
| Wazuh | ~$500 | ~$6,000 | Infrastructure only (VMs, storage) |
| Splunk ES | ~$30,000 | ~$360,000 | On-premise license |
| AWS Security Lake | ~$5,000 | ~$60,000 | S3 + OpenSearch + Athena |
Scenario: 1 TB/day ingestion, 1-year retention
| Platform | Monthly Cost | Annual Cost | Notes |
|---|---|---|---|
| Elastic Cloud | ~$33,000 | ~$396,000 | Hot: 30d, Warm: 90d, Cold: 245d |
| Microsoft Sentinel | ~$75,000 | ~$900,000 | 1 TB commitment tier |
| Wazuh | ~$2,500 | ~$30,000 | Infrastructure (larger VMs, storage) |
| Splunk ES | ~$150,000 | ~$1,800,000 | On-premise license |
| AWS Security Lake | ~$25,000 | ~$300,000 | S3 + OpenSearch (larger) + Athena |
Cost Optimization Tips
Elastic SIEM:
- Use hot/warm/cold tiering aggressively
- Sample non-critical logs (reduce volume)
- Self-host for larger deployments (cost-effective >500 GB/day)
Microsoft Sentinel:
- Use 50 GB commitment tier for SMBs
- Enable data retention archiving (cheaper long-term storage)
- Filter noisy logs before ingestion
Wazuh:
- Use cloud VMs with reserved instances (60% savings)
- Archive to S3/Azure Blob/GCS (cheap long-term storage)
- Scale horizontally (add nodes as needed)
Splunk:
- Negotiate volume discounts (40%+ discounts possible)
- Use Splunk Cloud for smaller deployments (<500 GB/day)
- Implement data tiering (reduce hot storage)
AWS Security Lake:
- Use S3 Intelligent-Tiering (automatic cost optimization)
- Partition data by date (reduce Athena scan costs)
- Use OpenSearch reserved instances (30% savings)
---
Summary
Choose Elastic SIEM when:
- Multi-cloud or hybrid environment
- DevOps/engineering team with Elasticsearch skills
- Need customization and extensibility
- Want open-source foundation with commercial options
Choose Microsoft Sentinel when:
- Heavy Azure/Microsoft 365 investment
- Want cloud-native, fully managed SIEM
- Need built-in SOAR capabilities
- Small to large organizations with scalable pricing
Choose Wazuh when:
- Tight budget (startups, SMBs, non-profits)
- Want full control and open-source solution
- Need compliance features (PCI DSS, HIPAA, GDPR)
- Comfortable managing infrastructure
Choose Splunk when:
- Large enterprise with unlimited budget
- Need proven scalability at massive scale
- Require enterprise support and SLAs
- Complex use cases requiring deep customization
Choose AWS Security Lake when:
- AWS-heavy organization (90%+ AWS infrastructure)
- Want centralized security data lake
- Need flexible analysis options
- Budget-conscious (avoid per-GB ingestion fees)
#!/usr/bin/env python3
"""
SIEM Cost Calculator
Estimates SIEM costs based on log volume, retention period, and storage tiering strategy.
"""
import argparse
from decimal import Decimal
def calculate_costs(daily_gb, retention_days, hot_days=30, warm_days=90):
"""
Calculate SIEM storage costs with hot/warm/cold tiering.
Args:
daily_gb: Daily log volume in GB
retention_days: Total retention period in days
hot_days: Days to keep in hot tier (default: 30)
warm_days: Days to keep in warm tier (default: 90)
Returns:
Dictionary with cost breakdown
"""
# Storage costs per GB/month
hot_cost = Decimal('0.10') # SSD, high-IOPS
warm_cost = Decimal('0.05') # HDD, lower IOPS
cold_cost = Decimal('0.01') # S3 Glacier, archival
daily_gb = Decimal(str(daily_gb))
# Calculate total data per tier
hot_total_gb = daily_gb * min(hot_days, retention_days)
warm_total_gb = daily_gb * max(0, min(warm_days, retention_days - hot_days))
cold_total_gb = daily_gb * max(0, retention_days - hot_days - warm_days)
# Convert to TB for readability
hot_tb = hot_total_gb / 1000
warm_tb = warm_total_gb / 1000
cold_tb = cold_total_gb / 1000
# Calculate monthly costs
hot_monthly = (hot_total_gb * hot_cost).quantize(Decimal('0.01'))
warm_monthly = (warm_total_gb * warm_cost).quantize(Decimal('0.01'))
cold_monthly = (cold_total_gb * cold_cost).quantize(Decimal('0.01'))
total_monthly = hot_monthly + warm_monthly + cold_monthly
total_annual = total_monthly * 12
# Calculate hot-only costs for comparison
total_gb = daily_gb * retention_days
hot_only_monthly = (total_gb * hot_cost).quantize(Decimal('0.01'))
hot_only_annual = hot_only_monthly * 12
savings_monthly = hot_only_monthly - total_monthly
savings_annual = hot_only_annual - total_annual
savings_percent = ((savings_annual / hot_only_annual) * 100).quantize(Decimal('0.1'))
return {
'hot': {
'days': hot_days,
'tb': float(hot_tb),
'monthly': float(hot_monthly),
'annual': float(hot_monthly * 12)
},
'warm': {
'days': warm_days - hot_days if warm_days > hot_days else 0,
'tb': float(warm_tb),
'monthly': float(warm_monthly),
'annual': float(warm_monthly * 12)
},
'cold': {
'days': retention_days - warm_days if retention_days > warm_days else 0,
'tb': float(cold_tb),
'monthly': float(cold_monthly),
'annual': float(cold_monthly * 12)
},
'total': {
'monthly': float(total_monthly),
'annual': float(total_annual)
},
'comparison': {
'hot_only_monthly': float(hot_only_monthly),
'hot_only_annual': float(hot_only_annual),
'savings_monthly': float(savings_monthly),
'savings_annual': float(savings_annual),
'savings_percent': float(savings_percent)
}
}
def print_cost_report(costs, daily_gb, retention_days):
"""Print formatted cost report."""
print("\n" + "="*60)
print(f"SIEM Cost Estimation")
print("="*60)
print(f"\nInput Parameters:")
print(f" Daily Log Volume: {daily_gb} GB/day")
print(f" Total Retention: {retention_days} days")
print(f"\nStorage Tiering Breakdown:")
print(f" Hot Tier ({costs['hot']['days']} days):")
print(f" Volume: {costs['hot']['tb']:.2f} TB")
print(f" Cost: ${costs['hot']['monthly']:,.2f}/month (${costs['hot']['annual']:,.2f}/year)")
print(f" Warm Tier ({costs['warm']['days']} days):")
print(f" Volume: {costs['warm']['tb']:.2f} TB")
print(f" Cost: ${costs['warm']['monthly']:,.2f}/month (${costs['warm']['annual']:,.2f}/year)")
print(f" Cold Tier ({costs['cold']['days']} days):")
print(f" Volume: {costs['cold']['tb']:.2f} TB")
print(f" Cost: ${costs['cold']['monthly']:,.2f}/month (${costs['cold']['annual']:,.2f}/year)")
print(f"\nTotal Cost:")
print(f" Monthly: ${costs['total']['monthly']:,.2f}")
print(f" Annual: ${costs['total']['annual']:,.2f}")
print(f"\nComparison (Hot-Only Storage):")
print(f" Hot-Only Monthly: ${costs['comparison']['hot_only_monthly']:,.2f}")
print(f" Hot-Only Annual: ${costs['comparison']['hot_only_annual']:,.2f}")
print(f" Savings: ${costs['comparison']['savings_annual']:,.2f}/year ({costs['comparison']['savings_percent']:.1f}%)")
print("="*60 + "\n")
def main():
parser = argparse.ArgumentParser(
description='Calculate SIEM storage costs with tiering strategy'
)
parser.add_argument(
'--daily-gb',
type=float,
required=True,
help='Daily log volume in GB'
)
parser.add_argument(
'--retention-days',
type=int,
required=True,
help='Total retention period in days'
)
parser.add_argument(
'--hot-days',
type=int,
default=30,
help='Days to keep in hot tier (default: 30)'
)
parser.add_argument(
'--warm-days',
type=int,
default=90,
help='Total days in hot+warm tier (default: 90)'
)
args = parser.parse_args()
costs = calculate_costs(
args.daily_gb,
args.retention_days,
args.hot_days,
args.warm_days
)
print_cost_report(costs, args.daily_gb, args.retention_days)
if __name__ == '__main__':
main()
Related skills
FAQ
What is a SIGMA rule?
A universal detection rule format that compiles to any SIEM query language, including Elastic EQL, Splunk SPL and Microsoft KQL.
Which SIEM should I pick on a tight budget?
Wazuh, a free open-source XDR/SIEM, for budgets under 50k per year.