Log Analysis

log-analysis

Purpose

This skill analyzes system and application logs to identify anomalies, security threats, and patterns indicative of breaches, supporting blue-team operations by providing actionable insights for incident response.

When to Use

Use this skill during forensic investigations, real-time monitoring, or post-incident reviews. Apply it when logs show unusual activity, such as repeated failed logins or unexpected network traffic, or for routine security audits in environments like cloud servers or on-premise systems.

Key Capabilities

• Detect anomalies using machine learning-based pattern recognition on log data.
• Identify threats like SQL injection attempts or malware indicators via signature matching.
• Parse multiple log formats (e.g., syslog, Apache access logs) and extract metadata for correlation.
• Generate reports with severity levels and recommendations for mitigation.
• Support for filtering logs by time, IP, or user ID to narrow down investigations.

Usage Patterns

Invoke this skill via CLI for quick analysis or integrate it into scripts for automated workflows. Always provide log input paths or streams, and specify analysis parameters. For example, pipe logs from a file or API, then apply filters before running detection. Use environment variables for authentication, like $LOG_ANALYSIS_API_KEY, to secure API calls. Ensure logs are in plain text or JSON format for optimal parsing.

Common Commands/API

Use the CLI tool with commands like log-analysis scan for basic operations. For API integration, call endpoints like POST /api/v1/logs/analyze with a JSON payload.

• CLI Example: Scan a local file for threats:

  log-analysis scan --file /var/log/syslog --anomaly true --output report.json

This flags anomalies and saves results to a file.

• API Example: Send logs for analysis:

  curl -X POST https://api.openclaw.com/api/v1/logs/analyze \
  -H "Authorization: Bearer $LOG_ANALYSIS_API_KEY" \
  -d '{"logs": [{"line": "ERROR: Unauthorized access"}], "filters": {"ip": "192.168.1.1"}}'

This analyzes specified logs and returns JSON with detected threats.

• Common Flags: --file <path> for input, --anomaly for ML-based detection, --threshold 0.8 for sensitivity level.
• Config Format: Use JSON for custom rules, e.g.:

  {
    "rules": [
      {"pattern": "Failed login", "severity": "high"}
    ]
  }

Load via --config config.json.

Integration Notes

Integrate with tools like ELK Stack or Splunk by streaming logs via API. Set $LOG_ANALYSIS_API_KEY as an environment variable for authentication. For example, in a Python script, import the SDK and authenticate:

  import log_analysis_sdk
  client = log_analysis_sdk.Client(api_key=os.environ['LOG_ANALYSIS_API_KEY'])
  response = client.analyze(logs_data)

Handle webhooks for real-time alerts by registering a callback URL. Ensure compatibility by matching log formats; convert non-standard logs using tools like jq.

Error Handling

Check for errors like invalid log formats or authentication failures. Use try-catch blocks in scripts:

  try:
      result = log_analysis.scan('--file invalid.log')
  except FileNotFoundError:
      print("Error: Log file not found. Verify path and permissions.")

Common errors include 401 Unauthorized (missing $LOG_ANALYSIS_API_KEY) or 400 Bad Request (malformed JSON). Log errors with timestamps and retry transient issues up to 3 times with exponential backoff. Validate inputs before commands, e.g., check if files exist using os.path.exists().

Concrete Usage Examples

1. Detect anomalies in server logs: Run log-analysis scan --file /var/log/auth.log --anomaly true to identify potential brute-force attacks. Review the output JSON for entries like {"event": "Failed login", "count": 5, "threat": "high"}, then correlate with firewall logs for further investigation.

2. Integrate with a monitoring script: In a Bash script, fetch logs from a remote server and analyze:

   logs=$(curl http://server/logs.txt)
   echo "$logs" | log-analysis scan --stream --threshold 0.7

This detects threats in real-time streams, outputting alerts for anomalies like unusual user activity, and pipes results to a SIEM for automated response.

Graph Relationships

• Related to: forensics (via shared tags for log parsing), security (for threat detection overlap), blue-team cluster (as a core component for operations).
• Links: Connects to skills like "incident-response" for follow-up actions and "network-monitoring" for correlated data analysis.