Bug Analysis

Bug Analysis

Overview

This skill provides systematic bug analysis to identify root causes, assess impact, classify severity, and generate actionable fix recommendations. It helps triage bugs efficiently and provides structured analysis for development teams.

Core Analysis Workflow

Step 1: Initial Triage & Information Gathering

Collect Essential Information:

• Bug description and symptoms
• Reproduction steps (verify they work)
• Expected vs actual behavior
• Environment details (OS, browser, version, config)
• Error messages, stack traces, logs
• Screenshots or videos
• User impact and frequency

Quick Assessment:

• Severity: Critical/High/Medium/Low
• Type: Functional/Performance/Security/UI/Data/Integration/Configuration/Regression
• Priority: Based on severity + business impact
• Potential duplicates: Search existing issues

Step 2: Bug Categorization

Severity Classification (see severity-guidelines.md for detailed criteria):

Critical (P0) - Response: Immediate (<1 hour)

• System outage, data loss, security breach, no workaround

High (P1) - Response: Same day

• Major feature broken, significant user impact (>25%), difficult workaround

Medium (P2) - Response: Within 1 week

• Feature partially broken, moderate impact, workaround available

Low (P3) - Response: Backlog

• Minor issue, cosmetic problem, minimal impact

Bug Type Categories:

• Functional: Feature not working as specified
• Performance: Slow response, timeouts, resource issues
• Security: Vulnerabilities, unauthorized access
• UI/UX: Visual glitches, usability problems
• Data: Corruption, loss, incorrect processing
• Integration: API failures, third-party issues
• Configuration: Environment or deployment issues
• Regression: Previously working feature broken

Step 3: Root Cause Analysis

Investigation Process:

1. Review Error Evidence

• Parse stack traces to identify failure point
• Map error codes to known issues
• Check recent code changes (git blame, commit history)

2. Reproduce the Issue

• Validate reproduction steps
• Test in different environments
• Vary inputs to identify boundaries
• Document consistent reproduction method

3. Trace Execution Flow

• Follow code path from entry to failure
• Identify where actual diverges from expected
• Check data transformations and control flow
• Review relevant code sections

4. Analyze Dependencies

• Verify library and framework versions
• Check for known issues in dependencies
• Review integration points
• Test with different dependency versions

Common Root Cause Patterns:

• Logic errors (incorrect conditions, calculations)
• Null/undefined reference errors
• Race conditions and timing issues
• Memory leaks
• Boundary conditions (off-by-one, overflow)
• Configuration issues
• Dependency problems
• Integration failures

For detailed analysis techniques, see analysis-techniques.md for:

• Five Whys technique
• Stack trace analysis
• Differential analysis
• Data flow tracing
• Hypothesis testing
• Evidence collection methods

Step 4: Impact Assessment

Evaluate Impact Across Dimensions:

User Impact:

• Number/percentage of affected users
• User workflows disrupted
• User segments affected

Business Impact:

• Revenue loss or risk
• SLA violations
• Customer satisfaction impact
• Reputation risk

System Impact:

• Performance degradation
• Resource consumption
• Cascading failures
• Data integrity risks

Security Impact (if applicable):

• Confidentiality: Data exposure level
• Integrity: Unauthorized modifications
• Availability: Service disruptions
• Exploit potential

Scope Definition:

• Affected versions/releases
• Affected platforms/browsers
• Affected features/workflows
• Regression scope

Step 5: Fix Recommendation

Generate Structured Fix Strategy:

1. Immediate Mitigation (if not already done):

• Workarounds for users
• Configuration changes to reduce impact
• Feature flags to disable problematic code
• Rollback options if recent regression

2. Permanent Solution:

• Specific code changes needed
• Files to modify with line numbers
• Design changes required
• Database migrations or cleanup needed
• Configuration updates required

3. Testing Requirements:

• Unit tests to add
• Integration tests needed
• Regression tests to prevent recurrence
• Performance/security tests if applicable

4. Prevention Measures:

• Code review focus areas
• Additional validation needed
• Monitoring/alerting to add
• Documentation updates
• Process improvements

Output Format

Provide structured analysis using these templates:

Standard Bug Analysis: Use template from output-templates.md including:

• Bug summary with severity and priority
• Environment and reproduction steps
• Root cause analysis with evidence
• Impact assessment
• Recommended fix with testing plan

Specialized Reports (see output-templates.md):

• Security Vulnerability Report: CVSS scoring, attack vectors, disclosure plan
• Performance Bug Report: Metrics, profiling results, optimization strategy
• Crash Analysis Report: Stack traces, memory state, crash triggers

Special Analysis Scenarios

Security Vulnerabilities

For security issues:

1. Assess using CVSS: Attack vector, complexity, privileges, impact 2. Identify exploit potential: Remote exploitation, authentication required 3. Plan containment: Immediate patches, access restrictions, monitoring 4. Disclosure strategy: Timeline, notifications, compliance (CVE, GDPR, PCI-DSS)

See severity-guidelines.md for security-specific triage.

Performance Issues

For performance bugs:

1. Establish baseline: Expected metrics, SLA thresholds 2. Identify bottlenecks: CPU profiling, memory patterns, I/O, database queries 3. Quantify degradation: Response time increase, throughput reduction 4. Optimization strategy: Code optimization, caching, indexing, architecture changes

Crash Analysis

For application crashes:

1. Analyze crash dump: Exception type, stack trace, thread states 2. Identify trigger: User action, system condition, data input, timing 3. Assess stability impact: Frequency, affected scenarios, data loss risk 4. Recovery strategy: Crash handling, graceful degradation, monitoring

Investigation Tools & Commands

For detailed command references, see investigation-commands.md:

Version Control:

• git bisect - Find commit that introduced bug
• git blame - See who last modified code
• git log -S "text" - Find when code changed

Log Analysis:

• grep -A 5 -B 5 "error" app.log - Find errors with context
• tail -f app.log | grep ERROR - Monitor errors real-time
• Log parsing with awk and analysis scripts

Database Investigation:

• PostgreSQL: Slow query analysis, index usage
• MySQL: Process list, deadlock detection
• MongoDB: Operation profiling, collection stats

System Monitoring:

• Process monitoring: top, ps, htop
• Memory analysis: free, pmap, valgrind
• Network analysis: netstat, tcpdump, curl -v

Application Debugging:

• Node.js: node --inspect, profiling, heap snapshots
• Python: pdb, cProfile, memory profiling
• Java: jmap, jstack, flight recorder
• Docker/Kubernetes: Container logs, exec, debugging

Best Practices

Investigation Principles

• Evidence-based: Base conclusions on concrete data, not assumptions
• Systematic: Follow logical investigation process
• Hypothesis-driven: Form hypotheses, test them, verify results
• Document everything: Record findings, reasoning, and decisions
• Consider multiple causes: Don't fixate on first theory

Effective Communication

• Use clear language: Avoid jargon with non-technical stakeholders
• Provide context: Explain why the bug matters
• Set expectations: Realistic timelines and complexity
• Offer workarounds: Help users immediately when possible
• Follow up: Update stakeholders on progress

Prevention Focus

After fixing bugs:

• Identify patterns: Common causes across multiple bugs
• Improve testing: Add coverage for bug scenarios
• Enhance monitoring: Add alerts for similar issues
• Update processes: Code review checklists, deployment procedures
• Document lessons: Update knowledge base

Quick Reference

Common Root Causes Checklist

• [ ] Null/undefined reference
• [ ] Off-by-one error or boundary condition
• [ ] Race condition or timing issue
• [ ] Memory leak
• [ ] Missing validation or error handling
• [ ] Configuration issue
• [ ] Dependency version mismatch
• [ ] API contract change
• [ ] Database schema mismatch
• [ ] Incorrect permissions
• [ ] Resource exhaustion
• [ ] Caching issue
• [ ] Timezone/date handling
• [ ] Character encoding problem

Quick Diagnosis Commands

# Service status
systemctl status service_name

# Resource usage
top -bn1 | head -20                    # CPU
ps aux --sort=-%mem | head -10         # Memory
du -sh /* | sort -rh | head -10        # Disk

# Recent changes
git log --since="1 day ago" --oneline

# Error analysis
tail -100 /var/log/app.log | grep -i error
grep ERROR /var/log/app.log | wc -l

Integration with Development Workflow

Bug Lifecycle:

1. New → Report received 2. Triage → Analysis and prioritization (use this skill) 3. Confirmed → Reproduced, root cause identified 4. Assigned → Developer assigned 5. In Progress → Fix being implemented 6. Code Review → Fix under review 7. Testing → QA validation 8. Fixed → Deployed 9. Closed → Verified resolved

Documentation Requirements:

• Link to code: Files and line numbers
• Link to tests: Verify fix test cases
• Link to monitoring: Dashboards or alerts
• Link to related issues: Duplicates, related bugs
• Update documentation: If user-facing changes

Reference Files

Load reference files based on analysis needs:

• Severity Guidelines: See severity-guidelines.md when:
• Determining bug severity and priority
• Understanding triage criteria
• Need severity/priority matrix
• Security vulnerability classification

• Analysis Techniques: See analysis-techniques.md when:
• Need detailed RCA methodologies
• Applying Five Whys or other techniques
• Conducting hypothesis testing
• Performing evidence collection
• Using specific debugging strategies

• Output Templates: See output-templates.md when:
• Creating bug reports
• Writing RCA documents
• Documenting security vulnerabilities
• Reporting performance issues
• Analyzing crashes
• Marking duplicates

• Investigation Commands: See investigation-commands.md when:
• Need specific command syntax
• Working with version control (git)
• Analyzing logs and databases
• Monitoring system resources
• Debugging applications
• Using container tools (Docker, Kubernetes)

Bug Analysis Techniques

Root Cause Analysis Methods

1. Five Whys Technique

Ask "Why?" repeatedly to drill down to root cause:

Example:

• Bug: User login fails
• Why? Authentication token is invalid
• Why? Token expired before validation
• Why? Token lifetime is too short
• Why? Configuration set to 5 minutes instead of 30 minutes
• Why? Default configuration was not updated during deployment
• Root Cause: Missing deployment checklist item for configuration validation

2. Stack Trace Analysis

Steps:

1. Identify Exception Type

• What kind of error occurred?
• Is it a system exception or application exception?

2. Find Failure Point

• Which line/method threw the exception?
• What was the code doing at that point?

3. Trace Call Chain

• How did execution reach this point?
• What were the calling methods?
• What were the inputs?

4. Identify Root Caller

• What user action triggered this?
• What was the entry point?

Example Analysis:

Exception: NullPointerException at UserService.getUserProfile(UserService.java:45)
  at ProfileController.getProfile(ProfileController.java:23)
  at HttpRequest.handle(HttpRequest.java:102)

Analysis:
- Line 45 in UserService.getUserProfile() tried to access null object
- Called from ProfileController.getProfile() at line 23
- Triggered by HTTP request handler
- Likely: user ID passed to getUserProfile was valid but user record was null
- Root cause: Missing null check after database query

3. Differential Analysis

Compare working vs non-working scenarios:

What Changed?

• Code changes (git diff)
• Configuration changes
• Data changes
• Environment changes
• Dependency updates

Matrix Approach:

Analysis: Issue only in production with real data → likely data-specific problem

4. Reproduction Analysis

Systematic Reproduction:

1. Establish Baseline

• Can you reproduce the bug?
• How consistently? (Always/Sometimes/Rare)

2. Vary One Factor at a Time

• Different user accounts
• Different data inputs
• Different browsers/devices
• Different times/loads
• Different environment configs

3. Identify Minimal Reproduction

• What's the simplest way to reproduce?
• What's the minimal test case?
• What are the exact prerequisites?

4. Document Reproduction Steps

• Clear, numbered steps
• Expected vs actual result
• Screenshots or videos
• Relevant system state

5. Timing Analysis

For race conditions and performance issues:

Questions to Ask:

• Does the bug occur at specific times?
• Is it load-dependent?
• Does it happen during concurrent operations?
• Is there a delay or timing window?

Investigation Techniques:

• Add logging with timestamps
• Use debugger with breakpoints
• Profile execution time
• Check for locks and synchronization
• Review async operations

6. Data Flow Analysis

Track data through the system:

Steps:

1. Identify Input Data

• What data enters the system?
• What format is it in?

2. Trace Transformations

• How is data processed at each step?
• What validations are applied?
• What conversions occur?

3. Check State Changes

• How does data change?
• What's stored in database?
• What's cached?

4. Verify Output

• What data is returned?
• Is it correct?
• Where did it diverge from expected?

Example:

Input: {"amount": "1,234.56"}
→ Parser: parseFloat("1,234.56") = NaN (Bug: doesn't handle comma)
→ Expected: Remove comma first: "1234.56" → 1234.56

---

Evidence Collection

Log Analysis

What to Look For:

• Error messages before the bug
• Warnings that might indicate issues
• Unusual patterns or frequencies
• Missing expected log entries
• Correlation with other events

Techniques:

• Search for error patterns: grep -i "error" app.log
• Find related errors: grep -A 5 -B 5 "specific_error" app.log
• Count occurrences: grep "error" app.log | wc -l
• Timeline analysis: Sort by timestamp and look for sequences

Database Analysis

Queries to Run:

• Check data integrity
• Look for missing or duplicate records
• Verify relationships (foreign keys)
• Check for unexpected null values
• Review recent data changes

Example Queries:

-- Find orphaned records
SELECT * FROM orders WHERE user_id NOT IN (SELECT id FROM users);

-- Check for duplicates
SELECT email, COUNT(*) FROM users GROUP BY email HAVING COUNT(*) > 1;

-- Find recent problematic data
SELECT * FROM transactions WHERE status = 'failed' AND created_at > NOW() - INTERVAL 1 DAY;

Performance Analysis

Profiling:

• CPU profiling (identify hot spots)
• Memory profiling (identify leaks)
• I/O profiling (identify bottlenecks)
• Database query profiling (slow queries)

Metrics to Collect:

• Response times (p50, p95, p99)
• Throughput (requests per second)
• Error rates
• Resource utilization (CPU, memory, disk)

Network Analysis

Tools:

• Packet capture (Wireshark, tcpdump)
• Network monitoring (latency, packet loss)
• API request/response logging
• Browser developer tools (Network tab)

What to Check:

• Request/response headers
• Payload sizes
• HTTP status codes
• Timing (DNS, connection, SSL, transfer)
• Failed requests

---

Hypothesis Testing

Forming Hypotheses

Good Hypothesis Characteristics:

• Specific and testable
• Based on evidence
• Falsifiable
• Explains observed symptoms

Example:

• ❌ Bad: "There's a bug in the code"
• ✅ Good: "The null pointer exception occurs when getUserProfile() is called with a valid user ID but the user record has been soft-deleted, because the query doesn't filter deleted records"

Testing Hypotheses

1. Code Review

• Read the suspected code section
• Look for the hypothesized issue
• Check for similar patterns elsewhere

2. Experiment

• Modify code to test hypothesis
• Add logging to verify assumptions
• Create unit test that should fail

3. Verify

• Does the fix resolve the issue?
• Does it explain all symptoms?
• Does it work in all scenarios?

Multiple Hypotheses

When multiple possible causes exist:

Prioritize by:

• Likelihood (how probable)
• Impact (how severe if true)
• Ease of testing (quick to verify)

Document All Hypotheses:

• Hypothesis description
• Evidence supporting it
• Test performed
• Result (confirmed/rejected)

---

Common Root Cause Patterns

Null/Undefined Errors

Symptoms:

• NullPointerException, TypeError
• "Cannot read property of undefined"
• Unexpected null values

Common Causes:

• Missing null checks
• Uninitialized variables
• Incorrect default values
• Optional fields not handled
• Database query returning null

Investigation:

• Where was the variable assigned?
• What conditions lead to null?
• Are all code paths checked?

Race Conditions

Symptoms:

• Intermittent failures
• Works most of the time
• Fails under load
• Timing-dependent

Common Causes:

• Concurrent access to shared resources
• Missing synchronization
• Incorrect locking
• Atomic operation violations

Investigation:

• What resources are shared?
• Are there multiple threads/processes?
• Is synchronization proper?
• Review async operations

Memory Leaks

Symptoms:

• Memory usage grows over time
• Out of memory errors
• Performance degradation
• System slowdown

Common Causes:

• Objects not being garbage collected
• Event listeners not removed
• Circular references
• Cache without eviction
• Unclosed connections/resources

Investigation:

• Memory profiling over time
• Check for retained objects
• Review lifecycle management
• Look for resource cleanup

Configuration Issues

Symptoms:

• Works in some environments, not others
• Fails after deployment
• Environment-specific behavior

Common Causes:

• Missing configuration values
• Incorrect environment variables
• Wrong API endpoints
• Credential issues
• Feature flags misconfigured

Investigation:

• Compare configurations across environments
• Verify all required configs present
• Check configuration precedence
• Review deployment logs

Dependency Problems

Symptoms:

• Works locally but fails in production
• "Cannot find module" errors
• Incompatible versions
• Unexpected behavior after updates

Common Causes:

• Version mismatches
• Transitive dependency conflicts
• Breaking changes in dependencies
• Missing dependencies

Investigation:

• Check package lock files
• Review dependency versions
• Check for breaking changes in changelogs
• Test with specific dependency versions

---

Debugging Strategies

Divide and Conquer

Break down the problem:

1. Identify the system boundaries (input → output) 2. Find the midpoint in the flow 3. Check if data is correct at midpoint 4. If yes: bug is in second half; if no: bug is in first half 5. Repeat until bug is isolated

Binary Search Debugging

For finding which commit introduced a bug:

Use git bisect:

git bisect start
git bisect bad  # Current version has bug
git bisect good <commit>  # Known good commit
# Git will checkout commits to test
# Mark each as good or bad
git bisect good/bad
# Git will find the problematic commit

Rubber Duck Debugging

Explain the problem out loud:

1. Describe what the code should do 2. Explain what it actually does 3. Walk through the logic step by step 4. Often you'll spot the issue while explaining

Add Logging

Strategic logging:

• Log inputs at function entry
• Log outputs before function exit
• Log intermediate values in complex calculations
• Log error paths
• Log state changes

Good log message:

// Bad
console.log(user);

// Good
console.log('[UserService.getProfile] Fetching profile for userId:', userId, 'timestamp:', Date.now());

Breakpoint Debugging

Use debugger effectively:

• Set breakpoint before suspected issue
• Step through code line by line
• Inspect variable values
• Check call stack
• Watch expressions
• Conditional breakpoints for specific cases

---

Anti-Patterns to Avoid

Random Changes

❌ Making changes without understanding root cause ✅ Form hypothesis, test it, verify fix

Incomplete Investigation

❌ Stopping at first potential cause ✅ Verify the cause explains all symptoms

Fixing Symptoms

❌ Patching the visible issue without fixing root cause ✅ Address the underlying problem

No Reproduction

❌ Attempting to fix without reproducing ✅ Always reproduce before fixing

One-Size-Fits-All

❌ Applying the same fix pattern to all similar bugs ✅ Analyze each bug individually

Ignoring Evidence

❌ Dismissing data that doesn't fit your hypothesis ✅ Follow the evidence wherever it leads

Investigation Commands and Tools

Version Control Investigation

Git Commands for Bug Investigation

Find when a bug was introduced:

# Binary search to find problematic commit
git bisect start
git bisect bad                    # Current version has bug
git bisect good v1.2.0           # Known good version
# Test each commit git shows
git bisect good/bad              # Mark each commit
git bisect reset                 # When done

# Find when specific line was changed
git blame path/to/file.js

# Show changes to specific file
git log -p path/to/file.js

# Find commits mentioning specific text
git log --all --grep="bug keyword"

# Show commits in date range
git log --since="2024-01-01" --until="2024-01-15"

# Show commits by author
git log --author="john@example.com"

# Show file history with renames
git log --follow path/to/file.js

# Compare branches
git diff main..feature-branch path/to/file.js

# Show what changed in a commit
git show abc123def

# Find who changed specific lines
git log -S "function_name" -p path/to/file.js

---

Log Analysis

Searching Logs

Basic grep patterns:

# Find errors in logs
grep -i "error" /var/log/app.log

# Case-insensitive search
grep -i "exception" app.log

# Show context (5 lines before and after)
grep -A 5 -B 5 "error" app.log

# Recursive search in directory
grep -r "NullPointerException" /var/log/

# Count occurrences
grep "error" app.log | wc -l

# Show only matching part
grep -o "Error: [^$]*" app.log

# Multiple patterns
grep -E "error|exception|fatal" app.log

# Exclude patterns
grep "error" app.log | grep -v "IgnoredError"

# Search with line numbers
grep -n "error" app.log

Advanced log analysis:

# Find errors in last hour
grep "$(date -d '1 hour ago' '+%Y-%m-%d %H')" app.log | grep "ERROR"

# Sort and count error types
grep "ERROR" app.log | sort | uniq -c | sort -rn

# Extract timestamps of errors
grep "ERROR" app.log | cut -d' ' -f1-2

# Find correlations between errors
grep -A 10 "DatabaseError" app.log | grep "ConnectionTimeout"

# Analyze error patterns over time
for hour in {00..23}; do
  echo "Hour $hour: $(grep "2024-01-15 $hour:" app.log | grep ERROR | wc -l)"
done

Log Parsing with awk

# Extract specific fields
awk '{print $1, $4, $5}' access.log

# Filter by condition
awk '$9 >= 400 {print $0}' access.log

# Calculate statistics
awk '{sum+=$10; count++} END {print "Average:", sum/count}' access.log

# Group and count
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -10

Monitoring Logs in Real-time

# Tail logs with follow
tail -f /var/log/app.log

# Tail multiple files
tail -f /var/log/app.log /var/log/error.log

# Tail with grep
tail -f app.log | grep --line-buffered "ERROR"

# Tail last 100 lines
tail -n 100 app.log

# Watch log file size
watch -n 5 'ls -lh /var/log/app.log'

---

Database Investigation

PostgreSQL

-- Find slow queries
SELECT query, calls, total_time, mean_time
FROM pg_stat_statements
ORDER BY mean_time DESC
LIMIT 20;

-- Check table sizes
SELECT schemaname, tablename, 
       pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) AS size
FROM pg_tables
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC
LIMIT 10;

-- Check index usage
SELECT schemaname, tablename, indexname, idx_scan
FROM pg_stat_user_indexes
WHERE idx_scan = 0
ORDER BY pg_relation_size(indexrelid) DESC;

-- Find missing indexes
SELECT schemaname, tablename, seq_scan, seq_tup_read,
       idx_scan, seq_tup_read / seq_scan AS avg_tuples
FROM pg_stat_user_tables
WHERE seq_scan > 0
ORDER BY seq_tup_read DESC
LIMIT 10;

-- Check database connections
SELECT datname, usename, application_name, client_addr, state, query
FROM pg_stat_activity
WHERE state != 'idle';

-- Find blocking queries
SELECT blocked_locks.pid AS blocked_pid,
       blocking_locks.pid AS blocking_pid,
       blocked_activity.query AS blocked_statement,
       blocking_activity.query AS blocking_statement
FROM pg_catalog.pg_locks blocked_locks
JOIN pg_catalog.pg_stat_activity blocked_activity ON blocked_activity.pid = blocked_locks.pid
JOIN pg_catalog.pg_locks blocking_locks ON blocking_locks.locktype = blocked_locks.locktype
JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid
WHERE NOT blocked_locks.granted;

-- Check for deadlocks
SELECT * FROM pg_stat_database WHERE datname = 'your_db';

-- Analyze table statistics
ANALYZE tablename;

-- Check vacuum status
SELECT schemaname, tablename, last_vacuum, last_autovacuum
FROM pg_stat_user_tables;

MySQL

-- Show slow queries
SELECT * FROM mysql.slow_log ORDER BY query_time DESC LIMIT 20;

-- Current queries
SHOW FULL PROCESSLIST;

-- Table sizes
SELECT table_schema, table_name,
       ROUND((data_length + index_length) / 1024 / 1024, 2) AS size_mb
FROM information_schema.tables
ORDER BY (data_length + index_length) DESC
LIMIT 10;

-- Index usage
SELECT * FROM sys.schema_unused_indexes;

-- Deadlocks
SHOW ENGINE INNODB STATUS;

-- Connection count
SHOW STATUS LIKE 'Threads_connected';

MongoDB

// Find slow queries
db.system.profile.find({millis: {$gt: 100}}).sort({millis: -1}).limit(10)

// Collection stats
db.collection.stats()

// Index usage
db.collection.aggregate([{$indexStats: {}}])

// Current operations
db.currentOp()

// Kill long-running operation
db.killOp(operationId)

// Database statistics
db.stats()

---

System Resource Monitoring

Linux/macOS

Process monitoring:

# Top processes by CPU
top
# Interactive mode: Press '1' for per-CPU, 'M' for memory sort

# Process list with details
ps aux | sort -rk 3 | head -10  # By CPU
ps aux | sort -rk 4 | head -10  # By memory

# Monitor specific process
watch -n 1 'ps aux | grep processname'

# Process tree
pstree -p

# Detailed process info
lsof -p <PID>  # Open files
strace -p <PID>  # System calls

Memory analysis:

# Memory usage
free -h

# Detailed memory info
cat /proc/meminfo

# Memory by process
ps aux --sort=-%mem | head

# Check for memory leaks
valgrind --leak-check=full ./program

# Memory map of process
pmap <PID>

Disk analysis:

# Disk usage
df -h

# Directory sizes
du -sh */ | sort -rh | head -10

# Find large files
find /path -type f -size +100M -exec ls -lh {} \;

# Disk I/O
iostat -x 1

# Who is using disk
iotop

Network analysis:

# Network connections
netstat -tulpn  # Listening ports
netstat -an | grep ESTABLISHED  # Active connections

# Network traffic
tcpdump -i eth0 port 80
tcpdump -i eth0 -w capture.pcap

# Bandwidth usage
iftop -i eth0

# DNS lookup
nslookup domain.com
dig domain.com

# Trace route
traceroute domain.com

# Check port connectivity
telnet hostname port
nc -zv hostname port

# HTTP request debugging
curl -v https://api.example.com

---

Application-Specific Tools

Node.js

Debugging:

# Run with debugger
node --inspect app.js
node --inspect-brk app.js  # Break on first line

# Memory profiling
node --inspect --expose-gc app.js

# CPU profiling
node --prof app.js
node --prof-process isolate-0x*.log

# Heap snapshot
node --heapsnapshot-signal=SIGUSR2 app.js
# Then: kill -SIGUSR2 <PID>

Package debugging:

# List installed packages
npm ls

# Check for vulnerabilities
npm audit

# Check for updates
npm outdated

# View package info
npm info package-name

# Check peer dependencies
npm ls package-name

Python

Debugging:

# Run with debugger
python -m pdb script.py

# Profile execution
python -m cProfile -s cumtime script.py

# Memory profiling
python -m memory_profiler script.py

# Line profiling
kernprof -l script.py
python -m line_profiler script.py.lprof

Package debugging:

# List installed packages
pip list

# Show package details
pip show package-name

# Check for updates
pip list --outdated

# Verify dependencies
pip check

Java

Debugging:

# Heap dump
jmap -dump:format=b,file=heap.bin <PID>

# Thread dump
jstack <PID> > threads.txt

# GC monitoring
jstat -gc <PID> 1000

# Java process info
jps -lvm

# Flight recorder
jcmd <PID> JFR.start duration=60s filename=recording.jfr

Docker

Container debugging:

# Container logs
docker logs container_name
docker logs -f --tail 100 container_name

# Execute command in container
docker exec -it container_name bash
docker exec -it container_name sh

# Container stats
docker stats container_name

# Inspect container
docker inspect container_name

# Check container processes
docker top container_name

# Copy files from container
docker cp container_name:/path/to/file ./local/path

# Container events
docker events --filter container=container_name

Kubernetes

Pod debugging:

# Get pod logs
kubectl logs pod-name
kubectl logs -f pod-name  # Follow
kubectl logs --previous pod-name  # Previous instance

# Execute command in pod
kubectl exec -it pod-name -- bash

# Describe pod (check events)
kubectl describe pod pod-name

# Get pod details
kubectl get pod pod-name -o yaml

# Port forward for debugging
kubectl port-forward pod-name 8080:80

# Debug with ephemeral container
kubectl debug pod-name -it --image=busybox

# Check pod resources
kubectl top pods

# Get events
kubectl get events --sort-by=.metadata.creationTimestamp

---

Performance Analysis Tools

Web Performance

Browser DevTools:

• Network tab: Request timing, headers, payload
• Performance tab: Timeline, CPU/memory usage
• Lighthouse: Performance audit

Command-line tools:

# Load testing
ab -n 1000 -c 10 https://example.com/api

# Advanced load testing
wrk -t4 -c100 -d30s https://example.com

# HTTP benchmarking
siege -c 50 -t 1M https://example.com

# DNS timing
time nslookup example.com

# SSL handshake timing
openssl s_time -connect example.com:443

APM Tools

Common Application Performance Monitoring tools:

• New Relic: Transaction tracing, error tracking
• Datadog: Metrics, logs, traces correlation
• AppDynamics: Business transaction monitoring
• Dynatrace: AI-powered root cause analysis
• Elastic APM: Open-source APM with ELK stack

Profilers

• Node.js: node --prof, clinic.js, 0x
• Python: cProfile, py-spy, austin
• Java: JProfiler, YourKit, VisualVM
• Ruby: ruby-prof, stackprof
• .NET: dotTrace, PerfView

---

Automation Scripts

Bug Reproduction Script Template

#!/bin/bash
# Bug reproduction script for BUG-XXXX

set -e  # Exit on error

echo "Setting up environment..."
# Setup steps
export ENV_VAR=value
cd /path/to/project

echo "Starting services..."
# Start required services
docker-compose up -d

echo "Waiting for services..."
sleep 5

echo "Running reproduction steps..."
# Step 1
curl -X POST https://api.example.com/endpoint -d '{"data":"value"}'

# Step 2
./run_script.sh

# Step 3
# Check for bug
if grep -q "ERROR" /var/log/app.log; then
    echo "BUG REPRODUCED: Error found in logs"
    grep "ERROR" /var/log/app.log
    exit 1
else
    echo "Bug not reproduced"
    exit 0
fi

Log Collection Script

#!/bin/bash
# Collect logs and diagnostic info for bug report

BUG_ID="BUG-XXXX"
OUTPUT_DIR="bug_report_${BUG_ID}_$(date +%Y%m%d_%H%M%S)"

mkdir -p "$OUTPUT_DIR"

echo "Collecting logs for $BUG_ID..."

# Application logs
cp /var/log/app/*.log "$OUTPUT_DIR/"

# System info
uname -a > "$OUTPUT_DIR/system_info.txt"
df -h > "$OUTPUT_DIR/disk_usage.txt"
free -h > "$OUTPUT_DIR/memory_usage.txt"

# Process info
ps aux > "$OUTPUT_DIR/processes.txt"

# Network info
netstat -tulpn > "$OUTPUT_DIR/network.txt"

# Docker info (if applicable)
if command -v docker &> /dev/null; then
    docker ps > "$OUTPUT_DIR/docker_containers.txt"
    docker logs app_container > "$OUTPUT_DIR/docker_app.log"
fi

# Package to send
tar -czf "${OUTPUT_DIR}.tar.gz" "$OUTPUT_DIR"
echo "Logs collected in ${OUTPUT_DIR}.tar.gz"

---

Cheat Sheet

Quick Diagnosis Commands

# Is the service running?
systemctl status service_name

# What's using all the CPU?
top -bn1 | head -20

# What's using all the memory?
ps aux --sort=-%mem | head -10

# What's using all the disk?
du -sh /* | sort -rh | head -10

# What's the network doing?
netstat -tunlp

# What's in the logs?
tail -100 /var/log/app.log | grep -i error

# What changed recently?
git log --since="1 day ago" --oneline

# What's the error rate?
grep ERROR /var/log/app.log | wc -l

Bug Report Templates

Standard Bug Report Template

## Bug Report

## Summary
**Bug ID**: [BUG-XXXX]
**Title**: [Concise, descriptive title]
**Reporter**: [Name/Email]
**Date**: [YYYY-MM-DD]
**Severity**: [Critical/High/Medium/Low]
**Priority**: [P0/P1/P2/P3]
**Type**: [Functional/Performance/Security/UI/Data/Integration/Configuration/Regression]
**Status**: [New/Confirmed/In Progress/Fixed/Closed]

### Description
[Clear, detailed description of the bug including what's wrong and why it matters]

### Environment
- **Operating System**: [Windows 10/macOS 13/Ubuntu 22.04/etc.]
- **Browser/Client**: [Chrome 120/Firefox 121/Safari 17/Mobile App 2.1]
- **Application Version**: [v2.3.1]
- **Environment**: [Production/Staging/Development]
- **Configuration**: [Any relevant settings or features enabled]
- **Other**: [Database version, API version, etc.]

### Reproduction Steps
1. [First action]
2. [Second action]
3. [Continue with numbered steps]
4. [Observe the result]

**Expected Behavior**: [What should happen]

**Actual Behavior**: [What actually happens]

**Reproducibility**: [Always (100%) / Intermittent (50%) / Rare (<10%)]

### Supporting Evidence
- **Screenshots**: [Attach or link]
- **Videos**: [Screen recording if helpful]
- **Error Messages**: 

[Paste exact error messages or stack traces]

- **Logs**:

[Relevant log entries with timestamps]

- **Network Requests**: [API calls, responses]

### Impact Assessment
**Users Affected**: [Number, percentage, or user segment description]
**Frequency**: [How often does this occur?]
**Business Impact**: [Revenue loss, SLA violation, customer satisfaction, etc.]
**Workaround Available**: [Yes/No - describe if yes]

### Related Information
- **Related Issues**: [Links to similar bugs, duplicates, or related features]
- **Recent Changes**: [Recent deployments, config changes, or updates that might be related]
- **External References**: [Documentation, API specs, design docs]

---

Root Cause Analysis Report Template

## Root Cause Analysis

### Bug Summary
**Bug ID**: [BUG-XXXX]
**Title**: [Bug title]
**Analyzed By**: [Your name]
**Analysis Date**: [YYYY-MM-DD]

### Symptoms
[Description of observable symptoms and behavior]

### Root Cause
**Primary Cause**: [Main underlying cause of the bug]

**Technical Details**:
- **Location**: [File path, method name, line numbers]
- **Issue**: [Specific code/logic problem]
- **Contributing Factors**: [Additional factors that contributed]
- **Why It Occurred**: [Explanation of mechanism]

### Evidence
**Error Messages**:

[Stack traces, error logs]

**Code Reference**:

// Problematic code [paste relevant code snippet]

**Data Evidence**:

- [Database queries showing problematic data]
- [API responses showing issues]
- [Log entries showing sequence of events]

**Timeline**:

- [When bug was introduced]
- [When it was first reported]
- [Pattern of occurrences]

### Analysis Process

1. **Initial Hypothesis**: [What you initially suspected]
2. **Investigation Steps**: [What you checked and tested]
3. **Findings**: [What you discovered]
4. **Conclusion**: [How you confirmed the root cause]

### Impact Analysis

**User Impact**:

- Affected users: [number/percentage]
- User workflows disrupted: [description]
- User experience impact: [severity description]

**Business Impact**:

- Revenue impact: [$amount or percentage]
- SLA violations: [Yes/No - details]
- Customer satisfaction: [description]
- Reputation risk: [assessment]

**System Impact**:

- Performance degradation: [metrics]
- Resource consumption: [details]
- Cascading failures: [related systems affected]
- Data integrity: [any data issues]

**Security Impact**:

- Confidentiality: [any data exposure]
- Integrity: [any unauthorized modifications]
- Availability: [any service disruptions]

### Recommended Fix

**Immediate Mitigation** (if not already done):

1. [Emergency fix or workaround]
2. [Configuration changes]
3. [Feature flag to disable problematic code]
4. [Rollback considerations]

**Permanent Solution**:

**Code Changes Required**:

// Before (problematic code) [current code]

// After (proposed fix) [fixed code with explanation]

**Files to Modify**:

- `src/path/to/file1.js` - [description of changes]
- `src/path/to/file2.js` - [description of changes]

**Database Changes** (if any):

-- Migration script or data fixes [SQL or migration commands]

**Configuration Changes** (if any):

- [Environment variables to update]
- [Config files to modify]

**Testing Requirements**:

- **Unit Tests**:
  - [Test case 1]
  - [Test case 2]
- **Integration Tests**:
  - [Test scenario 1]
  - [Test scenario 2]
- **Regression Tests**:
  - [Test to prevent recurrence]
  - [Tests for related functionality]
- **Manual Testing**:
  - [Steps to verify fix in each environment]

**Deployment Plan**:

1. [Deploy to dev/test environment]
2. [Verify fix works]
3. [Deploy to staging]
4. [Run regression tests]
5. [Deploy to production during maintenance window]
6. [Monitor for issues]

**Rollback Plan**:
[Steps to rollback if fix causes issues]

### Prevention Measures

**Immediate Actions**:

- [ ] Add validation for [specific input]
- [ ] Add error handling for [specific scenario]
- [ ] Add logging for [specific events]
- [ ] Update documentation for [specific feature]

**Long-term Improvements**:

- **Code Quality**:
  - [Refactoring needed]
  - [Design improvements]
  - [Code review focus areas]
  
- **Testing**:
  - [Test coverage gaps to address]
  - [New test scenarios to add]
  - [Testing process improvements]
  
- **Monitoring**:
  - [Metrics to track]
  - [Alerts to add]
  - [Dashboards to create]
  
- **Process**:
  - [Development process improvements]
  - [Code review checklist updates]
  - [Deployment checklist updates]

- **Documentation**:
  - [Documentation gaps to fill]
  - [Knowledge base articles to create]
  - [Training needs identified]

### Lessons Learned

- [Key takeaway 1]
- [Key takeaway 2]
- [Patterns to watch for in future]

### Estimated Effort

**Fix Implementation**: [X hours/days]
**Testing**: [X hours/days]
**Deployment**: [X hours]
**Total**: [X hours/days]

**Resources Required**: [Team members, approvals, infrastructure access, etc.]

---

Security Vulnerability Report Template

## Security Vulnerability Analysis

### Classification
**CVE ID**: [If assigned]
**Severity**: [Critical/High/Medium/Low based on CVSS]
**CVSS Score**: [X.X]
**Type**: [SQL Injection/XSS/CSRF/Authentication Bypass/Authorization Issue/etc.]
**Affected Versions**: [Version range]

### Vulnerability Description
[Detailed description of the security issue]

### Attack Vector
**Attack Complexity**: [Low/High]
**Privileges Required**: [None/Low/High]
**User Interaction**: [None/Required]
**Scope**: [Unchanged/Changed]

**Attack Scenario**:
1. [Step-by-step description of how attack would work]
2. [What attacker needs]
3. [What attacker gains]

### Proof of Concept

[Code or commands demonstrating the vulnerability]

**Example Exploit**:
[Real or hypothetical example showing exploitation]

### Impact Assessment

**Confidentiality Impact**: [None/Low/High]

- [What data can be exposed]

**Integrity Impact**: [None/Low/High]

- [What data/systems can be modified]

**Availability Impact**: [None/Low/High]

- [What services can be disrupted]

**Scope**: [Limited to component / Affects other components]

### Current Security Controls

[What security measures are currently in place (if any)]

### Recommended Fix

**Immediate Actions** (within 24-48 hours):

1. [Emergency mitigation - WAF rules, rate limiting, etc.]
2. [Access restrictions]
3. [Monitoring for exploitation attempts]
4. [Incident response readiness]

**Permanent Fix**:

- [Secure code changes needed]
- [Security controls to implement]
- [Input validation requirements]
- [Output encoding requirements]
- [Authentication/authorization fixes]

**Secure Code Example**:

// Vulnerable code [problematic code]

// Secure code [fixed code with security best practices]

### Verification Testing

- [ ] Verify attack no longer works
- [ ] Test edge cases
- [ ] Verify no bypass methods
- [ ] Security scan results clean
- [ ] Penetration test passed

### Disclosure Plan

**Timeline**:

- Day 0: Vulnerability discovered
- Day 1-2: Initial analysis and triage
- Day 2-7: Develop and test fix
- Day 7-14: Deploy fix to production
- Day 30: Public disclosure (if applicable)

**Notifications**:

- [ ] Internal security team notified
- [ ] Development team notified
- [ ] Management notified
- [ ] Affected customers notified (if required)
- [ ] Public disclosure prepared (if required)

**Compliance Requirements**:

- [ ] GDPR notification (if data breach)
- [ ] PCI-DSS reporting (if payment data)
- [ ] HIPAA reporting (if health data)
- [ ] CVE request (if applicable)

### References

- [OWASP guidelines]
- [CWE reference]
- [Related CVEs]
- [Security advisories]

---

Performance Bug Report Template

## Performance Issue Analysis

### Performance Problem Summary
**Issue**: [Concise description of performance problem]
**Severity**: [Critical/High/Medium/Low]
**Affected Component**: [System/service/feature]

### Performance Metrics

**Current Performance**:
- Response Time: [p50: Xms, p95: Yms, p99: Zms]
- Throughput: [N requests/second]
- Error Rate: [X%]
- Resource Usage: [CPU: X%, Memory: Y GB, Disk: Z%]

**Expected Performance**:
- Response Time: [Target SLA]
- Throughput: [Target capacity]
- Error Rate: [Target < X%]

**Degradation**:
- Response time increased by: [X%]
- Throughput decreased by: [Y%]
- Started occurring: [Date/time]

### Reproduction
**Load Conditions**:
- Concurrent users: [N]
- Request rate: [X req/s]
- Data volume: [Y records]
- Duration: [Z minutes]

**Steps to Reproduce**:
1. [Setup conditions]
2. [Apply load]
3. [Measure metrics]
4. [Observe degradation]

### Performance Analysis

**Profiling Results**:
- **CPU Bottleneck**: [Function/method taking most CPU time]
- **Memory Issues**: [Memory usage patterns, leaks]
- **I/O Bottleneck**: [Disk/network operations]
- **Database**: [Slow queries, N+1 problems]

**Evidence**:

[Profiling output, slow query logs, flame graphs]

### Root Cause
**Primary Bottleneck**: [Specific performance issue]

**Technical Details**:
- [Algorithm complexity issue]
- [Inefficient query]
- [Missing index]
- [Resource leak]
- [Synchronous blocking]

### Recommended Optimization

**Quick Wins** (immediate improvements):
1. [Add database index]
2. [Increase cache TTL]
3. [Add query pagination]
4. [Enable compression]

**Long-term Solutions**:
- [Algorithm optimization]
- [Database schema changes]
- [Caching strategy]
- [Architecture changes]
- [Horizontal scaling]

**Expected Improvement**:
- Response time: [Reduce by X%]
- Throughput: [Increase by Y%]
- Resource usage: [Reduce by Z%]

### Testing Plan
- Load testing scenarios
- Stress testing requirements
- Performance benchmarks
- Monitoring during rollout

---

Crash Analysis Report Template

## Crash Analysis Report

### Crash Summary
**Crash ID**: [Unique identifier]
**Occurrence**: [First seen / Total occurrences / Frequency]
**Affected Versions**: [Version range]
**Platforms**: [OS/Browser/Device affected]

### Crash Details

**Exception/Signal**:

[Exception type, error code, signal number]

**Stack Trace**:

[Full stack trace from crash dump]

**Thread State**:

[Thread information, deadlocks, race conditions]

**Memory State**:
- Heap usage: [X MB]
- Stack usage: [Y KB]
- Memory pressure: [High/Normal/Low]

### Crash Trigger

**User Action**:
[What was the user doing when crash occurred]

**System Condition**:
- [Low memory]
- [High CPU load]
- [Network issue]
- [Specific data input]

**Timing**:
- [Time-dependent]
- [Load-dependent]
- [Sequence-dependent]

### Impact
- **Crash Rate**: [X crashes per Y sessions]
- **Users Affected**: [N users / X%]
- **Data Loss**: [Yes/No - description]
- **Recovery**: [Automatic/Manual/None]

### Root Cause
[Detailed explanation of what causes the crash]

### Fix Recommendation
**Immediate**:
- [Crash handling to prevent data loss]
- [Graceful degradation]
- [User notification]

**Permanent**:
- [Bug fix]
- [Resource management]
- [Error handling]
- [Crash reporting improvements]

### Prevention
- [Input validation]
- [Resource limits]
- [Error boundaries]
- [Monitoring and alerts]

---

Duplicate Bug Template

## Duplicate Bug Report

**This bug is a duplicate of**: [Link to original bug #XXXX]

### Verification
- [ ] Same symptoms observed
- [ ] Same component affected
- [ ] Same root cause (if known)
- [ ] Same reproduction steps

### Additional Information
[Any new information from this report that might be useful:]
- Different environment or configuration
- Additional affected versions
- Alternative reproduction steps
- More detailed error messages
- Different workaround suggestions

### Action Taken
- Merged comments and attachments to original issue
- Notified reporter and redirected to original
- Updated affected version list in original
- Added any new reproduction steps to original

Severity Guidelines and Triage

Severity Classification

Critical (P0)

Criteria:

• Complete system outage or crash
• Data loss or corruption
• Security breach or critical vulnerability
• Payment/transaction processing failure
• No workaround available
• Affects all or majority of users

Response Time: Immediate (< 1 hour)

Examples:

• Database connection failure causing site downtime
• Payment gateway not processing transactions
• User data exposed to unauthorized access
• System crashes on startup for all users
• Critical security vulnerability actively exploited

Required Actions:

• Immediate team notification
• Escalate to on-call engineer
• Create incident war room
• Implement emergency fix or rollback
• Post-mortem required

---

High (P1)

Criteria:

• Major feature completely broken
• Significant user impact (>25% of users)
• Moderate security vulnerability
• Difficult or complex workaround
• Revenue-impacting issue
• SLA violation risk

Response Time: Same day (< 4 hours)

Examples:

• Login failure for subset of users
• Core feature (search, checkout) not working
• API returning errors for multiple clients
• Performance degradation causing timeouts
• Data sync failures between systems

Required Actions:

• Assign to senior engineer
• Provide status updates every 2 hours
• Implement fix in hotfix branch
• Deploy outside normal release cycle
• Notify affected customers

---

Medium (P2)

Criteria:

• Feature partially broken
• Moderate user impact (<25% of users)
• Reasonable workaround available
• Non-critical functionality affected
• Minor performance degradation

Response Time: Within 1 week

Examples:

• Secondary feature not working correctly
• UI element not displaying properly
• Export feature failing for specific file types
• Search results occasionally incorrect
• Minor memory leak not causing immediate issues

Required Actions:

• Include in next sprint planning
• Fix in regular release cycle
• Add to regression test suite
• Document workaround in knowledge base

---

Low (P3)

Criteria:

• Minor issue or edge case
• Cosmetic problem
• Minimal user impact
• Easy workaround
• Nice-to-have enhancement

Response Time: Backlog/next release

Examples:

• Spelling/grammar errors
• Minor alignment issues
• Console warnings (non-breaking)
• Rare edge case handling
• Legacy browser compatibility

Required Actions:

• Add to backlog
• Fix when convenient
• May be combined with other small fixes
• Consider for future releases

---

Priority vs Severity Matrix

Bug Categories

By Type

Functional Bugs:

• Feature not working as specified
• Incorrect business logic
• Missing validation
• Wrong calculations or data processing

Performance Bugs:

• Slow response times
• High resource consumption
• Memory leaks
• Inefficient algorithms

Security Bugs:

• Authentication/authorization failures
• Data exposure vulnerabilities
• Injection vulnerabilities (SQL, XSS, etc.)
• Insecure configurations

UI/UX Bugs:

• Visual glitches or misalignments
• Broken layouts
• Incorrect styling
• Accessibility issues

Data Bugs:

• Data corruption
• Data loss
• Incorrect data transformation
• Data inconsistency between systems

Integration Bugs:

• API failures
• Third-party service issues
• Message queue problems
• Webhook failures

Configuration Bugs:

• Environment-specific issues
• Incorrect settings
• Deployment problems
• Infrastructure issues

Regression Bugs:

• Previously working feature broken
• New code breaks existing functionality
• Deployment breaks production

By Root Cause Pattern

Logic Errors:

• Incorrect conditional logic
• Missing edge cases
• Wrong algorithm implementation
• Calculation errors

Race Conditions:

• Concurrent access issues
• Thread safety problems
• Timing-dependent bugs

Resource Issues:

• Memory leaks
• Connection pool exhaustion
• File handle leaks
• Deadlocks

Null/Undefined Errors:

• Null pointer exceptions
• Undefined variables
• Missing null checks

Boundary Conditions:

• Off-by-one errors
• Array index out of bounds
• Integer overflow
• Empty collection handling

Integration Failures:

• API contract changes
• Version incompatibilities
• Network failures
• Timeout issues

---

Triage Process

Step 1: Initial Assessment (5 minutes)

1. Read bug description 2. Check if reproducible 3. Identify affected component 4. Assess initial severity 5. Check for duplicates

Step 2: Categorization (5 minutes)

1. Assign bug type 2. Identify root cause pattern (if obvious) 3. Tag with relevant labels 4. Assign to appropriate team/component

Step 3: Priority Assignment (5 minutes)

Consider:

• Number of users affected
• Business impact (revenue, reputation)
• Security implications
• Workaround availability
• Effort to fix

Step 4: Assignment (2 minutes)

• P0/P1: Assign immediately to on-call or senior engineer
• P2: Add to current sprint
• P3: Add to backlog

Step 5: Stakeholder Communication

• P0/P1: Notify stakeholders immediately
• P2: Include in sprint planning
• P3: Update in weekly summary

---

Special Considerations

Security Vulnerabilities

Always escalate to security team for:

• Authentication bypasses
• Authorization failures
• Data exposure
• Injection vulnerabilities
• Known CVEs

Use CVSS scoring:

• 9.0-10.0: Critical
• 7.0-8.9: High
• 4.0-6.9: Medium
• 0.1-3.9: Low

Production vs Non-Production

Production bugs:

• Higher priority by default
• Require faster response
• May need hotfix deployment

Non-production bugs:

• Can wait for regular release
• Use for testing improvements
• May indicate test coverage gaps

Customer-Reported vs Internal

Customer-reported:

• Higher priority (affects users)
• Requires customer communication
• May need workaround documentation

Internal:

• Can be fixed proactively
• Opportunity to prevent customer impact
• Use for quality improvements