Python Performance

# Benchmarking Tools

Tools and techniques for benchmarking Python code performance.

## Custom Benchmark Decorator

```python
import time
from functools import wraps

def benchmark(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{func.__name__}: {elapsed:.6f}s")
        return result
    return wrapper

@benchmark
def my_function():
    # Code to benchmark
    pass
```

## pytest-benchmark

```python
# Install: pip install pytest-benchmark

def test_list_comprehension(benchmark):
    result = benchmark(lambda: [i**2 for i in range(10000)])
    assert len(result) == 10000

# Run: pytest --benchmark-compare
```


---
name: best-practices
description: Best practices and common pitfalls for Python performance optimization
category: performance
tags: [best-practices, guidelines, optimization, pitfalls]
dependencies: [profiling-tools, optimization-patterns]
estimated_tokens: 207
---

# Best Practices

Guidelines and common pitfalls for Python performance optimization.

## Best Practices

1. **Profile before optimizing** - Measure to find real bottlenecks
2. **Focus on hot paths** - Optimize frequently executed code
3. **Use appropriate data structures** - Dict for lookups, set for membership
4. **Cache expensive computations** - Use lru_cache
5. **Batch I/O operations** - Reduce system calls
6. **Use generators** for large datasets
7. **Consider NumPy** for numerical operations

## Common Pitfalls

- Optimizing without profiling
- Using global variables unnecessarily
- Creating unnecessary copies of data
- Not using connection pooling
- Ignoring algorithmic complexity
- Over-optimizing rare code paths

## Exit Criteria

- Profiled code to identify bottlenecks
- Applied appropriate optimization patterns
- Verified improvements with benchmarks
- Memory usage acceptable
- No performance regressions


---
name: memory-management
description: Memory optimization techniques including leak tracking with tracemalloc and weak references for caches
category: performance
tags: [memory, optimization, tracemalloc, weak-references]
dependencies: [profiling-tools]
estimated_tokens: 167
---

# Memory Management

Techniques for optimizing memory usage and preventing memory leaks.

## Tracking Memory Leaks

```python
import tracemalloc

tracemalloc.start()

# Your code here
result = memory_intensive_operation()

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')

print("Top 10 memory allocations:")
for stat in top_stats[:10]:
    print(stat)
```

## Weak References for Caches

```python
import weakref

# Allows garbage collection when no strong references
weak_cache = weakref.WeakValueDictionary()

def get_resource(key):
    resource = weak_cache.get(key)
    if resource is None:
        resource = create_expensive_resource(key)
        weak_cache[key] = resource
    return resource
```


---
name: optimization-patterns
description: Ten proven optimization patterns for Python code including list comprehensions, generators, caching, and parallel processing
category: performance
tags: [optimization, patterns, performance, best-practices]
dependencies: []
estimated_tokens: 811
---

# Optimization Patterns

Proven patterns for optimizing Python code performance.

## Pattern 1: List Comprehensions vs Loops

```python
# Slow
def slow_squares(n):
    result = []
    for i in range(n):
        result.append(i**2)
    return result

# Fast (~2x speedup)
def fast_squares(n):
    return [i**2 for i in range(n)]
```

## Pattern 2: Generator Expressions for Memory

```python
import sys

# Memory-intensive
list_data = [i**2 for i in