Python

Python 3.11 Best Practices

Comprehensive performance optimization guide for Python 3.11+ applications. Contains 42 rules across 8 categories, prioritized by impact to guide automated refactoring and code generation.

When to Apply

Reference these guidelines when:

• Writing new Python async I/O code
• Choosing data structures for collections
• Optimizing memory usage in data-intensive applications
• Implementing concurrent or parallel processing
• Reviewing Python code for performance issues

Rule Categories by Priority

Table of Contents

1. I/O & Async Patterns — CRITICAL

• 1.1 Defer await Until Value Needed — CRITICAL (2-5× faster for dependent operations)
• 1.2 Use aiofiles for Async File Operations — CRITICAL (prevents event loop blocking)
• 1.3 Use asyncio.gather() for Concurrent I/O — CRITICAL (2-10× throughput improvement)
• 1.4 Use Connection Pooling for Database Access — CRITICAL (100-200ms saved per connection)
• 1.5 Use Semaphores to Limit Concurrent Operations — CRITICAL (prevents resource exhaustion)
• 1.6 Use uvloop for Faster Event Loop — CRITICAL (2-4× faster async I/O)

2. Data Structure Selection — CRITICAL

• 2.1 Use bisect for O(log n) Sorted List Operations — CRITICAL (O(n) to O(log n) search)
• 2.2 Use defaultdict to Avoid Key Existence Checks — CRITICAL (eliminates redundant lookups)
• 2.3 Use deque for O(1) Queue Operations — CRITICAL (O(n) to O(1) for popleft)
• 2.4 Use Dict for O(1) Key-Value Lookup — CRITICAL (O(n) to O(1) lookup)
• 2.5 Use frozenset for Hashable Set Keys — CRITICAL (enables set-of-sets patterns)
• 2.6 Use Set for O(1) Membership Testing — CRITICAL (O(n) to O(1) lookup)

3. Memory Optimization — HIGH

• 3.1 Intern Repeated Strings to Save Memory — HIGH (reduces duplicate string storage)
• 3.2 Use __slots__ for Memory-Efficient Classes — HIGH (20-50% memory reduction per instance)
• 3.3 Use array.array for Homogeneous Numeric Data — HIGH (4-8× memory reduction for numbers)
• 3.4 Use Generators for Large Sequences — HIGH (100-1000× memory reduction)
• 3.5 Use weakref for Caches to Prevent Memory Leaks — HIGH (prevents unbounded cache growth)

4. Concurrency & Parallelism — HIGH

• 4.1 Use asyncio for I/O-Bound Concurrency — HIGH (300% throughput improvement for I/O)
• 4.2 Use multiprocessing for CPU-Bound Parallelism — HIGH (4-8× speedup on multi-core systems)
• 4.3 Use Queue for Thread-Safe Communication — HIGH (prevents race conditions)
• 4.4 Use TaskGroup for Structured Concurrency — HIGH (prevents resource leaks on failure)
• 4.5 Use ThreadPoolExecutor for Blocking Calls in Async — HIGH (prevents event loop blocking)

5. Loop & Iteration — MEDIUM

• 5.1 Hoist Loop-Invariant Computations — MEDIUM (avoids N× redundant work)
• 5.2 Use any() and all() for Boolean Aggregation — MEDIUM (O(n) to O(1) best case)
• 5.3 Use dict.items() for Key-Value Iteration — MEDIUM (single lookup vs double lookup)
• 5.4 Use enumerate() for Index-Value Iteration — MEDIUM (cleaner code, avoids index errors)
• 5.5 Use itertools for Efficient Iteration Patterns — MEDIUM (2-3× faster iteration patterns)
• 5.6 Use List Comprehensions Over Explicit Loops — MEDIUM (2-3× faster iteration)

6. String Operations — MEDIUM

• 6.1 Use f-strings for Simple String Formatting — MEDIUM (20-30% faster than .format())
• 6.2 Use join() for Multiple String Concatenation — MEDIUM (4× faster for 5+ strings)
• 6.3 Use str.startswith() with Tuple for Multiple Prefixes — MEDIUM (single call vs multiple comparisons)
• 6.4 Use str.translate() for Character-Level Replacements — MEDIUM (10× faster than chained replace())

7. Function & Call Overhead — LOW-MEDIUM

• 7.1 Reduce Function Calls in Tight Loops — LOW-MEDIUM (100ms savings per 1M iterations)
• 7.2 Use functools.partial for Pre-Filled Arguments — LOW-MEDIUM (50% faster debugging via introspection)
• 7.3 Use Keyword-Only Arguments for API Clarity — LOW-MEDIUM (prevents positional argument errors)
• 7.4 Use lru_cache for Expensive Function Memoization — LOW-MEDIUM (avoids repeated computation)

8. Python Idioms & Micro — LOW

• 8.1 Leverage Zero-Cost Exception Handling — LOW (zero overhead in happy path (Python 3.11+))
• 8.2 Prefer Local Variables Over Global Lookups — LOW (faster name resolution)
• 8.3 Use dataclass for Data-Holding Classes — LOW (reduces boilerplate by 80%)
• 8.4 Use Lazy Imports for Faster Startup — LOW (10-15% faster startup)
• 8.5 Use match Statement for Structural Pattern Matching — LOW (reduces branch complexity)
• 8.6 Use Walrus Operator for Assignment in Expressions — LOW (eliminates redundant computations)

References

1. Python 3.11 Release Notes 2. PEP 8 Style Guide 3. Python Wiki - Performance Tips 4. Real Python - Async IO 5. Real Python - LEGB Rule 6. Real Python - String Concatenation 7. Python Tutorial - Data Structures 8. CPython Exception Handling 9. DataCamp - Python Generators 10. JetBrains - Performance Hacks

Python

This curated skill mirrors SKILL.md. When maintaining it, keep the guidance focused on Python 3.11+ performance, data structures, asyncio, memory, concurrency, and idiomatic code.

Rule Title Here

Brief explanation (1-3 sentences) of WHY this matters. Focus on performance implications and cascade effects.

Incorrect (describe the problem/cost):

def example_function(data: list[str]) -> list[str]:
    # Comment explaining the cost on key line only
    result = []
    for item in data:
        result.append(process(item))  # This line has the problem
    return result

Correct (describe the benefit/solution):

def example_function(data: list[str]) -> list[str]:
    return [process(item) for item in data]
    # Minimal diff from incorrect - same variable names

Alternative (when applicable):

# Alternative approach for specific contexts

When NOT to use this pattern:

• Exception 1
• Exception 2

Benefits:

• Benefit 1
• Benefit 2

Reference: Reference Title

Sections

This file defines all sections, their ordering, impact levels, and descriptions. The section ID (in parentheses) is the filename prefix used to group rules.

---

1. I/O & Async Patterns (io)

Impact: CRITICAL Description: Blocking I/O is the #1 performance bottleneck. Async patterns eliminate sequential waits, yielding 2-10× throughput improvements for I/O-bound workloads.

2. Data Structure Selection (ds)

Impact: CRITICAL Description: Wrong data structure choice causes O(n) lookups instead of O(1). Set and dict lookups are 100× faster than list scans for large collections.

3. Memory Optimization (mem)

Impact: HIGH Description: Excessive allocations trigger garbage collection and increase memory footprint. Generators, __slots__, and object reuse reduce memory 20-50%.

4. Concurrency & Parallelism (conc)

Impact: HIGH Description: The GIL limits CPU-bound parallelism. Choosing asyncio vs threading vs multiprocessing correctly determines application throughput.

5. Loop & Iteration (loop)

Impact: MEDIUM Description: Comprehensions are 2-3× faster than explicit loops. Moving invariant work outside loops avoids N× overhead multiplication.

6. String Operations (str)

Impact: MEDIUM Description: String concatenation in loops is O(n²) due to immutability. Using join() is 4× faster for combining multiple strings.

7. Function & Call Overhead (func)

Impact: LOW-MEDIUM Description: Function calls cost 50-100ns each in CPython. In tight loops processing millions of items, reducing calls improves throughput.

8. Python Idioms & Micro (py)

Impact: LOW Description: Pythonic patterns leverage C-optimized internals. Local variables, built-in functions, and modern syntax yield incremental but measurable gains.

Use asyncio for I/O-Bound Concurrency

For I/O-bound workloads (network, disk), asyncio provides the best performance with minimal overhead. Threading adds context-switch costs; multiprocessing adds process overhead.

Incorrect (blocking synchronous I/O):

import requests

def fetch_all_apis(urls: list[str]) -> list[dict]:
    results = []
    for url in urls:
        response = requests.get(url)  # Blocks until complete
        results.append(response.json())
    return results
# 100 URLs × 200ms each = 20 seconds sequential

Correct (async concurrent I/O):

import asyncio
import aiohttp

async def fetch_all_apis(urls: list[str]) -> list[dict]:
    async with aiohttp.ClientSession() as session:
        async def fetch(url: str) -> dict:
            async with session.get(url) as response:
                return await response.json()

        return await asyncio.gather(*[fetch(url) for url in urls])
# 100 URLs × max(200ms) = ~200ms concurrent

When to use each model:

• asyncio: I/O-bound, high concurrency (thousands of connections)
• threading: I/O-bound, simpler code, moderate concurrency
• multiprocessing: CPU-bound, true parallelism needed

Reference: Real Python - asyncio

Use multiprocessing for CPU-Bound Parallelism

The GIL prevents true parallelism in threads for CPU-bound work. Use multiprocessing to bypass the GIL and utilize multiple cores.

Incorrect (GIL-limited threading):

from concurrent.futures import ThreadPoolExecutor

def compute_hashes(data_chunks: list[bytes]) -> list[str]:
    def hash_chunk(chunk: bytes) -> str:
        return hashlib.sha256(chunk).hexdigest()

    with ThreadPoolExecutor(max_workers=4) as executor:
        return list(executor.map(hash_chunk, data_chunks))
    # GIL prevents parallel execution - effectively single-threaded

Correct (true parallelism):

from concurrent.futures import ProcessPoolExecutor

def compute_hashes(data_chunks: list[bytes]) -> list[str]:
    def hash_chunk(chunk: bytes) -> str:
        return hashlib.sha256(chunk).hexdigest()

    with ProcessPoolExecutor(max_workers=4) as executor:
        return list(executor.map(hash_chunk, data_chunks))
    # Each process has its own GIL - true parallel execution

Alternative (for large data):

import multiprocessing as mp

def compute_hashes_large(data_chunks: list[bytes]) -> list[str]:
    with mp.Pool(processes=4) as pool:
        return pool.map(hash_chunk, data_chunks, chunksize=100)
    # chunksize reduces IPC overhead for many small items

When NOT to use multiprocessing:

• I/O-bound tasks (use asyncio instead)
• Small datasets (process startup overhead dominates)
• When sharing large state between workers

Reference: multiprocessing documentation

Use Queue for Thread-Safe Communication

Sharing mutable state between threads causes race conditions. Use queue.Queue for thread-safe producer-consumer patterns.

Incorrect (shared list with race condition):

import threading

results = []  # Shared mutable state

def worker(items: list[str]) -> None:
    for item in items:
        processed = process_item(item)
        results.append(processed)  # Race condition!

threads = [threading.Thread(target=worker, args=(chunk,)) for chunk in chunks]
for t in threads:
    t.start()
for t in threads:
    t.join()
# results may have corrupted or missing data

Correct (thread-safe queue):

import threading
from queue import Queue

def worker(input_queue: Queue, output_queue: Queue) -> None:
    while True:
        item = input_queue.get()
        if item is None:  # Poison pill
            break
        output_queue.put(process_item(item))
        input_queue.task_done()

input_queue = Queue()
output_queue = Queue()

threads = [threading.Thread(target=worker, args=(input_queue, output_queue))
           for _ in range(4)]
for t in threads:
    t.start()

for item in items:
    input_queue.put(item)

input_queue.join()  # Wait for all items processed

for _ in threads:
    input_queue.put(None)  # Signal workers to stop
for t in threads:
    t.join()

results = [output_queue.get() for _ in range(len(items))]

Note: For async code, use asyncio.Queue instead.

Reference: queue documentation

Use TaskGroup for Structured Concurrency

asyncio.gather() doesn't cancel remaining tasks on error by default. TaskGroup (Python 3.11+) provides structured concurrency with automatic cancellation.

Incorrect (tasks continue after failure):

async def fetch_all_data(user_ids: list[int]) -> list[dict]:
    tasks = [fetch_user(uid) for uid in user_ids]
    results = await asyncio.gather(*tasks)  # If one fails, others continue
    return results
    # Exception from one task doesn't stop others

Correct (automatic cancellation on error):

async def fetch_all_data(user_ids: list[int]) -> list[dict]:
    async with asyncio.TaskGroup() as tg:
        tasks = [tg.create_task(fetch_user(uid)) for uid in user_ids]
    # If any task fails, all others are cancelled
    # ExceptionGroup raised with all errors
    return [task.result() for task in tasks]

Alternative (gather with return_exceptions):

async def fetch_all_data(user_ids: list[int]) -> list[dict | Exception]:
    tasks = [fetch_user(uid) for uid in user_ids]
    results = await asyncio.gather(*tasks, return_exceptions=True)
    # Exceptions returned as values, not raised
    return [r for r in results if not isinstance(r, Exception)]

Benefits of TaskGroup:

• Automatic cancellation on first error
• Proper cleanup of all tasks
• Clear lifetime boundaries
• ExceptionGroup for handling multiple errors

Reference: asyncio.TaskGroup documentation

Use ThreadPoolExecutor for Blocking Calls in Async

Blocking calls in async code freeze the entire event loop. Use run_in_executor() to offload blocking operations to a thread pool.

Incorrect (blocks event loop):

import asyncio

async def process_image(image_path: str) -> bytes:
    # PIL operations are blocking - freezes all other coroutines
    with Image.open(image_path) as img:
        img = img.resize((800, 600))
        buffer = io.BytesIO()
        img.save(buffer, format="JPEG")
        return buffer.getvalue()

Correct (offloads to thread pool):

import asyncio
from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor(max_workers=4)

def _process_image_sync(image_path: str) -> bytes:
    with Image.open(image_path) as img:
        img = img.resize((800, 600))
        buffer = io.BytesIO()
        img.save(buffer, format="JPEG")
        return buffer.getvalue()

async def process_image(image_path: str) -> bytes:
    loop = asyncio.get_running_loop()
    return await loop.run_in_executor(executor, _process_image_sync, image_path)

Alternative (default executor):

async def process_image(image_path: str) -> bytes:
    loop = asyncio.get_running_loop()
    return await loop.run_in_executor(None, _process_image_sync, image_path)
    # None uses default ThreadPoolExecutor

Reference: asyncio.loop.run_in_executor

Use bisect for O(log n) Sorted List Operations

Linear search through a sorted list wastes the sorted property. The bisect module provides O(log n) binary search operations.

Incorrect (O(n) linear search):

def find_price_tier(price: float, thresholds: list[float]) -> int:
    # thresholds = [10.0, 25.0, 50.0, 100.0, 250.0] (sorted)
    tier = 0
    for i, threshold in enumerate(thresholds):  # O(n) scan
        if price >= threshold:
            tier = i + 1
        else:
            break
    return tier

Correct (O(log n) binary search):

import bisect

def find_price_tier(price: float, thresholds: list[float]) -> int:
    # thresholds = [10.0, 25.0, 50.0, 100.0, 250.0] (sorted)
    return bisect.bisect_right(thresholds, price)  # O(log n)

Alternative (maintaining sorted order):

import bisect

def add_score_sorted(scores: list[int], new_score: int) -> None:
    bisect.insort(scores, new_score)  # O(n) insert but maintains order
    # Better than: scores.append(new_score); scores.sort()  # O(n log n)

Note: bisect_left finds leftmost position, bisect_right finds rightmost for equal values.

Reference: bisect documentation

Use defaultdict to Avoid Key Existence Checks

Checking if a key exists before modifying it requires two lookups. defaultdict auto-initializes missing keys, reducing code and improving performance.

Incorrect (double lookup per key):

def group_orders_by_user(orders: list[dict]) -> dict[int, list[dict]]:
    grouped = {}
    for order in orders:
        user_id = order["user_id"]
        if user_id not in grouped:  # First lookup
            grouped[user_id] = []
        grouped[user_id].append(order)  # Second lookup
    return grouped

Correct (single lookup):

from collections import defaultdict

def group_orders_by_user(orders: list[dict]) -> dict[int, list[dict]]:
    grouped = defaultdict(list)
    for order in orders:
        grouped[order["user_id"]].append(order)  # Single lookup, auto-creates list
    return dict(grouped)

Alternative (setdefault):

def group_orders_by_user(orders: list[dict]) -> dict[int, list[dict]]:
    grouped = {}
    for order in orders:
        grouped.setdefault(order["user_id"], []).append(order)
    return grouped

Note: Convert back to regular dict if you need strict KeyError behavior later.

Reference: collections.defaultdict documentation

Use deque for O(1) Queue Operations

List pop(0) is O(n) because all remaining elements must shift. collections.deque provides O(1) operations on both ends.

Incorrect (O(n) popleft):

def process_tasks(tasks: list[str]) -> list[str]:
    queue = tasks.copy()
    results = []
    while queue:
        task = queue.pop(0)  # O(n) - shifts all elements left
        results.append(execute_task(task))
    return results
# n tasks × O(n) shift = O(n²) total

Correct (O(1) popleft):

from collections import deque

def process_tasks(tasks: list[str]) -> list[str]:
    queue = deque(tasks)  # O(n) conversion once
    results = []
    while queue:
        task = queue.popleft()  # O(1) - doubly-linked list
        results.append(execute_task(task))
    return results
# n tasks × O(1) = O(n) total

Benefits:

• appendleft() and popleft() are O(1)
• append() and pop() are O(1)
• Thread-safe for single append/pop operations
• Optional maxlen for fixed-size buffers

Reference: collections.deque documentation

Use Dict for O(1) Key-Value Lookup

Searching a list of tuples or objects for a key is O(n). Converting to a dict provides O(1) lookup by key, critical for repeated access patterns.

Incorrect (O(n) search per lookup):

def get_user_emails(user_ids: list[int], users: list[tuple[int, str]]) -> list[str]:
    emails = []
    for user_id in user_ids:
        for uid, email in users:  # O(n) scan for each user_id
            if uid == user_id:
                emails.append(email)
                break
    return emails

Correct (O(1) lookup):

def get_user_emails(user_ids: list[int], users: list[tuple[int, str]]) -> list[str]:
    user_map = {uid: email for uid, email in users}  # One-time O(n) conversion
    return [user_map[user_id] for user_id in user_ids if user_id in user_map]

Alternative (with default value):

def get_user_emails(user_ids: list[int], users: list[tuple[int, str]]) -> list[str]:
    user_map = dict(users)
    return [user_map.get(user_id, "unknown@example.com") for user_id in user_ids]

Reference: Python Data Structures

Use frozenset for Hashable Set Keys

Regular sets are mutable and unhashable, so they cannot be dict keys or set members. Use frozenset for immutable, hashable sets.

Incorrect (unhashable set as key):

def find_duplicate_permission_groups(users: list[dict]) -> list[set]:
    seen = {}
    duplicates = []
    for user in users:
        perms = set(user["permissions"])
        if perms in seen:  # TypeError: unhashable type: 'set'
            duplicates.append(perms)
        seen[perms] = user["id"]
    return duplicates

Correct (hashable frozenset):

def find_duplicate_permission_groups(users: list[dict]) -> list[set]:
    seen = {}
    duplicates = []
    for user in users:
        perms = frozenset(user["permissions"])  # Immutable and hashable
        if perms in seen:  # O(1) lookup works
            duplicates.append(set(perms))
        seen[perms] = user["id"]
    return duplicates

Alternative (caching computed sets):

from functools import cache

@cache
def compute_dependencies(package: frozenset[str]) -> frozenset[str]:
    # frozenset enables caching of set-based inputs
    return frozenset(resolve_deps(package))

Reference: frozenset documentation

Use Set for O(1) Membership Testing

List membership testing with in is O(n), scanning every element. Set membership is O(1) using hash lookup, making it 100× faster for large collections.

Incorrect (O(n) per lookup):

def filter_valid_users(user_ids: list[int], valid_ids: list[int]) -> list[int]:
    result = []
    for user_id in user_ids:
        if user_id in valid_ids:  # O(n) scan on every iteration
            result.append(user_id)
    return result
# 10,000 users × 10,000 valid IDs = 100M comparisons

Correct (O(1) per lookup):

def filter_valid_users(user_ids: list[int], valid_ids: list[int]) -> list[int]:
    valid_set = set(valid_ids)  # One-time O(n) conversion
    result = []
    for user_id in user_ids:
        if user_id in valid_set:  # O(1) hash lookup
            result.append(user_id)
    return result
# 10,000 users × O(1) = 10,000 operations

Even better (comprehension):

def filter_valid_users(user_ids: list[int], valid_ids: list[int]) -> list[int]:
    valid_set = set(valid_ids)
    return [user_id for user_id in user_ids if user_id in valid_set]

Reference: Python Wiki - Time Complexity

Use Keyword-Only Arguments for API Clarity

Functions with multiple boolean or similar-typed arguments are error-prone when called positionally. Keyword-only arguments (after *) enforce explicit naming.

Incorrect (ambiguous positional args):

def create_user(name: str, admin: bool, active: bool, verified: bool) -> User:
    return User(name=name, admin=admin, active=active, verified=verified)

# Easy to get wrong:
user = create_user("alice", True, False, True)  # Which bool is which?
user = create_user("bob", False, True, False)  # Confusing

*Correct (keyword-only after ):**

def create_user(
    name: str,
    *,
    admin: bool = False,
    active: bool = True,
    verified: bool = False,
) -> User:
    return User(name=name, admin=admin, active=active, verified=verified)

# Forces clarity:
user = create_user("alice", admin=True, verified=True)
user = create_user("bob", active=True)
# create_user("charlie", True, False, True)  # TypeError!

Positional-only (Python 3.8+):

def calculate_distance(x1: float, y1: float, x2: float, y2: float, /) -> float:
    # / means all args before it are positional-only
    return ((x2 - x1) ** 2 + (y2 - y1) ** 2) ** 0.5

Reference: PEP 3102 - Keyword-Only Arguments

Use lru_cache for Expensive Function Memoization

Functions called with the same arguments repeatedly waste computation. @lru_cache stores results automatically, returning cached values on subsequent calls.

Incorrect (recomputes every call):

def calculate_fibonacci(n: int) -> int:
    if n < 2:
        return n
    return calculate_fibonacci(n - 1) + calculate_fibonacci(n - 2)
# fib(35) = 9,227,465 recursive calls

def get_user_permissions(user_id: int) -> set[str]:
    user = fetch_user_from_db(user_id)  # DB call every time
    return compute_effective_permissions(user)

Correct (cached results):

from functools import lru_cache

@lru_cache(maxsize=128)
def calculate_fibonacci(n: int) -> int:
    if n < 2:
        return n
    return calculate_fibonacci(n - 1) + calculate_fibonacci(n - 2)
# fib(35) = 35 unique calls, rest cached

@lru_cache(maxsize=1000)
def get_user_permissions(user_id: int) -> frozenset[str]:
    user = fetch_user_from_db(user_id)  # Cached after first call
    return frozenset(compute_effective_permissions(user))

For unhashable arguments:

from functools import cache  # Python 3.9+, unbounded cache

@cache
def expensive_computation(x: int, y: int) -> int:
    return x ** y

Note: Arguments must be hashable. Use frozenset instead of set, tuple instead of list.

Reference: functools.lru_cache documentation

Use functools.partial for Pre-Filled Arguments

When you need a function with some arguments pre-filled, partial is cleaner than lambdas and provides better debugging information.

Incorrect (lambda wrapper):

def process_items(items: list[str], processor) -> list[str]:
    return [processor(item) for item in items]

# Lambda obscures the actual function
results = process_items(
    items,
    lambda x: format_string(x, uppercase=True, strip=True)
)

Correct (partial application):

from functools import partial

def process_items(items: list[str], processor) -> list[str]:
    return [processor(item) for item in items]

# Partial shows the actual function
format_upper = partial(format_string, uppercase=True, strip=True)
results = process_items(items, format_upper)

# Better for debugging: partial has __name__ and __func__
print(format_upper.func.__name__)  # 'format_string'
print(format_upper.keywords)  # {'uppercase': True, 'strip': True}

Common use cases:

from functools import partial

# Pre-configure logging
debug_log = partial(log_message, level="DEBUG")
error_log = partial(log_message, level="ERROR")

# Pre-configure API client
prod_client = partial(api_request, base_url="https://api.example.com")
test_client = partial(api_request, base_url="https://test.example.com")

Reference: functools.partial documentation

Reduce Function Calls in Tight Loops

Each Python function call costs 50-100ns. In loops processing millions of items, this overhead accumulates significantly.

Incorrect (function call per iteration):

def process_values(values: list[float]) -> list[float]:
    def transform(x: float) -> float:
        return x * 2.5 + 10

    return [transform(v) for v in values]
# 1M values × 100ns = 100ms in call overhead alone

Correct (inline simple operations):

def process_values(values: list[float]) -> list[float]:
    return [v * 2.5 + 10 for v in values]
# No function call overhead

For method calls, cache the lookup:

# Before (3 lookups per iteration)
for item in items:
    result.append(processor.transform(item))

# After (1 lookup total)
append = result.append
transform = processor.transform
for item in items:
    append(transform(item))

When NOT to inline:

• When it hurts readability significantly
• When the function is complex
• When profiling shows the call overhead is negligible

Reference: Python Wiki - Performance Tips

Use aiofiles for Async File Operations

Standard file operations block the event loop, preventing other coroutines from running. Use aiofiles for non-blocking file I/O in async applications.

Incorrect (blocks event loop):

async def process_log_files(log_paths: list[str]) -> list[dict]:
    results = []
    for path in log_paths:
        with open(path, "r") as f:  # Blocks entire event loop
            content = f.read()
            results.append(parse_log(content))
    return results

Correct (non-blocking):

import aiofiles

async def process_log_files(log_paths: list[str]) -> list[dict]:
    async def read_and_parse(path: str) -> dict:
        async with aiofiles.open(path, "r") as f:  # Non-blocking
            content = await f.read()
            return parse_log(content)

    return await asyncio.gather(*[read_and_parse(path) for path in log_paths])

Alternative (thread pool for sync I/O):

async def process_log_files(log_paths: list[str]) -> list[dict]:
    loop = asyncio.get_running_loop()

    def read_sync(path: str) -> dict:
        with open(path, "r") as f:
            return parse_log(f.read())

    tasks = [loop.run_in_executor(None, read_sync, path) for path in log_paths]
    return await asyncio.gather(*tasks)

Reference: aiofiles documentation

Use asyncio.gather() for Concurrent I/O

When multiple I/O operations have no dependencies, execute them concurrently with asyncio.gather(). Sequential awaits create waterfalls where each operation waits for the previous one to complete.

Incorrect (sequential execution, 3 round trips):

async def fetch_user_data(user_id: str) -> dict:
    profile = await fetch_profile(user_id)
    orders = await fetch_orders(user_id)
    preferences = await fetch_preferences(user_id)
    # Total time: profile + orders + preferences
    return {"profile": profile, "orders": orders, "preferences": preferences}

Correct (concurrent execution, 1 round trip):

async def fetch_user_data(user_id: str) -> dict:
    profile, orders, preferences = await asyncio.gather(
        fetch_profile(user_id),
        fetch_orders(user_id),
        fetch_preferences(user_id),
    )
    # Total time: max(profile, orders, preferences)
    return {"profile": profile, "orders": orders, "preferences": preferences}

When NOT to use this pattern:

• When operations depend on each other's results
• When you need to handle individual failures differently (use return_exceptions=True or asyncio.TaskGroup)

Reference: Python asyncio documentation

Use Connection Pooling for Database Access

Creating database connections is expensive, typically taking 100-200ms. Connection pools maintain reusable connections, eliminating this overhead for each query.

Incorrect (new connection per query):

async def get_user(user_id: int) -> dict:
    conn = await asyncpg.connect(DATABASE_URL)  # 100-200ms overhead
    try:
        row = await conn.fetchrow("SELECT * FROM users WHERE id = $1", user_id)
        return dict(row)
    finally:
        await conn.close()

async def get_orders(user_id: int) -> list:
    conn = await asyncpg.connect(DATABASE_URL)  # Another 100-200ms overhead
    try:
        rows = await conn.fetch("SELECT * FROM orders WHERE user_id = $1", user_id)
        return [dict(row) for row in rows]
    finally:
        await conn.close()

Correct (shared connection pool):

pool: asyncpg.Pool | None = None

async def init_pool():
    global pool
    pool = await asyncpg.create_pool(DATABASE_URL, min_size=5, max_size=20)

async def get_user(user_id: int) -> dict:
    async with pool.acquire() as conn:  # Reuses existing connection
        row = await conn.fetchrow("SELECT * FROM users WHERE id = $1", user_id)
        return dict(row)

async def get_orders(user_id: int) -> list:
    async with pool.acquire() as conn:  # Reuses existing connection
        rows = await conn.fetch("SELECT * FROM orders WHERE user_id = $1", user_id)
        return [dict(row) for row in rows]

Reference: asyncpg documentation

Defer await Until Value Needed

Start async operations immediately but defer await until the value is actually needed. This allows multiple operations to run concurrently while the code proceeds.

Incorrect (blocks immediately):

async def process_order(order_id: str) -> dict:
    order = await fetch_order(order_id)  # Blocks here
    user = await fetch_user(order.user_id)  # Waits for order first
    inventory = await check_inventory(order.items)  # Waits for user
    # Total: order + user + inventory
    return {"order": order, "user": user, "inventory": inventory}

Correct (starts early, awaits late):

async def process_order(order_id: str) -> dict:
    order_task = asyncio.create_task(fetch_order(order_id))  # Starts immediately

    order = await order_task  # Now we need the order

    # Start both in parallel since they only need order data
    user_task = asyncio.create_task(fetch_user(order.user_id))
    inventory_task = asyncio.create_task(check_inventory(order.items))

    user = await user_task
    inventory = await inventory_task
    # Total: order + max(user, inventory)
    return {"order": order, "user": user, "inventory": inventory}

Note: Use asyncio.create_task() to start coroutines immediately. The task runs in the background until awaited.

Reference: Python asyncio.create_task

Use Semaphores to Limit Concurrent Operations

Unbounded concurrency can exhaust resources like file descriptors, memory, or API rate limits. Use asyncio.Semaphore to cap concurrent operations.

Incorrect (unbounded concurrency):

async def fetch_all_urls(urls: list[str]) -> list[str]:
    async def fetch(url: str) -> str:
        async with aiohttp.ClientSession() as session:
            async with session.get(url) as response:
                return await response.text()

    # Launches all requests simultaneously - may exhaust connections
    return await asyncio.gather(*[fetch(url) for url in urls])

Correct (bounded concurrency):

async def fetch_all_urls(urls: list[str], max_concurrent: int = 10) -> list[str]:
    semaphore = asyncio.Semaphore(max_concurrent)

    async def fetch(url: str) -> str:
        async with semaphore:  # Limits concurrent requests
            async with aiohttp.ClientSession() as session:
                async with session.get(url) as response:
                    return await response.text()

    return await asyncio.gather(*[fetch(url) for url in urls])

Alternative (connection pool with aiohttp):

async def fetch_all_urls(urls: list[str]) -> list[str]:
    connector = aiohttp.TCPConnector(limit=10)  # Built-in limiting
    async with aiohttp.ClientSession(connector=connector) as session:
        async def fetch(url: str) -> str:
            async with session.get(url) as response:
                return await response.text()

        return await asyncio.gather(*[fetch(url) for url in urls])

Reference: asyncio.Semaphore documentation

Use uvloop for Faster Event Loop

uvloop is a drop-in replacement for asyncio's event loop, built on libuv. It provides 2-4× faster I/O performance with a single configuration change.

Incorrect (default event loop):

import asyncio

async def main():
    results = await asyncio.gather(
        fetch_users(),
        fetch_orders(),
        fetch_inventory(),
    )
    return results

if __name__ == "__main__":
    asyncio.run(main())  # Uses default event loop

Correct (uvloop event loop):

import asyncio
import uvloop

async def main():
    results = await asyncio.gather(
        fetch_users(),
        fetch_orders(),
        fetch_inventory(),
    )
    return results

if __name__ == "__main__":
    uvloop.install()  # Single line change
    asyncio.run(main())  # Now uses uvloop

Alternative (set policy explicitly):

import asyncio
import uvloop

asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

When NOT to use this pattern:

• On Windows (uvloop is Unix-only)
• When debugging with asyncio debug mode

Reference: uvloop documentation

Use any() and all() for Boolean Aggregation

Manual loops for checking conditions iterate through all elements. any() and all() short-circuit on the first conclusive result.

Incorrect (checks all elements):

def has_admin_user(users: list[dict]) -> bool:
    found = False
    for user in users:
        if user["role"] == "admin":
            found = True
            # Continues iterating even after finding one!
    return found

Correct (short-circuits immediately):

def has_admin_user(users: list[dict]) -> bool:
    return any(user["role"] == "admin" for user in users)
    # Stops at first admin found

Common patterns:

# Check if all items meet condition
all_active = all(user["status"] == "active" for user in users)

# Check if any item fails condition
has_invalid = any(not validate_email(user["email"]) for user in users)

# Combine with filter-like logic
has_large_order = any(order["total"] > 1000 for order in orders if order["status"] == "completed")

Note: Use generator expressions (parentheses) not list comprehensions (brackets) to get short-circuit behavior.

Reference: any() documentation

Use List Comprehensions Over Explicit Loops

List comprehensions are optimized in C and avoid the overhead of repeated append() calls. They're 2-3× faster than equivalent for loops.

Incorrect (explicit loop with append):

def get_active_user_ids(users: list[dict]) -> list[int]:
    result = []
    for user in users:
        if user["status"] == "active":
            result.append(user["id"])  # Method lookup + call per iteration
    return result

Correct (list comprehension):

def get_active_user_ids(users: list[dict]) -> list[int]:
    return [user["id"] for user in users if user["status"] == "active"]
    # No append overhead, optimized bytecode

For complex transformations:

# Multiple operations are still cleaner as comprehension
active_emails = [
    user["email"].lower().strip()
    for user in users
    if user["status"] == "active" and user["email"]
]

When NOT to use comprehensions:

• Side effects needed (logging, database writes)
• Complex multi-step logic requiring intermediate variables
• Readability suffers with deeply nested conditions

Reference: Python Wiki - Performance Tips

Use dict.items() for Key-Value Iteration

Iterating over keys then looking up values performs two operations per entry. dict.items() provides both in a single iteration.

Incorrect (double lookup per item):

def transform_config(config: dict[str, str]) -> dict[str, str]:
    result = {}
    for key in config:  # First: iterate keys
        value = config[key]  # Second: lookup value
        result[key.upper()] = value.strip()
    return result

Correct (single lookup):

def transform_config(config: dict[str, str]) -> dict[str, str]:
    return {key.upper(): value.strip() for key, value in config.items()}
    # items() yields (key, value) tuples directly

Similarly for values only:

# When you only need values
total = sum(order["amount"] for order in orders.values())

# When you only need keys
active_keys = [k for k in cache.keys() if not k.startswith("_")]
# Or simply: [k for k in cache if not k.startswith("_")]

Reference: dict.items() documentation

Use enumerate() for Index-Value Iteration

Manual index tracking with range(len()) is error-prone and requires two lookups. enumerate() provides both index and value in one clean pattern.

Incorrect (manual index tracking):

def find_duplicates(items: list[str]) -> list[tuple[int, int]]:
    duplicates = []
    for i in range(len(items)):  # Index only
        for j in range(i + 1, len(items)):
            if items[i] == items[j]:  # Separate lookup
                duplicates.append((i, j))
    return duplicates

Correct (enumerate for index + value):

def find_duplicates(items: list[str]) -> list[tuple[int, int]]:
    duplicates = []
    for i, item_i in enumerate(items):  # Index and value together
        for j, item_j in enumerate(items[i + 1:], start=i + 1):
            if item_i == item_j:  # Direct comparison
                duplicates.append((i, j))
    return duplicates

With custom start index:

# Line numbers typically start at 1
for line_num, line in enumerate(file_lines, start=1):
    if "ERROR" in line:
        print(f"Error on line {line_num}: {line}")

Reference: enumerate documentation

Hoist Loop-Invariant Computations

Computations that don't change between iterations should be moved outside the loop to avoid repeating the same work N times.

Incorrect (recomputes constant every iteration):

def apply_discount(prices: list[float], discount_code: str) -> list[float]:
    result = []
    for price in prices:
        discount = get_discount_rate(discount_code)  # Same result every time
        threshold = calculate_threshold(discount)  # Same result every time
        if price > threshold:
            result.append(price * (1 - discount))
        else:
            result.append(price)
    return result
# 10,000 prices × 2 function calls = 20,000 redundant calls

Correct (compute once before loop):

def apply_discount(prices: list[float], discount_code: str) -> list[float]:
    discount = get_discount_rate(discount_code)  # Computed once
    threshold = calculate_threshold(discount)  # Computed once
    result = []
    for price in prices:
        if price > threshold:
            result.append(price * (1 - discount))
        else:
            result.append(price)
    return result
# 10,000 prices × 0 redundant calls

Also hoist attribute lookups:

# Before (attribute lookup each iteration)
for item in items:
    self.processor.transform(item)

# After (single lookup)
transform = self.processor.transform
for item in items:
    transform(item)

Reference: Python Performance Tips

Use itertools for Efficient Iteration Patterns

The itertools module provides C-optimized functions for common iteration patterns, avoiding Python loop overhead.

Incorrect (manual nested loops):

def generate_combinations(colors: list[str], sizes: list[str]) -> list[tuple]:
    result = []
    for color in colors:
        for size in sizes:
            result.append((color, size))
    return result

Correct (itertools.product):

from itertools import product

def generate_combinations(colors: list[str], sizes: list[str]) -> list[tuple]:
    return list(product(colors, sizes))

Common itertools patterns:

from itertools import chain, groupby, islice, batched

# Flatten nested lists (faster than nested comprehension)
flat = list(chain.from_iterable(nested_lists))

# Group consecutive items
for key, group in groupby(sorted(orders, key=lambda x: x["status"]), key=lambda x: x["status"]):
    print(f"{key}: {list(group)}")

# Slice iterators without loading all into memory
first_1000 = list(islice(huge_generator, 1000))

# Batch items (Python 3.12+)
for batch in batched(items, 100):
    process_batch(batch)

Reference: itertools documentation

Use array.array for Homogeneous Numeric Data

Lists store pointers to boxed Python objects (~28 bytes per integer). array.array stores raw values compactly (4-8 bytes per number).

Incorrect (boxed integers in list):

def load_sensor_readings(filepath: str) -> list[int]:
    readings = []
    with open(filepath) as f:
        for line in f:
            readings.append(int(line.strip()))
    return readings
# 1M integers × 28 bytes = ~28MB

Correct (compact array storage):

from array import array

def load_sensor_readings(filepath: str) -> array:
    readings = array("i")  # 'i' = signed 32-bit integers
    with open(filepath) as f:
        for line in f:
            readings.append(int(line.strip()))
    return readings
# 1M integers × 4 bytes = ~4MB

Common type codes:

• 'b' - signed char (1 byte)
• 'i' - signed int (4 bytes)
• 'l' - signed long (4-8 bytes)
• 'f' - float (4 bytes)
• 'd' - double (8 bytes)

When NOT to use array.array:

• When you need mixed types
• When you need NumPy operations
• For small datasets where overhead doesn't matter

Reference: array documentation

Use Generators for Large Sequences

Lists store all elements in memory simultaneously. Generators produce values on-demand, using constant memory regardless of sequence size.

Incorrect (loads entire dataset into memory):

def process_large_file(filepath: str) -> list[dict]:
    with open(filepath) as f:
        lines = f.readlines()  # Loads entire file into memory

    results = []
    for line in lines:
        parsed = parse_line(line)
        if parsed["status"] == "active":
            results.append(transform(parsed))
    return results
# 1GB file = 1GB+ memory usage

Correct (constant memory usage):

def process_large_file(filepath: str):
    with open(filepath) as f:
        for line in f:  # Yields one line at a time
            parsed = parse_line(line)
            if parsed["status"] == "active":
                yield transform(parsed)
# 1GB file = ~100KB memory usage

# Use the generator
for result in process_large_file("data.csv"):
    save_to_database(result)

Alternative (generator expression):

def get_active_users(users: list[dict]):
    return (user for user in users if user["status"] == "active")
    # Generator expression uses minimal memory

When NOT to use generators:

• When you need random access to elements
• When you need to iterate multiple times

Reference: Python Wiki - Generators

Intern Repeated Strings to Save Memory

When the same string appears thousands of times (e.g., status codes, keys), each occurrence normally creates a new object. String interning reuses the same object.

Incorrect (duplicate string objects):

def process_events(events: list[dict]) -> list[dict]:
    results = []
    for event in events:
        results.append({
            "type": event["type"],  # "click" repeated 1M times = 1M objects
            "status": event["status"],  # "success" repeated = more objects
            "timestamp": event["ts"],
        })
    return results

Correct (interned strings):

import sys

def process_events(events: list[dict]) -> list[dict]:
    results = []
    for event in events:
        results.append({
            "type": sys.intern(event["type"]),  # Reuses single "click" object
            "status": sys.intern(event["status"]),  # Reuses single "success"
            "timestamp": event["ts"],
        })
    return results

Alternative (pre-intern known values):

STATUS_SUCCESS = sys.intern("success")
STATUS_FAILURE = sys.intern("failure")
TYPE_CLICK = sys.intern("click")
TYPE_VIEW = sys.intern("view")

def create_event(event_type: str, status: str) -> dict:
    return {"type": event_type, "status": status}

Note: Python automatically interns string literals and identifiers. Use sys.intern() for runtime-generated strings with high repetition.

Reference: sys.intern documentation

Use __slots__ for Memory-Efficient Classes

By default, Python stores instance attributes in a __dict__ dictionary. __slots__ replaces this with a fixed-size array, reducing memory and speeding up attribute access.

Incorrect (dict-based attributes):

class Point:
    def __init__(self, x: float, y: float, z: float):
        self.x = x
        self.y = y
        self.z = z

# Each instance uses ~296 bytes for __dict__
points = [Point(i, i, i) for i in range(100_000)]
# Total: ~30MB

Correct (slot-based attributes):

class Point:
    __slots__ = ("x", "y", "z")

    def __init__(self, x: float, y: float, z: float):
        self.x = x
        self.y = y
        self.z = z

# Each instance uses ~64 bytes (fixed slots)
points = [Point(i, i, i) for i in range(100_000)]
# Total: ~6.4MB

Benefits:

• 20-50% memory reduction per instance
• 10-20% faster attribute access
• Prevents accidental attribute creation

When NOT to use __slots__:

• When you need dynamic attribute creation
• When subclasses need their own __dict__
• For classes with few instances

Reference: Python Wiki - Using Slots

Use weakref for Caches to Prevent Memory Leaks

Strong references in caches prevent garbage collection, causing memory to grow unboundedly. Weak references allow cached objects to be collected when no longer used elsewhere.

Incorrect (strong reference cache):

class ImageProcessor:
    _cache: dict[str, Image] = {}

    def get_image(self, path: str) -> Image:
        if path not in self._cache:
            self._cache[path] = load_image(path)  # Strong reference
        return self._cache[path]
        # Images never freed even after UI closes them

Correct (weak reference cache):

import weakref

class ImageProcessor:
    _cache: weakref.WeakValueDictionary[str, Image]

    def __init__(self):
        self._cache = weakref.WeakValueDictionary()

    def get_image(self, path: str) -> Image:
        image = self._cache.get(path)
        if image is None:
            image = load_image(path)
            self._cache[path] = image  # Weak reference
        return image
        # Images freed when no other references exist

Alternative (LRU cache with size limit):

from functools import lru_cache

@lru_cache(maxsize=100)  # Bounded cache size
def get_image(path: str) -> Image:
    return load_image(path)

Note: WeakValueDictionary holds weak references to values; WeakKeyDictionary holds weak references to keys.

Reference: weakref documentation

Use dataclass for Data-Holding Classes

Classes that primarily hold data require boilerplate __init__, __repr__, __eq__, etc. @dataclass generates these automatically with optimizations.

Incorrect (manual boilerplate):

class User:
    def __init__(self, name: str, email: str, age: int, active: bool = True):
        self.name = name
        self.email = email
        self.age = age
        self.active = active

    def __repr__(self) -> str:
        return f"User(name={self.name!r}, email={self.email!r}, age={self.age}, active={self.active})"

    def __eq__(self, other: object) -> bool:
        if not isinstance(other, User):
            return NotImplemented
        return (self.name, self.email, self.age, self.active) == (other.name, other.email, other.age, other.active)

Correct (dataclass):

from dataclasses import dataclass

@dataclass
class User:
    name: str
    email: str
    age: int
    active: bool = True
    # __init__, __repr__, __eq__ auto-generated

With slots for memory efficiency:

@dataclass(slots=True)  # Python 3.10+
class Point:
    x: float
    y: float
    z: float

Frozen for immutability:

@dataclass(frozen=True)  # Hashable, immutable
class Coordinate:
    lat: float
    lng: float

Reference: dataclasses documentation

Use Lazy Imports for Faster Startup

Top-level imports execute at module load time, slowing startup. Import heavy modules inside functions when they're only needed occasionally.

Incorrect (always imports heavy module):

import pandas as pd  # Imports at module load, even if never used
import numpy as np
from sklearn.ensemble import RandomForestClassifier

def simple_stats(values: list[float]) -> dict:
    return {"mean": sum(values) / len(values)}

def advanced_ml_analysis(data: list[dict]) -> dict:
    # Only called rarely, but pandas/sklearn always loaded
    df = pd.DataFrame(data)
    model = RandomForestClassifier()
    return {"prediction": model.fit_predict(df)}

Correct (lazy import when needed):

def simple_stats(values: list[float]) -> dict:
    return {"mean": sum(values) / len(values)}

def advanced_ml_analysis(data: list[dict]) -> dict:
    import pandas as pd  # Only imports when function called
    from sklearn.ensemble import RandomForestClassifier

    df = pd.DataFrame(data)
    model = RandomForestClassifier()
    return {"prediction": model.fit_predict(df)}

For frequently called functions:

_pandas = None

def get_dataframe(data: list[dict]):
    global _pandas
    if _pandas is None:
        import pandas
        _pandas = pandas
    return _pandas.DataFrame(data)

Note: Python caches imports, so subsequent calls don't re-import.

Reference: Python 3.11 - Faster Startup

Prefer Local Variables Over Global Lookups

Python resolves names using LEGB (Local, Enclosing, Global, Built-in). Local variables are stored in a fixed-size array with O(1) index access, while globals require dictionary lookups.

Incorrect (global lookup each iteration):

MULTIPLIER = 2.5
OFFSET = 10

def transform_values(values: list[float]) -> list[float]:
    result = []
    for v in values:
        result.append(v * MULTIPLIER + OFFSET)  # Global lookup × 2 per iteration
    return result

Correct (local variable cache):

MULTIPLIER = 2.5
OFFSET = 10

def transform_values(values: list[float]) -> list[float]:
    multiplier = MULTIPLIER  # Cache as local
    offset = OFFSET
    result = []
    for v in values:
        result.append(v * multiplier + offset)  # Local lookup (faster)
    return result

For built-in functions:

# Before (built-in lookup each call)
for item in items:
    result.append(len(item))

# After (local cache)
_len = len
for item in items:
    result.append(_len(item))

Note: This optimization matters in tight loops with millions of iterations. For typical code, readability is more important.

Reference: Real Python - LEGB Rule

Use match Statement for Structural Pattern Matching

Python 3.10+ match statement provides structural pattern matching that's clearer and often faster than chained if/elif for complex conditions.

Incorrect (verbose if/elif chain):

def process_event(event: dict) -> str:
    event_type = event.get("type")
    if event_type == "click":
        if "target" in event and "position" in event:
            return f"Click on {event['target']} at {event['position']}"
        return "Invalid click event"
    elif event_type == "keypress":
        if "key" in event:
            return f"Key pressed: {event['key']}"
        return "Invalid keypress event"
    elif event_type == "scroll":
        return f"Scroll by {event.get('delta', 0)}"
    else:
        return "Unknown event"

Correct (structural pattern matching):

def process_event(event: dict) -> str:
    match event:
        case {"type": "click", "target": target, "position": pos}:
            return f"Click on {target} at {pos}"
        case {"type": "click"}:
            return "Invalid click event"
        case {"type": "keypress", "key": key}:
            return f"Key pressed: {key}"
        case {"type": "keypress"}:
            return "Invalid keypress event"
        case {"type": "scroll", "delta": delta}:
            return f"Scroll by {delta}"
        case {"type": "scroll"}:
            return "Scroll by 0"
        case _:
            return "Unknown event"

With guards:

match user:
    case {"role": "admin", "active": True}:
        grant_admin_access()
    case {"role": role} if role in ("editor", "moderator"):
        grant_limited_access()
    case _:
        grant_read_only()

Reference: PEP 634 - Structural Pattern Matching

Use Walrus Operator for Assignment in Expressions

The walrus operator (:=) assigns a value while also returning it, avoiding duplicate computations or function calls.

Incorrect (duplicate computation):

def process_data(items: list[str]) -> list[str]:
    results = []
    for item in items:
        if len(item.strip()) > 10:  # Computes strip() once
            results.append(item.strip())  # Computes strip() again
    return results

Correct (single computation with walrus):

def process_data(items: list[str]) -> list[str]:
    results = []
    for item in items:
        if len(stripped := item.strip()) > 10:  # Assign and test
            results.append(stripped)  # Reuse assigned value
    return results

Common patterns:

# Regex match and use
if match := pattern.search(text):
    print(f"Found: {match.group()}")

# Read until empty
while chunk := file.read(8192):
    process(chunk)

# Filter with computed value
valid_users = [user for user in users if (age := calculate_age(user)) >= 18]

Note: Introduced in Python 3.8 (PEP 572).

Reference: PEP 572 - Assignment Expressions

Leverage Zero-Cost Exception Handling

Python 3.11+ implements zero-cost exception handling where try/except blocks have no overhead when no exception is raised. This makes EAFP (Easier to Ask Forgiveness than Permission) efficient.

Incorrect (LBYL style, always checks):

def get_user_value(data: dict, key: str) -> str | None:
    if key in data:  # Always performs check
        value = data[key]
        if isinstance(value, str):  # Another check
            return value.strip()
    return None

Correct (EAFP style, zero cost when key exists):

def get_user_value(data: dict, key: str) -> str | None:
    try:
        return data[key].strip()  # Zero overhead if key exists
    except (KeyError, AttributeError):
        return None

When EAFP is better:

• Key/attribute usually exists (happy path is common)
• Multiple conditions would need checking
• Race conditions between check and use

When LBYL is better:

• Operation has side effects (file creation)
• Check is cheap, exception is expensive to create
• Failure is common (50%+ of cases)

# LBYL better here - side effect
if not path.exists():
    path.mkdir()

# EAFP better here - usually exists
try:
    config = load_config()
except FileNotFoundError:
    config = default_config()

Reference: CPython Exception Handling

Use f-strings for Simple String Formatting

f-strings (formatted string literals) are the fastest option for simple string formatting, outperforming % formatting and .format().

Incorrect (slower formatting methods):

def format_user_greeting(name: str, age: int) -> str:
    # Old-style % formatting
    return "Hello, %s! You are %d years old." % (name, age)

def format_user_greeting_v2(name: str, age: int) -> str:
    # .format() method
    return "Hello, {}! You are {} years old.".format(name, age)

Correct (f-string):

def format_user_greeting(name: str, age: int) -> str:
    return f"Hello, {name}! You are {age} years old."

f-string features:

# Expressions
total = f"Total: ${price * quantity:.2f}"

# Alignment and padding
header = f"{'Name':<20} {'Age':>5} {'Score':^10}"

# Debug format (Python 3.8+)
debug = f"{user_id=}, {status=}"  # Outputs: "user_id=42, status='active'"

# Multiline
message = f"""
Dear {name},
Your order #{order_id} has been shipped.
"""

Reference: PEP 498 - Literal String Interpolation

Use join() for Multiple String Concatenation

String concatenation with + in loops is O(n²) because strings are immutable—each concatenation creates a new string. join() pre-allocates the final size for O(n) performance.

Incorrect (O(n²) concatenation):

def build_csv_row(values: list[str]) -> str:
    result = ""
    for i, value in enumerate(values):
        if i > 0:
            result += ","  # Creates new string
        result += value  # Creates another new string
    return result
# 100 values = ~5,000 string allocations

Correct (O(n) join):

def build_csv_row(values: list[str]) -> str:
    return ",".join(values)
    # Single allocation of final size

For conditional inclusion:

def build_query_params(params: dict[str, str]) -> str:
    return "&".join(f"{key}={value}" for key, value in params.items() if value)

Note: For 2-3 strings, + or f-strings are fine. Use join() when concatenating 5+ strings or in loops.

Reference: Real Python - String Concatenation

Use str.startswith() with Tuple for Multiple Prefixes

Checking multiple prefixes with or requires multiple string scans. startswith() accepts a tuple of prefixes, checking all in one optimized call.

Incorrect (multiple comparisons):

def is_system_file(filename: str) -> bool:
    return (filename.startswith(".") or
            filename.startswith("__") or
            filename.startswith("~"))

Correct (tuple of prefixes):

def is_system_file(filename: str) -> bool:
    return filename.startswith((".", "__", "~"))

Works with endswith too:

def is_image_file(filename: str) -> bool:
    return filename.lower().endswith((".png", ".jpg", ".jpeg", ".gif", ".webp"))

def is_config_file(path: str) -> bool:
    return path.endswith((".yaml", ".yml", ".json", ".toml"))

Note: The argument must be a tuple, not a list. Lists are not supported.

Reference: str.startswith documentation

Use str.translate() for Character-Level Replacements

Multiple replace() calls each create a new string and scan the entire input. str.translate() performs all replacements in a single pass.

Incorrect (multiple passes):

def sanitize_filename(name: str) -> str:
    result = name.replace("/", "_")  # Pass 1
    result = result.replace("\\", "_")  # Pass 2
    result = result.replace(":", "_")  # Pass 3
    result = result.replace("*", "_")  # Pass 4
    result = result.replace("?", "_")  # Pass 5
    result = result.replace('"', "_")  # Pass 6
    result = result.replace("<", "_")  # Pass 7
    result = result.replace(">", "_")  # Pass 8
    result = result.replace("|", "_")  # Pass 9
    return result
# 9 passes over the string

Correct (single pass):

SANITIZE_TABLE = str.maketrans({
    "/": "_", "\\": "_", ":": "_", "*": "_",
    "?": "_", '"': "_", "<": "_", ">": "_", "|": "_"
})

def sanitize_filename(name: str) -> str:
    return name.translate(SANITIZE_TABLE)  # Single pass

For removing characters:

# Remove all digits
REMOVE_DIGITS = str.maketrans("", "", "0123456789")
clean = text.translate(REMOVE_DIGITS)

# Remove punctuation
import string
REMOVE_PUNCT = str.maketrans("", "", string.punctuation)
clean = text.translate(REMOVE_PUNCT)

Reference: str.translate documentation