Python Development

Python Development

Core Python language concepts, idioms, and best practices.

When to Use This Skill

Core Expertise

• Python Language: Modern Python 3.10+ features and idioms
• Best Practices: Pythonic code, design patterns, SOLID principles
• Debugging: Interactive debugging and profiling techniques
• Performance: Optimization strategies and profiling
• Async Programming: async/await patterns and asyncio

Modern Python Features (3.10+)

Type Hints

# Modern syntax (Python 3.10+)
def process_items(
    items: list[str],                    # Not List[str]
    mapping: dict[str, int],             # Not Dict[str, int]
    optional: str | None = None,         # Not Optional[str]
) -> tuple[bool, str]:                   # Not Tuple[bool, str]
    """Process items with modern type hints."""
    return True, "success"

# Type aliases
type UserId = int
type UserDict = dict[str, str | int]

def get_user(user_id: UserId) -> UserDict:
    return {"id": user_id, "name": "Alice"}

Pattern Matching (3.10+)

def handle_command(command: dict) -> str:
    match command:
        case {"action": "create", "item": item}:
            return f"Creating {item}"
        case {"action": "delete", "item": item}:
            return f"Deleting {item}"
        case {"action": "list"}:
            return "Listing items"
        case _:
            return "Unknown command"

Structural Pattern Matching

def process_response(response):
    match response:
        case {"status": 200, "data": data}:
            return process_success(data)
        case {"status": 404}:
            raise NotFoundError()
        case {"status": code} if code >= 500:
            raise ServerError(code)

Python Idioms

Context Managers

# File handling
with open("file.txt") as f:
    content = f.read()

# Custom context manager
from contextlib import contextmanager

@contextmanager
def database_connection():
    conn = create_connection()
    try:
        yield conn
    finally:
        conn.close()

with database_connection() as conn:
    conn.execute("SELECT * FROM users")

List Comprehensions

# List comprehension
squares = [x**2 for x in range(10)]

# Dict comprehension
word_lengths = {word: len(word) for word in ["hello", "world"]}

# Set comprehension
unique_lengths = {len(word) for word in ["hello", "world", "hi"]}

# Generator expression
sum_of_squares = sum(x**2 for x in range(1000000))  # Memory efficient

Iterators and Generators

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

# Use generator
fib = fibonacci()
first_ten = [next(fib) for _ in range(10)]

# Generator expression
even_squares = (x**2 for x in range(10) if x % 2 == 0)

Debugging

Interactive Debugging

import pdb

def problematic_function():
    value = calculate()
    pdb.set_trace()  # Debugger breakpoint
    return process(value)

# Debug on error
python -m pdb script.py

# pytest with debugger
uv run pytest --pdb                     # Drop into pdb on failure
uv run pytest --pdb --pdbcls=IPython.terminal.debugger:TerminalPdb

Performance Profiling

# CPU profiling
uv run python -m cProfile -s cumtime script.py | head -20

# Line-by-line profiling (temporary dependency)
uv run --with line-profiler kernprof -l -v script.py

# Memory profiling (temporary dependency)
uv run --with memory-profiler python -m memory_profiler script.py

# Real-time profiling (ephemeral tool)
uvx py-spy top -- python script.py

# Quick profiling with scalene
uv run --with scalene python -m scalene script.py

Built-in Debugging Tools

# Trace execution
import sys

def trace_calls(frame, event, arg):
    if event == 'call':
        print(f"Calling {frame.f_code.co_name}")
    return trace_calls

sys.settrace(trace_calls)

# Memory tracking
import tracemalloc

tracemalloc.start()
# ... code to profile
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
for stat in top_stats[:10]:
    print(stat)

Async Programming

Basic async/await

import asyncio

async def fetch_data(url: str) -> dict:
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        return response.json()

async def main():
    result = await fetch_data("https://api.example.com")
    print(result)

asyncio.run(main())

Concurrent Tasks

async def process_multiple():
    # Run concurrently
    results = await asyncio.gather(
        fetch_data("url1"),
        fetch_data("url2"),
        fetch_data("url3"),
    )
    return results

# With timeout
async def with_timeout():
    try:
        result = await asyncio.wait_for(fetch_data("url"), timeout=5.0)
    except asyncio.TimeoutError:
        print("Request timed out")

Design Patterns

Dependency Injection

from typing import Protocol

class Database(Protocol):
    def query(self, sql: str) -> list: ...

def get_users(db: Database) -> list:
    return db.query("SELECT * FROM users")

Factory Pattern

def create_handler(handler_type: str):
    match handler_type:
        case "json":
            return JSONHandler()
        case "xml":
            return XMLHandler()
        case _:
            raise ValueError(f"Unknown handler: {handler_type}")

Decorator Pattern

from functools import wraps
import time

def timer(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(f"{func.__name__} took {end - start:.2f}s")
        return result
    return wrapper

@timer
def slow_function():
    time.sleep(1)

Best Practices

SOLID Principles

Single Responsibility:

# Bad: Class does too much
class User:
    def save(self): pass
    def send_email(self): pass
    def generate_report(self): pass

# Good: Separate concerns
class User:
    def save(self): pass

class EmailService:
    def send_email(self, user): pass

class ReportGenerator:
    def generate(self, user): pass

Fail Fast

def process_data(data: dict) -> str:
    # Validate early
    if not data:
        raise ValueError("Data cannot be empty")
    if "required_field" not in data:
        raise KeyError("Missing required field")

    # Process with confidence
    return data["required_field"].upper()

Functional Approach

# Prefer immutable transformations
def process_items(items: list[int]) -> list[int]:
    return [item * 2 for item in items]  # New list

# Over mutations
def process_items_bad(items: list[int]) -> None:
    for i in range(len(items)):
        items[i] *= 2  # Mutates input

Project Structure (src layout)

my-project/
├── pyproject.toml
├── README.md
├── src/
│   └── my_project/
│       ├── __init__.py
│       ├── core.py
│       ├── utils.py
│       └── models.py
└── tests/
    ├── conftest.py
    ├── test_core.py
    └── test_utils.py

See Also

• uv-run - Running scripts, temporary dependencies, PEP 723
• uv-project-management - Project setup and dependency management
• uv-tool-management - Installing CLI tools globally
• python-testing - Testing with pytest
• python-code-quality - Linting and type checking with ruff/ty
• python-packaging - Building and publishing packages
• uv-python-versions - Managing Python interpreters

References

• Python docs: https://docs.python.org/3/
• Type hints: https://docs.python.org/3/library/typing.html
• Async: https://docs.python.org/3/library/asyncio.html
• Detailed guide: See REFERENCE.md

Python Development - Detailed Reference

Complete pyproject.toml Configuration

[project]
name = "my-awesome-project"
version = "0.1.0"
description = "A modern Python project"
readme = "README.md"
authors = [
    { name = "Your Name", email = "you@example.com" }
]
license = { text = "MIT" }
requires-python = ">=3.10"
keywords = ["python", "modern", "uv"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
]

dependencies = [
    "requests>=2.31.0",
    "pydantic>=2.0.0",
]

[dependency-groups]
dev = [
    "pytest>=8.0.0",
    "pytest-cov>=4.0.0",
    "ty>=0.0.10",
    "ruff>=0.1.0",
]
docs = [
    "sphinx>=7.0.0",
    "sphinx-rtd-theme>=2.0.0",
]
security = [
    "bandit>=1.7.0",
    "safety>=3.0.0",
]

[project.urls]
Homepage = "https://github.com/user/project"
Documentation = "https://project.readthedocs.io"
Repository = "https://github.com/user/project.git"
Issues = "https://github.com/user/project/issues"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/my_awesome_project"]

Advanced ruff Configuration

[tool.ruff]
target-version = "py310"
line-length = 88
fix = true

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # pyflakes
    "I",    # isort
    "B",    # flake8-bugbear
    "C4",   # flake8-comprehensions
    "UP",   # pyupgrade
    "ARG",  # flake8-unused-arguments
    "SIM",  # flake8-simplify
    "TCH",  # flake8-type-checking
    "PTH",  # flake8-use-pathlib
]
ignore = [
    "E501",  # line too long, handled by formatter
    "B008",  # do not perform function calls in argument defaults
]

[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401"]  # unused imports in __init__.py
"tests/**/*.py" = ["ARG", "S101"]  # allow assert statements in tests

[tool.ruff.lint.isort]
known-first-party = ["my_awesome_project"]

[tool.ruff.format]
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false

Type Checking Configuration

[tool.ty]
python-version = "3.10"
exclude = [
    "**/__pycache__",
    "**/.venv",
]

[tool.ty.rules]
possibly-unbound = "warn"

pytest Configuration

[tool.pytest.ini_options]
minversion = "8.0"
addopts = [
    "--strict-markers",
    "--strict-config",
    "--cov=src",
    "--cov-report=term-missing:skip-covered",
    "--cov-report=html",
    "--cov-report=xml",
    "--cov-fail-under=95",
]
testpaths = ["tests"]
markers = [
    "slow: marks tests as slow",
    "integration: marks tests as integration tests",
]

Advanced Debugging Patterns

# Strategic debugging with pdb
import pdb

def problematic_function(data):
    pdb.set_trace()  # Breakpoint for interactive debugging
    # Or use breakpoint() in Python 3.7+
    result = complex_operation(data)
    return result

# Conditional breakpoints
if suspicious_condition:
    breakpoint()  # Only debug when condition is met

# Post-mortem debugging
import sys
import pdb

try:
    risky_operation()
except Exception:
    pdb.post_mortem()  # Debug at exception point

# Memory leak detection
import tracemalloc
tracemalloc.start()

# ... code to profile ...

current, peak = tracemalloc.get_traced_memory()
print(f"Current memory usage: {current / 10**6:.1f} MB")
tracemalloc.stop()

# Exception context preservation
import traceback

try:
    problematic_code()
except Exception as e:
    tb_str = traceback.format_exc()
    # Log or analyze the complete traceback

Django/Flask Debugging

# Django settings.py for debugging
DEBUG = True
INTERNAL_IPS = ['127.0.0.1']  # For debug toolbar

# Django debug toolbar
INSTALLED_APPS += ['debug_toolbar']
MIDDLEWARE += ['debug_toolbar.middleware.DebugToolbarMiddleware']

# Flask debugging
app = Flask(__name__)
app.debug = True
app.config['PROPAGATE_EXCEPTIONS'] = True

# Werkzeug debugger PIN
import os
os.environ['WERKZEUG_DEBUG_PIN'] = 'off'  # Disable PIN in development

CI/CD Integration

Pre-commit Configuration

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.1.9
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

  - repo: https://github.com/astral-sh/ty
    rev: v0.0.10
    hooks:
      - id: ty

  - repo: https://github.com/PyCQA/bandit
    rev: 1.7.5
    hooks:
      - id: bandit
        args: ["-c", "pyproject.toml"]

GitHub Actions Workflow

# .github/workflows/ci.yml
name: CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v4

      - name: Install uv
        uses: astral-sh/setup-uv@v3
        with:
          version: "latest"

      - name: Set up Python
        run: uv python install ${{ matrix.python-version }}

      - name: Install dependencies
        run: uv sync --all-extras --dev

      - name: Run ruff
        run: uv run ruff check .

      - name: Run type checking
        run: uv run ty check --hide-progress

      - name: Run tests
        run: uv run pytest --cov --cov-report=xml

      - name: Upload coverage
        uses: codecov/codecov-action@v3

Migration from Poetry

# Migrate existing Poetry project
uv init --python $(poetry env info --python)
uv add $(poetry export --dev | sed 's/==.*//')
rm poetry.lock pyproject.toml.bak

Troubleshooting Guide

# Dependency conflicts
uv lock --resolution=highest           # Prefer newer versions
uv add package --no-sync              # Add without syncing

# Environment issues
uv clean                               # Clean cache
uv python install --force 3.12        # Reinstall Python

# Build issues
uv build --sdist                       # Build source distribution only
uv build --wheel                       # Build wheel only