Python Testing

Python Testing Hub

Testing standards for pytest configuration, fixture management, and TDD implementation.

Table of Contents

1. Quick Start 2. When to Use 3. Modules

Quick Start

1. Dependencies: pip install pytest pytest-cov pytest-asyncio pytest-mock 2. Configuration: Add the following to pyproject.toml:

    [tool.pytest.ini_options]
    testpaths = ["tests"]
    addopts = "--cov=src"

3. Verification: Run pytest to confirm discovery of files matching test_*.py.

When To Use

• Constructing unit and integration tests for Python 3.9+ projects.
• Isolating external dependencies using pytest-mock or custom monkeypatching.
• Validating asynchronous logic with pytest-asyncio markers and event loop management.
• Configuring project-wide coverage thresholds and reporting.

When NOT To Use

• Evaluating test

quality - use pensive:test-review instead

• Infrastructure test

config - use leyline:pytest-config

Modules

This skill uses modular loading to manage the system prompt budget.

Core Implementation

• See modules/unit-testing.md - AAA (Arrange-Act-Assert) pattern, basic test structure, and exception validation.
• See modules/fixtures-and-mocking.md - Request-scoped fixtures, parameterization, and boundary mocking.
• See modules/async-testing.md - Coroutine testing, async fixtures, and concurrency validation.

Infrastructure & Workflow

• See modules/test-infrastructure.md - Directory standards, conftest.py management, and coverage tools.
• See modules/testing-workflows.md - Local execution patterns and GitHub Actions integration.

Standards

• See modules/test-quality.md - Identification of common anti-patterns like broad exception catching or shared state between tests.

Exit Criteria

• Tests implement the AAA pattern.
• Coverage reaches the 80% project minimum.
• Individual tests are independent and do not rely on execution order.
• Fixtures are scoped appropriately (function, class, or session) to prevent side effects.
• Mocking is restricted to external system boundaries.

Troubleshooting

• Test Discovery: Verify filenames match the test_*.py pattern. Use pytest --collect-only to debug discovery paths.
• Import Errors: Ensure the local source directory is in the path, typically by installing in editable mode with pip install -e ..
• Async Failures: Confirm that pytest-asyncio is installed and that async tests use the @pytest.mark.asyncio decorator or corresponding auto-mode configuration.

Async Testing

Patterns for testing asynchronous Python code with pytest-asyncio.

Table of Contents

• Basic Async Tests
• Async Fixtures
• Testing Concurrent Operations
• Mocking Async Functions
• Testing Timeouts
• Testing Exception Handling
• Async Context Managers
• Async Generators
• Configuration
• Best Practices
• Common Pitfalls

Basic Async Tests

Test async functions using pytest.mark.asyncio:

import pytest

@pytest.mark.asyncio
async def test_async_function():
    result = await async_operation()
    assert result == "success"

Verify: Run pytest tests/test_async.py -v to execute async tests.

Async Fixtures

Create async fixtures for setup/teardown:

import pytest
from typing import AsyncGenerator

@pytest.fixture
async def async_client() -> AsyncGenerator[AsyncClient, None]:
    """Async fixture with setup and teardown."""
    client = AsyncClient()
    await client.connect()
    yield client
    await client.disconnect()

@pytest.mark.asyncio
async def test_with_async_client(async_client):
    response = await async_client.get("/users")
    assert response.status == 200

Verify: Run pytest tests/test_async_fixtures.py -v to test async fixtures with proper setup/teardown.

Testing Concurrent Operations

Test multiple async operations:

import asyncio
import pytest

@pytest.mark.asyncio
async def test_concurrent_requests():
    async with AsyncAPIClient() as client:
        tasks = [
            client.get_user(1),
            client.get_user(2),
            client.get_user(3),
        ]
        results = await asyncio.gather(*tasks)
        assert len(results) == 3
        assert all(r is not None for r in results)

Mocking Async Functions

Mock async dependencies:

from unittest.mock import AsyncMock, patch

@pytest.mark.asyncio
@patch("aiohttp.ClientSession.get")
async def test_async_api_call(mock_get):
    mock_response = AsyncMock()
    mock_response.json.return_value = {"id": 1, "name": "Test"}
    mock_get.return_value.__aenter__.return_value = mock_response

    client = AsyncAPIClient()
    user = await client.get_user(1)

    assert user["id"] == 1

Testing Timeouts

Test timeout behavior:

@pytest.mark.asyncio
async def test_operation_timeout():
    with pytest.raises(asyncio.TimeoutError):
        await asyncio.wait_for(slow_operation(), timeout=0.1)

Testing Exception Handling

Test async exception handling:

@pytest.mark.asyncio
async def test_async_error_handling():
    client = AsyncAPIClient()

    with pytest.raises(ValueError, match="Invalid user ID"):
        await client.get_user(-1)

Async Context Managers

Test async context managers:

@pytest.mark.asyncio
async def test_async_context_manager():
    async with DatabaseConnection() as conn:
        result = await conn.execute("SELECT 1")
        assert result == [(1,)]

Async Generators

Test async generators:

@pytest.mark.asyncio
async def test_async_generator():
    results = []
    async for item in async_stream():
        results.append(item)
        if len(results) >= 3:
            break

    assert len(results) == 3

Configuration

Add to pyproject.toml:

[tool.pytest.ini_options]
asyncio_mode = "auto"  # Automatically detect async tests

Best Practices

1. Use `asyncio_mode = "auto"` - Automatically detect async tests 2. Clean up resources - Use async fixtures for proper teardown 3. Test concurrency - Verify behavior under concurrent execution 4. Mock async dependencies - Use AsyncMock for async mocks 5. Test timeout scenarios - Verify timeout handling 6. Avoid mixing sync/async - Keep async tests separate from sync tests

Common Pitfalls

# Bad: Forgetting await
@pytest.mark.asyncio
async def test_bad():
    result = async_function()  # Returns coroutine, not result
    assert result == "success"  # Fails

# Good: Awaiting properly
@pytest.mark.asyncio
async def test_good():
    result = await async_function()
    assert result == "success"

# Bad: Not awaiting in fixture
@pytest.fixture
async def bad_fixture():
    client = AsyncClient()
    client.connect()  # Missing await
    yield client

# Good: Awaiting in fixture
@pytest.fixture
async def good_fixture():
    client = AsyncClient()
    await client.connect()
    yield client
    await client.disconnect()

Fixtures and Mocking

Advanced patterns for test setup, teardown, parameterization, and mocking external dependencies.

Table of Contents

• Fixtures for Setup/Teardown
• Fixture Scopes
• Parameterized Tests
• Multiple Parameters
• Mocking External Dependencies
• Mock Patterns
• Mocking Best Practices
• Fixture Composition

Fixtures for Setup/Teardown

Fixtures provide reusable setup and teardown logic:

import pytest
from typing import Generator

@pytest.fixture
def db_session() -> Generator[Session, None, None]:
    """Fixture that provides database session."""
    session = Session()
    session.begin()
    yield session
    session.rollback()
    session.close()

def test_user_creation(db_session):
    user = User(name="Test")
    db_session.add(user)
    db_session.flush()
    assert user.id is not None

Verify: Run pytest tests/test_fixtures.py -v to confirm fixtures handle setup/teardown correctly.

Fixture Scopes

@pytest.fixture(scope="function")  # Default: new instance per test
def user():
    return User(name="Test")

@pytest.fixture(scope="class")  # Shared across test class
def api_client():
    return APIClient()

@pytest.fixture(scope="module")  # Shared across module
def database():
    db = Database()
    db.connect()
    yield db
    db.disconnect()

@pytest.fixture(scope="session")  # Once per test session
def app_config():
    return load_config()

Parameterized Tests

Test multiple inputs efficiently:

@pytest.mark.parametrize("email,is_valid", [
    ("user@example.com", True),
    ("test.user@domain.co.uk", True),
    ("invalid.email", False),
    ("@example.com", False),
    ("", False),
])
def test_email_validation(email, is_valid):
    assert validate_email(email) == is_valid

Multiple Parameters

@pytest.mark.parametrize("input_value,expected", [
    (0, "zero"),
    (1, "one"),
    (5, "many"),
])
@pytest.mark.parametrize("locale", ["en", "es", "fr"])
def test_number_formatting(input_value, expected, locale):
    result = format_number(input_value, locale)
    assert result is not None  # Locale-specific checks

Mocking External Dependencies

Mock external services and APIs:

from unittest.mock import Mock, patch

@patch("requests.get")
def test_api_client(mock_get):
    mock_get.return_value.json.return_value = {"id": 1, "name": "Test"}
    mock_get.return_value.raise_for_status.return_value = None

    client = APIClient("https://api.example.com")
    user = client.get_user(1)

    assert user["id"] == 1
    mock_get.assert_called_once()

Mock Patterns

# Mock with return value
mock_service = Mock(return_value={"status": "success"})

# Mock with side effects
mock_service = Mock(side_effect=[ValueError(), {"data": "ok"}])

# Mock attributes
mock_obj = Mock()
mock_obj.user.email = "test@example.com"

# Verify mock calls
mock_service.assert_called_once_with(param="value")
mock_service.assert_called_with(param="value")
assert mock_service.call_count == 2

Mocking Best Practices

1. Mock at boundaries - Only mock external dependencies (APIs, databases, file systems) 2. Don't over-mock - Avoid mocking simple calculations or pure functions 3. Use patch decorators - Apply @patch at the usage location, not definition 4. Verify interactions - Use assert_called_* to verify expected calls 5. Return realistic data - Mock responses should match actual API responses

Fixture Composition

Combine fixtures for complex setups:

@pytest.fixture
def admin_user(db_session):
    user = User(name="Admin", role="admin")
    db_session.add(user)
    db_session.flush()
    return user

@pytest.fixture
def authenticated_client(admin_user):
    client = APIClient()
    client.authenticate(admin_user)
    return client

def test_admin_endpoint(authenticated_client):
    response = authenticated_client.get("/admin/users")
    assert response.status_code == 200

Test Infrastructure

Configuration and structure for Python test suites.

Table of Contents

• pyproject.toml Configuration
• Test Directory Structure
• conftest.py
• Dependencies
• Coverage Configuration

pyproject.toml Configuration

Complete pytest configuration in pyproject.toml:

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
addopts = [
    "-v",
    "--cov=src",
    "--cov-report=term-missing",
    "--cov-fail-under=80",
]
markers = [
    "slow: marks tests as slow",
    "integration: marks integration tests",
    "unit: marks unit tests",
]

[tool.coverage.run]
source = ["src"]
omit = ["*/tests/*", "*/migrations/*"]

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "raise NotImplementedError",
]

Verify: Run pytest --collect-only to confirm test discovery works with this configuration.

Test Directory Structure

Organized test layout:

tests/
├── conftest.py           # Shared fixtures
├── unit/                 # Unit tests
│   ├── test_models.py
│   └── test_utils.py
├── integration/          # Integration tests
│   ├── test_api.py
│   └── test_database.py
└── fixtures/             # Test data
    └── sample_data.json

conftest.py

Shared fixtures and configuration:

import pytest
from pathlib import Path

@pytest.fixture
def fixtures_path() -> Path:
    """Path to test fixtures directory."""
    return Path(__file__).parent / "fixtures"

@pytest.fixture
def sample_data(fixtures_path):
    """Load sample test data."""
    import json
    with open(fixtures_path / "sample_data.json") as f:
        return json.load(f)

# Pytest configuration hooks
def pytest_configure(config):
    """Configure pytest."""
    config.addinivalue_line("markers", "smoke: Quick smoke tests")

def pytest_collection_modifyitems(items):
    """Modify test collection."""
    for item in items:
        if "integration" in item.nodeid:
            item.add_marker(pytest.mark.slow)

Dependencies

Add testing dependencies to pyproject.toml:

[project.optional-dependencies]
dev = [
    "pytest>=8.0.0",
    "pytest-cov>=4.1.0",
    "pytest-asyncio>=0.23.0",
    "pytest-xdist>=3.5.0",  # Parallel execution
    "pytest-mock>=3.12.0",
]

Coverage Configuration

Fine-tune coverage reporting:

[tool.coverage.run]
source = ["src"]
branch = true  # Branch coverage
omit = [
    "*/tests/*",
    "*/migrations/*",
    "*/__init__.py",
]

[tool.coverage.report]
precision = 2
skip_empty = true
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "def __str__",
    "raise AssertionError",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
    "@abstractmethod",
]

Test Quality

Guidelines for writing high-quality, maintainable tests.

Table of Contents

• Best Practices
• Anti-Patterns to Avoid
• Testing Private Methods Directly
• Over-mocking Simple Calculations
• Shared Mutable State
• Order-Dependent Tests
• Assertions Without Clear Messages
• Exit Criteria

Best Practices

1. Test behavior, not implementation - Focus on public interfaces

   # Good: Test behavior
   def test_user_can_login():
       user = authenticate("user@example.com", "password")
       assert user.is_authenticated

   # Bad: Test implementation details
   def test_password_hash_algorithm():
       assert user._hash_password("pass").startswith("$2b$")

2. One assertion per test - Keep tests focused

   # Good: Single concern
   def test_user_creation_sets_email():
       user = User.create(email="test@example.com")
       assert user.email == "test@example.com"

   def test_user_creation_generates_id():
       user = User.create(email="test@example.com")
       assert user.id is not None

3. Independent tests - No shared state between tests

   # Good: Each test is independent
   @pytest.fixture
   def user():
       return User(name="Test")

   # Bad: Tests share state
   shared_user = User(name="Test")

4. Descriptive names - Make test intent clear

   # Good: Clear intent
   def test_user_creation_with_invalid_email_raises_value_error():
       with pytest.raises(ValueError):
           User.create(email="invalid")

   # Bad: Unclear
   def test_user_error():
       ...

5. Use fixtures - Avoid setup duplication 6. Mock at boundaries - Only mock external dependencies 7. Measure coverage - Aim for meaningful, not just high

Anti-Patterns to Avoid

Testing Private Methods Directly

# Bad: Testing private methods
def test_private_hash_function():
    assert User._hash_password("test") == "..."

# Good: Test through public interface
def test_password_verification():
    user = User.create(password="test")
    assert user.verify_password("test")

Over-mocking Simple Calculations

# Bad: Mocking simple logic
@patch("math.sqrt")
def test_calculation(mock_sqrt):
    mock_sqrt.return_value = 3
    assert calculate_distance(0, 0, 3, 4) == 5

# Good: Test actual calculation
def test_calculation():
    assert calculate_distance(0, 0, 3, 4) == 5

Shared Mutable State

# Bad: Shared mutable state
cache = {}

def test_cache_set():
    cache["key"] = "value"
    assert cache["key"] == "value"

def test_cache_empty():  # Fails if test_cache_set runs first
    assert len(cache) == 0

Order-Dependent Tests

# Bad: Tests depend on order
def test_step_1():
    global state
    state = "initialized"

def test_step_2():
    assert state == "initialized"  # Depends on test_step_1

Assertions Without Clear Messages

# Bad: Unclear failure
assert len(users) > 0

# Good: Clear failure message
assert len(users) > 0, f"Expected users but got empty list"

Exit Criteria

Before considering tests complete:

• [ ] Tests follow AAA pattern
• [ ] Coverage meets project threshold (≥80%)
• [ ] All tests independent and reproducible
• [ ] CI/CD integration configured
• [ ] Clear test naming and organization
• [ ] No anti-patterns present
• [ ] Fixtures used appropriately
• [ ] Mocking only at boundaries

Testing Workflows

Running tests effectively and integrating with CI/CD pipelines.

Table of Contents

• Running Tests
• CI/CD Integration
• GitHub Actions
• GitLab CI
• Pre-commit Hooks
• Makefile Integration
• Coverage Reporting
• HTML Report
• Terminal Report
• XML Report (for CI)
• Debugging Tests

Running Tests

Common pytest commands:

# Run all tests
pytest

# Run with coverage
pytest --cov=src --cov-report=html

# Run specific markers
pytest -m unit
pytest -m "not slow"

# Run failed tests first
pytest --failed-first

# Parallel execution
pytest -n auto

# Run specific test file
pytest tests/unit/test_models.py

# Run specific test function
pytest tests/unit/test_models.py::test_user_creation

# Verbose output
pytest -v

# Show print statements
pytest -s

# Stop on first failure
pytest -x

# Run last failed tests
pytest --lf

CI/CD Integration

GitHub Actions

Complete workflow in .github/workflows/test.yml:

name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}

      - name: Install dependencies
        run: |
          pip install -e ".[dev]"

      - name: Run tests
        run: pytest --cov --cov-report=xml

      - name: Upload coverage
        uses: codecov/codecov-action@v4
        if: matrix.python-version == '3.12'

GitLab CI

.gitlab-ci.yml configuration:

test:
  image: python:3.12
  before_script:
    - pip install -e ".[dev]"
  script:
    - pytest --cov --cov-report=term --cov-report=xml
  coverage: '/TOTAL.*\s+(\d+%)$/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.xml

Pre-commit Hooks

Add testing to .pre-commit-config.yaml:

repos:
  - repo: local
    hooks:
      - id: pytest-check
        name: pytest
        entry: pytest
        language: system
        pass_filenames: false
        always_run: true
        stages: [commit]

Makefile Integration

Convenient test commands in Makefile:

.PHONY: test test-fast test-cov test-watch

test:
	pytest -v

test-fast:
	pytest -m "not slow" -x

test-cov:
	pytest --cov=src --cov-report=html --cov-report=term

test-watch:
	pytest-watch -c

Coverage Reporting

HTML Report

pytest --cov=src --cov-report=html
open htmlcov/index.html  # View in browser

Terminal Report

pytest --cov=src --cov-report=term-missing

XML Report (for CI)

pytest --cov=src --cov-report=xml

Debugging Tests

# Drop into debugger on failure
pytest --pdb

# Drop into debugger at test start
pytest --trace

# Show local variables on failure
pytest -l

# Show full diff output
pytest -vv

Unit Testing Fundamentals

Core patterns for writing effective unit tests with pytest.

Test Structure (AAA Pattern)

The Arrange-Act-Assert pattern provides clear test organization:

def test_user_creation():
    # Arrange
    data = {"email": "test@example.com", "name": "Test User"}

    # Act
    user = User.create(**data)

    # Assert
    assert user.email == data["email"]
    assert user.is_valid()

Basic pytest Tests

Simple test examples demonstrating pytest fundamentals:

import pytest

class Calculator:
    def divide(self, a: float, b: float) -> float:
        if b == 0:
            raise ValueError("Cannot divide by zero")
        return a / b

def test_division():
    calc = Calculator()
    assert calc.divide(6, 3) == 2

def test_division_by_zero():
    calc = Calculator()
    with pytest.raises(ValueError, match="Cannot divide by zero"):
        calc.divide(5, 0)

Key Principles

1. One test, one behavior - Each test validates a single behavior 2. Independent tests - Tests should not depend on each other 3. Descriptive names - Use names like test_user_creation_with_invalid_email_raises_error 4. Clear assertions - Make test intent obvious through assertion messages

Common Assertions

# Equality
assert result == expected

# Boolean checks
assert user.is_active
assert not user.is_deleted

# Exception testing
with pytest.raises(ValueError):
    validate_email("invalid")

# Exception with message matching
with pytest.raises(ValueError, match="Cannot divide by zero"):
    divide(5, 0)

# Collection membership
assert "admin" in user.roles
assert user.id not in deleted_ids