Pytest Advanced

Name: Pytest Advanced
Author: yonatangross

yonatangross/orchestkit

14 installs
213 repo stars
Updated August 4, 2026
yonatangross/orchestkit

Helps with testing & qa tasks.

About

pytest-advanced is a Claude Code skill for testing & qa. It helps solo builders move faster with AI-assisted coding.

pytest-advanced
Testing & QA
AI-coding skill

Pytest Advanced by the numbers

14 all-time installs (skills.sh)
Ranked #1,494 of 2,153 Testing & QA skills by installs in the Skillselion catalog
Data as of Aug 5, 2026 (Skillselion catalog sync)

npx skills add https://github.com/yonatangross/orchestkit --skill pytest-advanced

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/yonatangross/orchestkit/pytest-advanced.svg)](https://skillselion.com/skills/yonatangross/orchestkit/pytest-advanced)

Installs	14
repo stars	★ 213
Last updated	August 4, 2026
Repository	yonatangross/orchestkit ↗

What it does

Helps with testing & qa tasks.

Files

SKILL.mdMarkdownGitHub ↗

Advanced Pytest Patterns

Master pytest's advanced features for scalable, maintainable test suites.

Overview

Building custom test markers for categorization
Writing pytest plugins and hooks
Configuring parallel test execution with pytest-xdist
Creating reusable fixture patterns
Optimizing test collection and execution

Quick Reference

Custom Markers

# pyproject.toml
[tool.pytest.ini_options]
markers = [
    "slow: marks tests as slow (deselect with '-m \"not slow\"')",
    "integration: marks tests requiring external services",
    "smoke: critical path tests for CI/CD",
]

import pytest

@pytest.mark.slow
def test_complex_analysis():
    result = perform_complex_analysis(large_dataset)
    assert result.is_valid

# Run: pytest -m "not slow"  # Skip slow tests
# Run: pytest -m smoke       # Only smoke tests

See custom-plugins.md for plugin development.

Parallel Execution (pytest-xdist)

[tool.pytest.ini_options]
addopts = ["-n", "auto", "--dist", "loadscope"]

@pytest.fixture(scope="session")
def db_engine(worker_id):
    """Isolate database per worker."""
    db_name = "test_db" if worker_id == "master" else f"test_db_{worker_id}"
    engine = create_engine(f"postgresql://localhost/{db_name}")
    yield engine

See xdist-parallel.md for distribution modes.

Factory Fixtures

@pytest.fixture
def user_factory(db_session) -> Callable[..., User]:
    """Factory fixture for creating users."""
    created = []

    def _create(**kwargs) -> User:
        user = User(**{"email": f"u{len(created)}@test.com", **kwargs})
        db_session.add(user)
        created.append(user)
        return user

    yield _create
    for u in created:
        db_session.delete(u)

Key Decisions

Decision	Recommendation
Parallel execution	pytest-xdist with `--dist loadscope`
Marker strategy	Category (smoke, integration) + Resource (db, llm)
Fixture scope	Function default, session for expensive setup
Plugin location	conftest.py for project, package for reuse
Async testing	pytest-asyncio with auto mode

Anti-Patterns (FORBIDDEN)

# NEVER use expensive fixtures without session scope
@pytest.fixture  # WRONG - loads every test
def model():
    return load_ml_model()  # 5s each time!

# NEVER mutate global state
@pytest.fixture
def counter():
    global _counter
    _counter += 1  # WRONG - leaks between tests

# NEVER skip cleanup
@pytest.fixture
def temp_db():
    db = create_db()
    yield db
    # WRONG - missing db.drop()!

# NEVER use time.sleep (use mocking)
def test_timeout():
    time.sleep(5)  # WRONG - slows tests

Related Skills

unit-testing - Basic pytest patterns and AAA structure
integration-testing - Database and API testing patterns
property-based-testing - Hypothesis integration with pytest

References

Xdist Parallel - Parallel execution patterns
Custom Plugins - Plugin and hook development
Conftest Template - Production conftest.py

Capability Details

custom-markers

Keywords: pytest markers, test categorization, smoke tests, slow tests Solves: Categorize tests, run subsets in CI, skip expensive tests

pytest-xdist

Keywords: parallel, xdist, distributed, workers, loadscope Solves: Run tests in parallel, worker isolation, optimize distribution

pytest-hooks

Keywords: hook, plugin, conftest, pytest_configure, collection Solves: Customize pytest behavior, add timing reports, reorder tests

fixture-patterns

Keywords: fixture, factory, async fixture, cleanup, scope Solves: Factory fixtures, async fixtures, ensure cleanup runs

parametrize-advanced

Keywords: parametrize, indirect, cartesian, pytest.param, xfail Solves: Test multiple scenarios, fixtures with params, expected failures

Custom Pytest Plugins

Plugin Types

Local Plugins (conftest.py)

For project-specific functionality. Auto-loaded from any conftest.py.

# conftest.py
import pytest

def pytest_configure(config):
    """Run once at pytest startup."""
    config.addinivalue_line(
        "markers", "smoke: critical path tests"
    )

def pytest_collection_modifyitems(config, items):
    """Reorder tests: smoke first, slow last."""
    items.sort(key=lambda x: (
        0 if x.get_closest_marker("smoke") else
        2 if x.get_closest_marker("slow") else 1
    ))

Installable Plugins

For reusable functionality across projects.

# pytest_timing_plugin.py
import pytest
from datetime import datetime

class TimingPlugin:
    def __init__(self, threshold: float = 1.0):
        self.threshold = threshold
        self.slow_tests = []

    @pytest.hookimpl(hookwrapper=True)
    def pytest_runtest_call(self, item):
        start = datetime.now()
        yield
        duration = (datetime.now() - start).total_seconds()
        if duration > self.threshold:
            self.slow_tests.append((item.nodeid, duration))

    def pytest_terminal_summary(self, terminalreporter):
        if self.slow_tests:
            terminalreporter.write_sep("=", "Slow Tests Report")
            for nodeid, duration in sorted(self.slow_tests, key=lambda x: -x[1]):
                terminalreporter.write_line(f"  {duration:.2f}s - {nodeid}")

def pytest_configure(config):
    config.pluginmanager.register(TimingPlugin(threshold=1.0))

Hook Reference

Collection Hooks

def pytest_collection_modifyitems(config, items):
    """Modify collected tests."""

def pytest_generate_tests(metafunc):
    """Generate parametrized tests dynamically."""

Execution Hooks

@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    """Access test results."""
    outcome = yield
    report = outcome.get_result()
    if report.when == "call" and report.failed:
        # Handle failures
        pass

Setup/Teardown Hooks

def pytest_configure(config):
    """Startup hook."""

def pytest_unconfigure(config):
    """Shutdown hook."""

def pytest_sessionstart(session):
    """Session start."""

def pytest_sessionfinish(session, exitstatus):
    """Session end."""

Publishing a Plugin

# pyproject.toml
[project]
name = "pytest-my-plugin"
version = "1.0.0"

[project.entry-points.pytest11]
my_plugin = "pytest_my_plugin"

pytest-xdist Parallel Execution

Distribution Modes

loadscope (Recommended Default)

Groups tests by module for test functions and by class for test methods. Ideal when fixtures are expensive.

pytest -n auto --dist loadscope

loadfile

Groups tests by file. Good balance of parallelism and fixture sharing.

pytest -n auto --dist loadfile

loadgroup

Tests grouped by @pytest.mark.xdist_group(name="group1") marker.

@pytest.mark.xdist_group(name="database")
def test_create_user():
    pass

@pytest.mark.xdist_group(name="database")
def test_delete_user():
    pass

load

Round-robin distribution for maximum parallelism. Best when tests are truly independent.

pytest -n auto --dist load

Worker Isolation

Each worker is completely isolated:

Global state isn't shared
Environment variables are independent
Temp files/databases must be unique per worker

@pytest.fixture(scope="session")
def db_engine(worker_id):
    """Create isolated database per worker."""
    if worker_id == "master":
        db_name = "test_db"  # Not running in parallel
    else:
        db_name = f"test_db_{worker_id}"  # gw0, gw1, etc.

    engine = create_engine(f"postgresql://localhost/{db_name}")
    yield engine
    engine.dispose()

Resource Allocation

# Auto-detect cores (recommended)
pytest -n auto

# Specific count
pytest -n 4

# Use logical CPUs
pytest -n logical

Warning: Over-provisioning (e.g., -n 20 on 4 cores) increases overhead.

CI/CD Configuration

# GitHub Actions
- name: Run tests in parallel
  run: pytest -n auto --dist loadscope -v
  env:
    PYTEST_XDIST_AUTO_NUM_WORKERS: 4  # Override auto detection

Limitations

-s/--capture=no doesn't work with xdist
Some fixtures may need refactoring for parallelism
Database tests need worker-isolated databases

"""
Advanced pytest configuration template.

Includes:
- Custom markers configuration
- Worker isolation for pytest-xdist
- Factory fixtures
- Test reordering
- Timing reports
"""
import pytest
import time
from typing import Callable, Generator
from datetime import datetime


# =============================================================================
# CONFIGURATION
# =============================================================================

def pytest_configure(config):
    """Configure pytest at startup."""
    config.addinivalue_line("markers", "slow: marks tests as slow")
    config.addinivalue_line("markers", "integration: requires external services")
    config.addinivalue_line("markers", "smoke: critical path tests")
    config.addinivalue_line("markers", "db: requires database connection")
    config.addinivalue_line("markers", "llm: makes LLM API calls (expensive)")

    config.test_start_time = time.time()


def pytest_unconfigure(config):
    """Cleanup at pytest shutdown."""
    elapsed = time.time() - config.test_start_time
    print(f"\nTotal test time: {elapsed:.2f}s")


# =============================================================================
# TEST ORDERING
# =============================================================================

def pytest_collection_modifyitems(config, items):
    """Reorder tests: smoke first, slow last."""
    smoke_tests = []
    slow_tests = []
    other_tests = []

    for item in items:
        if item.get_closest_marker("smoke"):
            smoke_tests.append(item)
        elif item.get_closest_marker("slow"):
            slow_tests.append(item)
        else:
            other_tests.append(item)

    items[:] = smoke_tests + other_tests + slow_tests


# =============================================================================
# WORKER ISOLATION (pytest-xdist)
# =============================================================================

@pytest.fixture(scope="session")
def worker_id(request) -> str:
    """Get worker ID for parallel test isolation."""
    if hasattr(request.config, "workerinput"):
        return request.config.workerinput["workerid"]
    return "master"


@pytest.fixture(scope="session")
def db_name(worker_id: str) -> str:
    """Generate unique database name per worker."""
    if worker_id == "master":
        return "test_db"
    return f"test_db_{worker_id}"


# =============================================================================
# FACTORY FIXTURES
# =============================================================================

@pytest.fixture
def user_factory(db_session) -> Generator[Callable, None, None]:
    """
    Factory fixture for creating test users.

    Usage:
        def test_admin(user_factory):
            admin = user_factory(role="admin")
            user = user_factory(role="user")
    """
    created_users = []

    def _create_user(**kwargs):
        from app.models import User

        defaults = {
            "email": f"user_{len(created_users)}@test.com",
            "name": "Test User",
            "role": "user",
        }
        defaults.update(kwargs)

        user = User(**defaults)
        db_session.add(user)
        db_session.flush()
        created_users.append(user)
        return user

    yield _create_user

    # Cleanup
    for user in created_users:
        db_session.delete(user)
    db_session.flush()


# =============================================================================
# TIMING PLUGIN
# =============================================================================

class SlowTestReporter:
    """Track and report slow tests."""

    def __init__(self, threshold: float = 1.0):
        self.threshold = threshold
        self.slow_tests = []

    @pytest.hookimpl(hookwrapper=True)
    def pytest_runtest_call(self, item):
        start = datetime.now()
        yield
        duration = (datetime.now() - start).total_seconds()
        if duration > self.threshold:
            self.slow_tests.append((item.nodeid, duration))

    def pytest_terminal_summary(self, terminalreporter):
        if self.slow_tests:
            terminalreporter.write_sep("=", f"Slow Tests (>{self.threshold}s)")
            for nodeid, duration in sorted(self.slow_tests, key=lambda x: -x[1]):
                terminalreporter.write_line(f"  {duration:.2f}s - {nodeid}")


def pytest_configure_slow_reporter(config):
    """Register slow test reporter plugin (call from pytest_configure)."""
    config.pluginmanager.register(SlowTestReporter(threshold=1.0), "slow_reporter")