Python Architect

# Contributing to MyPackage

Thank you for your interest in contributing to MyPackage! This document provides guidelines and instructions for getting involved.

## Code of Conduct

This project adheres to the Contributor Covenant [Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to [maintainer-email@example.com].

## Getting Started

### Development Setup

1. **Clone the repository**
   ```bash
   git clone https://github.com/username/mypackage.git
   cd mypackage
   ```

2. **Create a virtual environment**
   ```bash
   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   ```

3. **Install development dependencies**
   ```bash
   pip install -e ".[dev]"
   ```

4. **Install pre-commit hooks**
   ```bash
   pre-commit install
   ```

### Testing Your Setup

Run the test suite to verify everything works:

```bash
pytest
```

## How to Contribute

### Report Bugs

**Before creating a bug report:**
- Check existing issues to avoid duplicates
- Verify the issue reproduces on the latest main branch

**When creating a bug report, include:**
- Clear, descriptive title
- Exact steps to reproduce
- Expected vs actual behavior
- Python version and OS
- Relevant code snippets
- Full error traceback if applicable

### Suggest Enhancements

**Before requesting a feature:**
- Check existing issues and discussions
- Ensure it aligns with the library's scope

**When suggesting an enhancement:**
- Clear use case and motivation
- Example of how you'd use the feature
- Possible drawbacks or alternatives

### Code Contributions

We follow a standard fork-and-pull-request workflow:

1. **Fork the repository** on GitHub

2. **Create a feature branch**
   ```bash
   git checkout -b feature/your-feature-name
   ```

3. **Make your changes**
   - Follow the code style guidelines below
   - Add tests for new functionality
   - Update documentation as needed

4. **Run checks locally**
   ```bash
   # Format code
   black src/ tests/

   # Lint code
   ruff check --fix src/ tests/

   # Type check
   mypy src/

   # Run tests
   pytest

   # Check coverage
   pytest --cov=src/mypackage
   ```

5. **Commit with clear messages**
   ```bash
   git commit -m "feat: Add new feature"
   git commit -m "fix: Resolve bug in validator"
   ```
   Use conventional commits: `feat:`, `fix:`, `docs:`, `test:`, `refactor:`, `chore:`

6. **Push and create a pull request**
   ```bash
   git push origin feature/your-feature-name
   ```
   Open a PR on GitHub with a clear description

## Code Style Guidelines

### Python Style

We use the following tools to maintain consistent code style:

- **Black** for code formatting (88-character line length)
- **Ruff** for linting
- **mypy** for type checking (strict mode)
- **isort** for import sorting

### Type Hints

All public APIs **must** have type hints:

```python
from typing import Optional
from collections.abc import Sequence

def process_items(
    items: Sequence[str],
    limit: Optional[int] = None
) -> dict[str, int]:
    """Process items and return frequency map.

    Args:
        items: Items to process
        limit: Optional limit on items to process

    Returns:
        Frequency map of items
    """
    ...
```

### Docstrings

Use Google-style docstrings for all public functions and classes:

```python
def calculate(x: int, y: int) -> int:
    """Calculate sum of two numbers.

    Args:
        x: First number
        y: Second number

    Returns:
        Sum of x and y

    Raises:
        TypeError: If inputs are not integers

    Example:
        >>> calculate(2, 3)
        5
    """
    ...
```

### Naming Conventions

- **Functions/variables**: `lowercase_with_underscores`
- **Classes**: `PascalCase`
- **Constants**: `UPPER_CASE_WITH_UNDERSCORES`
- **Private/internal**: `_leading_underscore`

## Testing Guidelines

### Writing Tests

Tests go in the `tests/` directory and use pytest:

```python
import pytest
from mypackage.core import process

def test_process_basic():
    """Test basic processing."""
    result = process("input")
    assert result == "expected_output"

@pytest.mark.parametrize("input,expected", [
    ("a", "A"),
    ("b", "B"),
])
def test_process_varied(input, expected):
    """Test with varied inputs."""
    assert process(input) == expected

def test_process_error():
    """Test error handling."""
    with pytest.raises(ValueError):
        process(None)
```

### Coverage Requirements

- Aim for >90% code coverage on public APIs
- All error paths should be tested
- Both success and failure cases

Run tests with coverage:

```bash
pytest --cov=src/mypackage --cov-report=html
```

## Documentation

### Updating Documentation

Documentation lives in `docs/source/`:

1. **API Documentation**: Generated from docstrings via Sphinx
   - Keep docstrings up-to-date with code changes
   - Use type hints and examples

2. **User Guides**: In `docs/source/guide/`
   - Explain concepts with examples
   - Show common use cases

3. **README.md**:
   - Keep quick-start examples current
   - Update feature list

### Building Documentation Locally

```bash
cd docs
make html
# View at docs/build/html/index.html
```

## Commit Message Conventions

Use [Conventional Commits](https://www.conventionalcommits.org/):

```
<type>(<scope>): <subject>

<body>

<footer>
```

Types:
- `feat`: New feature
- `fix`: Bug fix
- `docs`: Documentation
- `style`: Code style (formatting, etc.)
- `refactor`: Code refactoring
- `test`: Adding/updating tests
- `chore`: Build, dependencies, etc.

Example:
```
feat(processor): Add batch processing support

Implements parallel processing for large datasets.
Improves performance by 3x for typical use cases.

Closes #42
```

## Pull Request Process

1. **Update CHANGELOG.md** with your changes
2. **Ensure tests pass** and coverage is maintained
3. **Get review approval** from maintainers
4. **Squash commits** if requested (keep history clean)
5. **PR gets merged** once approved

### PR Guidelines

- **One feature per PR** (unless closely related)
- **Descriptive title and description**
- **Reference issues** with "Closes #123" or "Fixes #123"
- **Keep scope focused** (avoid mixing refactoring with features)

## Release Process

Maintainers handle releases following semantic versioning:

- **MAJOR**: Breaking changes
- **MINOR**: New features (backwards compatible)
- **PATCH**: Bug fixes (backwards compatible)

Releases are tagged, published to PyPI, and documented in CHANGELOG.md.

## Asking for Help

- **Questions about code**: Open a discussion on GitHub
- **Design feedback**: Comment on relevant issues
- **Need guidance**: Check documentation or ask maintainers
- **Found a problem**: Open an issue with details

## Recognition

Contributors are recognized in:
- CHANGELOG.md (for significant contributions)
- GitHub contributors page
- Release notes

Thank you for contributing! 🎉

"""Example configuration patterns for Python libraries.

This file demonstrates best practices for handling configuration
in production-grade Python libraries.
"""

import os
from dataclasses import dataclass, field
from typing import Optional
from pathlib import Path


# Pattern 1: Dataclass-based Configuration
# ==========================================

@dataclass
class DatabaseConfig:
    """Database connection configuration.

    Attributes:
        host: Database host address
        port: Database port
        username: Database username
        password: Database password (use env var, never in code)
        database: Database name
        timeout: Connection timeout in seconds
        pool_size: Connection pool size
        ssl: Whether to use SSL for connection
    """

    host: str = "localhost"
    port: int = 5432
    username: str = "user"
    password: str = ""  # Set from environment
    database: str = "mydb"
    timeout: int = 30
    pool_size: int = 10
    ssl: bool = True

    def __post_init__(self):
        """Validate configuration after initialization."""
        if self.port < 1 or self.port > 65535:
            raise ValueError(f"Invalid port number: {self.port}")
        if self.timeout <= 0:
            raise ValueError(f"Timeout must be positive, got {self.timeout}")
        if self.pool_size < 1:
            raise ValueError(f"Pool size must be positive, got {self.pool_size}")


@dataclass
class LibraryConfig:
    """Main library configuration.

    Attributes:
        debug: Enable debug mode
        log_level: Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
        cache_enabled: Whether to enable caching
        cache_ttl: Cache time-to-live in seconds
        database: Database configuration
        timeout: Default operation timeout in seconds
        max_retries: Maximum number of retries for operations
        retry_backoff: Exponential backoff multiplier for retries
    """

    debug: bool = False
    log_level: str = "INFO"
    cache_enabled: bool = True
    cache_ttl: int = 3600  # 1 hour
    database: DatabaseConfig = field(default_factory=DatabaseConfig)
    timeout: int = 30
    max_retries: int = 3
    retry_backoff: float = 2.0

    @classmethod
    def from_env(cls) -> "LibraryConfig":
        """Create configuration from environment variables.

        Environment variables:
            MYLIB_DEBUG: Set to "1", "true", "yes" for debug mode
            MYLIB_LOG_LEVEL: Logging level (default: INFO)
            MYLIB_CACHE_ENABLED: Set to "0", "false", "no" to disable cache
            MYLIB_CACHE_TTL: Cache TTL in seconds (default: 3600)
            MYLIB_DB_HOST: Database host (default: localhost)
            MYLIB_DB_PORT: Database port (default: 5432)
            MYLIB_DB_USER: Database username (default: user)
            MYLIB_DB_PASSWORD: Database password (required if used)
            MYLIB_DB_NAME: Database name (default: mydb)
            MYLIB_TIMEOUT: Operation timeout (default: 30)
            MYLIB_MAX_RETRIES: Max retries (default: 3)
        """
        return cls(
            debug=os.getenv("MYLIB_DEBUG", "").lower() in ("1", "true", "yes"),
            log_level=os.getenv("MYLIB_LOG_LEVEL", "INFO"),
            cache_enabled=os.getenv("MYLIB_CACHE_ENABLED", "1").lower() not in ("0", "false", "no"),
            cache_ttl=int(os.getenv("MYLIB_CACHE_TTL", "3600")),
            database=DatabaseConfig(
                host=os.getenv("MYLIB_DB_HOST", "localhost"),
                port=int(os.getenv("MYLIB_DB_PORT", "5432")),
                username=os.getenv("MYLIB_DB_USER", "user"),
                password=os.getenv("MYLIB_DB_PASSWORD", ""),
                database=os.getenv("MYLIB_DB_NAME", "mydb"),
            ),
            timeout=int(os.getenv("MYLIB_TIMEOUT", "30")),
            max_retries=int(os.getenv("MYLIB_MAX_RETRIES", "3")),
        )

    @classmethod
    def from_file(cls, config_path: str) -> "LibraryConfig":
        """Load configuration from a YAML or JSON file.

        Args:
            config_path: Path to configuration file

        Returns:
            Loaded configuration

        Raises:
            FileNotFoundError: If config file doesn't exist
            ValueError: If config file format is invalid
        """
        import json
        from pathlib import Path

        path = Path(config_path)
        if not path.exists():
            raise FileNotFoundError(f"Configuration file not found: {config_path}")

        if path.suffix == ".json":
            with open(path) as f:
                data = json.load(f)
        elif path.suffix in (".yaml", ".yml"):
            try:
                import yaml
            except ImportError:
                raise ImportError("PyYAML required for YAML config. Install with: pip install pyyaml")
            with open(path) as f:
                data = yaml.safe_load(f)
        else:
            raise ValueError(f"Unsupported config format: {path.suffix}")

        # Convert dict to config object
        return cls(**data)


# Pattern 2: Builder Pattern for Complex Configuration
# ======================================================

class ConfigBuilder:
    """Builder for constructing complex configurations.

    Allows gradual configuration with sensible defaults:

        config = (ConfigBuilder()
            .with_debug(True)
            .with_database("postgresql://localhost/mydb")
            .with_cache_ttl(7200)
            .build())
    """

    def __init__(self):
        """Initialize builder with defaults."""
        self._debug = False
        self._log_level = "INFO"
        self._cache_enabled = True
        self._cache_ttl = 3600
        self._db_config = DatabaseConfig()
        self._timeout = 30
        self._max_retries = 3

    def with_debug(self, debug: bool) -> "ConfigBuilder":
        """Enable/disable debug mode."""
        self._debug = debug
        return self

    def with_log_level(self, level: str) -> "ConfigBuilder":
        """Set logging level."""
        if level not in ("DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"):
            raise ValueError(f"Invalid log level: {level}")
        self._log_level = level
        return self

    def with_database(self, host: str, port: int = 5432, **kwargs) -> "ConfigBuilder":
        """Configure database connection."""
        self._db_config = DatabaseConfig(
            host=host,
            port=port,
            **kwargs
        )
        return self

    def with_cache(self, enabled: bool = True, ttl: int = 3600) -> "ConfigBuilder":
        """Configure caching."""
        self._cache_enabled = enabled
        self._cache_ttl = ttl
        return self

    def with_timeout(self, timeout: int) -> "ConfigBuilder":
        """Set operation timeout."""
        if timeout <= 0:
            raise ValueError(f"Timeout must be positive, got {timeout}")
        self._timeout = timeout
        return self

    def build(self) -> LibraryConfig:
        """Build the final configuration."""
        return LibraryConfig(
            debug=self._debug,
            log_level=self._log_level,
            cache_enabled=self._cache_enabled,
            cache_ttl=self._cache_ttl,
            database=self._db_config,
            timeout=self._timeout,
        )


# Pattern 3: Configuration with Optional Dependencies
# ====================================================

@dataclass
class OptionalFeaturesConfig:
    """Configuration for optional features.

    Only load dependencies if features are actually used.
    """

    enable_caching: bool = True
    cache_backend: str = "memory"  # "memory", "redis"
    enable_compression: bool = False
    compression_level: int = 6

    def get_cache_backend(self):
        """Lazy-load cache backend only if needed."""
        if not self.enable_caching:
            return None

        if self.cache_backend == "memory":
            return MemoryCache()
        elif self.cache_backend == "redis":
            try:
                import redis
            except ImportError:
                raise ImportError(
                    "Redis cache backend requires 'redis' package. "
                    "Install with: pip install redis"
                )
            return RedisCache()
        else:
            raise ValueError(f"Unknown cache backend: {self.cache_backend}")


class MemoryCache:
    """Simple in-memory cache."""

    pass


class RedisCache:
    """Redis-backed cache."""

    pass


# Pattern 4: Validation and Defaults
# ===================================

@dataclass
class APIConfig:
    """API configuration with validation.

    Demonstrates setting sensible defaults and validating
    configuration values.
    """

    api_key: str
    api_secret: str
    base_url: str = "https://api.example.com"
    timeout: int = 30
    max_connections: int = 100
    verify_ssl: bool = True

    def __post_init__(self):
        """Validate configuration after initialization."""
        if not self.api_key:
            raise ValueError("api_key is required and cannot be empty")
        if not self.api_secret:
            raise ValueError("api_secret is required and cannot be empty")
        if not self.base_url:
            raise ValueError("base_url is required and cannot be empty")
        if self.timeout <= 0:
            raise ValueError(f"timeout must be positive, got {self.timeout}")
        if self.max_connections < 1:
            raise ValueError(f"max_connections must be at least 1, got {self.max_connections}")

    @classmethod
    def from_env(cls) -> "APIConfig":
        """Load API configuration from environment variables.

        Environment variables:
            MYLIB_API_KEY (required)
            MYLIB_API_SECRET (required)
            MYLIB_API_URL (optional, default: https://api.example.com)
            MYLIB_API_TIMEOUT (optional, default: 30)
            MYLIB_API_VERIFY_SSL (optional, default: true)
        """
        api_key = os.getenv("MYLIB_API_KEY")
        if not api_key:
            raise ValueError(
                "MYLIB_API_KEY environment variable is required. "
                "Set it with: export MYLIB_API_KEY=your_key"
            )

        api_secret = os.getenv("MYLIB_API_SECRET")
        if not api_secret:
            raise ValueError(
                "MYLIB_API_SECRET environment variable is required. "
                "Set it with: export MYLIB_API_SECRET=your_secret"
            )

        return cls(
            api_key=api_key,
            api_secret=api_secret,
            base_url=os.getenv("MYLIB_API_URL", "https://api.example.com"),
            timeout=int(os.getenv("MYLIB_API_TIMEOUT", "30")),
            verify_ssl=os.getenv("MYLIB_API_VERIFY_SSL", "true").lower() in ("1", "true", "yes"),
        )


# Usage Examples
# ==============

def example_dataclass_config():
    """Example: Create configuration with dataclass."""
    config = LibraryConfig(
        debug=True,
        log_level="DEBUG",
        database=DatabaseConfig(
            host="db.example.com",
            username="admin",
            password=os.getenv("DB_PASSWORD"),
        ),
        timeout=60,
    )
    print(f"Config: {config}")


def example_from_env():
    """Example: Load configuration from environment variables."""
    # Set environment variables first:
    # export MYLIB_DEBUG=1
    # export MYLIB_LOG_LEVEL=DEBUG
    # export MYLIB_DB_HOST=db.example.com

    config = LibraryConfig.from_env()
    print(f"Debug mode: {config.debug}")
    print(f"Database host: {config.database.host}")


def example_from_file():
    """Example: Load configuration from file."""
    # config.json:
    # {
    #   "debug": true,
    #   "log_level": "DEBUG",
    #   "database": {
    #     "host": "db.example.com",
    #     "password": "secret"
    #   }
    # }

    config = LibraryConfig.from_file("config.json")
    print(f"Config loaded from file: {config}")


def example_builder():
    """Example: Use builder pattern for fluent configuration."""
    config = (ConfigBuilder()
              .with_debug(True)
              .with_log_level("DEBUG")
              .with_database("db.example.com", port=5432, username="admin")
              .with_cache(enabled=True, ttl=7200)
              .with_timeout(60)
              .build())
    print(f"Config built with builder: {config}")


# Best Practices Summary
# ======================
"""
1. USE DATACLASSES FOR CONFIGURATION
   - Cleaner than writing __init__ manually
   - Type hints provide IDE support
   - Easy serialization/deserialization

2. PROVIDE MULTIPLE LOADING METHODS
   - from_env(): Load from environment variables
   - from_file(): Load from config files
   - Direct instantiation: Simple cases

3. VALIDATE IN __post_init__
   - Check required values
   - Validate ranges and constraints
   - Provide helpful error messages

4. USE SENSIBLE DEFAULTS
   - Default values prevent required arguments
   - Documents expected configuration
   - Simplifies common use cases

5. BUILDER PATTERN FOR COMPLEX CONFIGS
   - Fluent interface for readability
   - Gradual construction
   - Works well with optional features

6. NEVER HARDCODE CREDENTIALS
   - Use environment variables
   - Use config files (not in repo)
   - Document required env variables

7. LAZY LOAD OPTIONAL DEPENDENCIES
   - Only import when feature is used
   - Provide helpful error messages
   - Keep core library lightweight

8. CONSISTENT NAMING
   - Environment variables: MYLIB_FEATURE_SETTING
   - Config file keys: feature.setting
   - Python attributes: feature_setting (snake_case)
"""

"""Example custom exception hierarchy for a Python library.

This file demonstrates best practices for defining exception hierarchies
in production-grade Python libraries.
"""


class MyLibraryError(Exception):
    """Base exception for all mylib errors.

    All exceptions raised by the library inherit from this class,
    allowing users to catch all library-specific errors with:

        try:
            ...
        except MyLibraryError as e:
            ...
    """

    pass


# Configuration-related errors
class ConfigurationError(MyLibraryError):
    """Raised when configuration is invalid.

    Example:
        raise ConfigurationError(
            f"Missing required config key 'database_url'. "
            f"Provide via config parameter or DATABASE_URL env variable."
        )
    """

    pass


class MissingConfigError(ConfigurationError):
    """Raised when required configuration is missing."""

    pass


class InvalidConfigError(ConfigurationError):
    """Raised when configuration values are invalid."""

    pass


# Validation-related errors
class ValidationError(MyLibraryError):
    """Raised when input validation fails.

    Example:
        if not isinstance(age, int):
            raise ValidationError(
                f"Parameter 'age' must be an integer, got {type(age).__name__}"
            )
    """

    pass


class InputError(ValidationError):
    """Raised when user input is invalid."""

    pass


class DataFormatError(ValidationError):
    """Raised when data format is not as expected."""

    pass


# API/Communication errors
class APIError(MyLibraryError):
    """Raised when API operations fail.

    Example:
        try:
            response = requests.get(url)
            response.raise_for_status()
        except requests.RequestException as e:
            raise APIError(
                f"Failed to fetch from {url}: {str(e)}. "
                f"Check your network connection and URL."
            ) from e
    """

    pass


class NetworkError(APIError):
    """Raised when network operations fail."""

    pass


class TimeoutError(APIError):
    """Raised when operation times out.

    Note: This shadows the built-in TimeoutError. Consider using
    a different name like `OperationTimeout` if needed.
    """

    pass


# Resource-related errors
class ResourceError(MyLibraryError):
    """Raised when resource operations fail."""

    pass


class ResourceNotFoundError(ResourceError):
    """Raised when a required resource is not found.

    Example:
        if not os.path.exists(filepath):
            raise ResourceNotFoundError(
                f"Configuration file not found: {filepath}. "
                f"Create a config file at {default_path} or "
                f"pass 'config_path' parameter."
            )
    """

    pass


class PermissionError(ResourceError):
    """Raised when operation is not permitted.

    Note: This shadows the built-in PermissionError. Consider using
    a different name like `AccessDenied` if needed.
    """

    pass


class InsufficientResourcesError(ResourceError):
    """Raised when insufficient resources are available."""

    pass


# State-related errors
class StateError(MyLibraryError):
    """Raised when operation is invalid for current state.

    Example:
        if not self.is_initialized:
            raise StateError(
                "Client not initialized. Call client.initialize() first."
            )
    """

    pass


class NotInitializedError(StateError):
    """Raised when required initialization hasn't been performed."""

    pass


class AlreadyExistsError(StateError):
    """Raised when trying to create something that already exists."""

    pass


# Implementation errors
class NotImplementedError(MyLibraryError):
    """Raised when feature is not yet implemented."""

    pass


class UnsupportedOperationError(MyLibraryError):
    """Raised when operation is not supported.

    Example:
        if not hasattr(processor, 'process'):
            raise UnsupportedOperationError(
                f"Processor {type(processor).__name__} doesn't support "
                f"process operation. Use a different processor."
            )
    """

    pass


# Usage Examples
# ==============

def example_validation_error():
    """Example of raising a validation error with helpful message."""
    timeout = -5
    if timeout < 0:
        raise ValidationError(
            f"Parameter 'timeout' must be non-negative, got {timeout}. "
            f"Use timeout=30 for a 30-second timeout."
        )


def example_chained_exception():
    """Example of chaining exceptions to preserve context."""
    import requests

    try:
        response = requests.get("https://example.com", timeout=5)
        response.raise_for_status()
    except requests.RequestException as e:
        raise APIError(
            f"Failed to fetch data from API: {str(e)}. "
            f"Check your network connection and API endpoint."
        ) from e


def example_missing_resource():
    """Example of reporting missing resources clearly."""
    import os

    config_path = "/etc/myapp/config.json"
    if not os.path.exists(config_path):
        raise ResourceNotFoundError(
            f"Configuration file not found: {config_path}\n"
            f"Create a config file or set MYAPP_CONFIG_PATH environment variable.\n"
            f"See documentation at: https://docs.example.com/config"
        )


def example_state_error():
    """Example of reporting state-related errors."""

    class Client:
        def __init__(self):
            self._connected = False

        def query(self, sql):
            if not self._connected:
                raise StateError(
                    "Client not connected. Call client.connect() before "
                    "executing queries."
                )
            # Execute query...

    client = Client()
    try:
        client.query("SELECT * FROM users")
    except StateError as e:
        print(f"Connection error: {e}")


# Best Practices Summary
# ======================
"""
1. DEFINE A BASE EXCEPTION
   - All library exceptions inherit from it
   - Users can catch all library errors with: except MyLibraryError

2. CREATE SEMANTIC SUBCLASSES
   - Group related errors (ConfigurationError, ValidationError, etc.)
   - Allows specific error handling

3. WRITE HELPFUL ERROR MESSAGES
   - State what was wrong
   - Explain why it's wrong
   - Show how to fix it
   - Example: f"X must be Y, got {Z}. Use X=default for default behavior."

4. CHAIN EXCEPTIONS WHEN WRAPPING
   - Use `raise ... from e` to preserve context
   - Helps with debugging and tracing

5. DOCUMENT EXCEPTIONS
   - Add docstrings to exception classes
   - Include examples in docstrings
   - Document which operations raise which exceptions

6. AVOID SHADOWING BUILT-INS
   - Consider naming like TimeoutError -> OperationTimeout
   - Be careful with common exception names

7. USE EXCEPTIONS FOR EXCEPTIONAL CASES
   - Not for normal control flow
   - Return None or empty collection for "not found" when appropriate
   - Reserve exceptions for actual errors

8. CONSISTENCY
   - Same error for same condition across codebase
   - Consistent error message format
   - Clear exception hierarchy
"""

RECOMMENDED PROJECT STRUCTURE (src/ layout)
==========================================

mypackage/                      # Repository root
│
├── .github/                    # GitHub specific
│   ├── workflows/
│   │   ├── ci.yml             # CI/CD pipeline
│   │   └── release.yml        # Release automation
│   └── ISSUE_TEMPLATE/
│
├── src/                        # Source code (src/ layout required!)
│   └── mypackage/              # Main package
│       ├── __init__.py         # Public API exports
│       ├── __version__.py      # Version string
│       ├── exceptions.py       # Custom exception hierarchy
│       ├── core/               # Core functionality
│       │   ├── __init__.py
│       │   ├── processor.py
│       │   └── validator.py
│       ├── utils/              # Utility functions
│       │   ├── __init__.py
│       │   ├── helpers.py
│       │   └── decorators.py
│       ├── config.py           # Configuration classes
│       └── _internal/          # Private/internal modules
│           ├── __init__.py
│           └── backend.py
│
├── tests/                      # Test suite
│   ├── __init__.py
│   ├── conftest.py            # pytest fixtures and config
│   ├── unit/                  # Unit tests (test individual components)
│   │   ├── test_core.py
│   │   ├── test_validator.py
│   │   └── test_config.py
│   ├── integration/           # Integration tests (test workflows)
│   │   ├── test_workflow_a.py
│   │   └── test_workflow_b.py
│   └── fixtures/              # Test data and fixtures
│       └── sample_data.json
│
├── docs/                       # Documentation
│   ├── source/
│   │   ├── conf.py            # Sphinx config
│   │   ├── index.rst
│   │   ├── guide/
│   │   │   ├── installation.rst
│   │   │   ├── quickstart.rst
│   │   │   └── advanced.rst
│   │   ├── api/
│   │   │   └── index.rst      # API reference (auto-generated from docstrings)
│   │   └── changelog.rst
│   ├── build/                 # Built docs (generated)
│   └── Makefile              # Documentation build
│
├── scripts/                    # Development scripts
│   ├── check_types.sh
│   ├── run_tests.sh
│   └── build_release.sh
│
├── .gitignore                 # Git ignore rules
├── .gitattributes            # Git attributes
├── .pre-commit-config.yaml   # Pre-commit hooks
│
├── pyproject.toml            # Project metadata & build config (PEP 517/518)
├── README.md                 # Project overview
├── CHANGELOG.md              # Version history
├── CONTRIBUTING.md           # Contributing guidelines
├── CODE_OF_CONDUCT.md        # Community guidelines
├── LICENSE                   # License (MIT, Apache, etc.)
├── SECURITY.md               # Security policy
│
├── Makefile                  # Development convenience commands
└── tox.ini                   # Test automation config


KEY STRUCTURE DECISIONS
=======================

1. SRC/ LAYOUT (REQUIRED)
   - src/ prevents accidentally importing from source during tests
   - Ensures you always test the installed version
   - Industry standard for modern Python packages

2. INTERNAL UNDERSCORE CONVENTION
   - Public modules: no leading underscore (src/mypackage/core.py)
   - Private modules: leading underscore (src/mypackage/_internal.py)
   - Signals to users: this is implementation detail, don't depend on it

3. FLAT HIERARCHY
   - Avoid deep nesting (keep 2-3 levels maximum)
   - Package structure is implementation detail
   - Expose public API at package level via __init__.py

4. SEPARATION OF CONCERNS
   - core/: Main algorithm/logic
   - utils/: Helper functions and utilities
   - config.py: Configuration classes
   - exceptions.py: Custom exception hierarchy
   - _internal/: Implementation details

5. TEST STRUCTURE
   - unit/: Test individual components (fast, no I/O)
   - integration/: Test workflows (may be slower)
   - conftest.py: Shared fixtures and setup
   - fixtures/: Sample data needed for tests

6. DOCUMENTATION
   - README.md: Quick start and overview
   - docs/source/: Full documentation (Sphinx)
   - Docstrings: API documentation in code
   - CONTRIBUTING.md: How to contribute
   - CHANGELOG.md: Version history


PUBLIC API DESIGN
=================

In src/mypackage/__init__.py:

```python
"""MyPackage - [One-line description]."""

from mypackage._version import __version__
from mypackage.exceptions import MyPackageError, ValidationError
from mypackage.core.processor import Processor
from mypackage.core.validator import validate
from mypackage.config import Config

__all__ = [
    "__version__",
    "Config",
    "Processor",
    "validate",
    "MyPackageError",
    "ValidationError",
]
```

Users import like:
- `from mypackage import Processor`
- `import mypackage`
- `mypackage.Processor()`

NOT:
- `from mypackage.core.processor import Processor`  (internal detail)
- `from mypackage._internal import Backend`       (private)


MODULE ORGANIZATION PATTERNS
============================

Pattern 1: Function-based core
- core/
  ├── __init__.py (exports main functions)
  ├── processor.py (main algorithm)
  ├── validator.py (validation logic)
  └── transformer.py (transformation logic)

Pattern 2: Class-based core with strategy pattern
- core/
  ├── __init__.py (exports main classes)
  ├── base.py (abstract classes)
  ├── default.py (default implementation)
  └── specialized.py (specialized implementations)

Pattern 3: Large library with submodules
- core/
  - processing/
    - __init__.py
    - (modules)
  - validation/
    - __init__.py
    - (modules)
  - storage/
    - __init__.py
    - (modules)

All expose high-level API at package level!


SIZE GUIDELINES
===============

Small library (< 100 functions):
- src/mypackage/
  ├── __init__.py
  ├── core.py (main logic)
  └── utils.py (helpers)

Medium library (100-500 functions):
- src/mypackage/
  ├── __init__.py
  ├── core/
  ├── utils/
  └── cli/ (optional)

Large library (500+ functions):
- src/mypackage/
  ├── __init__.py
  ├── core/
  ├── processing/
  ├── storage/
  ├── utils/
  └── _internal/

Don't over-organize! Flat is better than nested.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "mypackage"
version = "1.0.0"
description = "A brief, one-sentence description of your library"
readme = "README.md"
requires-python = ">=3.10"
license = {text = "MIT"}
authors = [
    {name = "Your Name", email = "your@email.com"}
]
maintainers = [
    {name = "Your Name", email = "your@email.com"}
]
keywords = ["keyword1", "keyword2"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
]

dependencies = [
    # Add runtime dependencies with semantic version constraints
    # "requests>=2.28,<3.0",
    # "pydantic>=2.0,<3.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "pytest-cov>=4.0",
    "black>=22.0",
    "mypy>=1.0",
    "ruff>=0.1.0",
    "isort>=5.0",
]
docs = [
    "sphinx>=6.0",
    "sphinx-rtd-theme>=1.0",
    "sphinx-autodoc-typehints>=1.0",
]

[project.urls]
Homepage = "https://github.com/username/mypackage"
Documentation = "https://mypackage.readthedocs.io"
Repository = "https://github.com/username/mypackage.git"
Issues = "https://github.com/username/mypackage/issues"
Changelog = "https://github.com/username/mypackage/blob/main/CHANGELOG.md"

[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
addopts = "--cov=src/mypackage --cov-report=term-missing --strict-markers"
markers = [
    "unit: Unit tests",
    "integration: Integration tests",
    "slow: Slow running tests",
]

[tool.mypy]
python_version = "3.10"
strict = true
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true
check_untyped_defs = true

[tool.ruff]
line-length = 88
target-version = "py310"
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # Pyflakes
    "I",    # isort
    "N",    # pep8-naming
    "UP",   # pyupgrade
    "B",    # flake8-bugbear
    "C4",   # flake8-comprehensions
]
ignore = [
    "E501",  # Line too long (handled by black)
]

[tool.ruff.isort]
known-first-party = ["mypackage"]

[tool.black]
line-length = 88
target-version = ["py310"]
include = '\.pyi?$'
extend-exclude = '''
/(
  # directories
  \.eggs
  | \.git
  | \.hg
  | \.mypy_cache
  | \.tox
  | \.venv
  | build
  | dist
)/
'''

[tool.isort]
profile = "black"
line_length = 88
multi_line_mode = 3
include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
ensure_newline_before_comments = true
skip_gitignore = true

[tool.coverage.run]
source = ["src/mypackage"]
branch = true
omit = [
    "*/tests/*",
    "*/__main__.py",
]

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "raise AssertionError",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
    "if typing.TYPE_CHECKING:",
]
precision = 2

# MyPackage

[![PyPI version](https://badge.fury.io/py/mypackage.svg)](https://badge.fury.io/py/mypackage)
[![Python Versions](https://img.shields.io/pypi/pyversions/mypackage.svg)](https://pypi.org/project/mypackage)
[![CI Status](https://github.com/username/mypackage/workflows/CI/badge.svg)](https://github.com/username/mypackage/actions)
[![Coverage](https://codecov.io/gh/username/mypackage/branch/main/graph/badge.svg)](https://codecov.io/gh/username/mypackage)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A brief, one-sentence description of what your library does.

## Features

- ✨ Feature one with clear benefits
- ✨ Feature two with clear benefits
- ✨ Feature three with clear benefits

## Installation

Install from PyPI:

```bash
pip install mypackage
```

Or install from source for development:

```bash
git clone https://github.com/username/mypackage.git
cd mypackage
pip install -e ".[dev]"
```

## Quick Start

### Basic Usage

```python
import mypackage

# Simple example showing core functionality
result = mypackage.process(data="example")
print(result)
```

### More Examples

See the [documentation](https://mypackage.readthedocs.io) for comprehensive guides and API reference.

## Key Concepts

### Concept One

Explain the main concept your library introduces.

```python
# Example demonstrating the concept
obj = mypackage.Thing(param=value)
obj.do_something()
```

### Concept Two

Explain another key concept.

```python
# Example code
result = mypackage.transform(input_data)
```

## Configuration

Configure the library using environment variables or configuration objects:

```python
config = mypackage.Config(
    timeout=30,
    retries=3,
    debug=False
)
client = mypackage.Client(config=config)
```

## API Reference

Full API documentation is available at [mypackage.readthedocs.io](https://mypackage.readthedocs.io).

### Core Classes

- `mypackage.Thing` - Main class for [purpose]
- `mypackage.Config` - Configuration object
- `mypackage.Error` - Base exception class

### Main Functions

- `mypackage.process()` - Process data
- `mypackage.validate()` - Validate input
- `mypackage.transform()` - Transform data

## Error Handling

The library uses a custom exception hierarchy:

```python
try:
    result = mypackage.process(data)
except mypackage.ValidationError as e:
    print(f"Validation failed: {e}")
except mypackage.MyPackageError as e:
    print(f"Library error: {e}")
```

## Performance

- Process time: ~X ms for typical inputs
- Memory usage: ~X MB for standard operations
- Suitable for [use case description]

## Testing

Run the test suite:

```bash
# Run all tests
pytest

# Run with coverage
pytest --cov=src/mypackage

# Run specific test file
pytest tests/unit/test_core.py

# Run tests matching pattern
pytest -k "test_validation"
```

## Development

### Setup Development Environment

```bash
git clone https://github.com/username/mypackage.git
cd mypackage
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -e ".[dev]"
```

### Code Quality

Format and lint code:

```bash
black src/ tests/
ruff check --fix src/ tests/
mypy src/
```

### Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on:
- Reporting bugs
- Requesting features
- Submitting pull requests
- Code style and standards

## Changelog

See [CHANGELOG.md](CHANGELOG.md) for release notes and version history.

## License

This project is licensed under the MIT License - see [LICENSE](LICENSE) file for details.

## Support

- 📖 [Documentation](https://mypackage.readthedocs.io)
- 🐛 [Issue Tracker](https://github.com/username/mypackage/issues)
- 💬 [Discussions](https://github.com/username/mypackage/discussions)

## Acknowledgments

Thank you to [list key contributors, inspirations, or dependencies].

## Related Projects

- [Related Project 1](https://example.com)
- [Related Project 2](https://example.com)

RECOMMENDED TEST STRUCTURE
==========================

tests/
├── __init__.py                 # Makes tests a package
├── conftest.py                 # Shared fixtures and pytest config
│
├── unit/                       # Unit tests (fast, isolated)
│   ├── test_core.py
│   ├── test_validator.py
│   ├── test_config.py
│   └── test_utils.py
│
├── integration/                # Integration tests (workflows, slower)
│   ├── test_basic_workflow.py
│   ├── test_advanced_workflow.py
│   └── test_error_handling.py
│
├── fixtures/                   # Test data files
│   ├── sample_data.json
│   ├── sample_config.yaml
│   └── expected_output.json
│
└── helpers.py                  # Shared test utilities


TEST ORGANIZATION PHILOSOPHY
=============================

Unit Tests (tests/unit/):
- Test individual functions/classes in isolation
- Mock external dependencies
- Fast execution (milliseconds)
- High coverage (>90% target)
- Test success and error paths

Integration Tests (tests/integration/):
- Test complete workflows
- May use real objects/connections (or well-mocked)
- Test API contracts
- Slower but more realistic

Fixtures (tests/fixtures/):
- Sample data for tests
- Configuration files
- Expected outputs
- Keep separate from code


CONFTEST.PY TEMPLATE
====================

"""Shared pytest fixtures and configuration."""

import pytest
from mypackage.config import Config


@pytest.fixture
def sample_config():
    """Create a test configuration."""
    return Config(
        timeout=10,
        retries=2,
        debug=True
    )


@pytest.fixture
def sample_data():
    """Load sample data for tests."""
    return {
        "name": "test",
        "value": 42,
        "items": ["a", "b", "c"]
    }


@pytest.fixture
def mock_processor(mocker):
    """Create a mocked processor."""
    processor = mocker.Mock()
    processor.process.return_value = {"result": "success"}
    return processor


# Pytest configuration
def pytest_configure(config):
    """Configure pytest."""
    config.addinivalue_line(
        "markers", "unit: Unit tests (fast, isolated)"
    )
    config.addinivalue_line(
        "markers", "integration: Integration tests (may be slower)"
    )
    config.addinivalue_line(
        "markers", "slow: Slow running tests"
    )


UNIT TEST EXAMPLE
=================

# tests/unit/test_core.py

import pytest
from mypackage.core import process
from mypackage.exceptions import ValidationError


class TestProcess:
    """Test suite for process function."""

    def test_process_basic(self):
        """Test basic processing."""
        result = process("test_input")
        assert result == "expected_output"

    @pytest.mark.parametrize("input,expected", [
        ("input1", "output1"),
        ("input2", "output2"),
        ("", "default_output"),
    ])
    def test_process_varied_inputs(self, input, expected):
        """Test with varied inputs."""
        result = process(input)
        assert result == expected

    def test_process_with_config(self, sample_config):
        """Test with configuration."""
        result = process("input", config=sample_config)
        assert result is not None

    def test_process_invalid_input(self):
        """Test error handling for invalid input."""
        with pytest.raises(ValidationError):
            process(None)

    def test_process_large_input(self):
        """Test with large input (edge case)."""
        large_input = "x" * 10000
        result = process(large_input)
        assert result is not None

    def test_process_empty_input(self):
        """Test with empty input (edge case)."""
        with pytest.raises(ValidationError):
            process("")


INTEGRATION TEST EXAMPLE
========================

# tests/integration/test_workflow.py

import pytest
from mypackage import Client, Config


class TestBasicWorkflow:
    """Test basic client workflow."""

    def test_create_and_process(self):
        """Test creating client and processing data."""
        config = Config(timeout=30)
        client = Client(config=config)

        result = client.process("test_data")

        assert result is not None
        assert "success" in result

    def test_error_recovery(self):
        """Test client recovers from errors."""
        config = Config(timeout=5, retries=3)
        client = Client(config=config)

        # Should retry and eventually succeed
        result = client.process("flaky_data")
        assert result is not None

    @pytest.mark.slow
    def test_large_batch_processing(self):
        """Test processing large batches."""
        config = Config(timeout=60)
        client = Client(config=config)

        results = []
        for i in range(1000):
            result = client.process(f"item_{i}")
            results.append(result)

        assert len(results) == 1000
        assert all(r is not None for r in results)


PYTEST.INI CONFIGURATION
========================

[pytest]
# Test discovery
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*

# Markers
markers =
    unit: Unit tests (fast, isolated)
    integration: Integration tests (may be slower)
    slow: Slow running tests
    requires_network: Tests requiring network access

# Output
addopts =
    --verbose
    --strict-markers
    --cov=src/mypackage
    --cov-report=term-missing
    --cov-report=html
    --cov-fail-under=90

# Coverage
[coverage:run]
source = src/mypackage
branch = true

[coverage:report]
exclude_lines =
    pragma: no cover
    def __repr__
    raise AssertionError
    raise NotImplementedError
    if __name__ == .__main__.:
    if TYPE_CHECKING:
    if typing.TYPE_CHECKING:


KEY TESTING PRINCIPLES
======================

1. UNIT TESTS SHOULD BE FAST
   - Mock external dependencies (files, network, databases)
   - Test single functions/methods
   - Run in milliseconds
   - Can be run frequently during development

2. INTEGRATION TESTS VERIFY WORKFLOWS
   - Test interactions between components
   - May use slower operations
   - Test the complete flow
   - Run less frequently (but required before commits)

3. TEST ERROR PATHS
   - What happens with invalid input?
   - What happens with edge cases?
   - What happens when dependencies fail?
   - Test both success and failure

4. USE FIXTURES FOR SETUP
   - Avoid code duplication in tests
   - Conftest.py for shared fixtures
   - Fixture scopes: function, class, module, session

5. PARAMETRIZED TESTS FOR VARIATIONS
   - @pytest.mark.parametrize for similar tests with different inputs
   - Reduces code duplication
   - Clearer what variations are tested

6. MOCK EXTERNAL DEPENDENCIES
   - Use pytest-mock or unittest.mock
   - Don't depend on network/files in unit tests
   - Make tests deterministic and fast

7. CLEAR TEST NAMES
   - test_<function>_<scenario>
   - test_<function>_<input>_<expected>
   - Example: test_validate_empty_input_raises_error

8. ORGANIZE TESTS IN CLASSES
   - Group related tests in TestClass
   - Better organization and scoping
   - Share fixtures at class level


COVERAGE TARGETS
================

Public APIs:
- Aim for >90% line coverage
- 100% coverage of error paths
- All branches exercised

Private/internal code:
- High coverage encouraged (70%+)
- Less critical than public API
- Focus on complex logic

Performance-critical code:
- Extra test coverage
- Profile to verify behavior
- Test with realistic data sizes

Generated code:
- May exclude from coverage
- Document exclusions


MOCKING BEST PRACTICES
======================

# Mock external dependencies
def test_with_mock_api(mocker):
    mock_api = mocker.patch('mypackage.api.Client')
    mock_api.return_value.call.return_value = {"result": "success"}

    result = my_function_that_calls_api()
    assert result == {"result": "success"}

# Don't mock what you're testing
def test_validation():
    # Test real validation logic, don't mock it
    with pytest.raises(ValueError):
        validate(invalid_input)

# Verify interactions
def test_api_called_correctly(mocker):
    mock_api = mocker.patch('mypackage.api.call')

    my_function(param="value")

    mock_api.assert_called_once_with(param="value")