Python Packaging

Table of Contents

• Quick Start
• When to Use
• Core Decisions
• 1. Layout Choice
• 2. Project Structure
• Detailed Topics
• Best Practices
• Exit Criteria

Python Packaging

Modern Python packaging with pyproject.toml, uv, and best practices for distribution.

Quick Start

# Create new project with uv
uv init my-package
cd my-package

# Add dependencies
uv add requests click

# Build package
uv build

# Publish to PyPI
uv publish

Verification: Run the command with --help flag to verify availability.

When To Use

• Creating distributable Python libraries
• Building CLI tools
• Publishing to PyPI
• Setting up development environments
• Managing project dependencies

When NOT To Use

• Testing packages - use python-testing

instead

• Optimizing package performance - use python-performance
• Testing packages - use python-testing

instead

• Optimizing package performance - use python-performance

Core Decisions

1. Layout Choice

# Source layout (recommended)
src/my_package/
    __init__.py
    module.py

# Flat layout (simple)
my_package/
    __init__.py
    module.py

Verification: Run the command with --help flag to verify availability.

Source layout benefits:

• Clear separation of source and tests
• Prevents accidental imports of uninstalled code
• Better for packages with complex structure

2. Project Structure

Minimal Project:

**Verification:** Run `pytest -v` to verify tests pass.
my-project/
├── pyproject.toml
├── README.md
├── src/
│   └── my_package/
│       └── __init__.py
└── tests/
    └── test_init.py

Verification: Run pytest -v to verify tests pass.

Complete Project:

**Verification:** Run the command with `--help` flag to verify availability.
my-project/
├── pyproject.toml
├── README.md
├── LICENSE
├── .gitignore
├── src/
│   └── my_package/
│       ├── __init__.py
│       ├── cli.py
│       ├── core.py
│       └── utils.py
├── tests/
│   ├── conftest.py
│   └── test_core.py
└── docs/
    └── index.md

Verification: Run pytest -v to verify tests pass.

Detailed Topics

See modules for detailed information:

• [uv Workflow](modules/uv-workflow.md) - Complete uv commands and troubleshooting
• [pyproject.toml Patterns](modules/pyproject-patterns.md) - Configuration examples for different package types
• [Entry Points](modules/entry-points.md) - Console scripts, GUI scripts, and plugins
• [CI/CD Integration](modules/ci-cd-integration.md) - GitHub Actions and automated publishing

Best Practices

1. Use source layout for anything beyond simple packages 2. Pin direct dependencies with minimum versions 3. Use optional dependency groups for dev/docs/test 4. Include py.typed for type hint support 5. Add detailed README with usage examples 6. Use semantic versioning (MAJOR.MINOR.PATCH) 7. Test on multiple Python versions before publishing

Exit Criteria

• Modern pyproject.toml configuration
• Clear dependency specification
• Proper version management
• Tests included and passing
• Build process reproducible
• Publishing pipeline automated

CI/CD Integration

Automated testing, building, and publishing workflows.

GitHub Actions Publishing

# .github/workflows/publish.yml
name: Publish

on:
  release:
    types: [published]

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v4
      - run: uv build
      - run: uv publish
        env:
          UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN }}

Testing Workflow

# .github/workflows/test.yml
name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.9", "3.10", "3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v4
      - run: uv sync --all-extras
      - run: uv run pytest

Complete CI/CD Pipeline

# .github/workflows/ci.yml
name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  release:
    types: [published]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.9", "3.10", "3.11", "3.12"]
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v4
      - run: uv sync --all-extras
      - run: uv run pytest --cov
      - run: uv run ruff check
      - run: uv run mypy src

  publish:
    if: github.event_name == 'release'
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v4
      - run: uv build
      - run: uv publish
        env:
          UV_PUBLISH_TOKEN: ${{ secrets.PYPI_TOKEN }}

Best Practices

1. Test before publish - Always run tests in CI before publishing 2. Matrix testing - Test on multiple Python versions 3. Use trusted actions - Official actions from astral-sh, actions org 4. Secure tokens - Store PyPI tokens as GitHub secrets 5. Version tags - Trigger releases from git tags 6. Test on TestPyPI - Verify packages before production publish

Entry Points

Configure console scripts, GUI applications, and plugin systems.

Console Scripts

[project.scripts]
my-cli = "my_package.cli:main"
my-other-cmd = "my_package.other:run"

# src/my_package/cli.py
import click

@click.command()
@click.option("--name", default="World")
def main(name: str):
    click.echo(f"Hello, {name}!")

if __name__ == "__main__":
    main()

GUI Scripts

[project.gui-scripts]
my-gui = "my_package.gui:main"

GUI scripts are similar to console scripts but on Windows they don't open a console window.

Plugin Entry Points

[project.entry-points."myapp.plugins"]
plugin1 = "my_package.plugins:Plugin1"
plugin2 = "my_package.plugins:Plugin2"

# src/my_package/plugins.py
class Plugin1:
    def activate(self):
        print("Plugin 1 activated")

class Plugin2:
    def activate(self):
        print("Plugin 2 activated")

Discovering Plugins

# Application code to discover plugins
from importlib.metadata import entry_points

def load_plugins():
    plugins = entry_points(group="myapp.plugins")
    for plugin in plugins:
        plugin_class = plugin.load()
        instance = plugin_class()
        instance.activate()

Best Practices

1. Keep entry points simple - Minimal logic in entry point functions 2. Use click/argparse - Proper argument parsing for CLI tools 3. Handle errors gracefully - Exit codes and error messages 4. Support --help - Document all CLI options 5. Test entry points - Verify they work when installed

pyproject.toml Patterns

Configuration patterns for different Python package types.

Build Backend Choice

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Options:

• hatchling: Modern, fast, minimal config
• setuptools: Traditional, widely supported
• flit: Simple, pure Python packages
• poetry-core: Poetry ecosystem integration

Package Type Matrix

Simple Package

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-package"
version = "0.1.0"
description = "A simple Python package"
readme = "README.md"
requires-python = ">=3.9"
license = "MIT"
authors = [
    { name = "Your Name", email = "you@example.com" }
]
dependencies = [
    "requests>=2.0.0",
]

CLI Package

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "my-cli"
version = "0.1.0"
description = "A CLI tool"
requires-python = ">=3.9"
dependencies = [
    "click>=8.0.0",
    "rich>=13.0.0",
]

[project.scripts]
my-cli = "my_cli.main:cli"

Development Package

[project]
name = "my-package"
version = "0.1.0"
dependencies = ["requests>=2.0.0"]

[project.optional-dependencies]
dev = [
    "pytest>=7.0.0",
    "pytest-cov>=4.0.0",
    "ruff>=0.1.0",
    "mypy>=1.0.0",
]
docs = [
    "mkdocs>=1.5.0",
    "mkdocs-material>=9.0.0",
]

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = ["-v", "--cov=src"]

[tool.ruff]
line-length = 100
target-version = "py39"

[tool.mypy]
python_version = "3.9"
strict = true

Advanced Patterns

Namespace Packages

[project]
name = "mycompany-core"

[tool.hatch.build.targets.wheel]
packages = ["src/mycompany"]

Conditional Dependencies

[project]
dependencies = [
    "requests>=2.0.0",
    "typing-extensions>=4.0; python_version < '3.11'",
]

[project.optional-dependencies]
gpu = ["torch", "cuda-python"]
cpu = ["numpy", "scipy"]

Dynamic Version

[project]
dynamic = ["version"]

[tool.hatch.version]
path = "src/my_package/__init__.py"

# src/my_package/__init__.py
__version__ = "0.1.0"

Anti-Patterns to Avoid

# Don't: Complex setup.py with file reading
setup(
    version=open("version.txt").read(),
    install_requires=open("requirements.txt").readlines(),
)

# Do: Simple pyproject.toml
[project]
version = "0.1.0"
dependencies = ["requests>=2.0.0"]

uv Workflow

Complete workflows for using uv package manager in Python projects.

Project Setup

# Initialize new project
uv init my-project
cd my-project

# Create with specific Python version
uv init --python 3.12 my-project

Dependency Management

# Add dependencies
uv add requests click

# Add dev dependencies
uv add --dev pytest ruff mypy

# Add optional dependencies group
uv add --optional docs mkdocs

# Remove dependency
uv remove requests

# Sync environment
uv sync

Building and Publishing

# Build package
uv build

# Publish to TestPyPI
uv publish --publish-url https://test.pypi.org/legacy/

# Publish to PyPI
uv publish

Version Management

# Update version in pyproject.toml manually, then:
git tag v0.1.1
git push origin v0.1.1

Troubleshooting

Build Failures

# Check build isolation
uv build --no-isolation

# Clear build cache
rm -rf build/ dist/

Import Errors

# Verify package structure
python -c "import my_package"

# Install in development mode
uv pip install -e .

Version Conflicts

# Check dependency tree
uv tree

# Use specific versions
uv add "requests==2.28.2"