Python Ultimate

Python Ultimate

Single Python reference with quick routes for standards, tooling, workflows, and best practices.

Quick Start

Writing code? → Start with Coding Standards Checking naming? → Go to Naming Conventions Building a CLI? → Go to CLI Development Fixing linter errors? → Go to Linter Rules Writing tests? → Go to Testing Debugging a bug? → Go to Debugging Refactoring? → Go to Refactoring Reviewing code? → Go to Code Review Auditing codebase? → Go to Auditing Documenting? → Go to Documentation Planning a feature? → Go to Planning Bulk operations? → Go to Refactoring (10+ files)

Slash Commands

The /python-ultimate command accepts an optional sub-command argument to run a targeted guideline review. When invoked without a sub-command (e.g., just /python-ultimate), run a general antipattern check using the Antipatterns / Forbidden Styles Index section below.

Routing

Match the sub-command argument to one of the sections below and follow its workflow. If the argument does not match any known sub-command, explain the available options (you can output the table from python-ultimate help).

______________________________________________________________________

python-ultimate help

When the sub-command is help (or when the user asks for available commands), render the following table so the user sees all available options:

Sub-Command                    │ What It Reviews                                        │ Reference
───────────────────────────────┼────────────────────────────────────────────────────────┼──────────────────────────────────
python-ultimate naming         │ File/dir variable naming (_file/_dir/_path suffixes)  │ references/naming-conventions.md
python-ultimate type-checking  │ TYPE_CHECKING guards and Optional[T] usage             │ references/type-checking.md
python-ultimate imports        │ Required-vs-optional import patterns                    │ references/imports-optional-dependencies.md
python-ultimate coding-standards│ Type hints, f-strings, pathlib, docstrings, comments, data modeling │ references/coding-standards.md
python-ultimate linter-rules   │ Ruff violations (E402, B007, B008, S108, etc.)         │ references/linter-rules.md
python-ultimate debugging      │ Systematic 4-phase debugging process                   │ references/debugging.md
python-ultimate testing        │ Test organization, fixtures, mocking, TDD, coverage    │ references/testing.md
python-ultimate audit          │ 6-dimension codebase audit                             │ references/auditing.md
python-ultimate verification   │ Evidence-based completion claims                       │ references/verification.md
python-ultimate code-review    │ Code review feedback evaluation                        │ references/code-review.md

______________________________________________________________________

python-ultimate naming

Reviews file and directory variable naming conventions (_file / _dir / _path suffixes).

Workflow:

1. Open references/naming-conventions.md and load the "1. Files and Directories" section 2. Scan the codebase for bare path names (path, file, dir, output, source, target) used as Path variables 3. Check for prefix patterns (dir_output → should be output_dir) 4. Find generic path variable names missing suffixes (results → ambiguous) 5. Report findings using the standard antipattern response format

______________________________________________________________________

python-ultimate type-checking

Scans for TYPE_CHECKING guards and Optional[T] usage.

Workflow:

1. Open references/type-checking.md and load the "Rule: Never Use TYPE_CHECKING Guards" section 2. Search for TYPE_CHECKING imports: rg "TYPE_CHECKING" src/ 3. Search for Optional[ usage: rg "Optional\[" src/ 4. For each finding, identify the root cause (circular imports, type-only imports) 5. Recommend the appropriate alternative (shared types module, protocols, forward refs, local imports) 6. Report findings using the standard antipattern response format

______________________________________________________________________

python-ultimate imports

Reviews import patterns — distinguishes required vs optional dependencies.

Workflow:

1. Open references/imports-optional-dependencies.md and load the hard rule 2. Check pyproject.toml to determine which packages are required vs optional 3. Search for try/except ImportError patterns guarding required deps: rg "except ImportError" src/ 4. For each match, classify: required dep → normal top-level import; optional dep → localized handling 5. Report findings using the standard antipattern response format

______________________________________________________________________

python-ultimate coding-standards

Reviews compliance with coding standards: type hints, f-strings, pathlib, docstrings, comments, prohibited patterns, vague input/output types.

Workflow:

1. Open references/coding-standards.md and load relevant sections 2. For each prohibited pattern, search with targeted grep patterns:

• Optional\[ → must be T | None
• \.format\( or % formatting → must be f-strings
• os\.path\. → must be pathlib.Path
• # noqa → fix root issue

3. Check for vague input/output types with multiple isinstance checks 4. Report findings using the standard antipattern response format

______________________________________________________________________

python-ultimate linter-rules

Reviews and fixes specific Ruff linter violations using context-aware patterns.

Workflow:

1. Open references/linter-rules.md and load the relevant rule section 2. Run ruff check src/ to identify violations 3. For each violated rule, apply the context-specific fix pattern from the reference:

• E402 → Move import to top of module
• B007 → Prefix unused loop variable with _
• B008 → Use None sentinel (except Typer Annotated parameters)
• S108 → Use tempfile or tmp_path fixture
• PLC0415 → Move import to module level
• NPY002 → Use default_rng()
• S311 → Use secrets for security contexts

4. Re-run ruff check src/ to confirm fixes 5. Report findings using the standard antipattern response format

______________________________________________________________________

python-ultimate debugging

Initiates the systematic 4-phase debugging process.

Workflow:

1. Open references/debugging.md and load the full 4-phase process 2. Phase 1 — Root Cause: Reproduce the issue, read error messages, trace data flow from symptom to origin 3. Phase 2 — Pattern: Find working examples, compare against broken code, list every difference 4. Phase 3 — Hypothesis: Form a single testable hypothesis, make the smallest possible change to test it 5. Phase 4 — Implementation: Write a failing test first, implement the fix, verify all tests pass 6. Remember the iron law: No fixes without root cause investigation first. 7. If 3+ fixes have failed, stop and reassess architecture rather than continuing to guess

______________________________________________________________________

python-ultimate testing

Reviews test organization, coverage, fixtures, mocking, and TDD compliance.

Workflow:

1. Open references/testing.md for patterns and standards 2. Check test file naming: test_<module>.py convention 3. Check test class naming: Test<Name> PascalCase 4. Check test method naming: test_<description> snake_case 5. Run coverage: uv run pytest --cov=src --cov-report=term-missing 6. Review fixture quality (descriptive names, proper scope, teardown) 7. Report findings using the standard antipattern response format

______________________________________________________________________

python-ultimate audit

Runs a 6-dimension codebase audit.

Workflow:

1. Open references/auditing.md and load all six dimensions 2. For each dimension (Architecture, Quality, Security, Performance, Testing, Maintainability):

• Scan with grep/glob for relevant red flags
• Rate findings by severity (Critical, High, Medium, Low)

3. Synthesize into an audit report using the format from references/auditing.md 4. Include an executive summary with health score and top recommendation 5. Include an action plan with immediate/short-term/medium-term/backlog items

______________________________________________________________________

python-ultimate verification

Verifies that completion claims are backed by fresh evidence.

Workflow:

1. Open references/verification.md and load the iron law and gate function 2. For each claim, determine what command proves it 3. Run the full command, read the output, check the exit code 4. Only then state the result — with evidence, not assumptions 5. Forbidden words: should, probably, might, likely 6. Report results using the standard antipattern response format

______________________________________________________________________

python-ultimate code-review

Evaluates code review feedback and responds with technical rigor.

Workflow:

1. Open references/code-review.md and load the full workflow 2. Follow the READ → UNDERSTAND → VERIFY → EVALUATE → RESPOND → IMPLEMENT sequence 3. For each feedback item: verify against codebase reality, evaluate technical soundness 4. No performative agreement — respond with technical reasoning or push back with evidence 5. Push back when: suggestion breaks existing functionality, violates YAGNI, lacks full context

______________________________________________________________________

Expected /python-ultimate Response Format

For consistency across agents, format all sub-command responses as:

1. Summary — total findings by severity and category 2. Findings — one item per finding: file:line, matched pattern class, short rationale 3. Fix Guidance — preferred replacement pattern with one concrete before/after example 4. References — direct links to the relevant section in references/*.md 5. Verification — exact command(s) run and observed result

Use concise, technical language. Avoid performative agreement and avoid speculative wording.

Antipatterns / Forbidden Styles Index

Canonical quick-reference for common bad or forbidden patterns. Detailed rationale and examples stay in reference files.

______________________________________________________________________

Coding Standards

Core Python coding rules. See references/coding-standards.md for full details.

Type Hints

Mandatory everywhere. Use pipe syntax (T | None), never Optional[T]. Python 3.10+ required.

def process(data: str, limit: int | None = None) -> list[str]: ...

Never use `TYPE_CHECKING` guards. See references/type-checking.md for alternatives.

String Formatting

Use f-strings only. No .format() or % formatting.

Code Size Limits

Docstrings

Google style. Required for public functions and classes.

Prohibited Patterns

• TYPE_CHECKING guards
• os.path (use pathlib.Path)
• Optional[T] (use T | None)
• # noqa comments
• sys.path manipulation
• Defensive try/except ImportError for required dependencies (see references/imports-optional-dependencies.md)
• Vague or wide parameter/return types with hidden isinstance/hasattr checks and None-as-error returns (see references/coding-standards.md)
• Using pydantic for lightweight internal structs, or dataclass for untrusted/API data (see references/coding-standards.md)

______________________________________________________________________

Naming Conventions

Variable naming standards for clarity and consistency. See references/naming-conventions.md for full details.

Files and Directories

Anti-patterns (always invalid for path variables):

• Bare generic names: path, file, folder, dir, directory, output, input, source, target, dest
• Prefix instead of suffix: dir_output, file_config
• Missing suffix: results, data, config (ambiguous)

See references/naming-conventions.md for the complete anti-pattern list.

Test Naming

• Files: test_<module>.py
• Classes: Test<DataProcessor> (PascalCase with Test prefix)
• Methods: test_<description> (snake_case with test\_ prefix)

Automated Validation

# Check a variable name
uv run assets/check_path_naming.py output_file
# Output: is_file

# Scan for violations
uv run assets/check_path_naming.py --check-files src/

# Scan for core forbidden patterns
uv run assets/check_path_naming.py --check-forbidden src/

# Repro fixture scan (see assets/examples)
uv run assets/check_path_naming.py --check-forbidden assets/examples/

Reference fixture files and expected output: assets/examples/forbidden-scan-expected.md

______________________________________________________________________

CLI Development

Building Python CLIs with Typer or Click. See references/cli-development.md.

Framework Selection

Use Typer for new projects (type-hint driven, less boilerplate). Use Click for complex parameter handling.

Key Patterns

• Parameter validation with type hints
• Rich output formatting
• Environment variable integration
• Exit codes for error states

______________________________________________________________________

Linter Rules

Context-aware fixes for Ruff linter rules. See references/linter-rules.md.

Covered Rules

Typer Exception

B008 is allowed for Typer Annotated parameters. See references/linter-rules.md.

______________________________________________________________________

Testing

Test organization, fixtures, mocking, and TDD. See references/testing.md.

Quick Commands

uv run pytest -v --tb=short
uv run pytest --cov=src --cov-report=term-missing

Key Practices

• Co-located tests: <module>_test.py alongside implementation
• 90%+ coverage target
• Fixtures in conftest.py
• Parameterized testing for multiple inputs
• unittest.mock for external dependencies

TDD Cycle

Red → Green → Refactor. No production code without a failing test first. See references/testing.md.

______________________________________________________________________

Debugging

Systematic 4-phase debugging process. See references/debugging.md.

Iron Law

No fixes without root cause investigation first.

4-Phase Process

1. Root Cause — Reproduce, isolate, trace data flow 2. Pattern Analysis — Identify state changes, timing issues 3. Hypothesis — Form testable prediction 4. Implementation — Minimal fix, verify with test

Red Flags

• "Let me just try changing X"
• Fixing symptoms without understanding cause
• Multiple failed fix attempts

______________________________________________________________________

Refactoring

Find → Replace → Verify workflow. See references/refactoring.md.

Workflow

1. Find — Grep for target pattern 2. Replace — Edit with replace_all for bulk changes 3. Verify — Run tests, check for regressions

Code Transfer

Line-based code movement between files. See references/refactoring.md.

______________________________________________________________________

Code Review

Receiving and evaluating code review feedback. See references/code-review.md.

Workflow

Read → Understand → Verify → Evaluate → Respond → Implement

Key Principles

• No performative agreement
• Push back with technical reasoning
• Verify feedback before implementing
• Evaluate: is the suggestion correct?

______________________________________________________________________

Auditing

6-dimension codebase analysis. See references/auditing.md.

Dimensions

1. Architecture — Structure, modularity, dependencies 2. Quality — Readability, complexity, duplication 3. Security — Input validation, secrets, injection 4. Performance — Bottlenecks, memory, I/O 5. Testing — Coverage, quality, edge cases 6. Maintainability — Documentation, technical debt

Severity Ratings

Critical → High → Medium → Low

______________________________________________________________________

Documentation

10-section documentation structure. See references/documentation.md.

Workflow

Explore → Map → Read → Synthesize

Sections

Project Overview, Architecture, Key Components, Data Flow, API Reference, Configuration, Setup Guide, Development Guide, Testing, Deployment

Mermaid Diagrams

Use for architecture, sequence, and flowchart visualizations.

______________________________________________________________________

Planning

PLAN.md living document for feature implementation. See references/planning.md.

When to Use

Features spanning 3-15 prompts. Self-contained for fresh sessions.

Structure

Goal → Context → Phases → Validation → Progress → Decisions → Notes

______________________________________________________________________

Project Setup

Project structure, dependencies, and imports. See references/project-setup.md.

Key Tools

• uv for dependency management
• src layout for packages
• pyproject.toml for configuration

Import Order

1. Standard library 2. Third-party 3. Local (absolute imports)

______________________________________________________________________

File Analysis

Non-destructive file and codebase analysis. See references/file-analysis.md.

Tools

• stat for metadata
• wc for line counts
• Grep for pattern searching
• Glob for file discovery

______________________________________________________________________

Type Checking Alternatives

Never use TYPE_CHECKING guards. See references/type-checking.md.

Alternatives

1. Extract shared types to dedicated modules 2. Use protocols for structural typing 3. Forward references (string literals) 4. Local imports (last resort)

______________________________________________________________________

Bulk Operations

High-efficiency Python execution for 10+ file operations. 90-99% token savings vs. iterative approaches.

When to use:

• Bulk operations (10+ files)
• Complex multi-step workflows
• Iterative processing across many files
• User mentions efficiency/performance

Workflow pattern:

1. Analyze locally — Use metadata operations (file counts, grep patterns) 2. Process locally — Execute all transformations in Python 3. Return summary — Report counts, not full data

Example patterns:

# Bulk refactor across 50 files
from pathlib import Path
import re

files = list(Path('.').glob('**/*.py'))
modified = 0

for f in files:
    content = f.read_text()
    new_content = re.sub(r'old_pattern', 'new_pattern', content)
    if new_content != content:
        f.write_text(new_content)
        modified += 1

result = {'files_scanned': len(files), 'files_modified': modified}

# Code audit metadata extraction
from pathlib import Path
import ast

files = list(Path('src').glob('**/*.py'))
complexity_issues = []

for f in files:
    tree = ast.parse(f.read_text())
    for node in ast.walk(tree):
        if isinstance(node, ast.FunctionDef):
            # Calculate simple complexity metric
            nested = sum(1 for n in ast.walk(node) if isinstance(n, (ast.If, ast.For, ast.While)))
            if nested > 10:
                complexity_issues.append({'file': str(f), 'function': node.name, 'complexity': nested})

result = {'files_audited': len(files), 'high_complexity': len(complexity_issues)}

Best practices:

• ✅ Return summaries, not full data
• ✅ Batch operations where possible
• ✅ Use pathlib.Path for file operations
• ✅ Handle errors gracefully, return error counts
• ❌ Don't read full source into context when metadata suffices
• ❌ Don't process files one-by-one interactively

Token savings scale with file count:

______________________________________________________________________

Reference Files

All detailed content lives in references/. Load only what you need:

Forbidden Scan Fixtures: Expected Output

Use these fixtures to validate --check-forbidden behavior from assets/check_path_naming.py.

Command

uv run assets/check_path_naming.py --check-forbidden assets/examples/

Expected Matches by File

fixture_type_and_imports.py

Expected pattern classes:

• TYPE_CHECKING guard (import and conditional guard)
• Optional[T]

Expected count:

• 3 matches

fixture_paths_and_noqa.py

Expected pattern classes:

• os.path usage
• sys.path manipulation
• noqa suppression

Expected count:

• 4 matches

fixture_defensive_import.py

Expected pattern classes:

• defensive required import

Expected count:

• 1 match

Expected Aggregate

• Total matches: 8
• Exit code: 1 (findings present)

The scanner prints file paths, line numbers, pattern class labels, and a short remediation reason for each match.

Code Auditing Reference

Comprehensive 6-dimension code audit methodology for codebase analysis.

Table of Contents

1. Audit Dimensions

• 1. Architecture
• 2. Code Quality
• 3. Security
• 4. Performance
• 5. Testing
• 6. Maintainability

2. Severity Ratings 3. Audit Report Format 4. When to Audit

______________________________________________________________________

Audit Dimensions

1. Architecture

Analyze overall structure and design decisions.

Check:

• Overall structure and organization
• Design patterns in use
• Module boundaries and separation of concerns
• Dependency management (depth, circular dependencies)
• Architectural decisions and trade-offs
• Inter-module communication patterns

Red Flags:

• God modules with many responsibilities
• Circular dependencies between modules
• Unclear boundaries between layers
• Hidden coupling through global state

______________________________________________________________________

2. Code Quality

Assess readability, complexity, and duplication.

Check:

• Cyclomatic complexity hotspots
• Code duplication (DRY violations)
• Naming conventions and consistency
• Documentation coverage
• Function length and Single Responsibility Principle
• Dead code and unused imports

Red Flags:

• Functions exceeding 50 lines
• Repeated switch/if-else chains
• Inconsistent naming schemes
• Magic numbers or hardcoded values
• Missing docstrings on public APIs

______________________________________________________________________

3. Security

Identify vulnerabilities and protection gaps.

Check:

• Input validation and sanitization
• Authentication and authorization patterns
• Secrets management (no hardcoded credentials)
• SQL injection and parameterization
• Dependency vulnerabilities (outdated or known-vulnerable packages)
• OWASP Top 10 issues

Red Flags:

• User input used in queries or commands without validation
• API keys or passwords in source code
• Missing access control checks
• Unvalidated file paths or content
• Insecure deserialization

______________________________________________________________________

4. Performance

Find bottlenecks and resource inefficiencies.

Check:

• Algorithmic complexity issues (O(n²) or worse in hot paths)
• Database query optimization (N+1 queries, missing indexes)
• Memory usage patterns (leaks, excessive allocation)
• Caching opportunities
• Resource leaks (unclosed files, connections)
• Unnecessary iterations or copies

Red Flags:

• Nested loops processing large datasets
• Loading entire datasets into memory
• Missing pagination or streaming
• Unnecessary object creation in hot paths
• Unclosed resources in error paths

______________________________________________________________________

5. Testing

Evaluate test coverage and quality.

Check:

• Test coverage percentage and trends
• Test quality and effectiveness
• Missing test scenarios (happy path, edge cases, error paths)
• Test isolation and independence
• Integration vs unit test balance
• Mock and fixture practices

Red Flags:

• Coverage below 70%
• Tests that only assert one thing
• Shared mutable state between tests
• Tests depending on execution order
• Missing error condition tests

______________________________________________________________________

6. Maintainability

Assess technical debt and long-term health.

Check:

• Technical debt assessment
• Module coupling and cohesion
• Ease of future changes
• Onboarding friendliness (code clarity)
• Documentation quality (README, API docs, inline comments)
• Configuration complexity

Red Flags:

• Duplicate code across modules
• Complex conditional logic without explanation
• Missing or outdated documentation
• Over-engineered abstractions
• Tight coupling preventing independent changes

______________________________________________________________________

Severity Ratings

______________________________________________________________________

Audit Report Format

# Code Audit Report

## Executive Summary
- Overall health score: X/10
- Critical issues: N
- High priority issues: N
- Top recommendation

## Findings

### Critical
- [Issue]: [File:Line]
  - Impact: [Description]
  - Fix: [Recommendation]

### High
- [Issue]: [File:Line]
  - Impact: [Description]
  - Fix: [Recommendation]

### Medium
- [Issue]: [File:Line]
  - Impact: [Description]
  - Fix: [Recommendation]

### Low
- [Issue]: [File:Line]
  - Impact: [Description]
  - Fix: [Recommendation]

## Action Plan

1. **Immediate** (< 1 day): [Critical fixes]
2. **Short-term** (1-5 days): [High priority]
3. **Medium-term** (1-2 weeks): [Medium priority]
4. **Backlog**: [Low priority improvements]

## Metrics
- Files analyzed: X
- Lines of code: Y
- Test coverage: Z%
- Complexity hotspots: N

______________________________________________________________________

When to Audit

Trigger an audit when:

• Starting a major refactoring effort
• Investigating recurring bugs or incidents
• Onboarding to unfamiliar codebase
• Planning significant feature additions
• Security compliance requirements
• Pre-release quality check
• Post-mortem after production issues

Audit depth levels:

• Quick (15-30 min): Critical issues only, high-level scan
• Standard (30-60 min): Full 6-dimension analysis
• Deep (60+ min): Exhaustive with line-by-line review

______________________________________________________________________

Audit Workflow

1. Explore the codebase structure thoroughly 2. Identify patterns using Grep and Glob 3. Read critical files in detail 4. Run static analysis tools if available 5. Synthesize findings into actionable report

Tools:

• grep / rg — Pattern matching for code smells
• glob — Find files by type or pattern
• read — Detailed file analysis
• bash — Run linters, complexity analyzers, coverage tools
• LSP diagnostics — IDE-level issue detection

CLI Development Reference

Universal patterns for developing command-line interfaces in Python.

Table of Contents

1. Framework Selection 2. Project Structure 3. Parameter Handling 4. Output Formatting 5. Environment Variables Integration 6. Error Handling 7. Help Text and Documentation 8. Testing CLI Applications

______________________________________________________________________

1. Framework Selection

Choose the right framework based on project complexity:

# Typer - Simple, type-annotated CLIs
import typer
from typing import Optional

def main(
    input_file: str = typer.Argument(..., help="Input file path"),
    output_file: Optional[str] = typer.Option(None, "-o", "--output", help="Output file path"),
    verbose: bool = typer.Option(False, "-v", "--verbose", help="Verbose output")
):
    """Universal CLI entry point"""
    typer.echo(f"Processing {input_file}")

if __name__ == "__main__":
    typer.run(main)

# Click - Complex CLIs with nested commands
import click

@click.group()
def cli():
    """Main CLI entry point"""
    pass

@cli.command()
@click.argument("input_file")
@click.option("-o", "--output", help="Output file path")
def process(input_file, output):
    """Process a file"""
    click.echo(f"Processing {input_file}")

if __name__ == "__main__":
    cli()

# Argparse - Standard library approach
import argparse

parser = argparse.ArgumentParser(description="Universal CLI")
parser.add_argument("input_file", help="Input file path")
parser.add_argument("-o", "--output", help="Output file path")
args = parser.parse_args()

______________________________________________________________________

2. Project Structure

Organize CLI applications for maintainability:

project/
├── src/
│   └── project/
│       ├── __init__.py
│       ├── cli/
│       │   ├── __init__.py
│       │   ├── main.py          # Entry point
│       │   ├── commands/        # Command modules
│       │   │   ├── __init__.py
│       │   │   ├── process.py
│       │   │   └── config.py
│       │   └── shared/          # Shared utilities
│       │       ├── __init__.py
│       │       ├── options.py   # Common options
│       │       └── formatting.py # Output formatting
│       └── core/
│           └── ...
├── tests/
│   └── test_cli/
├── pyproject.toml
└── README.md

# src/project/cli/main.py
import typer
from typing import Optional

app = typer.Typer(help="Project CLI", no_args_is_help=True)

@app.command()
def process(
    input_file: str = typer.Argument(..., help="Input file path"),
    output: Optional[str] = typer.Option(None, "-o", "--output", help="Output file"),
    verbose: bool = typer.Option(False, "-v", "--verbose", help="Verbose output")
):
    """Process the input file"""
    ...

if __name__ == "__main__":
    app()

______________________________________________________________________

3. Parameter Handling

Structure parameters consistently and validate early:

# Parameter handling with validation
import os
from dataclasses import dataclass, field
from typing import Optional

@dataclass
class CLIParameters:
    """Structured CLI parameters with validation"""
    input_file: str
    output_file: Optional[str] = None
    verbose: bool = False
    timeout: int = 30

    @classmethod
    def from_args(cls, args) -> "CLIParameters":
        """Parse arguments into structured parameters"""
        output = args.output or cls._default_output(args.input_file)
        return cls(
            input_file=args.input_file,
            output_file=output,
            verbose=args.verbose
        )

    @staticmethod
    def _default_output(input_file: str) -> str:
        """Generate default output filename"""
        name, ext = os.path.splitext(input_file)
        return f"{name}_out{ext}"

    def validate(self) -> None:
        """Validate parameters before execution"""
        if not os.path.exists(self.input_file):
            raise FileNotFoundError(f"Input file not found: {self.input_file}")
        if self.timeout <= 0:
            raise ValueError("Timeout must be positive")

Parameter Types:

# Positional arguments (required)
input_file: str = typer.Argument(..., help="Input file path")

# Options (with defaults)
output: Optional[str] = typer.Option(None, "-o", "--output", help="Output file")
verbose: bool = typer.Option(False, "-v", "--verbose", help="Verbose output")
count: int = typer.Option(1, "-c", "--count", help="Number of iterations")

# Flags with explicit True/False
force: bool = typer.Option(False, "--force/--no-force", help="Force operation")

# Multiple values
files: list[str] = typer.Option([], "-f", "--file", help="Input files (multiple allowed)")

______________________________________________________________________

4. Output Formatting

Use Rich for beautiful, informative output:

# Rich output formatting
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
from rich.progress import Progress, SpinnerColumn, TextColumn

console = Console()

# Status messages
console.print("[green]✓[/green] Operation completed successfully")
console.print("[red]✗[/red] Error: File not found")
console.print("[yellow]⚠[/yellow] Warning: Deprecated feature used")
console.print("[cyan]?[/cyan] Processing: file.txt")

# Panel output
panel = Panel.fit(
    "Result content",
    title="Results",
    border_style="blue",
    padding=(1, 2)
)
console.print(panel)

# Table output
table = Table(title="Files")
table.add_column("Name", style="cyan")
table.add_column("Size", justify="right", style="green")
table.add_column("Status", style="yellow")
table.add_row("file1.txt", "1.2 KB", "OK")
table.add_row("file2.txt", "3.4 KB", "OK")
console.print(table)

# Progress indicators
with Progress(
    SpinnerColumn(),
    TextColumn("[progress.description]{task.description}"),
    console=console
) as progress:
    task = progress.add_task("Processing...", total=100)
    # Work here
    progress.update(task, advance=50)

______________________________________________________________________

5. Environment Variables Integration

Manage configuration via environment variables:

# Environment variable patterns
import os
from dotenv import load_dotenv
from dataclasses import dataclass

# Load .env file in development
load_dotenv()

@dataclass
class EnvConfig:
    """Environment-based configuration"""
    debug: bool = False
    timeout: int = 30
    api_key: str = ""
    max_retries: int = 3

    @classmethod
    def from_env(cls) -> "EnvConfig":
        """Load configuration from environment variables"""
        return cls(
            debug=os.getenv("DEBUG", "false").lower() == "true",
            timeout=int(os.getenv("TIMEOUT", "30")),
            api_key=os.getenv("API_KEY", ""),
            max_retries=int(os.getenv("MAX_RETRIES", "3"))
        )

    def validate(self) -> None:
        """Validate required environment variables"""
        if not self.api_key and not self.debug:
            raise EnvironmentError("API_KEY environment variable required")

Accessing env vars in Click:

# Click with environment variables
import click
from dotenv import load_dotenv

load_dotenv()

@click.option("--config", envvar="APP_CONFIG", help="Config file path")
@click.option("--debug", envvar="DEBUG", is_flag=True, help="Debug mode")
def process(config, debug):
    ...

______________________________________________________________________

6. Error Handling

Provide clear, actionable error messages:

# Error handling with user-friendly messages
from rich.console import Console
import sys

console = Console()

def handle_cli_error(error, context="CLI", verbose: bool = False) -> int:
    """Handle errors with user-friendly messages, return exit code"""
    if isinstance(error, FileNotFoundError):
        console.print(f"[red]✗[/red] File not found: {error.filename}")
        suggest_similar_files(error.filename)
        return 1
    elif isinstance(error, PermissionError):
        console.print(f"[red]✗[/red] Permission denied: {error.filename}")
        console.print("[yellow]⚠[/yellow] Try running with elevated privileges")
        return 1
    elif isinstance(error, ValueError) as e:
        console.print(f"[red]✗[/red] Invalid value: {str(e)}")
        return 1
    else:
        console.print(f"[red]✗[/red] {context} error: {str(error)}")
        if verbose:
            console.print_exception()
        return 1

def suggest_similar_files(filename: str) -> None:
    """Suggest similar files when file not found"""
    import os
    directory = os.path.dirname(filename)
    basename = os.path.basename(filename)
    if os.path.exists(directory):
        matches = [f for f in os.listdir(directory) if basename.lower() in f.lower()]
        if matches:
            console.print(f"[cyan]Did you mean:[/cyan] {', '.join(matches)}")

# Main entry point with error handling
def main():
    try:
        cli_params = CLIParameters.from_args(args)
        cli_params.validate()
        run_process(cli_params)
    except Exception as e:
        sys.exit(handle_cli_error(e, verbose=args.verbose))

Exit codes:

______________________________________________________________________

7. Help Text and Documentation

Write clear, useful help text:

# Good help text patterns
def main(
    input_file: str = typer.Argument(
        ...,
        help="Path to input file",
        show_default=False
    ),
    output: Optional[str] = typer.Option(
        None,
        "-o", "--output",
        help="Output file path [default: auto-generated from input]",
        show_default=False
    ),
    workers: int = typer.Option(
        4,
        "-w", "--workers",
        help="Number of parallel workers",
        min=1,
        max=32
    ),
    verbose: bool = typer.Option(
        False,
        "-v", "--verbose",
        help="Enable verbose output"
    )
):
    """
    Process INPUT_FILE and generate formatted output.

    Examples:

        python cli.py process data.txt -o result.txt

        python cli.py process data.txt --workers 8 --verbose
    """
    ...

Help text guidelines:

• Keep descriptions concise (one line if possible)
• Use imperative mood ("Process file" not "Processes file")
• Show defaults when relevant
• Provide examples for complex commands
• Document exit codes for scripts

______________________________________________________________________

8. Testing CLI Applications

Test CLI applications thoroughly:

# Testing CLI applications
import pytest
from typer.testing import CliRunner
from project.cli.main import app

runner = CliRunner()

def test_cli_basic_invocation():
    """Test basic CLI invocation"""
    result = runner.invoke(app, ["--help"])
    assert result.exit_code == 0
    assert "--help" in result.output

def test_cli_process_file(tmp_path):
    """Test file processing command"""
    input_file = tmp_path / "input.txt"
    input_file.write_text("test content")

    result = runner.invoke(app, ["process", str(input_file)])
    assert result.exit_code == 0
    assert "Processing" in result.output

def test_cli_missing_file():
    """Test error handling for missing file"""
    result = runner.invoke(app, ["process", "/nonexistent/file.txt"])
    assert result.exit_code == 1
    assert "not found" in result.output.lower()

def test_cli_with_options(tmp_path):
    """Test CLI with various options"""
    input_file = tmp_path / "input.txt"
    input_file.write_text("test content")

    result = runner.invoke(app, [
        "process",
        str(input_file),
        "-o", str(tmp_path / "output.txt"),
        "-v"
    ])
    assert result.exit_code == 0

# Fixture for CLI runner
@pytest.fixture
def cli_runner():
    """Provide a CLI runner for tests"""
    return CliRunner()

# Isolated filesystem tests
def test_cli_isolated_filesystem(cli_runner):
    """Test CLI with isolated filesystem"""
    with cli_runner.isolated_filesystem():
        # Create test files
        os.mkdir("input")
        # Run CLI
        result = cli_runner.invoke(app, ["process", "input/file.txt"])
        # Assertions

Code Review Reception

Table of Contents

1. Workflow 2. No Performative Agreement 3. Push Back with Technical Reasoning 4. Evaluating Feedback Quality 5. When to Implement vs When to Push Back 6. Responding to Unclear Feedback

______________________________________________________________________

Workflow

READ → UNDERSTAND → VERIFY → EVALUATE → RESPOND → IMPLEMENT

1. READ – Complete feedback without reacting 2. UNDERSTAND – Restate requirement in own words (or ask) 3. VERIFY – Check against codebase reality 4. EVALUATE – Technically sound for THIS codebase? 5. RESPOND – Technical acknowledgment or reasoned pushback 6. IMPLEMENT – One item at a time, test each

Implementation order for multi-item feedback:

• Blocking issues (breaks, security)
• Simple fixes (typos, imports)
• Complex fixes (refactoring, logic)

______________________________________________________________________

No Performative Agreement

NEVER:

• "You're absolutely right!"
• "Great point!" / "Excellent feedback!"
• "Thanks for catching that!"
• Any gratitude expression

INSTEAD:

• Restate the technical requirement
• Ask clarifying questions
• Push back with technical reasoning if wrong
• Just start working (actions > words)

Why: Actions speak. Just fix it. The code itself shows you heard the feedback.

______________________________________________________________________

Push Back with Technical Reasoning

Push back when:

• Suggestion breaks existing functionality
• Reviewer lacks full context
• Violates YAGNI (unused feature)
• Technically incorrect for this stack
• Legacy/compatibility reasons exist
• Conflicts with architectural decisions

How to push back:

• Use technical reasoning, not defensiveness
• Ask specific questions
• Reference working tests/code
• Escalate to decision-maker if architectural

Signal if uncomfortable pushing back: "Strange things are afoot at the Circle K"

______________________________________________________________________

Evaluating Feedback Quality

Before implementing external feedback:

1. Technically correct for THIS codebase? 2. Breaks existing functionality? 3. Reason for current implementation? 4. Works on all platforms/versions? 5. Does reviewer understand full context?

YAGNI Check:

IF reviewer suggests unused feature:
  grep codebase for actual usage
  IF unused: "This endpoint isn't called. Remove it (YAGNI)?"

______________________________________________________________________

When to Implement vs When to Push Back

Acknowledging correct feedback:

✅ "Fixed. [Brief description]"
✅ "Good catch - [specific issue]. Fixed in [location]."
✅ [Just fix it and show the code]

If you were wrong:

✅ "You were right - I checked [X] and it does [Y]. Implementing now."
✅ "Verified, you're correct. My initial understanding was wrong because [reason]. Fixing."

______________________________________________________________________

Responding to Unclear Feedback

IF any item is unclear:
  STOP – do not implement anything yet
  ASK for clarification on unclear items

Why: Items may be related. Partial understanding = wrong implementation.

Example:

Partner: "Fix items 1-6"
You understand 1,2,3,6. Unclear on 4,5.

❌ WRONG: Implement 1,2,3,6 now, ask about 4,5 later
✅ RIGHT: "I understand items 1,2,3,6. Need clarification on 4 and 5 before proceeding."

When can't verify:

"I can't verify this without [X]. Should I [investigate/ask/proceed]?"

______________________________________________________________________

GitHub Thread Replies

Reply in the comment thread (gh api repos/{owner}/{repo}/pulls/{pr}/comments/{id}/replies), not as a top-level PR comment.

______________________________________________________________________

The Bottom Line

External feedback = suggestions to evaluate, not orders to follow.

Verify. Question. Then implement.

No performative agreement. Technical rigor always.

Coding Standards Reference

Table of Contents

• Coding Standards Reference
• Table of Contents
• 1. Type Hints (Mandatory)
• 2. String Formatting
• 3. Data Structures
• Comprehensions
• pathlib.Path
• enumerate()
• Resource Management
• 3.5. Naming Conventions
• 4. Logging
• 5. Code Size Limits
• 6. Docstrings
• 7. Comments
• 8. Library Preferences
• 9. Version Control
• Commit Message Format
• Branch Naming
• 10. Boolean Flags and Environment Variables
• Boolean Flags
• Environment Variables
• 11. Error Handling
• 12. Prohibited Patterns
• TYPE_CHECKING
• Prohibited Linter Rules
• os.path
• [Optional[T]](#optionalt)
• Linting and Formatting

______________________________________________________________________

1. Type Hints (Mandatory)

Type hints are required for all function parameters and return values.

from typing import Any

def process_data(item_id: str, group_id: int) -> dict[str, Any]:
    """Process a data item and return metadata."""
    return {"id": item_id, "group": group_id}

# Use pipe syntax for optional types
def fetch_data(item_id: str | None) -> dict[str, Any] | None:
    """Fetch data metadata, returns None if not found."""
    if item_id is None:
        return None
    return {"id": item_id}

Rules:

• Use T | None instead of Optional[T]
• Use Any from typing instead of object
• Include type hints in docstrings for parameters and returns

Null comparisons: Use is and is not for None, not == or !=.

______________________________________________________________________

2. String Formatting

Use f-strings for all string formatting.

# CORRECT
name = "S4-123456"
meeting = "SA4#99e"
print(f"Processing {name} from {meeting}")

# WRONG
print("Processing {} from {}".format(name, meeting))
print("Processing %s from %s" % (name, meeting))

______________________________________________________________________

3. Data Structures

Comprehensions

# List comprehension
ids = [i.id for i in items if i.active]

# Dictionary comprehension
item_map = {i.id: i for i in items}

pathlib.Path

from pathlib import Path

# CORRECT
cache_dir = Path.home() / ".project-cache" / "data"
cache_dir.mkdir(parents=True, exist_ok=True)

# WRONG
import os.path
cache_dir = os.path.join(os.path.expanduser("~"), ".project-cache", "data")

enumerate()

# CORRECT - when you need both index and value
for i, item in enumerate(items):
    print(f"{i}: {item.id}")

# WRONG
for i in range(len(items)):
    print(f"{i}: {items[i].id}")

Resource Management

Use with statements when working with files.

# CORRECT
from pathlib import Path
with open("data.json") as f:
    data = json.load(f)

# WRONG
f = open("data.json")
data = json.load(f)
f.close()

______________________________________________________________________

3.5. Naming Conventions

See naming-conventions.md for comprehensive naming conventions, including:

• Files and directories (suffix-based naming: _file, _dir, _path)
• Test naming conventions
• Fixture naming conventions
• Unused loop variables
• Constants and enums

______________________________________________________________________

4. Logging

Use the logging module instead of print().

import logging

logger = logging.getLogger(__name__)

logger.debug("Processing item: %s", item_id)
logger.info("Successfully fetched metadata for %s", item_id)
logger.warning("Retry attempt %d for %s", retry_count, item_id)
logger.error("Failed to process %s: %s", item_id, exc_info=True)

______________________________________________________________________

5. Code Size Limits

Keep modules and symbols small to preserve maintainability:

Refactor when limits are exceeded:

• Split large functions into smaller helper functions
• Extract common logic to separate utilities
• Split large classes into composition of smaller classes

______________________________________________________________________

6. Docstrings

Write clear, concise docstrings using Google style.

def fetch_item_metadata(item_id: str, source: str) -> dict[str, Any] | None:
    """Fetch data metadata from the specified source.

    Args:
        item_id: The unique identifier (e.g., "DOC-123456")
        source: The metadata source ("api", "database", or "cache")

    Returns:
        Dictionary with metadata fields (title, author, timestamp, etc.),
        or None if the item is not found.

    Raises:
        ValueError: If source is not recognized
        ConnectionError: If the API endpoint is unreachable

    Examples:
        >>> fetch_item_metadata("DOC-123456", "api")
        {"id": "DOC-123456", "title": "Example document", ...}
    """

______________________________________________________________________

7. Comments

Comments should explain intent or subtle constraints, not restate what's obvious.

# GOOD - explains WHY
# IDs are case-insensitive, so normalize to uppercase
item_id = item_id.upper()

# GOOD - documents a subtle constraint
# The portal API rate-limits to 10 requests/second
time.sleep(0.1)

# BAD - restates what the code does
# Get the ID from the item object
id = item.id

DO NOT:

• Repeat obvious names (variable/function names already describe what they are)
• Include "what you did" (belongs in commit messages)
• Use decorative headings (===== MIGRATION TOOLS =====)
• Number steps (// Step 3: Fetch data)
• Use emojis or special Unicode characters

______________________________________________________________________

8. Library Preferences

______________________________________________________________________

8.5. Data Modeling — Dataclass vs. Pydantic

Choose the right tool based on what the data does, not just what it holds.

Decision Framework

Rules

1. Default to `@dataclass` for purely internal data. If the data never crosses a trust boundary (user input, network, file), does not need JSON serialization, and does not need runtime validation, use @dataclass. It is faster, has no extra dependency, and frozen=True gives you value semantics for free.

2. Use `pydantic` at trust boundaries. Any time data comes from outside the program (HTTP request, CLI args parsed to a model, config file, database row), use pydantic.BaseModel so invalid data fails loudly and early with a descriptive error.

3. Don't mix the two for the same concept. If a type starts as @dataclass and later needs validation, convert it to pydantic.BaseModel rather than bolting on manual __post_init__ validation in a dataclass.

Examples

from dataclasses import dataclass

# CORRECT — lightweight internal point, never leaves the process
@dataclass(frozen=True, slots=True)
class Point:
    x: float
    y: float

from pydantic import BaseModel, Field, EmailStr

# CORRECT — user-facing data with validation at a trust boundary
class UserRegistration(BaseModel):
    name: str = Field(min_length=1, max_length=100)
    email: EmailStr
    age: int = Field(ge=0, le=150)

from dataclasses import dataclass

# CORRECT — CLI parameter bundle, validated by Typer before reaching this struct
@dataclass
class RenderOptions:
    width: int
    height: int
    output_file: Path

from typing import TypedDict

# CORRECT — dict-shaped data from an external JSON API you don't control
class GithubRepo(TypedDict):
    id: int
    name: str
    full_name: str

Anti-patterns

# WRONG — pydantic is overkill for a simple internal struct
from pydantic import BaseModel
class Point(BaseModel):
    x: float
    y: float

# WRONG — dataclass provides no runtime validation for untrusted input
@dataclass
class UserInput:
    email: str  # Accepts "not-an-email" silently

# CORRECT
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class Point:
    x: float
    y: float

from pydantic import BaseModel, EmailStr
class UserInput(BaseModel):
    email: EmailStr

______________________________________________________________________

9. Version Control

Commit Message Format

Type: feat, fix, docs, style, refactor, perf, test, chore
Scope: optional module/component
Subject: imperative, present tense, <50 chars
Body: optional detailed explanation
Footer: optional breaking changes, issue references

Branch Naming

Feature branches: feature/<name>
Bug fixes: fix/<description>
Documentation: docs/<topic>
Releases: release/v<version>

______________________________________________________________________

10. Boolean Flags and Environment Variables

Boolean Flags

class FeatureFlags:
    def __init__(self, **flags):
        self._flags = flags

    def is_enabled(self, flag_name: str) -> bool:
        return self._flags.get(flag_name, False)

    def enable(self, flag_name: str) -> None:
        self._flags[flag_name] = True

    def disable(self, flag_name: str) -> None:
        self._flags[flag_name] = False

Environment Variables

import os

def get_env_var(name: str, default: str | None = None) -> str:
    """Get environment variable with validation."""
    value = os.environ.get(name, default)
    if value is None:
        raise ValueError(f"Required environment variable {name} not set")
    return value

______________________________________________________________________

11. Error Handling

def safe_operation(func, *args, **kwargs):
    """Execute function with universal error handling."""
    try:
        return func(*args, **kwargs)
    except Exception as e:
        logger.error("Operation %s failed: %s", func.__name__, str(e), exc_info=True)
        return get_fallback_value(func)

______________________________________________________________________

12. Prohibited Patterns

TYPE_CHECKING

Never use TYPE_CHECKING as a permanent solution for circular imports. Refactor to eliminate circular dependencies.

Canonical deep-dive guidance for this rule lives in type-checking.md. Keep this section concise and use the dedicated reference for alternatives and migration steps.

# WRONG - permanent TYPE_CHECKING guard
from typing import TYPE_CHECKING
if TYPE_CHECKING:
    from my_project.database import ItemDatabase

# RIGHT - lazy import (temporary fix) or refactor
def _resolve_data_id(db_file: Path) -> int:
    from my_project.database import ItemDatabase  # Lazy import
    with ItemDatabase(db_file) as db:
        return db.resolve_data_id(name)

Prohibited Linter Rules

Never suppress linter issues with # noqa. Fix the underlying issue instead.

Never introduce these rules:

• PLC0415, E402 - Import at top of file
• ANN001 - Missing type annotation for function argument
• E402 - Module-level import not at top of file
• ANN201 - Missing return type annotation for public function
• ANN202 - Missing return type annotation for private function

os.path

Use pathlib.Path instead of os.path.

Optional[T]

Use T | None instead of Optional[T].

Vague Input/Output Types

Never design functions with wide, ambiguous parameter or return types that force the caller to handle many implicit cases. Such signatures are a sign that the function is doing too much input validation and output normalization internally — responsibilities that should belong to the caller.

Anti-pattern — all of these are wrong:

def fear_to_failure_func(
    arg1: int | str | None,
    arg2: str | None = None,
    arg3: Any | None = None,
) -> int | None:
    if arg1 is None:
        return None
    if isinstance(arg1, str):
        try:
            arg1 = int(arg1)
        except ValueError:
            return None
    # ... boilerplate for arg2, arg3 ...
    if result > 0:
        return result
    return None

Why it's wrong:

• The function signature hides what it actually accepts, making callers guess.
• Repeated isinstance / hasattr checks inside the function are a code smell — they indicate the function is compensating for a vague signature.
• T | None for both input and output forces callers to handle many implicit branches.
• The function silently swallows cases (return None) instead of failing loudly, masking bugs.

Principles to apply instead:

1. Be precise. Use the narrowest type that describes the actual data, not the widest type you think might work. If arg1 should be an int, say int, not int | str | None. 2. Fail loudly. If a caller passes an invalid type, raise a clear TypeError or ValueError immediately rather than returning None. Let the caller handle the error at the call site. 3. Separate concerns. Input conversion and output normalization are the caller's responsibility. The function should trust its contract. 4. Avoid `Any` in signatures. Use it only in genuinely untyped contexts (e.g., **kwargs). Prefer protocols or base types for structured flexibility. 5. Single-responsibility functions. If you find yourself writing isinstance or hasattr checks inside a function, split it into two functions: one that validates/transforms, one that does the work.

Correct patterns:

# Input: accept only what you actually need
def process_item(item_id: int) -> dict[str, Any]:
    """Process a single item by its integer ID."""
    ...

# Output: use a dedicated result type instead of None-as-error
@dataclass
class ProcessResult:
    value: int
    reason: str | None = None

def process_item(item_id: int) -> ProcessResult:
    ...

When `T | None` is acceptable:

• Return type when None is a legitimate, documented outcome (e.g., "not found" vs "found"). Document it explicitly.
• Parameters that are genuinely optional in the business sense — e.g., timeout: float | None = None where None means "use default". Not because you are afraid of validation.

Quick self-check:

• Does the function body contain isinstance or hasattr on its own parameters? → Split or narrow the type.
• Does the function return None as a silent fallback for more than one condition? → Use a result type or raise an exception.
• Is Any in the signature because you don't know what the caller might pass? → The caller should convert first.

______________________________________________________________________

Linting and Formatting

Run linters after significant changes:

# Run linter
ruff check src/ tests/

# Format code
ruff format src/ tests/

# Sort imports
ruff check src tests --select I --fix

# Type check
ty src/

Always use uv run for Python commands to ensure the virtual environment is activated.

For more details, see: Ruff Rules

Systematic Debugging Reference

Table of Contents

1. Iron Law 2. 4-Phase Process

• Phase 1: Root Cause Analysis
• Phase 2: Pattern Analysis
• Phase 3: Hypothesis Formation
• Phase 4: Implementation

3. Red Flags 4. Common Rationalizations to Avoid 5. Debugging Tools and Techniques 6. When to Stop and Reassess

______________________________________________________________________

Iron Law

NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST

Violating the letter of this process is violating the spirit of debugging.

If you haven't completed Phase 1, you cannot propose fixes.

______________________________________________________________________

4-Phase Process

Complete each phase before proceeding to the next.

Phase 1: Root Cause Analysis

Before attempting ANY fix:

1. Read Error Messages Carefully

• Don't skip past errors or warnings
• Read stack traces completely
• Note line numbers, file paths, error codes

2. Reproduce Consistently

• Can you trigger it reliably?
• What are the exact steps?
• Does it happen every time?
• If not reproducible → gather more data, don't guess

3. Check Recent Changes

• What changed that could cause this?
• Git diff, recent commits
• New dependencies, config changes

4. Trace Data Flow (for errors deep in call stack)

• Where does bad value originate?
• What called this with bad value?
• Trace up until you find the source
• Fix at source, not at symptom

5. Gather Evidence in Multi-Component Systems For each component boundary:

• Log what data enters component
• Log what data exits component
• Verify environment/config propagation
• Check state at each layer

Phase 2: Pattern Analysis

1. Find Working Examples

• Locate similar working code in same codebase
• What works that's similar to what's broken?

2. Compare Against References

• If implementing a pattern, read reference implementation COMPLETELY
• Don't skim - read every line

3. Identify Differences

• What's different between working and broken?
• List every difference, however small
• Don't assume "that can't matter"

4. Understand Dependencies

• What other components does this need?
• What settings, config, environment?

Phase 3: Hypothesis Formation

Apply the scientific method:

1. Form Single Hypothesis

• State clearly: "I think X is the root cause because Y"
• Write it down
• Be specific, not vague

2. Test Minimally

• Make the SMALLEST possible change to test hypothesis
• One variable at a time
• Don't fix multiple things at once

3. Verify Before Continuing

• Did it work? Yes → Phase 4
• Didn't work? Form NEW hypothesis
• DON'T add more fixes on top

Phase 4: Implementation

1. Create Failing Test Case First

• Simplest possible reproduction
• Automated test if possible
• MUST exist before fixing

2. Implement Single Fix

• Address the root cause identified
• ONE change at a time
• No "while I'm here" improvements

3. Verify Fix

• Test passes now?
• No other tests broken?
• Issue actually resolved?

______________________________________________________________________

Red Flags

STOP and return to Phase 1 if you catch yourself thinking:

• "Quick fix for now, investigate later"
• "Just try changing X and see if it works"
• "Add multiple changes, run tests"
• "Skip the test, I'll manually verify"
• "It's probably X, let me fix that"
• "I don't fully understand but this might work"
• Proposing solutions before tracing data flow
• "One more fix attempt" (when already tried 2+)
• Each fix reveals new problem in different place

______________________________________________________________________

Common Rationalizations to Avoid

______________________________________________________________________

Debugging Tools and Techniques

1. Instrumentation at Boundaries

• Log data entering and exiting each component
• Verify environment/config propagation
• Check state at each layer

2. Backward Tracing

• Start from where error manifests
• Trace data flow backward to origin
• Fix at source, not symptom

3. Minimal Test Cases

• Isolate the smallest possible reproduction
• One variable at a time

4. Compare Against Working State

• Git diff between known good state
• Compare configurations
• Check environmental differences

______________________________________________________________________

When to Stop and Reassess

If 3+ fixes failed, question the architecture:

Signs of architectural problems:

• Each fix reveals new shared state/coupling/problem in different place
• Fixes require "massive refactoring" to implement
• Each fix creates new symptoms elsewhere

STOP and question fundamentals:

• Is this pattern fundamentally sound?
• Should we refactor architecture vs. continue fixing symptoms?
• Discuss with your human partner before attempting more fixes

When process reveals "no root cause": If systematic investigation reveals issue is truly environmental, timing-dependent, or external:

1. Document what you investigated 2. Implement appropriate handling (retry, timeout, error message) 3. Add monitoring/logging for future investigation

Remember: 95% of "no root cause" cases are incomplete investigation.

Documentation Reference

Comprehensive guide for creating codebase documentation.

Table of Contents

01. Project Overview 02. Architecture 03. Key Components 04. Data Flow 05. API Reference 06. Configuration 07. Setup Guide 08. Development Guide 09. Testing 10. Deployment

______________________________________________________________________

Documentation Workflow

Follow this process to document a codebase:

1. Explore — Launch Explore agent for thorough codebase investigation 2. Map — Use Glob to map directory structure and entry points 3. Read — Analyze README, entry points, core modules, configuration 4. Synthesize — Merge findings into cohesive documentation

Depth Levels

______________________________________________________________________

1. Project Overview

Purpose: Explain what the project does and why it exists.

Include:

• Purpose & vision
• Target users
• Key features
• Technology stack
• Project status

______________________________________________________________________

2. Architecture

Purpose: Explain how components interact at a high level.

Include:

• High-level structure
• Design patterns
• Control flow
• Architectural decisions

Architecture Diagram

graph TB
    A[Entry Point] --> B[Core Module]
    B --> C[Service Layer]
    B --> D[Data Layer]
    C --> D
    D --> E[External APIs]

Component Relationships

graph LR
    A[Module A] --> B[Module B]
    A --> C[Module C]
    B --> D[Shared Utils]
    C --> D

______________________________________________________________________

3. Key Components

Purpose: Document major modules, classes, and functions.

Include:

• Major modules and their responsibilities
• Classes & functions with purpose
• Interactions between components
• Extension points
• Code examples

Component Template

````markdown

Module: <name>

Purpose: What this module does

Entry Points:

• function_name() — description

Key Classes:

• ClassName — description

Usage Example:

from module import function_name
result = function_name(arg1, arg2)

````

---

4. Data Flow

Purpose: Trace how data moves through the system.

Data Flow Diagram

flowchart LR
    A[Input] --> B[Processing]
    B --> C[Transformation]
    C --> D[Storage]
    D --> E[Output]

Sequence Diagram

sequenceDiagram
    participant Client
    participant API
    participant Service
    participant Database
    
    Client->>API: Request
    API->>Service: Process
    Service->>Database: Query
    Database-->>Service: Result
    Service-->>API: Response
    API-->>Client: Output

______________________________________________________________________

5. API Reference

Purpose: Document external and internal APIs.

Include:

• Endpoints with request/response formats
• Authentication requirements
• Error codes and handling
• Rate limits

Endpoint Template

````markdown

POST /endpoint

Description: What the endpoint does

Request:

{
  "field": "type"
}

Response:

{
  "result": "type"
}

````

---

6. Configuration

Purpose: Document all configuration options.

Include:

• Environment variables
• Config files and schemas
• Default values
• Required vs optional settings

Configuration Table

---

7. Setup Guide

Purpose: Get new developers running locally.

Include:

• Prerequisites
• Installation steps
• Configuration
• Running the application
• Troubleshooting

Quick Start

# Clone repository
git clone <repo-url>

# Install dependencies
uv sync

# Configure environment
cp .env.example .env

# Run application
uv run python src/main.py

______________________________________________________________________

8. Development Guide

Purpose: Standards for contributing code.

Include:

• Coding conventions
• Testing approach
• Error handling patterns
• Logging standards
• Security practices
• Performance guidelines

Coding Standards

• Follow PEP 8 style guidelines
• Use type hints for all function signatures
• Write docstrings for public APIs
• Keep functions focused (single responsibility)

Error Handling

try:
    result = risky_operation()
except SpecificError as e:
    logger.error(f"Failed: {e}")
    raise CustomError("Meaningful message") from e

______________________________________________________________________

9. Testing

Purpose: Document testing requirements and patterns.

Include:

• Test categories (unit, integration, e2e)
• Running tests
• Writing new tests
• Coverage requirements
• Mocking patterns

Test Structure

graph TB
    A[Tests] --> B[Unit Tests]
    A --> C[Integration Tests]
    A --> D[E2E Tests]
    
    B --> E[Fast, Isolated]
    C --> F[Component Interaction]
    D --> G[Full Flow]

Running Tests

# Run all tests
uv run pytest

# Run with coverage
uv run pytest --cov=src --cov-report=html

# Run specific category
uv run pytest tests/unit/

______________________________________________________________________

10. Deployment

Purpose: Document deployment process.

Include:

• Build process
• Deployment steps
• Environments (dev, staging, prod)
• Monitoring setup
• Rollback procedures

Deployment Flow

flowchart LR
    A[Build] --> B[Test]
    B --> C[Stage]
    C --> D[Production]
    
    D --> E[Monitor]
    E -->|Failure| F[Rollback]

______________________________________________________________________

When to Document

Use this skill when you hear:

• "explain this codebase"
• "document the architecture"
• "how does this code work"
• "create developer documentation"
• "generate codebase overview"
• "create onboarding docs"

______________________________________________________________________

Output Structure

Create documentation as:

docs/
├── README.md              # Overview and quick start
├── ARCHITECTURE.md        # System architecture
├── DEVELOPMENT.md         # Development guide
├── API.md                 # API documentation
├── DEPLOYMENT.md          # Deployment guide
└── CONTRIBUTING.md        # Contribution guidelines

Or a single comprehensive document if preferred.

______________________________________________________________________

Visual Elements

Include these for clarity:

• Mermaid diagrams — Architecture, flow charts, sequence diagrams
• Code examples — From actual codebase
• File references — Specific file:line citations
• Tables — Structured information
• Lists — Guidelines and requirements

______________________________________________________________________

Success Criteria

Good documentation includes:

• [ ] Complete coverage of all 10 sections
• [ ] Clear explanations with examples
• [ ] Visual diagrams for complex concepts
• [ ] Specific file:line references
• [ ] Actionable setup/development instructions
• [ ] New developer can onboard using only docs
• [ ] Organized, navigable structure
• [ ] Accurate and current information

File Analysis Reference

Comprehensive guide for non-destructive file analysis using read-only operations.

Table of Contents

1. Non-Destructive Principle 2. File Metadata 3. Line Counting 4. Pattern Searching 5. Content Statistics 6. Codebase Analysis Patterns

______________________________________________________________________

Non-Destructive Principle

Core Rule: Never modify files during analysis. Use only read-only operations.

Allowed Operations

Forbidden Operations

• sed, awk for in-place editing
• echo > file or output redirection for writing
• Any tool that modifies file content

______________________________________________________________________

File Metadata

Get Single File Metadata

stat -f "%z bytes, modified %Sm" [file_path]     # macOS
stat --printf="%s bytes, modified %y\n" [file_path]  # Linux

List Multiple Files with Sizes

ls -lh [directory]                              # Human-readable sizes
ls -ltr [directory]                            # Sort by modification time

Get Directory Disk Usage

du -h [file_path]                              # Human-readable
du -sh [directory]                             # Total only
du -ah [directory] | sort -rh | head -20       # Largest files

File Modification Times

stat [file_path]                               # Full stat output
ls -l [file_path]                              # Modification time

______________________________________________________________________

Line Counting

Single File

wc -l [file_path]                              # Line count

Multiple Files

wc -l [file1] [file2] [file3]                  # Count multiple files
wc -l *.py                                     # Wildcard expansion

Directory Totals

find [dir] -name "*.py" | xargs wc -l           # All Python files
find . -type f -name "*.py" -exec wc -l {} +    # Recursive with exec

Filtered Counting

find . -name "test_*.py" | xargs wc -l          # Test files only
find . -path ./node_modules -prune -o -type f -print | xargs wc -l  # Exclude directories

______________________________________________________________________

Pattern Searching

Count Matches

Grep(pattern="^def ", output_mode="count", path="src/")
Grep(pattern="^class ", output_mode="count", path="src/")
Grep(pattern="^import ", output_mode="count", path="src/")

Find with Line Numbers

Grep(pattern="TODO|FIXME|HACK", output_mode="content", -n=true)

Find Files by Pattern

Glob(pattern="**/*.py")
Glob(pattern="**/*.md", path="docs/")

Search with Inclusion Filter

Grep(pattern="function", include="*.js", path="src/")
Grep(pattern="class", include="*.ts", path=".")

Common Patterns for Code Analysis

______________________________________________________________________

Content Statistics

Analyze File Structure

1. Read the file to understand structure 2. Count functions: Grep(pattern="^def ", output_mode="count") 3. Count classes: Grep(pattern="^class ", output_mode="count") 4. Count imports: Grep(pattern="^import |^from ", output_mode="count")

Code Quality Metrics

# Total lines of code
find . -name "*.py" -not -path "./venv/*" | xargs wc -l

# Comment lines
Grep(pattern="^[[:space:]]*#", output_mode="count", include="*.py")

# Blank lines
Grep(pattern="^$", output_mode="count", include="*.py")

# Calculate comment ratio
# (comment lines / total lines) * 100

Find Largest Files

find . -type f -not -path "./node_modules/*" -not -path "./.git/*" \
  -exec du -h {} + | sort -rh | head -20

______________________________________________________________________

Codebase Analysis Patterns

Comprehensive File Analysis Workflow

1. Get metadata: stat -f "%z bytes, modified %Sm" file.py 2. Count lines: wc -l file.py 3. Read file: Read(file_path="file.py") 4. Count functions: Grep(pattern="^def ", output_mode="count") 5. Count classes: Grep(pattern="^class ", output_mode="count")

Compare File Sizes

1. Find files: Glob(pattern="src/**/*.py") 2. Get sizes: ls -lh src/**/*.py 3. Total size: du -sh src/

Project-wide Analysis

1. Count all source files: Glob(pattern="**/*.py") then count 2. Lines per file: find . -name "*.py" | xargs wc -l | sort -n 3. Identify largest files: du -ah . | sort -rh | head -20 4. Find duplicate names: find . -name "*.py" | xargs -I{} basename {} | sort | uniq -c | sort -rn

Integration with Other Skills

• code-auditor: Use file analysis before auditing
• code-transfer: Analyze large files before transfer decisions
• codebase-documenter: Use stats to understand file purposes

______________________________________________________________________

Best Practices

1. Always prefer read-only: Never modify files during analysis 2. Use efficient tools: Read small files fully, use Grep for large files 3. Context-aware: Compare to project averages 4. Partial reads for large files: Use Read with offset/limit 5. Chain operations: Combine Glob + Grep for targeted analysis 6. Sort and filter: Use sort, head, tail for useful outputs

Imports for Required vs Optional Dependencies

Rule for runtime imports and dependency groups.

Table of Contents

1. Hard Rule 2. Why This Matters 3. Patterns 4. Special Cases 5. How to Check

______________________________________________________________________

1. Hard Rule

Never wrap imports for required dependencies in `try/except ImportError`.

If a package is listed in [project.dependencies], import it normally at module top level. A missing required dependency is a setup error and should fail fast.

# CORRECT - required dependency import
import numpy as np
import requests

Allowed exception: optional or plugin dependencies that are intentionally not installed in every environment. Those belong in [project.optional-dependencies] and must be handled with explicit fallback behavior.

______________________________________________________________________

2. Why This Matters

Defensive imports for required packages create avoidable complexity:

• They hide environment problems that should be fixed at install time.
• They force repeated is not None checks throughout the code.
• They defer failures to unrelated runtime paths.
• They make behavior harder to reason about and test.

Fail fast for required dependencies. Degrade gracefully only for truly optional features.

______________________________________________________________________

3. Patterns

3.1 Anti-Pattern: Defensive Import for Required Dependency

# WRONG - required dependency should not be optionalized
try:
    import my_package
except ImportError:
    my_package = None


def run_feature() -> None:
    if my_package is not None:
        my_package.some_method()


def run_other_feature() -> None:
    if my_package is not None:
        my_package.other_method()
    else:
        raise RuntimeError("my_package not installed")

3.2 Correct: Required Dependency Imported Normally

# CORRECT - dependency is required by project configuration
import my_package


def run_feature() -> None:
    my_package.some_method()


def run_other_feature() -> None:
    my_package.other_method()

3.3 Allowed: Optional Dependency with Localized Handling

from collections.abc import Callable


def build_optional_renderer() -> Callable[[str], str] | None:
    """Return renderer when optional extra is installed, otherwise None."""
    try:
        import rich
    except ImportError:
        return None

    def render(message: str) -> str:
        return rich.markup.escape(message)

    return render

Guidelines for optional imports:

• Keep handling near one boundary (startup, factory, or plugin loader).
• Avoid scattering repeated if package is not None checks.
• Return explicit fallback behavior (None, no-op object, or clear error).
• Document why the dependency is optional.

______________________________________________________________________

4. Special Cases

Plugin Systems

Dynamic plugin imports can use guarded imports. On failure, log or report a clear plugin-level error and continue only if the plugin is optional.

Version Gates

Conditional imports based on Python version or platform are acceptable when they are runtime compatibility decisions, not missing dependency masking.

Development Tooling

Do not treat core development tools as optional in CI. If a required tool is missing, fail immediately and fix the environment.

______________________________________________________________________

5. How to Check

1. Open pyproject.toml. 2. If package is in [project.dependencies], use normal top-level import. 3. If package is in [project.optional-dependencies], guarded import is allowed with explicit fallback behavior. 4. Search for likely violations:

rg "try:\\n\\s+import .*\\nexcept ImportError" src tests

Then classify each match as required vs optional and refactor accordingly.

Ruff Linter Rules Reference

Table of Contents

• Overview
• E402: Module Level Import Not at Top of File
• B007: Loop Control Variable Not Used
• B008: Function Call in Default Argument
• S108: Hardcoded Temp File
• PLC0415: Import Outside Top-Level
• NPY002: Legacy NumPy Random Generation
• S311: Suspicious Non-Cryptographic Random Usage
• Typer CLI Exception for B008
• General Fix Strategy

______________________________________________________________________

Overview

Ruff is a fast Python linter written in Rust that identifies issues following PEP 8 and other best practices. It provides context-aware fixes that distinguish between production code, test code, and CLI frameworks.

When fixing linter errors:

1. Identify the rule code from the linter output 2. Select the appropriate fix pattern based on your code's context 3. Apply the fix and verify it resolves the issue without introducing new problems

______________________________________________________________________

E402: Module Level Import Not at Top of File

Rule Code: E402

Description: An import statement appears after code that is not an import, comment, or docstring.

Problem Pattern: Imports placed after executable code in a module.

Fix Pattern: Move all imports to the top of the module, grouped by standard library, third-party, then local imports.

Example:

# BEFORE (linter error)
def get_platform():
    import platform  # Error: import after code
    return platform.system()

# AFTER (fixed)
import platform

def get_platform():
    return platform.system()

______________________________________________________________________

B007: Loop Control Variable Not Used

Rule Code: B007

Description: A variable defined in a for loop is never used within the loop body.

Problem Pattern: Loop variables that are never referenced after assignment.

Fix Pattern: Prefix unused loop variables with an underscore (_) to indicate intentional non-use. See naming-conventions.md for complete guidelines.

Example:

# BEFORE (linter error)
for item in items:
    process_items()  # 'item' never used

# AFTER (fixed)
for _ in items:
    process_items()

______________________________________________________________________

B008: Function Call in Default Argument

Rule Code: B008

Description: A function call is used as a default argument value, evaluated only once at function definition time.

Problem Pattern: Mutable or dynamic return values captured at module load time instead of call time.

Fix Pattern: Use None as default and initialize inside the function body.

Example:

# BEFORE (linter error)
def create_list():
    return [1, 2, 3]

def process_data(items: list[int] = create_list()):
    items.append(4)
    return items

# AFTER (fixed)
def process_data(items: list[int] | None = None):
    if items is None:
        items = create_list()
    items.append(4)
    return items

______________________________________________________________________

S108: Hardcoded Temp File

Rule Code: S108

Description: Hardcoded paths like /tmp/, /var/tmp/, or C:\Temp\ for temporary files.

Problem Pattern: Using predictable temporary file paths instead of secure, platform-appropriate alternatives.

Fix Pattern (Production): Use tempfile.NamedTemporaryFile with delete=False.

Fix Pattern (Test): Use pytest's tmp_path fixture.

Example - Production:

# BEFORE (linter error)
with open("/tmp/data.txt", "w") as f:
    f.write(data)

# AFTER (fixed)
import tempfile

with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as f:
    f.write(data)
    temp_path = f.name

Example - Test:

# BEFORE (linter error)
def test_file_processing():
    test_file = "/tmp/test_data.txt"
    with open(test_file, "w") as f:
        f.write("test")

# AFTER (fixed)
def test_file_processing(tmp_path: Path):
    test_file = tmp_path / "test_data.txt"
    test_file.write_text("test")

______________________________________________________________________

PLC0415: Import Outside Top-Level

Rule Code: PLC0415

Description: Import statements placed inside functions, methods, or class definitions instead of at module level.

Problem Pattern: Imports buried within function bodies, making dependencies unclear.

Fix Pattern: Move imports to module level with proper grouping.

Example:

# BEFORE (linter error)
def print_version():
    import platform
    print(platform.python_version())

# AFTER (fixed)
import platform

def print_version():
    print(platform.python_version())

Standard Import Organization:

# 1. Standard library imports
import os
import sys
from pathlib import Path

# 2. Third-party imports
import typer
from rich.console import Console

# 3. Local application imports
from .utils import helpers
from .core import processors

______________________________________________________________________

NPY002: Legacy NumPy Random Generation

Rule Code: NPY002

Description: Usage of legacy numpy.random functions (np.random.seed, np.random.normal) instead of the modern Generator API.

Problem Pattern: Using frozen RandomState methods with global state and poorer statistical properties.

Fix Pattern: Use numpy.random.default_rng() for the modern Generator API.

Example:

# BEFORE (linter error)
import numpy as np

np.random.seed(1337)
data = np.random.normal(size=100)

# AFTER (fixed)
import numpy as np

rng = np.random.default_rng(1337)
data = rng.normal(size=100)

______________________________________________________________________

S311: Suspicious Non-Cryptographic Random Usage

Rule Code: S311

Description: Standard random module used in potentially sensitive contexts where cryptographic security is needed.

Problem Pattern: Using predictable pseudo-random generators for security-sensitive values.

Fix Pattern (Security Context): Use secrets module for tokens, passwords, or authentication.\ Fix Pattern (Non-Security Context): Use NumPy's default_rng() for better performance.

Example - Security Context:

# BEFORE (security risk)
import random
token = random.randrange(1000000)

# AFTER (secure)
import secrets
token = secrets.randbelow(1000000)

Example - Non-Security Context:

# Use NumPy for simulation/data analysis
import numpy as np
rng = np.random.default_rng()
data = rng.random(100)

______________________________________________________________________

Typer CLI Exception for B008

When using typer.Option() or typer.Argument(), function calls in defaults are intentional and required. Apply this special annotation pattern instead of fixing B008:

from typing import Annotated
import typer

def main(
    config: Annotated[
        str,
        typer.Option(help="Configuration file path")
    ] = "config.yaml",
) -> None:
    """CLI command with proper Typer annotations."""
    ...

Key Pattern: Use Annotated with typer.Option/typer.Argument, keeping the default value on the parameter itself.

______________________________________________________________________

General Fix Strategy

When in doubt:

1. Production vs Test: Choose based on module location and purpose 2. Security vs Non-Security: Default to secrets for any token, password, or key generation 3. Typer B008: Preserve the pattern, do not apply standard B008 fix 4. NumPy Random: Always prefer default_rng() for new code

Reference: Ruff Full Rule Index

Project Setup Reference

Universal Python project structure and dependency management guidelines.

Table of Contents

1. Project Structure 2. Dependency Management 3. Import Organization 4. Virtual Environments 5. Python Version 6. Package Configuration

______________________________________________________________________

Project Structure

project/
├── src/                  # Main source code
│   └── package/          # Importable package
├── tests/                # Test suite
├── docs/                 # Documentation
├── scripts/              # Utility scripts
├── pyproject.toml        # Project configuration
├── README.md             # Project overview
└── .gitignore            # Version control ignore

Key principles:

• Place all importable code under src/
• Keep tests alongside source or in dedicated tests/ directory
• Use pyproject.toml for all project metadata

______________________________________________________________________

Dependency Management

Use `uv` for all package operations:

uv add package-name          # Add production dependency
uv add package-name --dev    # Add development dependency
uv remove package-name       # Remove dependency
uv sync --all-extras -U     # Update all dependencies

`pyproject.toml` configuration:

[project]
name = "package-name"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
    "requests>=2.28",
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "ruff>=0.1.0",
]

______________________________________________________________________

Import Organization

Organize imports in three blocks, separated by blank lines:

# 1. Standard library imports
import os
import sys
from pathlib import Path
from typing import Any

# 2. Third-party imports
import numpy as np
import pandas as pd
from pydantic import BaseModel

# 3. Local application imports
from .utils import helpers
from .core import processors

Rules:

• Never use relative imports beyond a single level (prefer from .module over from ..module)
• Avoid import *
• Sort alphabetically within each block

______________________________________________________________________

Virtual Environments

# Create environment with uv
uv venv .venv

# Activate (Linux/macOS)
source .venv/bin/activate

# Activate (Windows PowerShell)
.venv\Scripts\Activate.ps1

# Install project with dependencies
uv sync

______________________________________________________________________

Python Version

Minimum: Python 3.10

Use modern type hint syntax:

# Use | instead of Union or Optional
def process_data(
    input_data: list[dict[str, int | str]],
    config: dict[str, str] | None = None
) -> dict[str, list[float]]:
    ...

______________________________________________________________________

Package Configuration

Standard `pyproject.toml` sections:

[project]
name = "package-name"
version = "0.1.0"
description = "Package description"
requires-python = ">=3.10"
dependencies = []

[project.optional-dependencies]
dev = ["pytest", "ruff"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.ruff]
line-length = 100
target-version = "py310"