Python Logging Best Practices

Python Logging Best Practices

Self-Evolving Skill: This skill improves through use. If instructions are wrong, parameters drifted, or a workaround was needed — fix this file immediately, don't defer. Only update for real, reproducible issues.

When to Use This Skill

Use this skill when:

• Setting up Python logging for any service or script
• Configuring structured JSONL logging for analysis
• Implementing log rotation
• Choosing between lightweight (zero-dep) and full-featured logging
• Adding logging to containerized, systemd, or local applications

Overview

Unified reference for Python logging patterns optimized for machine readability (Claude Code analysis) and operational reliability. Starts with the lightest viable approach and scales up only when needed.

Decision Heuristic: Start Light, Scale Up

Is it < 5 services on a single machine, < 1 event/sec?
  YES → Lightweight Pattern (print + JSONL telemetry)
  NO  → Is it containerized / serverless?
    YES → stdout JSON (any library), no file rotation
    NO  → Is OTel tracing required?
      YES → structlog + OTel
      NO  → loguru (CLI tools) or stdlib RotatingFileHandler

---

Preferred: Lightweight Pattern (Zero Dependencies)

For: < 5 systemd services, single server, single operator. Battle-tested in production by [ccmax-monitor](https://github.com/terrylica/ccmax-monitor).

This pattern uses a two-channel architecture:

• Channel 1: print(flush=True) → systemd journald (operational logs, human-readable)
• Channel 2: Append-only JSONL file (structured telemetry, machine-readable)

This maps to the 12-Factor App's "treat logs as event streams" principle. journald handles ops (rotation, filtering, metadata), while the JSONL file serves domain telemetry for post-mortem analysis.

Architecture: Three-Concern Separation

Complete Lightweight Example

"""Append-only JSONL telemetry logger with size-based rotation.

Zero external dependencies. Works with systemd journald for ops logging
and a separate JSONL file for structured machine-readable telemetry.
"""

import json
from datetime import datetime, timezone
from pathlib import Path

TELEMETRY_PATH = Path(__file__).parent / "telemetry.jsonl"
MAX_SIZE = 10 * 1024 * 1024  # 10 MB
BACKUP_COUNT = 3             # Keep 3 rotated backups (~30MB total)


def log_event(event_type: str, data: dict) -> None:
    """Append a structured JSON line to telemetry.jsonl."""
    entry = {
        "ts": datetime.now(timezone.utc).isoformat(),
        "type": event_type,
        **data,
    }
    line = json.dumps(entry, separators=(",", ":")) + "\n"

    try:
        try:
            if TELEMETRY_PATH.stat().st_size > MAX_SIZE:
                _rotate()
        except FileNotFoundError:
            pass

        with open(TELEMETRY_PATH, "a") as f:
            f.write(line)
    except OSError as e:
        # Fallback to stderr (captured by journald)
        print(f"[telemetry] write failed: {e}", file=__import__("sys").stderr, flush=True)


def _rotate() -> None:
    """Rotate telemetry files: .jsonl → .jsonl.1 → .jsonl.2 → .jsonl.3"""
    for i in range(BACKUP_COUNT, 1, -1):
        src = TELEMETRY_PATH.with_suffix(f".jsonl.{i - 1}")
        dst = TELEMETRY_PATH.with_suffix(f".jsonl.{i}")
        if src.exists():
            dst.unlink(missing_ok=True)
            src.rename(dst)
    backup = TELEMETRY_PATH.with_suffix(".jsonl.1")
    backup.unlink(missing_ok=True)
    TELEMETRY_PATH.rename(backup)


# === Ops logging (goes to journald via stdout) ===

def log(msg: str) -> None:
    """Human-readable operational log line. Captured by journald."""
    ts = datetime.now(timezone.utc).strftime("%H:%M:%S")
    print(f"[{ts}] {msg}", flush=True)

Usage:

# Operational (human reads via journalctl -u myservice -f)
log("Refreshing token for account X")
log("Switch: account A → account B (reason: 5h breach)")

# Telemetry (machine reads via jq/DuckDB/Claude Code)
log_event("token_refresh", {"account": "X", "expires_in_h": 8.0, "token_fp": "abc12345"})
log_event("account_switch", {"from": "A", "to": "B", "reason": "5h_breach"})

Security: Token Fingerprinting (Not Regex Redaction)

Never pass secrets through the logging pipeline. Log only a non-reversible fragment:

def _token_fingerprint(token: str) -> str:
    """Extract uniquely identifiable chars from a token's mid-section.

    The prefix (sk-ant-oat01-) and suffix (...AA) are common across tokens.
    Chars 14-22 (after the prefix) are the most unique per-token.
    Middle-slice avoids leaking type-prefix metadata that prefix-based
    approaches expose.
    """
    if len(token) > 25:
        return token[14:22]
    return token[:8] if token else ""

# Usage: log the fingerprint, never the token
log_event("token_refresh", {"account": name, "token_fp": _token_fingerprint(token)})

Why this is superior to regex redaction filters:

This aligns with OWASP Logging Cheat Sheet: "Ensure that no sensitive data is included in log entries." Major platforms (AWS, Stripe, GitHub) use separate non-secret identifiers or partial token display — never full tokens with regex scrubbing.

Regex filters remain useful as a defense-in-depth backstop, not a primary control.

Health Endpoints as Observability

For small deployments, rich JSON health endpoints replace log aggregation:

@app.get("/api/status")
def status():
    """White-box monitoring — current state on demand."""
    return {"active_account": ..., "accounts": [...], "polled_at": ...}

@app.get("/api/vault-health")
def vault_health():
    """Token health for all accounts."""
    return {name: {"status": "healthy", "expires_in": "7.5h", ...} for ...}

This is the Health Endpoint Monitoring Pattern (Microsoft Azure Architecture Center) / Health Check API Pattern (microservices.io). The dashboard IS the monitoring tool — no Grafana/Prometheus needed.

When the service itself serves its own operational state as structured JSON, you get:

• Real-time current state (not delayed by log ingestion pipelines)
• Zero infrastructure (no log shipper, storage, or query engine)
• AI-parseable (Claude Code can curl and analyze directly)

Post-Mortem with FOSS CLI Tools

No log aggregation stack needed. These single-binary tools work directly on JSONL:

# DuckDB — SQL analytics on JSONL (most powerful)
duckdb -c "SELECT type, count(*) FROM read_json_auto('telemetry.jsonl') GROUP BY 1 ORDER BY 2 DESC"

# jq — ad-hoc JSON filtering
jq 'select(.type == "token_refresh")' telemetry.jsonl

# journalctl — already exports JSONL natively
journalctl -u ccmax-switcher -o json --since "1h ago" | jq 'select(.PRIORITY == "3")'

# lnav — interactive terminal log viewer with SQL
lnav telemetry.jsonl

# llm (Simon Willison) — pipe to LLM for AI post-mortem
journalctl -u myservice --since "2h ago" --priority=err -o json | llm "analyze root cause"

When to Upgrade Beyond Lightweight

Upgrade to loguru/structlog when any of these become true:

• > 5 services across multiple hosts (need trace IDs for correlation)
• > 10 events/sec sustained (need async sinks, orjson)
• Multiple operators who need per-module log level filtering
• Compliance requirements that mandate structured audit trails with signatures
• Container/K8s deployment (stdout JSON is the standard)

---

Full-Featured: Loguru + JSONL Pattern

For CLI tools, scripts, and services that benefit from a logging library:

Log Rotation (ALWAYS CONFIGURE for local/CLI apps)

from loguru import logger

logger.add(
    log_path,
    rotation="10 MB",
    retention="7 days",
    compression="gz"
)

# stdlib alternative (zero-dep)
from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    log_path,
    maxBytes=100 * 1024 * 1024,  # 100MB
    backupCount=5
)

Container/serverless apps: Skip file rotation entirely. Log to stdout/stderr as JSON. Let the container runtime handle collection and rotation.

JSONL Format (Machine-Readable)

# One JSON object per line - jq-parseable
{"timestamp": "2026-01-14T12:45:23.456Z", "level": "info", "message": "..."}

File extension: Always use .jsonl (not .json or .log)

Performance: For >10k records/sec, use orjson instead of json.dumps():

import orjson

def json_formatter(record) -> str:
    log_entry = { ... }
    return orjson.dumps(log_entry).decode()

Regex Redaction (Defense-in-Depth)

Use as a backstop alongside token fingerprinting, not as the primary control:

import re

REDACT_PATTERNS = [
    (re.compile(r'AKIA[0-9A-Z]{16}'), '[REDACTED_AWS_KEY]'),
    (re.compile(r'sk-[a-zA-Z0-9]{48}'), '[REDACTED_API_KEY]'),
    (re.compile(r'(?i)bearer\s+[a-zA-Z0-9._~+/=-]+'), '[REDACTED_BEARER]'),
]

def redact_filter(record):
    for pattern, replacement in REDACT_PATTERNS:
        record["message"] = pattern.sub(replacement, record["message"])
    return True

logger.add(sink, filter=redact_filter)

Shutdown — Always Flush Enqueued Messages

import asyncio
from loguru import logger

async def main():
    logger.add("app.jsonl", enqueue=True)
    await logger.complete()

asyncio.run(main())
# Sync: logger.remove()

Complete Loguru + JSONL Example

#!/usr/bin/env python3
# /// script
# requires-python = ">=3.14"
# dependencies = ["loguru", "orjson"]
# ///

import re
import sys
from pathlib import Path
from uuid import uuid4

import orjson
from loguru import logger

REDACT_PATTERNS = [
    (re.compile(r'AKIA[0-9A-Z]{16}'), '[REDACTED_AWS_KEY]'),
    (re.compile(r'sk-[a-zA-Z0-9]{48}'), '[REDACTED_API_KEY]'),
]


def json_formatter(record) -> str:
    log_entry = {
        "timestamp": record["time"].strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z",
        "level": record["level"].name.lower(),
        "component": record["function"],
        "operation": record["extra"].get("operation", "unknown"),
        "operation_status": record["extra"].get("status", None),
        "trace_id": record["extra"].get("trace_id"),
        "message": record["message"],
        "context": {k: v for k, v in record["extra"].items()
                   if k not in ("operation", "status", "trace_id", "metrics")},
        "metrics": record["extra"].get("metrics", {}),
        "error": None
    }

    if record["exception"]:
        exc_type, exc_value, _ = record["exception"]
        log_entry["error"] = {
            "type": exc_type.__name__ if exc_type else "Unknown",
            "message": str(exc_value) if exc_value else "Unknown error",
        }

    return orjson.dumps(log_entry).decode()


def redact_filter(record):
    for pattern, replacement in REDACT_PATTERNS:
        record["message"] = pattern.sub(replacement, record["message"])
    return True


def setup_logger(app_name: str, log_dir: Path | None = None):
    logger.remove()
    logger.add(sys.stderr, format=json_formatter, filter=redact_filter, level="INFO")
    if log_dir is not None:
        log_dir.mkdir(parents=True, exist_ok=True)
        logger.add(
            str(log_dir / f"{app_name}.jsonl"),
            format=json_formatter,
            filter=redact_filter,
            rotation="10 MB",
            retention="7 days",
            compression="gz",
            level="DEBUG"
        )
    return logger

Semantic Fields Reference

Related Resources

• Health Endpoint Monitoring Pattern - Microsoft Azure Architecture Center
• OWASP Logging Cheat Sheet - Security best practices
• Write-Ahead Log pattern - Martin Fowler
• DuckDB JSON support - SQL analytics on JSONL
• lnav - Terminal log file navigator with SQL
• llm CLI - Pipe logs to LLMs for analysis
• structlog docs - Structured logging for production services
• Pydantic Logfire - AI/LLM observability built on OTel
• Langfuse - Open-source LLM observability (self-hostable)

Anti-Patterns to Avoid

1. Unbounded logs - Always configure rotation (local) or stdout (container) 2. Logging full secrets - Use token fingerprinting; regex redaction is a backstop, not primary 3. Adding loguru/structlog to < 5 low-volume services - print + JSONL is sufficient; dependency is not free 4. Bare except without logging - Catch specific exceptions, log them 5. Silent failures - Log errors before suppressing 6. `enqueue=True` without `logger.complete()` - Silent log loss on shutdown 7. `enqueue=True` with slow sinks - Unbounded memory growth 8. `json.dumps()` at >10k events/sec - Use orjson for 2-10x speedup 9. UUID4 trace IDs in OTel services - Use OTel-propagated trace IDs 10. Prometheus/Grafana for < 5 services - Health endpoints + Uptime Kuma is sufficient 11. Conflating WAL and telemetry - WAL is for crash recovery (ephemeral), telemetry is for audit (permanent)

---

Troubleshooting

Post-Execution Reflection

After this skill completes, check before closing:

1. Did the command succeed? — If not, fix the instruction or error table that caused the failure. 2. Did parameters or output change? — If the underlying tool's interface drifted, update Usage examples and Parameters table to match. 3. Was a workaround needed? — If you had to improvise (different flags, extra steps), update this SKILL.md so the next invocation doesn't need the same workaround.

Only update if the issue is real and reproducible — not speculative.

Evolution Log

Convention: Reverse chronological order (newest on top, oldest at bottom). Prepend new entries.

---

2026-04-05: SOTA Audit — Major Update

Status: Comprehensive audit against 2025-2026 best practices. All files updated.

Changes

SKILL.md:

• Added security section with loguru redaction filter pattern (regex-based secret scrubbing)
• Added enqueue=True OOM warning linking loguru#1419
• Added logger.complete() shutdown requirement for enqueued messages
• Replaced json.dumps() with orjson.dumps().decode() in JSONL formatter (2-10x speedup)
• Added OTel trace_id note — production services should use OTel-propagated IDs, not UUID4
• Expanded decision table: loguru, structlog, stdlib, Logfire, with decision heuristic
• Added container vs local distinction (stdout JSON, no rotation for containers)
• Added anti-patterns #6-9 (enqueue without complete, enqueue with slow sinks, json.dumps at volume, UUID4 in OTel services)
• Removed all platformdirs references — log_dir is now a caller-provided Path | None

loguru-patterns.md:

• Added async enqueue OOM warning with mitigations
• Added logger.complete() / logger.remove() shutdown patterns (async + sync)
• Added security redaction filter section with regex patterns
• Switched JSONL formatter to orjson
• Added best practices #6-9

logging-architecture.md:

• Added structlog as recommended for production services with OTel
• Added Pydantic Logfire for AI/LLM observability
• Added Kern for enterprise compliance
• Added container vs local comparison table
• Added Python 3.14 LoggerAdapter.merge_extra=True note
• Added structlog ContextVar reset pattern for async memory leak prevention
• Removed platformdirs references

migration-guide.md:

• Replaced platformdirs with simple Path-based log directory
• Switched to orjson in JSONL formatter
• Updated PEP 723 dependencies to ["loguru", "orjson"]

Deleted:

• references/platformdirs-xdg.md — platformdirs removed from skill

Sources

• loguru#1419 — enqueue unbounded memory
• structlog 25.x releases — ContextVars, exception groups
• Pydantic Logfire — OTel-based AI observability
• orjson benchmarks — 2-10x JSON serialization speedup
• OTel Python logging — auto-instrumentation
• pii-redactor — PII detection library

---

2026-02-26: Initial Evolution Log

Status: Skill is in use and maintained. Track improvements here.

Purpose

This evolution log tracks updates to the skill. Each entry should note:

• What changed (content, structure, tooling)
• Why it changed (bug fix, feature request, best practice)
• Files affected

How to Use

1. When updating SKILL.md or references, add an entry here with the date 2. Keep entries reverse-chronological (newest first) 3. Link to ADRs or GitHub issues when relevant 4. Reference specific line changes when helpful

---

Python Logging Architecture Guide

When to Use Which Approach

Decision Tree

Need logging?
├── Container or serverless?
│   └── YES → stdout/stderr as JSON (any library)
│       └── NO file rotation — let infrastructure handle it
├── Production service with tracing?
│   └── YES → structlog + OpenTelemetry
│       └── OTel auto-injects trace_id/span_id
├── AI/LLM app with Pydantic?
│   └── YES → Pydantic Logfire (built on OTel)
│       └── Token tracking, cost monitoring, SQL on logs
├── Stdlib-only required?
│   └── YES → RotatingFileHandler
│       └── Python 3.14: LoggerAdapter.merge_extra=True
├── Rich terminal output needed?
│   └── YES → Rich + RichHandler
│       └── Combine with structured file logging
└── CLI tool or script?
    └── YES → loguru + orjson
        └── This skill's recommended pattern

Container vs Local Logging

Approach Details

1. Loguru (Recommended for CLI/Scripts)

Best for: Modern Python scripts, CLI tools, local automation

from loguru import logger
from pathlib import Path

log_dir = Path.home() / ".local" / "log" / "my-app"
log_dir.mkdir(parents=True, exist_ok=True)

logger.add(
    str(log_dir / "app.jsonl"),
    rotation="10 MB",
    retention="7 days",
    compression="gz"
)

Advantages:

• Zero configuration to start
• Built-in rotation, retention, compression
• Structured logging with extra kwargs
• Exception formatting included

Caveats:

• enqueue=True has unbounded queue — OOM risk with slow sinks (loguru#1419)
• Always call logger.complete() or logger.remove() before shutdown

2. structlog (Recommended for Production Services)

Best for: Services with OpenTelemetry, async apps, production backends

import structlog

structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,        # Async-safe context
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),            # JSONL output
    ],
    wrapper_class=structlog.make_filtering_bound_logger(20),  # INFO+
)

log = structlog.get_logger()
log.info("request_handled", method="GET", path="/api/health", duration_ms=12)

Advantages:

• ContextVars for async-safe per-request context (5x throughput vs stdlib in async apps)
• Processor chains for transforming log records
• Native OTel integration via processors
• Exception group support (Python 3.11+)

ContextVar reset pattern (critical for async services):

# At request boundary — prevents memory leaks in long-running async services
structlog.contextvars.clear_contextvars()
structlog.contextvars.bind_contextvars(request_id="abc-123")

3. RotatingFileHandler (Stdlib)

Best for: LaunchAgent services, stdlib-only requirements

from logging.handlers import RotatingFileHandler
import logging

handler = RotatingFileHandler(
    "/path/to/app.log",
    maxBytes=100 * 1024 * 1024,  # 100MB
    backupCount=5
)
logging.getLogger().addHandler(handler)

Python 3.14 addition: LoggerAdapter gained merge_extra=True — call-level extras merge with adapter extras instead of replacing them:

adapter = logging.LoggerAdapter(logger, {"app": "my-app"}, merge_extra=True)
adapter.info("event", extra={"request_id": "abc"})
# Both app="my-app" and request_id="abc" are present

Reference: Python RotatingFileHandler

4. Pydantic Logfire (AI/LLM Observability)

Best for: AI/LLM applications, Pydantic-heavy services

• Built on OpenTelemetry — auto-exports to any OTel backend
• Purpose-built LLM features: conversation panels, token tracking, cost monitoring
• SQL queries on your logs
• 10M free spans/month

Reference: Pydantic Logfire

5. Rich Integration (Terminal Display)

Best for: Applications with rich terminal UI

from rich.logging import RichHandler
import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(message)s",
    handlers=[RichHandler(rich_tracebacks=True)]
)

Combine with a structured file handler for machine-readable output alongside pretty terminal display.

Output Format Recommendations

Common Patterns

Dual Output (Console + File)

from loguru import logger
import sys

# Human-readable to console
logger.add(sys.stderr, level="INFO")

# Machine-readable to file
logger.add("app.jsonl", format=json_formatter, level="DEBUG")

Environment-Based Configuration

import os

log_level = os.getenv("LOG_LEVEL", "INFO")
logger.add(sys.stderr, level=log_level)

Related Resources

• loguru-patterns.md - Loguru configuration
• migration-guide.md - From print() to logging
• structlog docs - Structured logging for production
• Pydantic Logfire - AI/LLM observability
• OpenTelemetry Python Logging - OTel auto-instrumentation

Loguru Configuration Patterns

Basic Setup

from loguru import logger
import sys

# Remove default handler
logger.remove()

# Add custom handlers
logger.add(sys.stderr, level="INFO")
logger.add("app.log", rotation="10 MB")

JSONL Output Pattern

import orjson

def json_formatter(record) -> str:
    """JSONL formatter — orjson is 2-10x faster than stdlib json."""
    return orjson.dumps({
        "timestamp": record["time"].strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z",
        "level": record["level"].name.lower(),
        "message": record["message"],
        "extra": record["extra"]
    }).decode()

logger.add(sys.stderr, format=json_formatter)

Why orjson? Native datetime/UUID/dataclass serialization, RFC 8259 compliant, 2-10x faster than json.dumps(). Use orjson.dumps().decode() since orjson returns bytes.

Structured Logging

# Add context to log messages
logger.info(
    "User logged in",
    operation="login",
    status="success",
    user_id=123,
    metrics={"duration_ms": 50}
)

Rotation Options

# Size-based rotation
logger.add("app.log", rotation="10 MB")

# Time-based rotation
logger.add("app.log", rotation="1 day")
logger.add("app.log", rotation="1 week")
logger.add("app.log", rotation="00:00")  # Midnight

# Count-based rotation
logger.add("app.log", rotation="100 records")

Retention Options

# Time-based retention
logger.add("app.log", retention="7 days")
logger.add("app.log", retention="1 month")

# Count-based retention
logger.add("app.log", retention=5)  # Keep 5 old files

Compression

# gzip compression (recommended)
logger.add("app.log", compression="gz")

# Other formats
logger.add("app.log", compression="bz2")
logger.add("app.log", compression="xz")
logger.add("app.log", compression="zip")

Exception Handling

# Log exceptions with traceback
try:
    raise ValueError("Something went wrong")
except ValueError:
    logger.exception("Error occurred")

# Or use opt() for more control
logger.opt(exception=True).error("Error with traceback")

Async Support & enqueue

# For async/multiprocess applications — use enqueue
logger.add("app.log", enqueue=True)

<!-- SSoT-OK: loguru version referenced for issue context, not as a dependency pin -->

WARNING — Unbounded memory risk: enqueue=True uses an internal multiprocessing.SimpleQueue with no max size. If the sink is slow (disk I/O, network), the queue grows unbounded until OOM. See loguru#1419. No upstream fix merged yet.

>

Mitigations:

>

- Monitor RSS in production when using enqueue=True

- Avoid slow sinks (network loggers, remote databases) with enqueue

- For high-throughput async services, consider structlog with ContextVars instead

- For simple CLI scripts, enqueue=False (default) is fine

Shutdown — logger.complete()

When using enqueue=True, always flush before exit to prevent silent log loss:

import asyncio
from loguru import logger

async def main():
    logger.add("app.jsonl", enqueue=True)
    # ... application logic ...
    await logger.complete()  # Flush all enqueued messages before exit

asyncio.run(main())

Synchronous alternative — logger.remove() implicitly flushes and closes all sinks:

def main():
    logger.add("app.jsonl", enqueue=True)
    try:
        # ... application logic ...
        pass
    finally:
        logger.remove()  # Flushes enqueued messages + closes sinks

Security — Redaction Filters

Scrub secrets at the filter level so they never reach any sink:

import re

REDACT_PATTERNS = [
    (re.compile(r'AKIA[0-9A-Z]{16}'), '[REDACTED_AWS_KEY]'),
    (re.compile(r'sk-[a-zA-Z0-9]{48}'), '[REDACTED_API_KEY]'),
    (re.compile(r'(?i)bearer\s+[a-zA-Z0-9._~+/=-]+'), '[REDACTED_BEARER]'),
    (re.compile(r'[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}'), '[REDACTED_EMAIL]'),
]

def redact_filter(record):
    """Scrub secrets from messages before they reach any sink."""
    for pattern, replacement in REDACT_PATTERNS:
        record["message"] = pattern.sub(replacement, record["message"])
    return True

# Apply to ALL sinks
logger.add("app.jsonl", filter=redact_filter)
logger.add(sys.stderr, filter=redact_filter)

Best practice: Don't log PII at all. Store PII in a vault, log tokens/hashes. Redaction filters are a safety net, not primary defense.

Filtering

# Filter by level
logger.add("errors.log", level="ERROR")

# Filter by function
def my_filter(record):
    return "sensitive" not in record["message"]

logger.add("filtered.log", filter=my_filter)

Best Practices

1. Always `logger.remove()` first - Removes default handler 2. Use rotation - Prevent unbounded growth (local/CLI apps only) 3. Use retention - Clean up old logs 4. Use compression - Save disk space 5. Use structured extras - Add context via kwargs 6. Use `redact_filter` - Scrub secrets from all sinks 7. Call `logger.complete()` - Flush enqueued messages before shutdown 8. Monitor RSS with `enqueue=True` - Unbounded queue risk with slow sinks 9. Use orjson - 2-10x faster JSONL serialization

Migration Guide: print() to Structured Logging

Overview

This guide covers migrating from print() statements to structured JSONL logging using loguru.

Quick Migration Table

Step-by-Step Migration

Step 1: Add Dependencies

# PEP 723 inline script metadata
# /// script
# requires-python = ">=3.14"
# dependencies = ["loguru", "orjson"]
# ///

Or via pyproject.toml:

uv add loguru orjson

Step 2: Add Logger Setup

import sys
from pathlib import Path
from loguru import logger
import orjson


def json_formatter(record) -> str:
    """JSONL formatter for machine-readable output."""
    return orjson.dumps({
        "timestamp": record["time"].strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3] + "Z",
        "level": record["level"].name.lower(),
        "message": record["message"],
        "extra": record["extra"]
    }).decode()


def setup_logger(app_name: str, log_dir: Path | None = None):
    logger.remove()

    # Console output
    logger.add(sys.stderr, format=json_formatter, level="INFO")

    # File output with rotation (if log_dir provided)
    if log_dir is not None:
        log_dir.mkdir(parents=True, exist_ok=True)
        logger.add(
            str(log_dir / f"{app_name}.jsonl"),
            format=json_formatter,
            rotation="10 MB",
            retention="7 days",
            compression="gz"
        )

Step 3: Call Setup Early

# At module level or in main()
setup_logger("my-app", log_dir=Path.home() / ".local" / "log" / "my-app")

Step 4: Replace Print Statements

Pattern 1: Simple status messages

# Before
print("Starting application...")
print("Application ready")

# After
logger.info("Application starting", operation="startup", status="started")
logger.info("Application ready", operation="startup", status="success")

Pattern 2: Variable debugging

# Before
print(f"[DEBUG] config = {config}")
print(f"[DEBUG] count = {len(items)}")

# After
logger.debug("Configuration loaded", config=config)
logger.debug("Items counted", metrics={"count": len(items)})

Pattern 3: Error reporting

# Before
try:
    do_something()
except Exception as e:
    print(f"[ERROR] Failed: {e}")

# After
try:
    do_something()
except ValueError as e:
    logger.error("Operation failed", operation="do_something", status="failed", error=str(e))

Pattern 4: Progress updates

# Before
for i, item in enumerate(items):
    print(f"Processing {i+1}/{len(items)}")

# After
total = len(items)
for i, item in enumerate(items):
    if i % 100 == 0:  # Log every 100 items
        logger.info("Processing progress", metrics={"current": i+1, "total": total})

Step 5: Remove Redundant Prints

After adding logger calls, remove the original print statements.

Common Anti-Patterns to Fix

Anti-Pattern 1: Silent Exception Handling

# BAD - Silent failure
try:
    result = parse_config(path)
except Exception:
    result = default_config  # No one knows this happened

# GOOD - Loud failure
try:
    result = parse_config(path)
except FileNotFoundError as e:
    logger.warning("Config not found, using defaults", path=str(path), error=str(e))
    result = default_config
except ValueError as e:
    logger.error("Config parse failed", path=str(path), error=str(e))
    raise

Anti-Pattern 2: Bare Except

# BAD
except:
    pass

# GOOD
except SpecificException as e:
    logger.error("Specific error occurred", error=str(e))

Anti-Pattern 3: Print to stdout

# BAD - Mixed with program output
print("Processing...")  # Goes to stdout

# GOOD - Logs to stderr
logger.info("Processing...")  # Goes to stderr, stdout clean for data

Verification

After migration, validate JSONL output:

# Run script and pipe stderr to file
python script.py 2> output.jsonl

# Validate JSON
cat output.jsonl | jq -c .

# Search logs
cat output.jsonl | jq 'select(.level == "error")'

Rollback

If issues arise, keep both temporarily:

# Parallel logging during migration
print(f"[INFO] {message}")  # Keep for now
logger.info(message)         # New structured log

Remove print statements once confident in new logging.