Python Data State

Python Data and State

Overview

Every data bug traces back to an unclear answer to three questions: who owns this data, where is it validated, and how far does it travel? This skill encodes boundary-first thinking — make ownership explicit, validate at the edge, and minimize what crosses a boundary.

Treat these defaults as starting points. When project constraints demand deviation, call out tradeoffs and compensating controls (tests, observability, rollback).

When to Use

• Data ownership is ambiguous or split across modules.
• Validation logic is scattered or duplicated instead of concentrated at ingress.
• Mutable state is shared across module or layer boundaries.
• Multiple modules participate in a single transaction.
• Configuration is stringly-typed, read lazily, or inconsistent across environments.
• Lifecycle of a data object (creation, transformation, persistence) spans unclear boundaries.

When NOT to Use

• Pure algorithmic or computational logic with no shared state.
• Prototypes or throwaway scripts where boundary discipline adds no value.
• UI/presentation-layer styling decisions with no data modeling impact.

Quick Reference

• Make module ownership and invariants explicit.
• Validate at ingress and normalize before domain decisions.
• Share minimal data across boundaries — prefer narrow, typed interfaces.
• Avoid cross-module transactions by default; if unavoidable, isolate coordination logic.
• Keep configuration typed, environment-driven, and startup-validated.

Common Mistakes

• Validating deep inside the call stack — pushing checks into domain logic instead of catching bad data at the boundary where it enters.
• Exposing internal models across module boundaries — sharing ORM objects or rich domain models instead of narrow DTOs or typed dicts.
• Relying on implicit ownership — assuming "whoever wrote it last" owns the data, leading to conflicting mutations and no single source of truth.
• Lazy, untyped config reads — calling os.getenv() at point-of-use with string defaults, so missing or malformed config surfaces as a runtime surprise instead of a startup failure.
• Wrapping unrelated modules in a shared transaction — coupling independent data stores for perceived consistency, creating hidden rollback complexity and contention.

Scope Note

• Treat these recommendations as preferred defaults for common cases, not universal rules.
• If a default conflicts with project constraints or worsens the outcome, suggest a better-fit alternative and explain why it is better for this case.
• When deviating, call out tradeoffs and compensating controls (tests, observability, migration, rollback).

Invocation Notice

• Inform the user when this skill is being invoked by name: python-data-state.

References

• references/data-lifecycle.md
• references/consistency-boundaries.md
• references/configuration.md

Configuration and Settings

Outcome

Configuration is explicit, validated early, and safe to operate.

Standards

• Use environment-driven configuration for deploy-specific values.
• Prefer typed settings with pydantic-settings (or equivalent typed schemas).
• Validate critical configuration at startup, not lazily during request handling.

Safety Rules

• Fail fast on missing required configuration.
• Fail fast on invalid configuration values and units.
• Avoid import-time configuration side effects; build settings in startup/bootstrap paths.
• Keep secrets out of source code, test fixtures, and logs.

Organization and Evolution

• Separate application settings from feature flags and runtime state.
• Use explicit defaults only for non-sensitive optional values.
• Allow test overrides via explicit injection/fixtures, not mutable module globals.
• Keep configuration schema documented and versioned when needed.

Consistency Boundaries and Cross-Module State

Outcome

Strong local consistency with resilient cross-module workflows.

Immediate Consistency Rules

• Treat each module as the unit of immediate consistency.
• Keep transactions scoped to a single module boundary.
• Enforce module invariants before publishing side effects.

Cross-Module Workflow Rules

• Do not span transactions across module boundaries.
• Coordinate via events, queued commands, or background jobs.
• Design cross-module flows for eventual consistency by default.

Request-Path Interaction Rules

• During request handling, avoid synchronous domain call chains across services/modules.
• Allow orchestration-only interactions: publish events, enqueue jobs, or call infrastructure services.
• Keep business decisions in the handling module's core logic.

Data and State Lifecycle

Outcome

Clear ownership and predictable data flow from ingress to persistence/egress.

Lifecycle Flow

1. Ingress at shell boundary: receive untrusted external input. 2. Validation: enforce schema and required fields at ingress. 3. Normalization: convert input to stable internal structures/value objects. 4. Domain decisions: run pure logic on normalized internal data. 5. Persistence and egress: map domain data to storage or transport schemas explicitly.

Ownership Rules

• Each module owns its data invariants.
• Keep persistence schemas and ORM models private to the owning module.
• Cross-module interaction must use explicit public interfaces.

Sharing and Mutability Rules

• Share the minimum data required through DTO/value objects.
• Prefer immutable or read-only data shapes for cross-boundary exchange.
• Minimize shared mutable state.
• Isolate stateful adapters at shell boundaries.

State Transition Reliability

• Validate before mutating state whenever possible.
• Make state transitions explicit and idempotent where retries can occur.
• Persist enough transition context for replay and failure diagnosis.