
Progressive Loading
- 92 installs
- 325 repo stars
- Updated August 2, 2026
- athola/claude-night-market
Design hub-and-spoke and advanced progressive-loading for large agent skills—adaptive tiers, DAG modules, caches, and load-failure recovery.
About
Progressive Loading is advanced documentation for skill authors who already understand basic hub-and-spoke loading and selection strategies. It explains when to escalate beyond simple splits: oversized skills that cannot be decomposed, module DAGs instead of flat lists, long sessions that require eviction, and graceful degradation when a module file is missing or fails to parse. Pattern one walks through an AdaptiveSelector that reads telemetry hits and demotes low-use modules to lower tiers. The module assumes familiarity with loading-patterns.md, selection-strategies.md, and performance-budgeting.md in the same night-market library. Solo builders shipping multi-file Claude skills use it to keep context lean without dropping capabilities. This is meta procedural knowledge—not a user-facing product generator—aimed at intermediate-to-advanced authors optimizing agent sessions.
- Covers adaptive loading driven by per-session module hit-rate telemetry
- Resolves directed acyclic graph dependencies across skill modules
- Documents multi-tier caches with eviction for long sessions
- Defines recovery when a module load fails mid-session instead of aborting
- Explicit gate: use only when skills exceed ~800–1500 token targets or need DAG/cache semantics
Progressive Loading by the numbers
- 92 all-time installs (skills.sh)
- Ranked #270 of 782 Skill Development skills by installs in the Skillselion catalog
- Security screen: LOW risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/athola/claude-night-market --skill progressive-loadingAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 92 |
|---|---|
| repo stars | ★ 325 |
| Security audit | 3 / 3 scanners passed |
| Last updated | August 2, 2026 |
| Repository | athola/claude-night-market ↗ |
What it does
Design hub-and-spoke and advanced progressive-loading for large agent skills—adaptive tiers, DAG modules, caches, and load-failure recovery.
Files
Table of Contents
- Overview
- When to Use
- Quick Start
- Basic Hub Pattern
- Progressive Loading
- Context-Based Selection
- Hub-and-Spoke Architecture
- Hub Responsibilities
- Spoke Characteristics
- Selection Strategies
- Loading Patterns
- Common Use Cases
- Best Practices
- Module References
- Integration with Other Skills
- Exit Criteria
Progressive Loading Patterns
Overview
Progressive loading provides standardized patterns for building skills that load modules dynamically based on context, user intent, and available token budget. This prevents loading unnecessary content while ensuring required functionality is available when needed.
The core principle: Start minimal, expand intelligently, monitor continuously.
When To Use
Use progressive loading when building skills that:
- Cover multiple distinct workflows or domains
- Need to manage context window efficiently
- Have modules that are mutually exclusive based on context
- Require MECW compliance for long-running sessions
- Want to optimize for common paths while supporting edge cases
When NOT To Use
- Project doesn't use the leyline infrastructure patterns
- Simple scripts without service architecture needs
Quick Start
Basic Hub Pattern
## Progressive Loading
**Context A**: Load `modules/loading-patterns.md` for scenario A
**Context B**: Load `modules/selection-strategies.md` for scenario B
**Always Available**: Core utilities, exit criteria, integration pointsVerification: Run the command with --help flag to verify availability.
Context-Based Selection
from leyline import ModuleSelector, MECWMonitor
selector = ModuleSelector(skill_path="my-skill/")
modules = selector.select_modules(
context={"intent": "git-catchup", "artifacts": ["git", "python"]},
max_tokens=MECWMonitor().get_safe_budget()
)Verification: Run the command with --help flag to verify availability.
Hub-and-Spoke Architecture
Hub Responsibilities
1. Context Detection: Identify user intent, artifacts, workflow type 2. Module Selection: Choose which modules to load based on context 3. Budget Management: Verify MECW compliance before loading 4. Integration Coordination: Provide integration points with other skills 5. Exit Criteria: Define completion criteria across all paths
Spoke Characteristics
1. Single Responsibility: Each module serves one workflow or domain 2. Self-Contained: Modules don't depend on other modules 3. Context-Tagged: Clear indicators of when module applies 4. Token-Budgeted: Known token cost for selection decisions 5. Independently Testable: Can be evaluated in isolation
Selection Strategies
See modules/selection-strategies.md for detailed strategies:
- Intent-based: Load based on detected user goals
- Artifact-based: Load based on detected files/systems
- Budget-aware: Load within available token budget
- Progressive: Load core first, expand as needed
- Mutually-exclusive: Load one path from multiple options
Loading Patterns
See modules/loading-patterns.md for implementation patterns:
- Conditional includes: Dynamic module references
- Lazy loading: Load on first use
- Tiered disclosure: Core → common → edge cases
- Context switching: Change loaded modules mid-session
- Preemptive unloading: Remove unused modules under pressure
Common Use Cases
- Multi-Domain Skills:
imbue:catchuploads git/docs/logs modules by context - Context-Heavy Analysis: Load relevant modules only, defer deep-dives, unload completed
- Plugin Infrastructure: Mix-and-match infrastructure modules with version checks
Best Practices
1. Design Hub First: Define all possible contexts and module boundaries 2. Tag Modules Clearly: Use YAML frontmatter to indicate context triggers 3. Measure Token Cost: Know the cost of each module for selection 4. Monitor Loading: Track which modules are actually used 5. Validate Paths: Verify all context paths have required modules 6. Document Triggers: Make context detection logic transparent
Module References
Core (always available to the hub)
- Selection Strategies: See
modules/selection-strategies.mdfor choosing modules - Loading Patterns: See
modules/loading-patterns.mdfor implementation techniques - Performance Budgeting: See
modules/performance-budgeting.mdfor token budget model and optimization workflow - Advanced Patterns: See
modules/advanced-patterns.mdfor nested hubs, multi-tier disclosure, and cross-skill module sharing - Troubleshooting: See
modules/troubleshooting.mdwhen modules fail to load, context detection misfires, or token budgets are exceeded
Context-Specific Pattern Modules
These modules are loaded on demand by the hub based on detected artifacts and user intent. They are listed in frontmatter so the selector can match them, but the hub should only load the ones whose activation context fires.
Operating-system patterns (load on detected platform):
modules/linux-patterns.md: Linux-specific shell, paths, and process patternsmodules/macos-patterns.md: macOS-specific tooling and platform quirksmodules/windows-patterns.md: Windows shell, path, and PowerShell patterns
Language and runtime patterns (load on detected ecosystem):
modules/modern-python.md: Python 3.11+ idioms, typing, asyncmodules/legacy-python.md: Python 2 / pre-3.8 compatibility patternsmodules/python-packaging.md: pyproject.toml, uv, pip, hatch, poetrymodules/python-patterns.md: General Python authoring patternsmodules/python-testing.md: pytest, fixtures, parametrization, mockingmodules/cargo-patterns.md: Rust Cargo workspace and dependency patternsmodules/rust-review.md: Rust code-review patterns
Workflow patterns (load on detected task):
modules/api-patterns.md: API design and endpoint conventionsmodules/api-review.md: API surface review patternsmodules/git-patterns.md: Git workflow and history patternsmodules/git-catchup-patterns.md: Catching up on a branch or PR diffmodules/document-analysis-patterns.md: Reading and analyzing documentsmodules/log-analysis-patterns.md: Parsing and reasoning over logsmodules/performance.md: Performance investigation patterns
Reference material (load only when explicitly cited):
modules/large-reference.md: Large reference tables and lookups (load
last; tokens are non-trivial)
Integration with Other Skills
This skill provides foundational patterns referenced by:
abstract:modular-skills- Uses progressive loading for skill designconserve:context-optimization- Uses for MECW-compliant loadingimbue:catchup- Uses for context-based module selection- Plugin authors building multi-workflow skills
Reference in your skill's frontmatter:
dependencies: [leyline:progressive-loading, leyline:mecw-patterns]
progressive_loading: trueVerification: Run the command with --help flag to verify availability.
Exit Criteria
- Hub clearly defines all module loading contexts
- Each module is tagged with activation context
- Module selection respects MECW constraints
- Token costs measured for all modules
- Context detection logic documented
- Loading paths validated for completeness
Advanced Patterns
This module covers progressive-loading techniques that go beyond the basic hub-and-spoke setup: adaptive loading driven by runtime signals, dependency graph resolution across modules, multi-tier caches with eviction, and recovery patterns when a module load fails mid-session. Read this after loading-patterns.md and selection-strategies.md are familiar.
When to Reach for Advanced Patterns
The basic patterns in loading-patterns.md cover most skills. Use the advanced patterns here only when one of the following is true:
- The skill exceeds the standard 800-1500 token target listed in
performance-budgeting.md and cannot be split.
- Modules form a directed acyclic graph (DAG) of dependencies,
not a flat list.
- Sessions are long enough that loaded modules outlive their
usefulness and must be evicted.
- A module load can fail (missing file, parse error, dependency
conflict) and the skill must degrade rather than abort.
Pattern 1: Adaptive Loading
Adaptive loading shifts module selection based on telemetry from the current session. The hub records which modules were actually read and which were never accessed. On the next activation, low hit-rate modules drop to a lower tier.
from collections import Counter
from pathlib import Path
class AdaptiveSelector:
def __init__(self, telemetry_path: Path) -> None:
self.telemetry_path = telemetry_path
self.hits: Counter[str] = self._load_hits()
def _load_hits(self) -> Counter[str]:
if not self.telemetry_path.exists():
return Counter()
return Counter(self.telemetry_path.read_text().splitlines())
def tier_for(self, module: str, default: str = "common") -> str:
count = self.hits.get(module, 0)
if count >= 10:
return "core"
if count >= 3:
return default
return "edge"This pattern is appropriate for skills used hundreds of times by the same user. For one-shot skills the overhead is not justified.
Pattern 2: Module DAG Resolution
When modules declare dependencies on other modules (see the dependencies field in the frontmatter shown in loading-patterns.md), the loader must resolve them topologically to avoid loading a module before its prerequisites.
from graphlib import TopologicalSorter
def resolve_load_order(modules: dict[str, list[str]]) -> list[str]:
sorter: TopologicalSorter[str] = TopologicalSorter()
for module, deps in modules.items():
sorter.add(module, *deps)
return list(sorter.static_order())graphlib is in the Python standard library since 3.9 and raises CycleError if the graph is not a DAG. Catch that exception and report which modules form the cycle rather than letting the loader hang.
Pattern 3: Tiered Cache with Eviction
For skills with many small modules, a two-tier cache keeps recent modules hot and evicts cold ones under context pressure.
from collections import OrderedDict
class TieredCache:
def __init__(self, hot_size: int = 5, warm_size: int = 15) -> None:
self.hot: OrderedDict[str, str] = OrderedDict()
self.warm: OrderedDict[str, str] = OrderedDict()
self.hot_size = hot_size
self.warm_size = warm_size
def get(self, key: str) -> str | None:
if key in self.hot:
self.hot.move_to_end(key)
return self.hot[key]
if key in self.warm:
value = self.warm.pop(key)
self._promote(key, value)
return value
return None
def _promote(self, key: str, value: str) -> None:
self.hot[key] = value
if len(self.hot) > self.hot_size:
evicted_key, evicted_value = self.hot.popitem(last=False)
self.warm[evicted_key] = evicted_value
if len(self.warm) > self.warm_size:
self.warm.popitem(last=False)The hot tier holds modules read in the last few turns. The warm tier holds older modules that may still be relevant. Eviction happens only when the warm tier overflows.
Pattern 4: Graceful Load Failure
If a module file is missing or malformed, the loader should log the failure, fall back to a known-good substitute when one is declared, and continue. Aborting the entire skill on a single missing module penalizes the user for an authoring mistake.
import logging
from pathlib import Path
logger = logging.getLogger(__name__)
def safe_load(module_path: Path, fallback: Path | None = None) -> str:
try:
return module_path.read_text(encoding="utf-8")
except FileNotFoundError:
logger.warning("module missing: %s", module_path)
if fallback and fallback.exists():
return fallback.read_text(encoding="utf-8")
return f"# Module unavailable: {module_path.name}\n"
except UnicodeDecodeError as exc:
logger.error("module decode failed: %s (%s)", module_path, exc)
return f"# Module unreadable: {module_path.name}\n"Pitfalls
1. Premature adaptation: Telemetry-driven loading needs many sessions of data. Do not enable it for new skills with no usage history. 2. DAG cycles in author error: If two modules both list each other as dependencies, the loader will reject the graph. Surface the cycle path, do not silently break it. 3. Cache invalidation on edits: A cached module persists even after the source file changes on disk. Stamp cache entries with the file mtime and re-read when it differs. 4. Hidden eviction: If a module disappears from context after being loaded, the user may not notice until the skill produces stale output. Log evictions at INFO level. 5. Fallback as silent feature: A fallback that always succeeds masks missing modules. Emit a warning every time the fallback activates so authors can fix the root cause.
Cross-Reference
See the parent SKILL.md for the hub-and-spoke overview and loading-patterns.md for the basic loading mechanisms these advanced patterns extend.
API Patterns
This module covers how to apply progressive-loading when a skill analyzes, designs, or reviews a public API surface. The driving question is which slices of API knowledge to load on demand: versioning rules, error envelopes, pagination, authentication, or surface inventory. Loading all of them at once wastes tokens when only one slice applies to the current task.
When This Module Applies
Load this module when the active task involves any of:
- Reviewing a REST, GraphQL, gRPC, or library API surface.
- Designing a new endpoint or public function signature.
- Auditing API consistency across an existing codebase.
- Generating client code, OpenAPI specs, or SDK bindings.
If the task is general code review with no API focus, prefer api-review.md for the audit workflow itself. This module is about how to chunk API content for progressive loading, not how to perform the review.
Slice the API Surface First
Before loading deep API knowledge, classify the surface in one pass. The classification drives the next module load.
from dataclasses import dataclass
from pathlib import Path
@dataclass
class ApiSlice:
style: str # "rest", "graphql", "grpc", "library"
transport: str # "http", "tcp", "in-process"
auth: str # "none", "bearer", "oauth2", "mtls"
versioned: bool
def classify(spec_path: Path) -> ApiSlice:
text = spec_path.read_text(encoding="utf-8").lower()
style = "rest"
if "type query" in text or "schema {" in text:
style = "graphql"
elif "service " in text and "rpc " in text:
style = "grpc"
auth = "bearer" if "authorization: bearer" in text else "none"
return ApiSlice(
style=style,
transport="http" if style != "grpc" else "tcp",
auth=auth,
versioned="version" in text or "/v1/" in text,
)The output of classify selects which detail modules to load. A REST API with bearer auth needs the REST conventions module and the OAuth/bearer module, not the gRPC streaming module.
Loading Map
A typical API skill keeps these modules separate so each loads only when relevant. The frontmatter triggers declared in selection-strategies.md controls activation.
| Slice | Load Trigger | Token Estimate |
|---|---|---|
| REST conventions | style == "rest" | 600 |
| GraphQL schema rules | style == "graphql" | 700 |
| gRPC service patterns | style == "grpc" | 500 |
| Pagination strategies | paginated field present | 400 |
| Auth: bearer/OAuth2 | auth in ("bearer","oauth2") | 500 |
| Versioning policy | versioned == True | 300 |
| Error envelopes | always (small) | 200 |
The error envelope module is small and always loaded because every API has errors. Everything else is gated.
Concrete Example: Bearer-Auth REST Endpoint
When the classifier returns ApiSlice(style="rest", auth="bearer", versioned=True), the hub loads three modules and skips the rest.
# hub frontmatter (illustrative; module names are placeholders
# the consuming skill author would supply for their own domain)
modules:
- modules/<rest-conventions>.md
- modules/<auth-bearer>.md
- modules/<versioning-policy>.md
- modules/<error-envelopes>.mdFor a GraphQL API with no auth, the load list shrinks to two modules plus the always-loaded error envelopes. The token saved by skipping REST and auth content is redirected to the actual review work.
Pitfalls
1. Loading by file extension alone: A .proto file might be a vendored copy in a REST project. Read the file content for classification, not just the suffix. 2. Treating "API" as one module: Authors who put REST, GraphQL, gRPC, and SDK guidance in one file force every API review to load all of it. Split by style first. 3. Skipping the error module: Error contracts are the most common review finding. Keep error guidance always loaded so reviewers see it without a second pass. 4. Hard-coding versioning into core: Some APIs are intentionally unversioned (internal RPCs, single-tenant tools). Gate versioning content on the versioned flag. 5. Re-classifying on every turn: Cache the classification per spec file. A REST API does not become a gRPC API mid-session.
Cross-Reference
See api-review.md for the review workflow that consumes these loading slices, and the parent SKILL.md for the hub-and-spoke contract these slices fit into.
API Review
This module shows how a progressive-loading hub can drive an API review without front-loading every checklist, exemplar, and language convention at once. It is the loading playbook that sits behind the pensive:api-review skill: when reviewing a public API, decide which detail modules to load and in what order, instead of pulling in every possible rule.
Scope
Use this module when:
- The user asks for an API review, design critique, or
consistency audit.
- The hub skill needs to choose between language-specific or
style-specific review modules.
- The session has already loaded a generic review skeleton and
needs to specialize.
For pure design exploration with no existing surface to audit, load api-patterns.md instead. That module covers slicing API content; this one covers running the review.
Three-Phase Loading
API review has three phases, and each phase needs a different module set. Loading all three at once defeats progressive loading.
Phase 1: Surface Inventory
Always load. The reviewer needs to know which symbols are public before anything else. A small inventory module suffices.
# Find Python public symbols (no leading underscore)
rg --type py '^(class|def) [A-Za-z]' -l
# Find Rust public items
rg --type rust '^pub (fn|struct|enum|trait) ' -lFor larger codebases, generate the inventory once into a session file and refer to it by path rather than reloading the rg output every turn.
Phase 2: Exemplar Comparison
Load on demand. Pick exemplars matching the surface style. For a Python data API, pandas and polars are reasonable references. For an HTTP client, requests and httpx. The exemplar list is in a separate module so it can be swapped without rewriting the review skeleton.
Phase 3: Consistency Audit
Load when discrepancies between surface and exemplars surface. The audit module enumerates the rules to check: naming conventions, error types, return shapes, pagination, idempotency, auth headers. Most reviews do not exercise every rule, so the audit module itself can be subdivided.
Loading Decision Table
| Signal | Load |
|---|---|
| Python files in surface | python-api-rules.md |
| Rust files in surface | rust-api-rules.md |
| OpenAPI spec found | openapi-conformance.md |
| Mentions of pagination | pagination-rules.md |
| Mentions of auth | auth-rules.md |
| Versioning question | versioning-policy.md |
The signals come from user input, scanned files, and earlier review turns. Cache the signal set per session so the loader does not re-scan the working tree on every check.
Example Loading Trace
A user asks: "Review the public functions in plugins/abstract/scripts/skills_auditor.py."
turn 1: load surface-inventory.md (always)
run rg for public defs in skills_auditor.py
turn 2: load python-api-rules.md (Python file detected)
turn 3: detect inconsistent return types
load consistency-audit.md
turn 4: user asks "what about errors?"
load error-envelopes.mdModules from later turns stay loaded until the session ends or context pressure forces eviction (see advanced-patterns.md).
Pitfalls
1. Loading every language module up front: A Python-only review does not need Rust rules. Detect language before loading. 2. Skipping inventory: Reviews without an explicit surface list drift into general code review. Always anchor in the inventory phase. 3. Stale exemplars: Library APIs evolve. If your exemplar module references a function signature that no longer exists in the upstream library, the review produces wrong findings. Date-stamp exemplar modules. 4. Re-running rg every turn: Inventory output is stable across the session unless the working tree changes. Cache it. 5. One mega-rules module: A 2000-token rule list violates the token target in performance-budgeting.md. Split by language and concern.
Cross-Reference
See api-patterns.md for slicing API spec content and the parent SKILL.md for the hub-and-spoke pattern these review phases plug into.
Cargo Patterns
This module covers progressive-loading inside skills that work with Rust crates managed by Cargo. The loading question is how to decide which Cargo-related modules to pull in: dependency audit, build configuration, workspace handling, or release publishing. Each is a separate slice with its own token cost.
When This Module Applies
Load this module when the active task touches:
- A
Cargo.tomlorCargo.lockfile. - A workspace root with multiple member crates.
- A
cargosubcommand:build,test,audit,publish,
tree, update, or bench.
- Dependency review for supply-chain risk.
If the task is reviewing Rust source for ownership or unsafe code, load rust-review.md instead. This module focuses on the Cargo tooling layer, not the language semantics.
Detect the Cargo Layout First
Cargo projects come in two shapes: single-crate and workspace. The loader needs to know which before pulling in workspace-only guidance.
# Single-crate project root
test -f Cargo.toml && ! grep -q '^\[workspace\]' Cargo.toml
# Workspace root
grep -q '^\[workspace\]' Cargo.tomlA workspace root delegates dependency versions to member crates or pins them in [workspace.dependencies]. The audit rules differ enough that they are in separate modules.
Loading Map
| Slice | Load Trigger | Token Estimate |
|---|---|---|
| Dependency audit | cargo audit mention or Cargo.lock review | 500 |
| Build profile review | [profile.*] table edits | 300 |
| Workspace coordination | [workspace] detected | 400 |
| Feature flag analysis | [features] table edits | 400 |
| Publish checklist | cargo publish or release task | 300 |
| MSRV check | rust-version field present | 200 |
The smallest slice (MSRV check) is cheap to keep loaded; the publish checklist is rarely needed and should stay deferred until the user explicitly asks about a release.
Concrete Cargo Commands the Modules Reference
Each loaded module documents its own commands; the hub keeps a small index so users know where to look.
# Inspect resolved dependency graph
cargo tree --duplicates
# Find unused features
cargo tree -e features
# Audit known advisories (requires cargo-audit)
cargo audit --json
# Check minimum supported Rust version
cargo +1.75 check # adjust version to declared rust-versioncargo audit is a real subcommand provided by the cargo-audit crate from RustSec. It is not a built-in subcommand. The dependency-audit module installs it once per toolchain with cargo install cargo-audit.
Workspace-Specific Loading
For workspaces, the hub also needs to decide whether to load guidance per member crate or once for the workspace. The cheap default is once per workspace, with member-crate detail loaded on explicit drill-down.
from pathlib import Path
import tomllib
def is_workspace(cargo_toml: Path) -> bool:
data = tomllib.loads(cargo_toml.read_text(encoding="utf-8"))
return "workspace" in data
def member_crates(cargo_toml: Path) -> list[str]:
data = tomllib.loads(cargo_toml.read_text(encoding="utf-8"))
return data.get("workspace", {}).get("members", [])tomllib is in the standard library since Python 3.11. For older Python, the tomli backport offers the same API.
Pitfalls
1. Loading workspace guidance for single crates: Single-crate projects do not have member coordination concerns. Skip the workspace module unless [workspace] is present. 2. Confusing `Cargo.lock` with dependency declarations: The lock file lists resolved versions. The declarations in Cargo.toml define the constraints. Audit the lock file for advisories and the toml for version policy. 3. Skipping MSRV when present: If rust-version is set, builds against newer toolchains may pass while CI on the declared MSRV fails. Always load the MSRV check when the field exists. 4. Caching `cargo tree` output across edits: Tree output changes whenever dependencies change. Re-run after any Cargo.toml or Cargo.lock edit. 5. Mixing audit and rust-review concerns: Cargo audit is about advisories on dependency versions. Rust review is about source code. Keep them in separate modules.
Cross-Reference
See rust-review.md for source-level review concerns, and the parent SKILL.md for how Cargo module loading fits the hub-and-spoke pattern.
Document Analysis Patterns
This module covers how progressive-loading applies when the skill processes prose documents: meeting notes, sprint summaries, markdown specs, RFCs, or imported PDFs. The driving question is which analysis modules to load based on document type, length, and target output.
When This Module Applies
Load this module when the task involves:
- Summarizing a markdown file or set of files.
- Extracting decisions, action items, or risks from prose.
- Producing a digest from meeting notes or sprint docs.
- Comparing two versions of a document for substantive change.
For git-based change summaries, load git-catchup-patterns.md instead. For log files and time-series data, load log-analysis-patterns.md. This module is for human-authored prose.
Detect Document Type Before Loading
Document type drives module selection. A short markdown spec needs different handling than a 100-page PDF.
from pathlib import Path
def classify_doc(path: Path) -> dict[str, object]:
suffix = path.suffix.lower()
size = path.stat().st_size
is_markdown = suffix in {".md", ".markdown"}
is_imported = suffix in {".pdf", ".docx", ".pptx", ".html"}
return {
"format": suffix,
"bytes": size,
"needs_conversion": is_imported,
"long_form": size > 50_000,
"markdown_native": is_markdown,
}The classification picks the next module: a long PDF triggers the conversion module first, then the long-form summarization module. A short markdown spec skips both and goes straight to extraction.
Loading Map
| Document Type | Load Module | Token Estimate |
|---|---|---|
| Short markdown (<10k bytes) | extraction-rules.md | 300 |
| Long markdown (>10k bytes) | chunked-summary.md | 500 |
| Meeting notes | decision-extraction.md | 400 |
| RFC or spec | requirements-extraction.md | 500 |
| PDF or DOCX | document-conversion.md then format module | 400+ |
| Two-version diff | prose-diff.md | 400 |
Conversion is the only module that depends on another module afterward. The hub loads conversion, runs it, then re-classifies the resulting markdown to pick the format module.
Real Conversion Path
For non-markdown inputs, the leyline:document-conversion skill provides a tiered fallback: MCP markitdown first, then native tools (pandoc, pdftotext), then a degraded text-only path. The conversion module loaded here calls into that skill rather than reimplementing the fallback.
# Pseudo-code for conversion handoff
def convert_to_markdown(path: Path) -> Path:
output = path.with_suffix(".md")
# Skill handoff: leyline:document-conversion
# owns the actual tool selection.
return outputAfter conversion, re-run classify_doc on the output path so the loader picks the right format module for the converted content.
Chunking for Long Documents
Long documents exceed safe per-turn token budgets. The chunked-summary module splits the document, summarizes each chunk, then merges the per-chunk summaries.
def chunk_by_heading(text: str) -> list[str]:
chunks: list[str] = []
current: list[str] = []
for line in text.splitlines():
if line.startswith("# ") and current:
chunks.append("\n".join(current))
current = [line]
else:
current.append(line)
if current:
chunks.append("\n".join(current))
return chunksHeading-based chunking preserves logical units. Byte-based chunking can split a code block or a table mid-row, breaking downstream parsing.
Pitfalls
1. Loading extraction rules before conversion: For PDFs the bytes you see are not the prose. Convert first, classify the converted markdown, then load extraction rules. 2. Treating all markdown the same: A 100-line spec and a 3000-line RFC need different summarization strategies. Use the size threshold. 3. Re-summarizing on every turn: If the document has not changed, the summary is stable. Cache by file mtime. 4. Losing tables to byte-chunking: Tables and code blocks are atomic units. Chunk by heading boundaries, not by byte count. 5. Skipping decision extraction for meeting notes: Meeting notes without an explicit decision pass produce a recap, not a useful digest. Always load decision-extraction.md for that document type.
Cross-Reference
See git-catchup-patterns.md for change-based summarization, log-analysis-patterns.md for time-series inputs, and the parent SKILL.md for how these analysis modules plug into the hub.
Git Catchup Patterns
This module covers progressive-loading for git-based catchup workflows: summarizing recent commits, surfacing what changed since a baseline, and producing a handoff for the next session. The loading question is which git tools and analysis modules to pull in based on the size of the diff and the depth of summary the user wants.
When This Module Applies
Load this module when the task is:
- "What changed since I left?" after a session break.
- Preparing a handoff summary for another developer.
- Summarizing the work on a feature branch before review.
- Catching up on a repo you have not touched in a while.
For deep diff analysis with risk scoring, load the imbue:diff-analysis skill instead. This module is about catchup loading, not full diff review.
Three Loading Tiers by Diff Size
The cost of analyzing a git range scales with the number of commits and changed files. The loader picks a tier first.
| Tier | Diff Size | Modules Loaded |
|---|---|---|
| Quick | <10 commits, <20 files | git-summary-quick.md |
| Standard | 10-50 commits, <100 files | quick and commit-grouping.md |
| Deep | >50 commits or >100 files | standard and chunked-analysis.md |
The tier is computed once at the start of catchup. If the user asks follow-up questions about specific files, the loader can upgrade to the next tier without redoing earlier work.
Establish Baseline First
Every catchup needs a baseline: "since when?". The default is the merge-base with the upstream branch. The user can override with a date, tag, or commit SHA.
# Default baseline: merge-base with upstream
BASE=$(git merge-base @ @{u} 2>/dev/null) || BASE=HEAD~10
# Show the size of the range
git log --oneline "$BASE"..HEAD | wc -l
git diff --stat "$BASE"...HEAD | tail -1The three-dot ...HEAD notation shows changes on the current branch since divergence. The two-dot ..HEAD shows commits reachable from HEAD but not from the baseline. Catchup uses both: dots-2 for commit lists, dots-3 for cumulative diffs.
Quick Tier: Single Pass Summary
For small ranges, one pass over the log is enough.
git log --pretty=format:'%h %s' "$BASE"..HEAD
git diff --stat "$BASE"...HEADThe quick module formats this into a short markdown summary. No grouping, no per-file analysis. Token cost stays under 500.
Standard Tier: Grouped by Subsystem
For medium ranges, group commits by subsystem (top-level directory or component). Grouping makes the summary scannable.
# List changed files grouped by top directory
git diff --name-only "$BASE"...HEAD \
| awk -F/ '{print $1}' \
| sort \
| uniq -c \
| sort -rnThe grouping module reads this output and produces a heading per subsystem with the relevant commits underneath.
Deep Tier: Chunked Analysis
For large ranges, even the log output exceeds safe budgets. The chunked module splits the range by week or by 20-commit windows and summarizes each window separately.
# Split the range into weekly windows
git log --pretty=format:'%ad %h %s' --date=format:'%Y-W%V' \
"$BASE"..HEAD \
| awk '{print $1}' | sort -uEach window summary is small. The merge step combines them into a single digest. This keeps any one turn under the per-turn token budget set in performance-budgeting.md.
Pitfalls
1. No baseline: Without an explicit baseline, catchup produces unstable output as HEAD moves. Always pin the baseline at the start of the workflow. 2. Skipping the size check: Loading the deep-tier module for a 5-commit range wastes tokens. Always check size before tier selection. 3. Reading every diff: A 10000-line diff overwhelms context. Use --stat for size, then drill into specific files only when asked. 4. Forgetting submodules: git diff --stat does not show submodule content changes by default. Add --submodule=diff when submodules are involved. 5. Mixing local and pushed history: Catchup on unpushed commits is a different workflow than catchup on the upstream branch. Document which baseline the summary covers.
Cross-Reference
See git-patterns.md for general git tooling patterns, and the parent SKILL.md for how catchup modules plug into the hub-and-spoke pattern.
Git Patterns
This module covers progressive-loading for skills that work with git repositories: workspace inspection, branch operations, commit analysis, and history rewrites. The driving question is which git tooling modules to load based on the operation, not on the file extension.
When This Module Applies
Load this module when the task involves:
- Inspecting working-tree state before a commit or PR.
- Branch operations: create, rebase, switch, delete.
- Commit-message generation or conventional-commit checks.
- Reading history with
git log,git blame, orgit bisect.
For catchup-style summaries of recent work, load git-catchup-patterns.md. This module covers general git operations beyond summarization.
Three Operation Buckets
Git operations split into three buckets that map to separate modules. Mixing them in one mega-module wastes tokens.
| Bucket | Examples | Module |
|---|---|---|
| Inspection | status, diff, log, blame | git-inspection.md |
| Mutation | commit, rebase, merge, reset | git-mutation.md |
| Plumbing | cat-file, rev-parse, update-ref | git-plumbing.md |
Plumbing commands are rarely needed. Defer their module unless the task explicitly calls for low-level git inspection.
Always Load: Workspace Sanity Check
A small inspection module is worth keeping always-loaded so the skill can verify state before any mutation.
# Are we in a git repo?
git rev-parse --is-inside-work-tree
# Current branch and tracking status
git status -sb | head -1
# Upstream tracking
git rev-parse --abbrev-ref --symbolic-full-name @{u} 2>/dev/null \
|| echo "no upstream"
# Are there local changes?
git diff --quiet || echo "modified"
git diff --cached --quiet || echo "staged"The --quiet flag returns nonzero on differences without producing output. This is the cheapest way to gate later work on a clean tree.
Mutation Module: Load Only When Needed
The mutation module covers operations that change repository state: commits, rebases, branch deletions. Loading it carries the risk that the model uses dangerous commands without intent. Gate it on explicit user request.
# Safe commit pattern: stage explicit files, never -A
git add path/to/file.py path/to/test.py
git commit -m "fix: explicit fix description"
# Branch creation from current HEAD
git switch -c feat/short-name
# Rebase onto remote main, preserving local commits
git fetch origin main
git rebase origin/mainThe mutation module documents safe defaults. The hub should declare in its frontmatter that this module is mutation: true so callers know loading it permits destructive commands.
Inspection Patterns the Module Should Cover
The inspection module documents the read-only patterns most analysis skills need. Examples:
# What changed in the last commit?
git show --stat HEAD
# Who last touched this line?
git blame -L 42,42 path/to/file.py
# Find the commit that introduced a string
git log -S 'function_name' -- path/to/file.py
# Show ancestors common to two branches
git merge-base feature-branch maingit log -S (the "pickaxe") finds commits that added or removed an exact string. git log -G accepts a regex but is slower. Use the pickaxe by default.
Pitfalls
1. Loading mutation by default: Read-only analysis does not need destructive commands available. Keep mutation behind an explicit gate. 2. Skipping the sanity check: Running git rebase in a non-git directory or on the wrong branch is a recoverable but expensive mistake. Always verify the workspace first. 3. Using `-A` or `.` for staging: git add -A and git add . capture untracked files that may include secrets or build artifacts. Stage explicit paths. 4. Treating force-push as safe: git push --force rewrites shared history. The mutation module should document --force-with-lease as the default, not --force. 5. Reading raw plumbing output as content: git cat-file returns blob content with no path context. Load the plumbing module only when the task is debugging git internals, not reading source.
Cross-Reference
See git-catchup-patterns.md for summarization workflows and the parent SKILL.md for how git modules plug into the hub-and-spoke pattern.
Large Reference
This module covers progressive-loading patterns for handling large reference content: API references, language standards, RFC bodies, or vendor documentation that exceeds the per-turn token budget by an order of magnitude. The technique is to load a small index always and pull in sections on demand.
When This Module Applies
Load this module when the skill needs to reference content that:
- Exceeds 2000 tokens in its complete form.
- Splits cleanly into self-contained sections.
- Is read in small slices per task, not whole.
- Updates rarely enough that a stable index is reasonable.
For dynamic content that changes frequently, this pattern is the wrong fit. Use a real document store and query it on demand.
The Index-and-Sections Pattern
Split the reference into two artifacts: a small index file with section names, anchors, and one-line descriptions, and a set of section files loaded only when their anchor is referenced.
modules/reference/
index.md # 200 tokens: list of sections and descriptions
errors.md # 400 tokens: error code reference
pagination.md # 350 tokens: pagination semantics
rate-limits.md # 300 tokens: throttling rules
webhooks.md # 600 tokens: webhook payloadsThe hub always loads index.md. When a turn needs error code detail, the loader resolves the anchor and pulls in errors.md. Other sections stay on disk.
Index File Format
The index lists each section with a stable anchor and a one-line purpose. The model uses the index to decide which section to load.
# Reference Index
## Sections
- `errors.md`: HTTP status codes, error envelope shape, and
retry guidance.
- `pagination.md`: Cursor and page-number variants, limits, and
ordering guarantees.
- `rate-limits.md`: Per-user and per-IP throttling, retry-after
header, burst rules.
- `webhooks.md`: Event types, signature verification, and
delivery retry policy.The index never describes implementation detail. It is a routing table, not a tutorial.
Section Loader
A small helper resolves anchors to file paths and reads the section content. Sections are cached by path so repeated references in one session do not re-read the disk.
from functools import lru_cache
from pathlib import Path
REFERENCE_ROOT = Path("modules/reference")
@lru_cache(maxsize=8)
def load_section(name: str) -> str:
path = REFERENCE_ROOT / f"{name}.md"
if not path.exists():
return f"# Missing section: {name}\n"
return path.read_text(encoding="utf-8")lru_cache(maxsize=8) keeps the eight most recently loaded sections in memory. For longer sessions, raise the size or use the tiered cache from advanced-patterns.md.
When Sections Need Subsections
If a single section grows past 1000 tokens, split it again rather than loading the whole thing. The split heuristic is the same as the index pattern: anchors with one-line descriptions.
# errors.md
## Sections
- `errors-4xx.md`: Client-side error codes (400-499).
- `errors-5xx.md`: Server-side error codes (500-599).
- `errors-envelope.md`: Common envelope shape across all codes.The recursion stops when each leaf section fits comfortably in one turn. A 200-400 token leaf is a good target.
Pitfalls
1. Loading the whole reference at start: Defeats the entire pattern. The index must be the only always-loaded artifact. 2. Index drift: If the index lists sections that no longer exist, the loader returns a missing-section stub and the user gets a confusing answer. Validate the index against disk during skill build. 3. Anchors that change between versions: Section names are public API. Renaming errors.md to error-codes.md breaks every cached anchor. Add an alias mechanism if you must rename. 4. Caching across edits: The lru_cache above does not notice file edits. For dev iteration, clear the cache when the source mtime changes. 5. Treating the index as content: The index is for routing. Putting tutorial prose in it bloats the always-loaded footprint and reintroduces the original problem.
Cross-Reference
See loading-patterns.md for the incremental-loading pattern this module implements at scale, and the parent SKILL.md for the hub-and-spoke contract.
Legacy Python
This module covers progressive-loading for skills that work with Python 3.8 through 3.10 codebases. The selection question is which language-feature modules to load when the runtime predates features like match statements, the tomllib standard library module, exception groups, and PEP 604 union syntax.
When This Module Applies
Load this module when:
- The target codebase pins
python_requires = ">=3.8,<3.11"
in setup.cfg or requires-python = ">=3.8,<3.11" in pyproject.toml.
- CI matrices target only 3.8, 3.9, or 3.10.
- The user mentions a specific legacy version constraint.
For Python 3.11+ work, load modern-python.md instead. The two modules are mutually exclusive (see the mutually-exclusive selection pattern in selection-strategies.md).
Detect the Version Floor
The version floor decides which features are available. Read it from the project metadata, not from the local interpreter.
import tomllib
from pathlib import Path
def python_floor(project_root: Path) -> tuple[int, int] | None:
pyproject = project_root / "pyproject.toml"
if not pyproject.exists():
return None
data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
requires = data.get("project", {}).get("requires-python", "")
if requires.startswith(">="):
major, minor = requires[2:].split(".")[:2]
return (int(major), int(minor.split(",")[0]))
return NoneFor projects on Python 3.10 or earlier, replace tomllib with the tomli backport. The legacy module should document this substitution explicitly.
What Legacy Python Lacks
The module loaded here documents the gaps so generated code stays valid. The most important gaps:
| Feature | Available From | Legacy Substitute |
|---|---|---|
match statement | 3.10 | if/elif chains |
| `int \ | str` unions | 3.10 |
tomllib stdlib | 3.11 | tomli backport |
ExceptionGroup | 3.11 | manual exception aggregation |
Self type | 3.11 | TypeVar bound to the class |
tomli_w integration | n/a | use tomli-w package |
For 3.8 specifically, dict[str, int] PEP 585 syntax requires from __future__ import annotations. Without that import, code fails at class definition time on 3.8.
Concrete Example: Type Hints That Work on 3.8
The legacy module shows the patterns that work across all supported versions.
from __future__ import annotations
from typing import Optional, Union
def first_match(values: list[str], pattern: str) -> Optional[str]:
return next((v for v in values if pattern in v), None)
def parse_int_or_str(value: Union[int, str]) -> int:
if isinstance(value, str):
return int(value)
return valueThe from __future__ import annotations at module top defers annotation evaluation to runtime, letting 3.8 parse PEP 585 generic syntax without crashing.
Concrete Example: tomli Backport
tomllib was added in 3.11. Legacy code uses tomli.
import sys
if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib # pip install tomli
with open("pyproject.toml", "rb") as fp:
data = tomllib.load(fp)Note that both modules require binary mode. tomli is a real PyPI package maintained by Hugo van Kemenade.
Pitfalls
1. Generating `match` for 3.8 targets: match statements are a syntax error before 3.10. Test generated code against the declared floor. 2. Forgetting `from __future__ import annotations`: Without it, PEP 585 generics raise TypeError at class definition time on 3.9 and earlier. 3. Assuming `tomllib` is always available: Stdlib tomllib exists only on 3.11+. Use the conditional import above. 4. Type hints with `Self`: typing.Self is 3.11+. For legacy targets, use a TypeVar bound to the class. 5. Loading both legacy and modern modules: They contradict each other on union syntax and stdlib modules. The mutually-exclusive selection pattern enforces a single choice.
Cross-Reference
See modern-python.md for the 3.11+ counterpart, and the parent SKILL.md for the mutually-exclusive selection pattern that picks between them.
Linux Patterns
This module covers progressive-loading for skills that target Linux as the host or production OS. The selection question is which OS-specific modules to load: file paths, process control, package managers, systemd services, or container runtimes. Linux distros differ enough that some sub-modules are themselves mutually exclusive.
When This Module Applies
Load this module when:
- The target machine runs Linux (Ubuntu, Debian, Fedora, RHEL,
Alpine, Arch, or similar).
- The task touches paths under
/etc,/var,/proc,/sys,
or ~/.config.
- The user mentions
systemd,apt,dnf,pacman, or a
Linux package manager.
- The deployment target is a Linux container.
For macOS-specific paths and tools, load macos-patterns.md. For Windows, load windows-patterns.md. The three are mutually exclusive per session unless the task explicitly compares platforms.
Detect the Distro Family Before Sub-Loading
Linux distros split into families with different package managers and init systems. The loader picks one sub-module per family.
# Distro detection (POSIX-portable)
. /etc/os-release && echo "$ID $ID_LIKE"
# Examples of expected output:
# ubuntu debian (Debian family, apt)
# fedora (Red Hat family, dnf)
# arch (Arch family, pacman)
# alpine (Alpine family, apk)/etc/os-release is standardized by systemd and present on every modern distro. The ID and ID_LIKE fields drive sub-module selection.
Loading Map
| Distro Family | Package Manager Module | Init Module |
|---|---|---|
| Debian/Ubuntu | apt-patterns.md | systemd-patterns.md |
| RHEL/Fedora | dnf-patterns.md | systemd-patterns.md |
| Arch | pacman-patterns.md | systemd-patterns.md |
| Alpine | apk-patterns.md | openrc-patterns.md |
systemd-patterns.md is shared across most families. Alpine and some minimal containers use OpenRC or no init system at all, so the init module is gated on detection rather than assumed.
Path Conventions
The shared Linux module documents the standard paths every sub-module references. The XDG Base Directory spec is the modern source.
# Configuration: ~/.config/<app>/
XDG_CONFIG_HOME="${XDG_CONFIG_HOME:-$HOME/.config}"
# Data: ~/.local/share/<app>/
XDG_DATA_HOME="${XDG_DATA_HOME:-$HOME/.local/share}"
# Cache: ~/.cache/<app>/
XDG_CACHE_HOME="${XDG_CACHE_HOME:-$HOME/.cache}"
# Runtime (per-user, cleared on logout)
XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"System-wide configs live under /etc, system data under /var/lib, and system cache under /var/cache. The path module documents both user and system layouts.
Process Inspection
Linux exposes process state through /proc. The process module documents the most useful entries.
# All processes for a user
ps -u "$USER" -o pid,pcpu,pmem,comm
# Open files for a process
ls -l /proc/PID/fd/
# Resource limits
cat /proc/PID/limits
# Memory map
cat /proc/PID/maps | headFor container introspection, /proc/1/cgroup reveals whether the process runs inside a cgroup-namespaced container. This is the canonical detection signal for "am I in Docker?".
Pitfalls
1. Assuming `apt` everywhere: A skill that runs apt install on Fedora fails. Always detect the distro family first. 2. Skipping XDG: Hard-coding paths under $HOME directly (e.g., ~/.myapp) ignores the XDG spec and conflicts with user customization. 3. Loading the systemd module on Alpine: Alpine ships with OpenRC by default. Detect the init system before loading. 4. Treating `/proc` as portable: macOS does not have /proc. The Linux module is the only place to reference it. 5. Mixing root and user contexts: XDG_RUNTIME_DIR has a sensible value only for interactive user sessions. System services need different runtime paths.
Cross-Reference
See macos-patterns.md and windows-patterns.md for the other platforms in the mutually-exclusive group, and the parent SKILL.md for the platform-selection contract.
Loading Patterns
Overview
Loading patterns define how modules are actually loaded, cached, unloaded, and switched during skill execution. These patterns implement the selection strategies with concrete mechanisms for managing module lifecycle.
Core Loading Patterns
1. Conditional Includes
Use hub SKILL.md to conditionally reference modules based on context.
Pattern:
# Skill Hub
## Progressive Loading
Load modules based on context:
**Git Workflow**: Load `modules/git-patterns.md` for git-based tasks
**Python Analysis**: Load `modules/python-patterns.md` for Python code
**API Review**: Load `modules/api-patterns.md` for API design review
**Always Available**: Core concepts, integration points, exit criteriaImplementation:
- Hub provides navigation map to modules
- Agent/user loads specific module when needed
- No all modules loaded upfront
When to Use:
- Simple, declarative loading logic
- Modules are mutually exclusive
- User/agent can select appropriate path
2. Lazy Loading
Load modules on first use, not at skill activation.
Pattern:
class SkillContext:
def __init__(self):
self._loaded_modules = {}
self._available_modules = self._scan_modules()
def get_module(self, module_name):
# Load on first access
if module_name not in self._loaded_modules:
module_path = self._available_modules[module_name]
self._loaded_modules[module_name] = self._load_module(module_path)
return self._loaded_modules[module_name]
# Usage
context = SkillContext()
# Module not loaded yet
git_patterns = context.get_module("git-patterns") # Loaded hereWhen to Use:
- Uncertain which modules will be needed
- Want fast skill activation
- Many optional modules available
Benefits:
- Faster initial load time
- Lower memory/context footprint
- Only pay for what you use
3. Tiered Disclosure
Load in predefined tiers: core → common → advanced.
Pattern:
# Skill Hub
## Overview
[Core concepts - always loaded]
## Quick Start
[Common patterns - ~80% of usage]
For advanced use cases, see:
- `modules/advanced-patterns.md` - Edge cases and optimization
- `modules/troubleshooting.md` - Debugging and fixes
- `modules/performance.md` - Benchmarking and tuningImplementation:
def load_tiered(context):
# Tier 1: Core (always)
load_content("SKILL.md#overview")
load_content("SKILL.md#quick-start")
# Tier 2: Common (if context suggests)
if context.complexity == "standard":
load_module("modules/common-patterns.md")
# Tier 3: Advanced (on explicit request)
if context.requires_advanced:
load_module("modules/advanced-patterns.md")
load_module("modules/troubleshooting.md")When to Use:
- Common path well-defined (80/20 rule applies)
- Most users need simple workflow
- Advanced features are truly optional
4. Context Switching
Change loaded modules mid-session based on workflow changes.
Pattern:
class ProgressiveSkill:
def __init__(self):
self.current_modules = []
self.context = None
def switch_context(self, new_context):
# Unload modules from old context
old_modules = self._get_modules_for_context(self.context)
new_modules = self._get_modules_for_context(new_context)
# Only reload what's different
to_unload = set(old_modules) - set(new_modules)
to_load = set(new_modules) - set(old_modules)
for module in to_unload:
self._unload_module(module)
for module in to_load:
self._load_module(module)
self.context = new_context
self.current_modules = new_modulesWhen to Use:
- Long-running sessions with workflow changes
- Context pressure requires module swapping
- User switches between different tasks
Example:
# Session Evolution
1. User: "Analyze this git history"
→ Load git-patterns.md
2. User: "Now review the Python code"
→ Unload git-patterns.md
→ Load python-patterns.md
3. User: "Write tests for the changes"
→ Keep python-patterns.md
→ Load testing-patterns.md5. Preemptive Unloading
Remove modules when context pressure rises or workflow completes.
Pattern:
from leyline import MECWMonitor
def manage_modules(skill, monitor):
pressure = monitor.get_pressure_level()
if pressure == "HIGH":
# Unload completed workflow modules
unload_completed_modules(skill)
elif pressure == "CRITICAL":
# Unload all but essential modules
unload_all_except_core(skill)
# Consider summarizing and context reset
if monitor.current_tokens > monitor.mecw_threshold * 0.8:
summarize_session()
reset_context()When to Use:
- Long sessions with growing context
- Multiple workflows executed sequentially
- MECW pressure approaching limits
Strategy:
def prioritize_for_unloading(modules):
# Unload in this order:
priorities = {
"completed": 1, # Finished workflows
"optional": 2, # Nice-to-have modules
"edge-case": 3, # Rarely used features
"common": 4, # Keep as long as possible
"core": float('inf') # Never unload
}
return sorted(modules, key=lambda m: priorities[m.type])Advanced Patterns
6. Dependency Resolution
Load required dependencies automatically.
Pattern:
# modules/api-review.md frontmatter
---
module_name: api-review
dependencies:
- error-patterns # From leyline
- design-principles
optional_dependencies:
- performance-patterns # Only if performance review requested
---def load_with_dependencies(module_name, skill_path, loaded=None):
if loaded is None:
loaded = set()
if module_name in loaded:
return # Already loaded
# Load dependencies first
module = read_module_metadata(skill_path, module_name)
for dep in module.dependencies:
load_with_dependencies(dep, skill_path, loaded)
# Load the module itself
load_module(skill_path, module_name)
loaded.add(module_name)When to Use:
- Modules have shared foundational content
- Want to avoid duplicate content across modules
- Complex module relationships
7. Caching and Memoization
Cache loaded modules for fast re-loading.
Pattern:
class ModuleCache:
def __init__(self):
self._cache = {}
self._access_count = {}
def get_module(self, module_path):
if module_path not in self._cache:
self._cache[module_path] = self._load_from_disk(module_path)
self._access_count[module_path] = 0
self._access_count[module_path] += 1
return self._cache[module_path]
def evict_least_used(self, keep_count=5):
# Keep most frequently accessed modules
sorted_modules = sorted(
self._access_count.items(),
key=lambda x: x[1],
reverse=True
)
for module_path, _ in sorted_modules[keep_count:]:
del self._cache[module_path]
del self._access_count[module_path]When to Use:
- Modules are expensive to load
- Same modules loaded repeatedly
- Want to optimize performance
8. Incremental Loading
Load large modules in chunks.
Pattern:
# modules/large-reference.md
## Core Concepts
[Load first - 200 tokens]
## Common Patterns
[Load on request - 500 tokens]
## Complete Reference
[Load only if needed - 2000 tokens]def load_incremental(module_path, section=None):
if section is None:
# Load just the overview
return load_section(module_path, "Core Concepts")
else:
# Load specific section
return load_section(module_path, section)
# Usage
core = load_incremental("large-reference.md") # 200 tokens
if need_more_detail:
patterns = load_incremental("large-reference.md", "Common Patterns") # +500 tokens
if need_complete:
full = load_incremental("large-reference.md", "Complete Reference") # +2000 tokensWhen to Use:
- Individual modules are very large
- Most uses need only part of module
- Want to support both quick reference and deep dives
Implementation Utilities
Module Metadata
# Every module should have frontmatter
---
module_name: git-patterns
skill: catchup
priority: common
estimated_tokens: 450
dependencies: []
optional_dependencies: [advanced-git]
triggers:
keywords: [git, commit, branch]
artifacts: [.git/]
mutually_exclusive_with: [document-patterns]
load_strategy: lazy
cache_ttl: 3600
---Loading Protocol
from leyline import MECWMonitor, estimate_tokens
class ModuleLoader:
def __init__(self, skill_path):
self.skill_path = skill_path
self.loaded = {}
self.monitor = MECWMonitor()
def load(self, module_name, strategy="lazy"):
# Check if already loaded
if module_name in self.loaded:
return self.loaded[module_name]
# Check MECW compliance
module_path = f"{self.skill_path}/modules/{module_name}.md"
tokens = estimate_tokens(module_path)
can_load, issues = self.monitor.can_handle_additional(tokens)
if not can_load:
raise ModuleLoadError(f"Cannot load {module_name}: {issues}")
# Load based on strategy
if strategy == "lazy":
content = self._load_on_access(module_path)
elif strategy == "eager":
content = self._load_immediately(module_path)
elif strategy == "incremental":
content = self._load_core_only(module_path)
# Track and return
self.loaded[module_name] = content
self.monitor.track_usage(self.monitor.current_tokens + tokens)
return content
def unload(self, module_name):
if module_name in self.loaded:
del self.loaded[module_name]Best Practices
1. Always Load Core: Hub SKILL.md should always load minimum viable content 2. Document Load Triggers: Make it clear when/why modules load 3. Respect MECW: Check budget before loading any module 4. Prefer Lazy: Load on-demand unless eager loading clearly better 5. Cache Smartly: Cache frequently accessed, stable modules 6. Measure Reality: Track which modules actually get loaded in practice 7. Fail Gracefully: Provide degraded functionality if module won't load
Anti-Patterns
Eager Loading Everything: Defeats progressive loading purpose Complex Load Logic: If loading is hard to debug, simplify Ignoring Dependencies: Load dependencies before dependents No Unloading: Memory/context grows unbounded Silent Load Failures: User should know if module unavailable Circular Dependencies: Modules should form DAG, not cycles
Integration Examples
With Imbue Catchup
# catchup/SKILL.md
## Progressive Loading
**Git Catchup**: Load `modules/git-catchup-patterns.md`
- Triggers: git commands, branch mentions, commit analysis
- Dependencies: leyline:mecw-patterns, sanctum:git-workspace-review
**Document Catchup**: Load `modules/document-analysis-patterns.md`
- Triggers: markdown files, meeting notes, sprint docs
- Dependencies: leyline:progressive-loading
**Log Catchup**: Load `modules/log-analysis-patterns.md`
- Triggers: log files, time-series data, event streams
- Dependencies: leyline:mecw-patternsWith Conservation Context-Optimization
# context-optimization/SKILL.md
## Progressive Loading
**MECW Assessment**: Always loaded (core module)
**Subagent Coordination**: Load when complexity high or context critical
**Advanced Optimization**: Load when MODERATE+ pressure detectedWith Abstract Modular-Skills
# modular-skills/SKILL.md
## Progressive Loading
**Core Workflow**: Always loaded - hub-and-spoke overview
**Implementation Patterns**: Load when user designing/implementing
**Migration Guide**: Load when user has existing monolithic skill
**Troubleshooting**: Load on explicit request or validation failuresValidation Checklist
- [ ] Hub SKILL.md defines all loading contexts
- [ ] Each module has frontmatter with loading metadata
- [ ] Dependencies declared and resolved correctly
- [ ] MECW compliance checked before loading
- [ ] Unloading strategy defined for long sessions
- [ ] Cache strategy appropriate for module access patterns
- [ ] Loading failures handled gracefully
- [ ] Performance measured (load time, token cost)
Log Analysis Patterns
This module covers progressive-loading for skills that read, parse, and summarize log files. The selection question is which parsing modules to load based on log format (JSON, syslog, custom plain text), volume (kilobytes vs gigabytes), and goal (error triage, performance review, audit trail).
When This Module Applies
Load this module when the task involves:
- Reading application or system log files.
- Filtering log streams for errors, warnings, or specific
events.
- Producing a digest of what happened during a window.
- Correlating events across multiple log sources.
For prose documents, load document-analysis-patterns.md. For git history, load git-catchup-patterns.md. This module is for machine-generated event streams.
Format Detection First
Log format dictates the parser. Misclassifying a syslog file as JSON wastes the first parse attempt and produces garbage.
from pathlib import Path
def detect_format(log_path: Path, sample_lines: int = 20) -> str:
with log_path.open("r", encoding="utf-8", errors="replace") as fp:
lines = [next(fp, "") for _ in range(sample_lines)]
sample = "\n".join(line for line in lines if line)
if sample.lstrip().startswith("{"):
return "json"
if " kernel:" in sample or " systemd[" in sample:
return "syslog"
if sample.startswith("[") and "INFO" in sample[:200]:
return "bracketed"
return "plain"Read only a small sample. A 5GB log file should never be opened in full just to detect the format.
Loading Map
| Format | Parser Module | Token Estimate |
|---|---|---|
| JSON lines | json-log-parser.md | 400 |
| Syslog | syslog-parser.md | 500 |
Bracketed ([LEVEL] msg) | bracketed-parser.md | 300 |
| Plain text | plain-text-parser.md | 500 |
| Mixed (multi-line stack traces) | multiline-parser.md | 600 |
The plain-text parser is the largest because it must handle arbitrary formats with regex heuristics. JSON is smaller because the structure is self-describing.
Volume-Based Loading
Log volume splits into three bands that drive different techniques.
| Band | Size | Strategy Module |
|---|---|---|
| Small | <10 MB | full-load.md (read everything) |
| Medium | 10 MB to 1 GB | streaming.md (line-at-a-time) |
| Large | >1 GB | sampled.md (head, tail, time windows) |
The strategy module loads after the format module, so the parser knows whether to expect a fully loaded file or a stream.
Streaming Pattern
Medium-volume logs need line-by-line streaming to keep memory bounded.
import json
from pathlib import Path
from typing import Iterator
def iter_json_logs(path: Path) -> Iterator[dict]:
with path.open("r", encoding="utf-8", errors="replace") as fp:
for line in fp:
line = line.strip()
if not line:
continue
try:
yield json.loads(line)
except json.JSONDecodeError:
continue # Skip malformed lines, do not abortSkipping malformed lines is intentional. Logs often contain partial writes at the tail when the file is rotated mid-read.
Sampling Pattern for Large Files
For multi-gigabyte logs, full reads are infeasible. The sampled module documents three slices that cover most analysis needs.
# First 1000 lines (startup events)
head -n 1000 /var/log/app.log
# Last 1000 lines (recent events)
tail -n 1000 /var/log/app.log
# Time window via grep on timestamp prefix
grep '^2026-05-03T1[0-2]' /var/log/app.log | head -n 5000For complex queries on large logs, the right answer is often to ingest into a real log store (grep, rg, awk, or a SIEM) rather than parse in-process.
Pitfalls
1. Loading entire log files: A 5 GB log file blows the process memory and the context budget. Always classify volume first. 2. Aborting on malformed lines: Logs are streaming data. Partial writes at the tail are normal. Skip and continue. 3. One regex for all plain-text: Plain-text logs vary by application. Use per-application parsers, not a single universal regex. 4. Ignoring multi-line entries: Stack traces and SQL queries span multiple lines. Single-line parsers split them incorrectly. Detect and load the multi-line parser when needed. 5. Treating timestamps as strings only: Sorting logs by string timestamp works for ISO-8601 but breaks for syslog format Mon DD HH:MM:SS. Parse to a datetime when ordering matters.
Cross-Reference
See document-analysis-patterns.md for prose documents and the parent SKILL.md for how log modules plug into the hub-and-spoke pattern.
macOS Patterns
This module covers progressive-loading for skills that target macOS as the development or deployment host. The selection question is which macOS-specific modules to load: file system layout (HFS+/APFS), process control via launchd, package managers (Homebrew, MacPorts), code signing, and the security sandbox.
When This Module Applies
Load this module when:
- The active machine reports
Darwinfromuname -s. - The task touches paths under
/Applications,~/Library, or
/System.
- The user mentions
brew,launchctl,codesign, or
xcode-select.
- The deployment target is macOS desktop or mac-hosted CI.
For Linux paths and tools, load linux-patterns.md. For Windows, load windows-patterns.md. The three are mutually exclusive per session unless the task is cross-platform.
Detect macOS Version Before Sub-Loading
Major macOS versions change defaults significantly. The version detection drives which sub-module loads.
# Get macOS version (e.g., "14.4.1")
sw_vers -productVersion
# Get the major version only
sw_vers -productVersion | cut -d. -f1macOS 11 (Big Sur) and later use APFS by default and ship with zsh as the user shell. macOS 10.15 (Catalina) introduced notarization requirements. The version-specific module documents what changed in each release relevant to skill behavior.
Path Conventions
macOS uses a layered file system that differs from Linux. The shared module documents the conventions.
# User configs (Apple convention)
~/Library/Application Support/<bundle-id>/
# User caches
~/Library/Caches/<bundle-id>/
# User logs
~/Library/Logs/<bundle-id>/
# Per-user launchd agents
~/Library/LaunchAgents/
# System-wide apps
/Applications/
# Command-line tools (Homebrew on Apple Silicon)
/opt/homebrew/bin/
# Command-line tools (Homebrew on Intel)
/usr/local/bin/Bundle IDs use reverse-DNS notation (com.example.myapp). For non-bundled tools, the convention is the tool name as a folder under Application Support.
launchd Service Pattern
Long-running services on macOS use launchd instead of systemd. The launchd sub-module documents the plist format.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.example.myagent</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/myagent</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>User-scoped agents go in ~/Library/LaunchAgents/. System-scoped daemons go in /Library/LaunchDaemons/ and require root. Load with launchctl load -w <path>, unload with launchctl unload.
Homebrew Detection
Homebrew is in different prefixes on Apple Silicon versus Intel. The Homebrew sub-module documents detection.
# Detect Homebrew prefix
if command -v brew >/dev/null 2>&1; then
BREW_PREFIX=$(brew --prefix)
elif [ -x /opt/homebrew/bin/brew ]; then
BREW_PREFIX=/opt/homebrew
elif [ -x /usr/local/bin/brew ]; then
BREW_PREFIX=/usr/local
fiApple Silicon Macs default to /opt/homebrew. Intel Macs use /usr/local. Skills installing tools via Homebrew should never hard-code one prefix.
Pitfalls
1. Hard-coding `/usr/local/bin`: This breaks on Apple Silicon Macs where Homebrew is at /opt/homebrew. Use the detection block above. 2. Treating `~/Library/Application Support` as `~/.config`: They are conceptually similar but the Apple convention uses bundle IDs and case-sensitive folder names with spaces. 3. Skipping notarization for distribution: macOS 10.15+ blocks unsigned and unnotarized executables from running. Distribution skills must load the codesign sub-module. 4. Using `service` or `systemctl`: macOS does not ship these commands. Use launchctl and the plist format. 5. Assuming bash: macOS 10.15+ ships zsh as the user default. Scripts that rely on bash 4 features need an explicit #!/usr/bin/env bash shebang and a Homebrew bash install (the system bash is 3.2).
Cross-Reference
See linux-patterns.md and windows-patterns.md for the other platforms in the mutually-exclusive group, and the parent SKILL.md for the platform-selection contract.
Modern Python
This module covers progressive-loading for skills targeting Python 3.11 or later. The selection question is which language-feature modules to load when the runtime supports match statements, tomllib in the standard library, exception groups, the Self type, and PEP 604 union syntax without backports.
When This Module Applies
Load this module when:
- The target codebase pins
requires-python = ">=3.11"in
pyproject.toml.
- CI matrices include 3.11, 3.12, or 3.13 only.
- The user explicitly requests modern Python features.
For Python 3.8-3.10 work, load legacy-python.md instead. The two modules contradict each other on syntax and stdlib availability, so the mutually-exclusive selection pattern in selection-strategies.md enforces a single choice.
Detect the Version Floor
Read the version constraint from the project metadata. The local interpreter version is irrelevant if the project must support older versions.
import tomllib
from pathlib import Path
def python_floor(project_root: Path) -> tuple[int, int] | None:
pyproject = project_root / "pyproject.toml"
if not pyproject.exists():
return None
data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
requires = data.get("project", {}).get("requires-python", "")
if requires.startswith(">="):
spec = requires[2:].split(",")[0]
major, minor = spec.split(".")[:2]
return (int(major), int(minor))
return Nonetomllib is available without backport on 3.11+, which is one reason this module assumes the modern floor.
Features Available on 3.11+
The module loaded here documents the features that justify the modern floor.
| Feature | Available From | Use Case |
|---|---|---|
match statement | 3.10 | Multi-way dispatch on shape |
| `int \ | str` unions | 3.10 |
tomllib stdlib | 3.11 | Parse TOML without tomli |
ExceptionGroup | 3.11 | Aggregate concurrent failures |
Self type | 3.11 | Annotate methods returning self |
LiteralString | 3.11 | SQL-injection-resistant types |
| Improved error locations | 3.11 | Per-character traceback markers |
tomllib.loads | 3.11 | String-mode TOML parsing |
3.12 adds the type statement (PEP 695 type aliases) and per-interpreter GIL support. 3.13 adds the experimental free-threaded build.
Concrete Example: Match on Shape
match statements replace verbose isinstance chains.
from dataclasses import dataclass
@dataclass
class Point:
x: float
y: float
def describe(value: object) -> str:
match value:
case Point(x=0, y=0):
return "origin"
case Point(x=0, y=y):
return f"on y-axis at {y}"
case Point(x=x, y=0):
return f"on x-axis at {x}"
case Point():
return "off-axis point"
case [first, *rest]:
return f"sequence starting with {first!r}"
case _:
return "unknown"Class patterns (Point(x=0, y=0)) require the dataclass to declare __match_args__, which @dataclass does automatically.
Concrete Example: Self Type
typing.Self removes the boilerplate of class-bound TypeVars.
from typing import Self
class Builder:
def __init__(self) -> None:
self.parts: list[str] = []
def add(self, part: str) -> Self:
self.parts.append(part)
return self
def build(self) -> str:
return " ".join(self.parts)Subclasses of Builder get the correct return type from add without redeclaring it.
Concrete Example: Exception Groups
ExceptionGroup aggregates multiple failures from concurrent work without losing any of them.
def collect_failures(tasks: list[callable]) -> None:
errors: list[Exception] = []
for task in tasks:
try:
task()
except Exception as exc:
errors.append(exc)
if errors:
raise ExceptionGroup("task failures", errors)Callers catch with except* ValueError as eg: to filter by exception type while preserving the group structure.
Pitfalls
1. Using `match` on 3.9 targets: match is 3.10+. Skills generating code for older floors must load legacy-python.md and use if/elif chains instead. 2. Assuming `tomllib` on every Python: It is 3.11+ only. Verify the floor before importing. 3. Class patterns on non-dataclass classes: Class patterns need __match_args__. Plain classes without this attribute match by keyword only. 4. `Self` without inheritance: Self is meaningful for subclassing. For final classes, the explicit class name in the annotation is equally clear. 5. Loading both legacy and modern modules: They contradict each other on union syntax and stdlib modules. The mutually-exclusive selection pattern enforces a single choice.
Cross-Reference
See legacy-python.md for the 3.8-3.10 counterpart, and the parent SKILL.md for the mutually-exclusive selection pattern that picks between them.
Performance Budgeting for Skills
Optimize Claude Code plugin performance through token budgeting and context-aware content delivery.
Token Budget Model
Budget Allocation (Claude Code v2.1.32+)
| Context Window | Budget (2%) | Per-Skill Target | Max Skills |
|---|---|---|---|
| 200k tokens | ~16,000 chars | 300-500 chars | ~40 |
| 1M tokens | ~20,000 chars | up to 160 chars | ~74 |
The SLASH_COMMAND_TOOL_CHAR_BUDGET env var overrides the default. The ecosystem validator uses 20,000 (matching 1M context GA).
Per-Skill Targets
| Skill Size | Token Range | Strategy |
|---|---|---|
| Minimal | <300 tokens | Single SKILL.md, no modules |
| Standard | 300-800 tokens | SKILL.md and 1-2 modules |
| Large | 800-1500 tokens | Progressive loading required |
| Oversize | >1500 tokens | Split into separate skills |
Core Principles
1. Metadata-first discovery - Claude scans ~100 tokens of frontmatter to decide relevance before loading full content 2. Progressive disclosure - Essential content loads immediately; advanced content is in modules loaded on-demand 3. Token budgeting - Track and enforce per-skill token limits aligned with the ecosystem budget 4. Context-aware delivery - Load depth matches task complexity
Optimization Workflow
Step 1: Measure
python plugins/abstract/scripts/validate_budget.pyStep 2: Identify Reduction Targets
- Move examples and edge cases to modules
- Compress repetitive patterns into tables
- Remove content duplicated from dependencies
- Replace verbose explanations with concise rules
Step 3: Restructure
- Extract sections >200 tokens into
modules/ - Ensure SKILL.md frontmatter description stays under 500 characters
- Add
progressive_loading: trueto frontmatter - List modules in frontmatter
modules:array
Step 4: Validate
python plugins/abstract/scripts/validate_budget.py
# Target: 50%+ reduction from originalQuality Checks
Before finalizing optimization:
- [ ] SKILL.md frontmatter description is under 500 characters
- [ ] Quick Start section provides enough info for basic use
- [ ] All modules are listed in frontmatter
modules:array - [ ]
estimated_tokensin frontmatter reflects actual measured value - [ ]
progressive_loading: trueis set when modules exist - [ ] No functionality lost - advanced content accessible via modules
- [ ] Token reduction target met (50%+ for large skills)
References
- ADR 0004: Skill Description Budget - The 2% budget rule scales with context window size
- Context Optimization (
conserve:context-optimization) - MECW principles for context management
Performance
This module covers progressive-loading patterns for performance analysis: which profiling, benchmarking, and complexity-review modules to load based on the symptom (slow runtime, high memory, high allocations) and the target language. Loading every performance tool at once wastes context when only one symptom is being investigated.
When This Module Applies
Load this module when the task involves:
- Profiling slow code or finding hot loops.
- Measuring memory use or allocation rate.
- Reviewing algorithmic complexity (big-O) of a function.
- Establishing performance baselines before a refactor.
For token-budget performance of skills themselves, load performance-budgeting.md. This module is about runtime performance of user code.
Symptom-Driven Loading
Performance work splits into three distinct symptom categories, each with its own toolchain. Load only the relevant module.
| Symptom | Tool Module | Token Estimate |
|---|---|---|
| Wall-clock slow | cpu-profiling.md | 500 |
| Memory high | memory-profiling.md | 500 |
| Algorithmic | complexity-review.md | 400 |
| Throughput low | benchmarking.md | 400 |
| Concurrency | async-profiling.md | 600 |
The user usually states the symptom first. If they say "this endpoint takes 8 seconds", load cpu-profiling.md. If they say "memory grows over time", load memory-profiling.md. Do not load both speculatively.
Language-Specific Sub-Loading
Each symptom module dispatches to a language-specific tool. The hub picks the language from project metadata.
# Python: cProfile, py-spy, scalene
python -m cProfile -s cumulative script.py
# Rust: cargo-flamegraph, perf, criterion
cargo flamegraph --bin myapp
# Node: --prof, clinic.js, 0x
node --prof script.jspy-spy and scalene are real PyPI packages. cargo flamegraph is provided by the flamegraph crate. clinic.js is on npm. The language sub-modules document install commands and read paths for the resulting profile artifacts.
Baseline Before Optimizing
A common failure mode is optimizing without a baseline. The benchmarking module enforces a measure-first protocol.
import time
from statistics import mean, stdev
def baseline(func, *args, runs: int = 10, **kwargs):
times: list[float] = []
for _ in range(runs):
start = time.perf_counter_ns()
func(*args, **kwargs)
times.append((time.perf_counter_ns() - start) / 1_000_000)
return {
"mean_ms": mean(times),
"stdev_ms": stdev(times) if len(times) > 1 else 0.0,
"min_ms": min(times),
"max_ms": max(times),
}time.perf_counter_ns is monotonic and high-resolution. For microsecond-scale operations, use timeit.repeat instead, which disables garbage collection between runs.
Complexity Review
For algorithmic concerns, the complexity module documents the common pitfalls.
# O(n^2): nested membership check on a list
def has_duplicate_slow(items: list[int]) -> bool:
for i, item in enumerate(items):
if item in items[i + 1:]: # O(n) inside O(n)
return True
return False
# O(n): set-based membership
def has_duplicate_fast(items: list[int]) -> bool:
seen: set[int] = set()
for item in items:
if item in seen:
return True
seen.add(item)
return FalseThe complexity module surfaces these patterns through static inspection rather than measurement. It pairs with the pensive:performance-review skill for a real AST walk.
Pitfalls
1. Profiling without a baseline: "It's faster" with no numbers is meaningless. Always record before/after with the benchmarking module loaded. 2. Loading the memory module for CPU symptoms: They use different tools and produce different artifacts. Match the module to the symptom. 3. Profiling production code unprepared: Profilers add overhead. cProfile in particular slows execution noticeably. Document the overhead in the profile output. 4. Treating mean as the only number: Mean hides tail latency. Always report at least mean, p99, and max. 5. Optimizing the wrong layer: A microsecond improvement in a function called once per request matters less than a millisecond in a function called per-row in a 10000-row loop. The complexity module addresses this by walking the call graph.
Cross-Reference
See performance-budgeting.md for skill token performance, troubleshooting.md for diagnostic workflows, and the parent SKILL.md for how performance modules plug into the hub-and-spoke pattern.
Python Packaging
This module covers progressive-loading for skills that work with Python package authoring and distribution: pyproject.toml authoring, build backend selection, entry points, dependency declaration, lockfile management, and PyPI publishing.
When This Module Applies
Load this module when the task touches:
- A
pyproject.toml,setup.py, orsetup.cfgfile. - Tool configuration tables:
[tool.uv],[tool.hatch],
[tool.poetry], [tool.ruff], [tool.pytest.ini_options].
- Console scripts or entry points (
[project.scripts]). - A
uv.lock,poetry.lock, orrequirements*.txtfile. - A PyPI publishing task or release pipeline.
For runtime Python idioms, load python-patterns.md. For tests, load python-testing.md. This module focuses on the packaging boundary.
Detect the Build Backend First
The build backend in pyproject.toml decides which tool sub-module to load. Different backends have different config shapes.
import tomllib
from pathlib import Path
def detect_backend(project_root: Path) -> str:
pyproject = project_root / "pyproject.toml"
if not pyproject.exists():
return "none"
data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
backend = data.get("build-system", {}).get("build-backend", "")
if "hatchling" in backend:
return "hatch"
if "poetry" in backend:
return "poetry"
if "setuptools" in backend:
return "setuptools"
if "flit" in backend:
return "flit"
if "pdm" in backend:
return "pdm"
return "unknown"The result drives which detail module loads next: hatch projects need [tool.hatch.build.targets.wheel] guidance, poetry projects need [tool.poetry.dependencies] guidance, and so on.
Loading Map
| Backend | Detail Module | Token Estimate |
|---|---|---|
| Hatch | hatch-config.md | 400 |
| Poetry | poetry-config.md | 500 |
| Setuptools | setuptools-config.md | 500 |
| PDM | pdm-config.md | 400 |
| Flit | flit-config.md | 300 |
The shared metadata module covers [project] table fields common across PEP 621-compliant backends. Keep it always loaded because every modern project uses it.
Concrete Example: Minimal pyproject.toml
The shared module documents the PEP 621 metadata that works across backends.
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "myapp"
version = "0.1.0"
description = "Short one-line description"
requires-python = ">=3.11"
dependencies = [
"click>=8.1.0",
"rich>=13.0.0",
]
[project.optional-dependencies]
dev = [
"pytest>=8.0.0",
"ruff>=0.4.0",
]
[project.scripts]
myapp = "myapp.cli:main"The [project.scripts] table generates console scripts at install time. The myapp command will invoke myapp.cli.main.
Lockfile Strategy
Lockfile choice depends on the resolver. The lockfile sub-module documents each.
| Tool | Lockfile | Generation Command |
|---|---|---|
| uv | uv.lock | uv lock |
| poetry | poetry.lock | poetry lock |
| pdm | pdm.lock | pdm lock |
| pip-tools | requirements.txt | pip-compile requirements.in |
uv is fast enough to be a drop-in resolver for pip-based projects. The uv sub-module documents uv pip compile as a pip-tools-compatible command.
Publishing
For PyPI publishing, the publish sub-module documents the two-stage workflow: build, then upload.
# Build artifacts (works with any PEP 517 backend)
python -m build
# Upload via twine (or backend-native command)
twine upload dist/*
twine upload --repository testpypi dist/* # test firstpython -m build is the official PyPA build front-end and works with hatch, poetry, setuptools, flit, and pdm backends.
Pitfalls
1. Editing `setup.py` when `pyproject.toml` exists: Modern projects declare metadata in pyproject.toml. Editing both is a source of drift. 2. Loading the wrong backend module: A Hatch project does not have [tool.poetry] tables. Detect the backend first. 3. Skipping the version floor: requires-python controls which features the package can use. Generated code that targets newer features than the floor breaks installs. 4. Hand-editing lockfiles: Lockfiles are generated. Edits break the resolver. Re-run the lock command instead. 5. Uploading to PyPI without testing: Always upload to TestPyPI first to verify the package installs cleanly before publishing the real version.
Cross-Reference
See python-patterns.md for runtime idioms, python-testing.md for test setup, and the parent SKILL.md for how packaging modules plug into the hub-and-spoke pattern.
Python Patterns
This module covers progressive-loading for skills that read, generate, or refactor Python source code. The selection question is which Python idiom modules to load: typing, dataclasses, context managers, iterators, or async. Each is a distinct slice that need not load together.
When This Module Applies
Load this module when the task involves:
- Reading or modifying
.pyfiles. - Generating new Python source code.
- Reviewing Python idioms for readability or correctness.
- Refactoring imperative code into more idiomatic forms.
For test-specific patterns, load python-testing.md. For packaging concerns, load python-packaging.md. For version selection between legacy and modern features, see legacy-python.md or modern-python.md.
Slice the Idiom Surface First
A "Python patterns" mega-module would cover everything from list comprehensions to async context managers. The progressive load splits by idiom family.
| Family | Module | Token Estimate |
|---|---|---|
| Type hints | typing-patterns.md | 500 |
| Dataclasses | dataclass-patterns.md | 400 |
| Context managers | context-manager-patterns.md | 300 |
| Iterators and generators | iterator-patterns.md | 400 |
Async (asyncio) | async-patterns.md | 600 |
| Pathlib | pathlib-patterns.md | 200 |
Most tasks need one or two families, not all six. The hub picks based on the imports and patterns already present in the file under review.
Detection by Existing Imports
A fast loader inspects the file's imports to pick the right sub-modules.
import ast
from pathlib import Path
def detect_idioms(py_path: Path) -> set[str]:
tree = ast.parse(py_path.read_text(encoding="utf-8"))
idioms: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
if alias.name == "asyncio":
idioms.add("async")
if alias.name == "pathlib":
idioms.add("pathlib")
elif isinstance(node, ast.ImportFrom):
if node.module == "dataclasses":
idioms.add("dataclass")
if node.module == "typing":
idioms.add("typing")
if node.module == "contextlib":
idioms.add("context-manager")
return idiomsThe set drives module selection. A file importing only pathlib and dataclasses does not need the async sub-module.
Concrete Example: Pathlib over os.path
The pathlib sub-module documents the most common substitutions since os.path operations have direct pathlib equivalents.
from pathlib import Path
# Read a file's text
content = Path("config.toml").read_text(encoding="utf-8")
# Build a path
log_dir = Path.home() / ".local" / "share" / "myapp" / "logs"
log_dir.mkdir(parents=True, exist_ok=True)
# Iterate matching files
for py_file in Path("src").rglob("*.py"):
print(py_file.relative_to("src"))
# Check existence and type
if log_dir.is_dir():
passPath.read_text and Path.write_text accept encoding= and default to the locale encoding. Always pass encoding="utf-8" for portable behavior.
Concrete Example: Dataclass with Defaults
The dataclass sub-module documents the field defaults that trip up most authors.
from dataclasses import dataclass, field
@dataclass(frozen=True, slots=True)
class Config:
name: str
tags: list[str] = field(default_factory=list)
extra: dict[str, str] = field(default_factory=dict)
timeout_s: float = 30.0field(default_factory=list) is required for mutable defaults. A bare tags: list[str] = [] triggers a ValueError at class definition because dataclass rejects shared mutable defaults.
frozen=True makes instances immutable and hashable. slots=True (Python 3.10+) saves memory and prevents accidental attribute addition.
Concrete Example: Context Manager via contextlib
For one-off context managers, contextlib.contextmanager is shorter than a class.
import time
from contextlib import contextmanager
from typing import Iterator
@contextmanager
def timed(label: str) -> Iterator[None]:
start = time.perf_counter()
try:
yield
finally:
elapsed = time.perf_counter() - start
print(f"{label}: {elapsed:.3f}s")
with timed("step 1"):
do_work()The try/finally around yield ensures cleanup runs even when the wrapped block raises.
Pitfalls
1. Mutable default arguments: def f(items=[]): shares the list across calls. The dataclass module documents the field(default_factory=...) fix; for plain functions, use items=None and assign inside the body. 2. String paths everywhere: os.path.join produces strings that lose type information. Use Path and convert to string at the boundary only when an external API requires it. 3. Loading async patterns for sync code: asyncio patterns add complexity that sync code does not need. Detect asyncio imports before loading. 4. Bare `except:`: Catches KeyboardInterrupt and SystemExit. Use except Exception: for broad catches and specific exception types when possible. 5. One mega-module: 2000 tokens of mixed Python guidance forces every Python task to load all of it. Split by family.
Cross-Reference
See python-testing.md for test patterns, python-packaging.md for distribution, and the parent SKILL.md for how Python modules plug into the hub-and-spoke pattern.
Python Testing
This module covers progressive-loading for skills that read, generate, or refactor Python tests. The selection question is which testing-tool modules to load: pytest fixtures, mocking, async tests, parameterization, coverage, or property-based testing.
When This Module Applies
Load this module when the task involves:
- Reading or writing files matching
test_*.pyor*_test.py. - Generating new tests for existing functions.
- Reviewing test quality, coverage, or fixture design.
- Setting up
pyproject.tomltest configuration.
For runtime Python patterns, load python-patterns.md. For packaging concerns including test extras, load python-packaging.md. This module focuses on the test layer.
Slice the Testing Surface First
Testing modules split by concern. Loading every concern at once adds 3000+ tokens to a routine test edit.
| Concern | Module | Token Estimate |
|---|---|---|
| pytest basics | pytest-basics.md | 400 |
| Fixtures and scope | fixture-patterns.md | 500 |
| Mocking | mock-patterns.md | 500 |
| Async tests | async-test-patterns.md | 400 |
| Parameterization | parametrize-patterns.md | 300 |
| Coverage | coverage-patterns.md | 400 |
| Property-based | hypothesis-patterns.md | 600 |
The hub picks based on the test file content and the project's declared test dependencies (pytest-asyncio, pytest-mock, hypothesis).
Detect Installed Test Plugins
The plugin set drives module selection. A project without pytest-asyncio cannot use async fixtures even if the source code is async.
import tomllib
from pathlib import Path
def test_extras(project_root: Path) -> set[str]:
pyproject = project_root / "pyproject.toml"
if not pyproject.exists():
return set()
data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
project = data.get("project", {})
deps = project.get("dependencies", []) + project.get(
"optional-dependencies", {}
).get("dev", []) + project.get("optional-dependencies", {}).get(
"test", []
)
return {dep.split(">=")[0].split("==")[0].strip() for dep in deps}Search the result for pytest-asyncio, pytest-mock, hypothesis, and friends. Load the matching sub-modules only.
Concrete Example: Fixture with Scope
The fixture sub-module documents scope choices.
import pytest
from pathlib import Path
@pytest.fixture(scope="session")
def shared_db_path(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Database path shared across the whole test session."""
return tmp_path_factory.mktemp("db") / "test.sqlite"
@pytest.fixture
def fresh_db_path(tmp_path: Path) -> Path:
"""Fresh database path per test."""
return tmp_path / "test.sqlite"tmp_path is function-scoped; tmp_path_factory is session- scoped. Mixing scopes causes subtle failures: a session-scoped fixture cannot depend on a function-scoped one.
Concrete Example: Parametrize
The parametrize sub-module documents the table-driven pattern.
import pytest
@pytest.mark.parametrize(
"value,expected",
[
("0", 0),
("42", 42),
("-1", -1),
("0xff", 255),
],
ids=["zero", "positive", "negative", "hex"],
)
def test_parse_int(value: str, expected: int) -> None:
assert parse_int(value) == expectedThe ids argument controls test names in pytest output. Without it, pytest generates names from the values, which can produce unreadable names for complex inputs.
Concrete Example: Async Test
The async sub-module documents the marker convention.
import asyncio
import pytest
@pytest.mark.asyncio
async def test_concurrent_fetches() -> None:
results = await asyncio.gather(
fetch("a"),
fetch("b"),
)
assert len(results) == 2pytest-asyncio provides the asyncio marker. The default mode is strict, which requires the marker on every async test. Set asyncio_mode = "auto" in pyproject.toml to mark all async functions automatically.
Concrete Example: pyproject Test Config
The test-config sub-module documents the pytest section.
[tool.pytest.ini_options]
minversion = "8.0"
addopts = [
"--strict-markers",
"--strict-config",
"--tb=short",
"-ra",
]
testpaths = ["tests"]
asyncio_mode = "auto"--strict-markers makes typo'd @pytest.mark.foo a hard error. --strict-config does the same for unknown options. -ra shows reasons for skipped, expected-fail, and error tests.
Pitfalls
1. Loading hypothesis without the dependency: Generated property-based tests fail at import time if hypothesis is not in the project. Detect dependencies before generating. 2. Mixing fixture scopes: A session-scoped fixture cannot request a function-scoped one. The fixture module documents the scope hierarchy. 3. Using `unittest.mock` blindly: unittest.mock.patch requires the import path of the function as it is used, rather than where it is defined. Mistakes here produce mocks that never fire. 4. Skipping `--strict-markers`: A typo in @pytest.mark.skipif becomes a silent no-op without strict markers. Always enable. 5. Async tests without the right marker: Without pytest-asyncio and the marker (or asyncio_mode = "auto"), async test functions are collected but never awaited, so they always pass.
Cross-Reference
See python-patterns.md for runtime idioms, python-packaging.md for distribution, and the parent SKILL.md for how testing modules plug into the hub-and-spoke pattern.
Rust Review
This module covers progressive-loading for Rust review skills: which review-concern modules to load when auditing Rust source for safety, ownership, concurrency, error handling, and idiomatic style. The companion to cargo-patterns.md, which covers the build-tool layer rather than the source layer.
When This Module Applies
Load this module when the task involves:
- Reviewing
.rsfiles for correctness, safety, or style. - Auditing
unsafeblocks for soundness. - Investigating ownership, lifetimes, or borrow-checker errors.
- Reviewing async or threaded code for race conditions.
For dependency or workspace concerns, load cargo-patterns.md. This module focuses on the source-code review surface.
Slice the Review Surface First
A "Rust review" mega-module covering every concern is too large. The progressive load splits by concern.
| Concern | Module | Token Estimate |
|---|---|---|
| Ownership and borrowing | ownership-rules.md | 600 |
| Unsafe audit | unsafe-rules.md | 700 |
| Error handling | error-handling.md | 500 |
| Concurrency | concurrency-rules.md | 700 |
Async (tokio, async-std) | async-rules.md | 600 |
| Idiomatic style | style-rules.md | 400 |
| Performance and allocation | performance-rules.md | 500 |
Most reviews exercise two or three concerns, not all seven. The hub selects based on the file content and explicit user signals.
Detect Concerns by Source Inspection
A simple regex pass over the file picks the relevant concerns.
# Unsafe blocks present
rg --type rust 'unsafe (fn|impl|trait|\{)' path/
# Async present
rg --type rust 'async fn|\.await\b' path/
# Concurrency primitives present
rg --type rust 'std::sync|std::thread|crossbeam|rayon' path/
# Custom error types
rg --type rust 'thiserror|impl Error for|enum.*Error' path/Each match triggers loading the matching sub-module. A file with no unsafe and no async only loads ownership and error handling.
Concrete Example: Result Handling
The error-handling sub-module documents the patterns that come up in every Rust review.
use std::fs;
use std::io;
use std::path::Path;
fn read_config(path: &Path) -> Result<String, io::Error> {
fs::read_to_string(path)
}
// Propagate with ?
fn load(path: &Path) -> Result<Config, ConfigError> {
let text = read_config(path).map_err(ConfigError::Io)?;
let config: Config = toml::from_str(&text).map_err(ConfigError::Parse)?;
Ok(config)
}
#[derive(Debug, thiserror::Error)]
enum ConfigError {
#[error("io error: {0}")]
Io(#[from] io::Error),
#[error("parse error: {0}")]
Parse(#[from] toml::de::Error),
}thiserror is a real crate by David Tolnay. The #[from] attribute generates From impls so the ? operator does the conversion automatically.
Concrete Example: Unsafe Audit Checklist
The unsafe sub-module documents the contract checks every reviewer must run.
// SAFETY: This block dereferences a raw pointer obtained from
// `Box::into_raw` exactly once, after which it is reboxed.
// The pointer is non-null and properly aligned because it
// came from a valid Box.
unsafe {
let b = Box::from_raw(ptr);
drop(b);
}The review rule is that every unsafe block must have a // SAFETY: comment documenting the invariants the caller is asserting. Blocks without the comment are review-blockers.
Concrete Example: Concurrency Smell
The concurrency sub-module documents common races.
use std::sync::{Arc, Mutex};
use std::thread;
// SMELL: lock held across .await
async fn bad(state: Arc<Mutex<Vec<u32>>>) {
let mut guard = state.lock().unwrap();
guard.push(1);
do_async_work().await; // holds lock across await
}
// FIX: drop guard before await
async fn good(state: Arc<Mutex<Vec<u32>>>) {
{
let mut guard = state.lock().unwrap();
guard.push(1);
}
do_async_work().await;
}Holding a std::sync::Mutex across an .await point is a common deadlock source. For locks held across awaits, use tokio::sync::Mutex instead.
Pitfalls
1. Loading every concern: A file with no unsafe does not need the unsafe sub-module. Detect concerns first. 2. Treating `unwrap` as a review block: In tests and examples, unwrap is acceptable. In library code paths, it is a finding. Context matters. 3. Skipping the `// SAFETY:` rule: Unsafe blocks without safety comments are the highest-value finding in a Rust review. Always load the unsafe sub-module when any unsafe block is present. 4. Confusing tokio Mutex and std Mutex: They have the same API surface but different blocking behavior. The async sub-module documents when to use each. 5. One review for source and dependencies: Cargo audit and source review are separate concerns with separate tools. Keep them in cargo-patterns.md and this module respectively.
Cross-Reference
See cargo-patterns.md for the build-tool layer and the parent SKILL.md for how Rust modules plug into the hub-and-spoke pattern.
Module Selection Strategies
Overview
Selection strategies determine which modules to load based on context signals, user intent, available resources, and workflow requirements. The goal is to load exactly what's needed while respecting token budgets and MECW constraints.
Core Strategies
1. Intent-Based Selection
Load modules based on detected user goals and explicit requests.
Pattern:
INTENT_MODULE_MAP = {
"git-analysis": ["git-catchup-patterns.md"],
"document-review": ["document-analysis-patterns.md"],
"log-analysis": ["log-analysis-patterns.md"],
"architecture-review": ["architecture-patterns.md", "design-principles.md"]
}
def select_by_intent(user_input, skill_modules):
# Detect intent from keywords, context, explicit requests
detected_intent = detect_intent(user_input)
# Map to modules
return INTENT_MODULE_MAP.get(detected_intent, [])When to Use:
- User provides clear task description
- Skill has distinct workflow paths
- Intent keywords are well-defined
Example:
## Progressive Loading
**Git Analysis**: User mentions "commits", "branches", "git log"
→ Load `modules/git-catchup-patterns.md`
**Document Review**: User mentions "meeting notes", "docs", "markdown"
→ Load `modules/document-analysis-patterns.md`2. Artifact-Based Selection
Load modules based on detected files, systems, or environmental signals.
Pattern:
def select_by_artifacts(detected_files, detected_systems):
modules = []
# File-based detection
if any(f.endswith('.git') for f in detected_files):
modules.append("git-workflow.md")
if any(f.endswith('.py') for f in detected_files):
modules.append("python-analysis.md")
# System-based detection
if "kubernetes" in detected_systems:
modules.append("k8s-patterns.md")
return modulesWhen to Use:
- Different file types require different analysis
- Environment determines workflow
- System presence indicates needed capabilities
Example:
## Progressive Loading
**Python Codebase Detected**: `.py` files, `pyproject.toml`, `requirements.txt`
→ Load `modules/python-testing.md`, `modules/python-packaging.md`
**Rust Codebase Detected**: `.rs` files, `Cargo.toml`
→ Load `modules/rust-review.md`, `modules/cargo-patterns.md`3. Budget-Aware Selection
Load modules within available token budget, prioritizing by importance.
Pattern:
from leyline import MECWMonitor, estimate_tokens
def select_by_budget(available_modules, max_tokens):
monitor = MECWMonitor()
selected = []
total_tokens = 0
# Sort by priority (core → common → edge cases)
prioritized = sort_by_priority(available_modules)
for module in prioritized:
module_cost = estimate_tokens(module.path)
if total_tokens + module_cost <= max_tokens:
selected.append(module)
total_tokens += module_cost
else:
# Skip lower-priority modules if budget exceeded
break
return selectedWhen to Use:
- Context pressure is moderate to high
- Skills have many optional modules
- Need MECW compliance
- Want to prioritize common paths
Example:
## Progressive Loading
**LOW Pressure** (< 30%): Load all relevant modules
**MODERATE Pressure** (30-50%): Load core + common modules only
**HIGH Pressure** (> 50%): Load core modules only, defer rest4. Progressive (Tiered) Selection
Load in stages: minimal core, then expand based on actual needs.
Pattern:
def progressive_select(context, stage="core"):
if stage == "core":
# Always load: essential concepts, integration, exit criteria
return ["core-workflow.md", "integration.md"]
elif stage == "common":
# Load for typical use cases
modules = progressive_select(context, "core")
modules.extend(["common-patterns.md", "examples.md"])
return modules
elif stage == "advanced":
# Load for edge cases, advanced features
modules = progressive_select(context, "common")
modules.extend(["advanced-patterns.md", "troubleshooting.md"])
return modulesWhen to Use:
- Uncertain which modules will be needed
- Want to start fast and expand as needed
- Support both quick tasks and deep dives
Example:
## Progressive Loading
**Stage 1 (Core)**: Always loaded
- Overview, quick start, integration, exit criteria
**Stage 2 (Common)**: Load when basic workflow confirmed
- Common patterns, examples, typical use cases
**Stage 3 (Advanced)**: Load on explicit request or edge case
- Advanced patterns, troubleshooting, optimization techniques5. Mutually-Exclusive Selection
Load one module from a set of alternatives based on context.
Pattern:
def select_mutually_exclusive(context, module_groups):
selected = []
for group in module_groups:
# Only one module from each group
for module in group.modules:
if module.matches_context(context):
selected.append(module)
break # Don't load other modules from this group
return selectedWhen to Use:
- Multiple workflows that never happen together
- Platform-specific implementations
- Version-specific patterns
Example:
## Progressive Loading
**Platform Selection** (mutually exclusive):
- Linux detected → Load `modules/linux-patterns.md`
- macOS detected → Load `modules/macos-patterns.md`
- Windows detected → Load `modules/windows-patterns.md`
**Version Selection** (mutually exclusive):
- Python 3.8-3.10 → Load `modules/legacy-python.md`
- Python 3.11+ → Load `modules/modern-python.md`Combining Strategies
Real-world skills often combine multiple strategies:
def select_modules(context):
# 1. Check budget first
safe_budget = MECWMonitor().get_safe_budget()
# 2. Detect intent and artifacts
intent_modules = select_by_intent(context.user_input)
artifact_modules = select_by_artifacts(context.files)
# 3. Combine and deduplicate
candidate_modules = list(set(intent_modules + artifact_modules))
# 4. Apply budget constraints
selected = select_by_budget(candidate_modules, safe_budget)
# 5. validate core modules always loaded
ensure_core_modules(selected)
return selectedSelection Metadata
Tag modules with selection hints in frontmatter:
---
# Module: git-catchup-patterns.md
module_name: git-catchup-patterns
priority: common
estimated_tokens: 450
triggers:
keywords: [git, commit, branch, diff, log]
artifacts: [.git/, .gitignore]
intents: [git-analysis, catchup, history-review]
mutually_exclusive_with: [document-analysis-patterns, log-analysis-patterns]
requires_budget_minimum: 400
---Context Signal Detection
Common signals for module selection:
User Input Signals
- Keywords: Explicit mentions of technologies, workflows, tasks
- Question patterns: "How do I...", "Show me...", "Analyze..."
- Command requests: Direct requests for specific operations
Environmental Signals
- File presence: Detected files indicate domain
- Directory structure: Project layout suggests patterns
- Tool availability: Installed tools suggest workflows
Session Signals
- Previous modules: What's already loaded
- Task history: What user has been working on
- Error patterns: Repeated failures suggest different module needed
Resource Signals
- Token budget: Available MECW budget
- Context pressure: Current utilization level
- Time constraints: Need for quick vs detailed loading
Best Practices
1. Default to Minimal: Start with smallest useful set, expand on demand 2. Document Triggers: Make selection logic transparent in hub SKILL.md 3. Measure Accuracy: Track which modules are actually used vs loaded 4. Provide Overrides: Let users force-load specific modules 5. Fail Gracefully: If budget insufficient, load core and warn 6. Cache Decisions: Don't re-evaluate for same context repeatedly
Anti-Patterns
Loading Everything: Defeats purpose of progressive loading Complex Selection Logic: If selection is hard to understand, simplify Ignoring Budget: Selection must respect MECW constraints Silent Failures: If module can't load, inform user why Implicit Dependencies: Module loading should be deterministic from context
Integration Points
With MECW Monitoring
from leyline import MECWMonitor
monitor = MECWMonitor()
if monitor.get_pressure_level() == "HIGH":
# Select minimal modules only
modules = select_by_budget(candidates, monitor.get_safe_budget())With Token Estimation
from leyline import estimate_tokens
total_cost = sum(estimate_tokens(m) for m in selected_modules)
if total_cost > safe_budget:
# Re-select with lower priority threshold
selected_modules = select_by_budget(candidates, safe_budget)With Module Loader
from leyline import progressive_load
modules = progressive_load(
skill="my-skill",
context={"intent": "analysis", "artifacts": [".py"]},
strategy="budget-aware",
max_tokens=safe_budget
)Validation Checklist
- [ ] All modules have selection metadata (triggers, priority, cost)
- [ ] Selection logic documented in hub SKILL.md
- [ ] Budget constraints enforced in all strategies
- [ ] Mutually-exclusive groups identified and enforced
- [ ] Core modules always loaded regardless of context
- [ ] Selection deterministic for same context
- [ ] Override mechanism available for explicit requests
Troubleshooting
This module covers progressive-loading patterns for diagnostic work: which diagnostic modules to load when something is broken, how to gather evidence without overloading context, and how to escalate from quick checks to deep investigation. The driving question is "what is the cheapest read that confirms or disconfirms my current hypothesis?".
When This Module Applies
Load this module when:
- A test fails unexpectedly and the cause is not obvious.
- A skill produces wrong output and you need to find why.
- A loaded module fails to apply, or a hook rejects an action.
- The user reports an error and you must reproduce it.
For routine code review without an active failure, load the review-specific modules instead. This module is for active debugging.
Three Diagnostic Tiers
Diagnostic work splits into three tiers by cost. Always start at tier 1 and escalate only when needed.
| Tier | Focus | Module | Token Budget |
|---|---|---|---|
| 1 | Read error output | error-reading.md | 200 |
| 2 | Reproduce locally | reproduction.md | 400 |
| 3 | Bisect or instrument | bisect-and-instrument.md | 600 |
Most failures resolve at tier 1: the error message names the problem. Tier 2 covers cases where the error is opaque or non-deterministic. Tier 3 is for bugs that survive both.
Tier 1: Read the Error Carefully
The cheapest diagnostic is reading the error message and the surrounding context. The tier-1 module documents what to extract.
# Re-run with verbose output and capture full traceback
pytest -xvs path/to/test_file.py::test_name
# For shell scripts, add -x to trace each command
bash -x ./script.sh
# For Python, show the full traceback (not the truncated form)
python -X tracebackshow=longest script.pypytest -x stops at the first failure. -v shows test names. -s disables output capture so print statements appear.
Tier 2: Reproduce in Isolation
If the error is non-obvious or intermittent, reduce to a minimal reproduction. The tier-2 module documents the bisection of inputs.
# Halving strategy: if a 1000-line file fails, try the first
# 500. If that passes, try 500-1000. Halve until you find the
# minimal failing line range.
def reduce_input(input_lines: list[str], test_fn) -> list[str]:
if len(input_lines) <= 1:
return input_lines
mid = len(input_lines) // 2
if test_fn(input_lines[:mid]):
return reduce_input(input_lines[mid:], test_fn)
if test_fn(input_lines[mid:]):
return reduce_input(input_lines[:mid], test_fn)
return input_lines # both halves neededThis is a manual delta-debugging step. Real delta debugging tools (e.g., creduce for C, picireny for Python) automate the process for complex inputs.
Tier 3: Bisect History
For regressions, git bisect finds the introducing commit without manual narrowing.
# Mark current HEAD as bad
git bisect start
git bisect bad
# Mark a known-good commit
git bisect good v1.9.3
# bisect runs the test on each midpoint commit
git bisect run pytest -x path/to/test_file.py::test_name
# When done
git bisect resetbisect run automates the process by running a command at each step. The command must exit 0 for "good" and nonzero for "bad".
Tier 3: Instrument with Logging
If bisect cannot find the cause (the bug existed forever or is non-deterministic), add temporary logging at suspected points.
import logging
import sys
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s %(levelname)s %(name)s %(message)s",
stream=sys.stderr,
)
logger = logging.getLogger(__name__)
def suspect_function(value):
logger.debug("entering with value=%r", value)
result = expensive_compute(value)
logger.debug("returning %r", result)
return resultUse %r formatting for inputs so quotes and types are visible. A debug line that says "entering with value=hello" hides whether the value was the string "hello" or b"hello".
Loading Decision
The hub picks the tier based on signals from the user.
- "Test fails, here's the output": tier 1.
- "It works for me but not in CI": tier 2.
- "It worked yesterday, broken now": tier 3 with bisect.
- "It fails 1 in 10 runs": tier 3 with instrumentation.
Pitfalls
1. Jumping to tier 3 immediately: Bisecting takes minutes when reading the error takes seconds. Always start at tier 1. 2. Removing the original error context: When reproducing, keep the original error output. The reproduction may produce a different error and confuse the diagnosis. 3. Permanent debug logging: Tier-3 logging is temporary. Remove or downgrade to TRACE before merging. 4. Bisecting a flaky test: If the test is non-deterministic, bisect will mark commits randomly as good or bad. Stabilize the test first or skip bisection for that bug class. 5. Reading the wrong error: Long pipelines (CI, hooks, subagent dispatch) show errors from the wrapper, not the root cause. Drill to the deepest stack frame before forming a hypothesis.
Cross-Reference
See performance.md for performance-specific diagnostics and the parent SKILL.md for how troubleshooting modules plug into the hub-and-spoke pattern.
Windows Patterns
This module covers progressive-loading for skills that target Windows as the development or deployment host. The selection question is which Windows-specific modules to load: file paths (drive letters, UNC paths), shells (PowerShell vs cmd), package managers (winget, Chocolatey, Scoop), services, and WSL bridging.
When This Module Applies
Load this module when:
- The active machine reports
Windows_NTfrom$env:OSor
MSYS_NT-* from uname -s.
- The task touches paths with drive letters (
C:\...) or UNC
prefixes (\\server\share).
- The user mentions PowerShell, cmd, winget, or WSL.
- The deployment target is Windows desktop or Windows Server.
For Linux paths and tools, load linux-patterns.md. For macOS, load macos-patterns.md. The three are mutually exclusive per session unless the task is explicitly cross-platform.
Detect Shell and Subsystem First
Windows ships multiple shells with different syntax. The shell detection drives the syntax sub-module.
# PowerShell detection (v5.1+ on Windows by default, v7+ if installed)
$PSVersionTable.PSVersion.Major
# WSL detection from PowerShell
wsl --status
# From inside WSL, detect the host
test -f /proc/sys/kernel/osrelease && \
grep -qi microsoft /proc/sys/kernel/osrelease && echo "WSL"PowerShell 5.1 ships with Windows; PowerShell 7+ is a separate install. Their syntax differs subtly (e.g., null handling, parallel execution). The sub-module documents what works in both.
Loading Map
| Sub-Concern | Module | Token Estimate |
|---|---|---|
| PowerShell scripting | powershell-rules.md | 500 |
| cmd / batch scripting | cmd-rules.md | 300 |
| WSL interop | wsl-bridge.md | 400 |
| Path conventions | windows-paths.md | 300 |
| Package managers | windows-packages.md | 400 |
| Services | windows-services.md | 400 |
The path conventions sub-module is small enough to keep always-loaded when targeting Windows.
Path Conventions
Windows paths differ from POSIX in three ways: drive letters, backslash separators, and case-insensitive comparison.
# User profile (PowerShell)
$env:USERPROFILE # C:\Users\alex
$env:APPDATA # C:\Users\alex\AppData\Roaming
$env:LOCALAPPDATA # C:\Users\alex\AppData\Local
$env:PROGRAMDATA # C:\ProgramData
# Programs
$env:PROGRAMFILES # C:\Program Files
${env:PROGRAMFILES(X86)} # C:\Program Files (x86)For cross-platform Python, Path accepts both separators on Windows but normalizes to backslash on output. Use forward slashes in source code and let Path handle the conversion.
Package Manager Detection
Windows has three popular package managers, each with its own defaults.
# winget (Microsoft, ships with Windows 10+ since 2020)
winget --version
# Chocolatey (community, requires admin install)
choco --version
# Scoop (community, user-scope by default)
scoop --versionFor new tooling, prefer winget because it ships by default. The package sub-module documents install commands for each.
WSL Bridging
WSL2 lets Linux processes run alongside Windows. The bridge sub-module documents the path translation rules.
# Run a WSL command from PowerShell
wsl ls /home/user
# Access WSL files from Windows
\\wsl$\Ubuntu\home\user
# Access Windows files from WSL
/mnt/c/Users/alex/DocumentsFile operations across the boundary are slow. For performance, keep files on the side that uses them most: WSL files for Linux tooling, Windows paths for Windows-native tools.
Concrete Example: PowerShell Path Handling
The PowerShell sub-module documents the path operators.
# Join paths portably
$config = Join-Path $env:APPDATA "myapp\config.toml"
# Test existence
Test-Path $config
# Read text with explicit encoding (UTF-8 without BOM)
$text = Get-Content $config -Raw -Encoding UTF8
# Write with explicit encoding
Set-Content -Path $config -Value $text -Encoding UTF8 -NoNewlineGet-Content -Raw returns the file as one string. Without -Raw, it returns an array of lines, which surprises authors expecting POSIX semantics.
Pitfalls
1. Hard-coding `/` separators in shell scripts: cmd.exe does not accept forward slashes for executables (only arguments). Use Path in Python and Join-Path in PowerShell. 2. Assuming POSIX line endings: Windows files default to CRLF. Reading a Windows file with splitlines() works, but writing back without specifying line endings produces mixed content. 3. Skipping the encoding flag: PowerShell 5.1 defaults to UTF-16 LE for Out-File. Always specify -Encoding UTF8 for portable output. 4. Confusing PowerShell 5 and 7: Some cmdlets behave differently. The PowerShell sub-module documents the major differences. 5. Treating WSL as fully Linux: WSL has caveats around file permissions, network namespaces, and /proc content. Test cross-boundary workflows explicitly.
Cross-Reference
See linux-patterns.md and macos-patterns.md for the other platforms in the mutually-exclusive group, and the parent SKILL.md for the platform-selection contract.
Related skills
FAQ
Is Progressive Loading safe to install?
skills.sh reports 3 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.