Context Engineering Collection

Book SFT Pipeline

A complete system for converting books into SFT datasets and training style-transfer models. This skill teaches the pipeline from raw ePub to a model that writes in any author's voice.

When to Activate

Activate this skill when:

• Building fine-tuning datasets from literary works
• Creating author-voice or style-transfer models
• Preparing training data for Tinker or similar SFT platforms
• Designing text segmentation pipelines for long-form content
• Training small models (8B or less) on limited data

Core Concepts

The Three Pillars of Book SFT

1. Intelligent Segmentation Text chunks must be semantically coherent. Breaking mid-sentence teaches the model to produce fragmented output. Target: 150-400 words per chunk, always at natural boundaries.

2. Diverse Instruction Generation Use multiple prompt templates and system prompts to prevent overfitting. A single prompt style leads to memorization. Use 15+ prompt templates with 5+ system prompts.

3. Style Over Content The goal is learning the author's rhythm and vocabulary patterns, not memorizing plots. Synthetic instructions describe what happens without quoting the text.

Pipeline Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    ORCHESTRATOR AGENT                           │
│  Coordinates pipeline phases, manages state, handles failures   │
└──────────────────────┬──────────────────────────────────────────┘
                       │
       ┌───────────────┼───────────────┬───────────────┐
       ▼               ▼               ▼               ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│  EXTRACTION  │ │ SEGMENTATION │ │  INSTRUCTION │ │   DATASET    │
│    AGENT     │ │    AGENT     │ │    AGENT     │ │   BUILDER    │
│ ePub → Text  │ │ Text → Chunks│ │ Chunks →     │ │ Pairs →      │
│              │ │ 150-400 words│ │ Prompts      │ │ JSONL        │
└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
                       │
       ┌───────────────┴───────────────┐
       ▼                               ▼
┌──────────────┐               ┌──────────────┐
│   TRAINING   │               │  VALIDATION  │
│    AGENT     │               │    AGENT     │
│ LoRA on      │               │ AI detector  │
│ Tinker       │               │ Originality  │
└──────────────┘               └──────────────┘

Phase 1: Text Extraction

Critical Rules

1. Always source ePub over PDF - OCR errors become learned patterns 2. Use paragraph-level extraction - Extract from <p> tags to preserve breaks 3. Remove front/back matter - Copyright and TOC pollute the dataset

# Extract text from ePub paragraphs
from epub2 import EPub
from bs4 import BeautifulSoup

def extract_epub(path):
    book = EPub(path)
    chapters = []
    for item in book.flow:
        html = book.get_chapter(item.id)
        soup = BeautifulSoup(html, 'html.parser')
        paragraphs = [p.get_text().strip() for p in soup.find_all('p')]
        chapters.append('\n\n'.join(p for p in paragraphs if p))
    return '\n\n'.join(chapters)

Phase 2: Intelligent Segmentation

Smaller Chunks + Overlap

Smaller chunks (150-400 words) produce more training examples and better style transfer than larger chunks (250-650).

def segment(text, min_words=150, max_words=400):
    paragraphs = text.split('\n\n')
    chunks, buffer, buffer_words = [], [], 0
    
    for para in paragraphs:
        words = len(para.split())
        if buffer_words + words > max_words and buffer_words >= min_words:
            chunks.append('\n\n'.join(buffer))
            # Keep last paragraph for overlap
            buffer = [buffer[-1], para] if buffer else [para]
            buffer_words = sum(len(p.split()) for p in buffer)
        else:
            buffer.append(para)
            buffer_words += words
    
    if buffer:
        chunks.append('\n\n'.join(buffer))
    return chunks

Expected Results

For an 86,000-word book:

• Old method (250-650 words): ~150 chunks
• New method (150-400 + overlap): ~300 chunks
• With 2 variants per chunk: 600+ training examples

Phase 3: Diverse Instruction Generation

The Key Insight

Using a single prompt template causes memorization. Diverse templates teach the underlying style.

SYSTEM_PROMPTS = [
    "You are an expert creative writer capable of emulating specific literary styles.",
    "You are a literary writer with deep knowledge of classic prose styles.",
    "You are a creative writer skilled at emulating distinctive authorial voices.",
    "You write prose that captures the essence of modernist literature.",
    "You are a talented writer who can channel classic American authors.",
]

PROMPT_TEMPLATES = [
    "Write a passage in the style of {author}: {desc}",
    "Channel {author}'s voice to write about: {desc}",
    "In {author}'s distinctive prose style, describe: {desc}",
    "Write this scene as {author} would have: {desc}",
    "Using {author}'s repetitive technique, describe: {desc}",
    "Capture the rhythm of {author} in this passage: {desc}",
    "Write like {author}: {desc}",
    "In the voice of {author}, write: {desc}",
    "This is a literary exercise. Write like {author}: {desc}",
    "Can you write in {author}'s style? {desc}",
]

Instruction Generation

INSTRUCTION_PROMPT = """Describe what is happening in this excerpt in 2-3 sentences.
Focus on: characters present, actions, emotions, setting.
Do NOT quote the text directly.

Excerpt:
{text}
"""

# Use a fast, cheap LLM (e.g., Gemini Flash)
instruction = llm_call(INSTRUCTION_PROMPT.format(text=chunk))

Phase 4: Dataset Construction

Message Format

{
    "messages": [
        {"role": "system", "content": "You are an expert creative writer..."},
        {"role": "user", "content": "Write in the style of Author: Scene description..."},
        {"role": "assistant", "content": "The actual book text from chunk..."}
    ]
}

Multiple Variants Per Chunk

def build_examples(chunk, instruction, author, variants=2):
    examples = []
    for i in range(variants):
        system = SYSTEM_PROMPTS[i % len(SYSTEM_PROMPTS)]
        template = PROMPT_TEMPLATES[(chunk.id + i) % len(PROMPT_TEMPLATES)]
        user = template.format(author=author, desc=instruction)
        examples.append({"messages": [
            {"role": "system", "content": system},
            {"role": "user", "content": user},
            {"role": "assistant", "content": chunk.text}
        ]})
    return examples

Phase 5: LoRA Training on Tinker

Configuration

CONFIG = {
    "model_name": "Qwen/Qwen3-8B-Base",  # Base, not instruct
    "lora_rank": 32,                      # 352MB adapter
    "learning_rate": 5e-4,                # Higher for LoRA
    "batch_size": 4,
    "epochs": 3,
}

Why Base Model?

Use base (pretrained) models, not instruction-tuned versions:

• Base models are more malleable for new styles
• Instruct models have patterns that resist overwriting
• Style is a low-level pattern that base models capture better

Training Loop

import tinker
from tinker import types

training_client = await service_client.create_lora_training_client_async(
    base_model="Qwen/Qwen3-8B-Base",
    rank=32
)

for epoch in range(3):
    for batch in batches:
        await training_client.forward_backward_async(batch, loss_fn="cross_entropy")
        await training_client.optim_step_async(types.AdamParams(learning_rate=5e-4))

result = await training_client.save_weights_for_sampler_async(name="final")

Phase 6: Validation

Modern Scenario Test

Test with scenarios that couldn't exist in the original book:

TEST_PROMPTS = [
    "Write about a barista making lattes",
    "Describe lovers communicating through text messages",
    "Write about someone anxious about climate change",
]

If the model applies style markers to modern scenarios, it learned style, not content.

Originality Verification

# Search training data for output phrases
grep "specific phrase from output" dataset.jsonl
# Should return: No matches

AI Detector Testing

Test outputs with GPTZero, Pangram, or ZeroGPT.

Known Issues and Solutions

Character Name Leakage

Symptom: Model uses original character names in new scenarios. Cause: Limited name diversity from one book. Solution: Train on multiple books or add synthetic examples.

Model Parrots Exact Phrases

Symptom: Outputs contain exact sentences from training data. Cause: Too few prompt variations or too many epochs. Solution: Use 15+ templates, limit to 3 epochs.

Fragmented Outputs

Symptom: Sentences feel incomplete. Cause: Poor segmentation breaking mid-thought. Solution: Always break at paragraph boundaries.

Guidelines

1. Always source ePub over PDF - OCR errors become learned patterns 2. Never break mid-sentence - Boundaries must be grammatically complete 3. Use diverse prompts - 15+ templates, 5+ system prompts 4. Use base models - Not instruct versions 5. Use smaller chunks - 150-400 words for more examples 6. Reserve test set - 50 examples minimum 7. Test on modern scenarios - Proves style transfer vs memorization 8. Verify originality - Grep training data for output phrases

Expected Results

Cost Estimate

Integration with Context Engineering Skills

This example applies several skills from the Agent Skills for Context Engineering collection:

project-development

The pipeline follows the staged, idempotent architecture pattern:

• Acquire: Extract text from ePub
• Prepare: Segment into training chunks
• Process: Generate synthetic instructions
• Parse: Build message format
• Render: Output Tinker-compatible JSONL
• Train: LoRA fine-tuning
• Validate: Modern scenario testing

Each phase is resumable and produces intermediate artifacts for debugging.

context-compression

Segmentation is a form of context compression for training. The core insight from context-compression applies: information density matters more than information quantity. Smaller, coherent chunks (150-400 words) produce better style transfer than larger, diluted chunks.

The two-tier strategy mirrors context compression evaluation:

• Tier 1: Fast, deterministic compression
• Tier 2: LLM-assisted for edge cases

multi-agent-patterns

The pipeline uses the supervisor/orchestrator pattern:

• Orchestrator coordinates phases and manages state
• Specialized agents (Extraction, Segmentation, Instruction, Builder) have isolated contexts
• Each agent receives only the information needed for its task

This matches the principle that sub-agents exist primarily to isolate context rather than simulate roles.

evaluation

Validation follows the end-state evaluation pattern:

• Functional testing: Does output match expected style markers?
• Originality verification: Is content genuinely generated?
• External validation: AI detector scores

The "modern scenario" test is a form of out-of-distribution evaluation that proves generalization.

context-fundamentals

Prompt diversity prevents attention collapse on single patterns. When training with identical prompt structures, the model memorizes the instruction-response mapping. Diverse templates force attention across the style patterns themselves.

References

Internal references:

• Segmentation Strategies - Text chunking patterns
• Tinker Format Specification - Datum structure
• Tinker API Documentation - Full API reference

Related skills from Agent Skills for Context Engineering:

• project-development - Pipeline architecture patterns
• context-compression - Compression strategies
• multi-agent-patterns - Agent coordination
• evaluation - Evaluation frameworks
• context-fundamentals - Attention and information density

External resources:

• Research Paper - Chakrabarty et al. 2025
• Dataset on Hugging Face
• Gertrude Stein Case Study - Complete working example

---

Skill Metadata

Created: 2025-12-26 Last Updated: 2025-12-28 Author: Muratcan Koylan Version: 2.0.0 Standalone: Yes (separate from main context-engineering collection)

AGENTS.md

Workspace memory for agents collaborating on this repository. Keep entries durable and broadly applicable; one-off task state belongs in chat or in a run thread, not here.

Learned User Preferences

• For autonomous research and repo-improvement work in this workspace, prefer proceeding through concrete research loops, subagents, validation, and edits when the scope is clear rather than asking broad process questions.
• Avoid stale regex or keyword-list heuristics in skills and scripts; prefer mechanism-level criteria, rubrics, and evidence-backed validation.
• Never push to GitHub or merge a PR without explicit user approval. Preparing branches, commits, and PRs is permitted only when the user has approved that specific action.
• Tone is technical CTO: direct, no marketing language, no exclamation marks, no emojis, no em dashes. State trade-offs and complexity upfront.
• When the scope spans multiple architectural decisions or irreversible changes, propose a plan first instead of executing.
• For benchmarks and evaluation work, hold to research-paper-grade methodology (statistical discipline, bias mitigation, ablations, reproducibility) over speed. Don't rush.

Learned Workspace Facts

• This repo is an autonomous research-to-skill organization. External AI research is curated through rubrics and distilled into context-engineering and harness-engineering skill updates.
• researcher/ is repo-native and file-based so agents can resume, audit, validate, and prepare PR-ready skill changes without a hosted scheduler.
• Per-run state lives in researcher/runs/<run-id>/run-state.json with explicit transitions (initialized -> retrieved -> evaluated -> proposed -> novelty_checked -> validated -> pr_ready -> closed). Use research_loop.py subcommands to advance state, never hand-edit run-state.json.
• Repo health (validate_repo.py) and per-run readiness (validate_run.py) are different questions. CI runs validate_repo.py --strict, run_benchmarks.py, and check_activation_cases.py on every PR via .github/workflows/validate.yml.
• The mechanism registry (researcher/mechanisms/registry.jsonl) is the encyclopedia backbone. Promotion is gated by research_loop.py promote-mechanisms with a recorded reviewer; ledgers live under researcher/mechanisms/ledgers/.
• Claim provenance for numeric or volatile claims lives in researcher/claims/index.jsonl. Add an entry for any new benchmark or volatility-sensitive claim.
• The corpus index (researcher/corpus/index.json) is the machine-readable map of skills, activation scenarios, mechanisms, and claims. Update it when adding or restructuring skills.
• The continuous loop (researcher/scripts/loop_*.py) runs from launchd via researcher/orchestration/launchd/. It never invokes paid LLMs; HTTP retrieval is stdlib-only with a 1.5 MB cap and a 30-second timeout.
• Runtime state is not committed: researcher/queue/*.jsonl, researcher/queue/.locks/, researcher/reports/{logs,snapshots,loop-events.jsonl,loop-failures.jsonl,status.md,parked-review.md}, and researcher/runs/*/ are gitignored. The seed run 20260515-035228-executable-autonomous-research-frameworks is the only committed run; it is closed as reference-only and serves as a worked example.
• The current published version is 2.3.0 across .claude-plugin/marketplace.json, .plugin/plugin.json, and root SKILL.md. There are 15 skills (latent-briefing covers KV cache sharing between agents).
• Detailed lessons from building the researcher OS live in researcher/insights/auto-research-experiment.md (engineering rationale) and researcher/insights/how-we-built-this.md (project narrative and sharing templates); read both before extending the harness or writing release-facing prose.
• Benchmarks are staged in researcher/benchmarks/: Stage 0 deterministic harness (shipped), Stage 1 per-skill health via researcher/scripts/skill_health.py (shipped; output researcher/reports/skill-health.json is gitignored), Stage 2 router (shipped; results in researcher/benchmarks/router/results-published/), Stage 3 effectiveness (scaffolded, one task built), Stage 4 composition (future). researcher/benchmarks/PLAN.md is the methodology source of truth.
• Current corpus-hardening baseline: 16 accepted mechanisms, 12 provenance-tracked claims, 19 activation cases, and strict skill-health score 0.9117 with 0 flagged skills. Do not describe a skill improvement as complete unless the prose, mechanism registry, claim index, corpus index, activation fixtures, and validators all agree.
• Benchmark execution uses the Cursor SDK runner at researcher/benchmarks/sdk-runner/ (TypeScript, @cursor/sdk 1.0.13). The runner supports --concurrency N, --no-resume, per-run progress logging, format-failure retry, and worst-case retry-aware cost forecasting; default behavior is to resume by skipping plan items that already have result files. Result artifacts under researcher/benchmarks/{router,effectiveness}/results/ and history JSONLs (router-history.jsonl, effectiveness-history.jsonl) are gitignored.
• Published Stage 2 router-benchmark results: researcher/benchmarks/router/results-published/2026-05-15.md (baseline), researcher/benchmarks/router/results-published/2026-05-15-v2.md (post-rewrite with delta-vs-baseline table), and researcher/benchmarks/router/results-published/2026-05-19.md (post-corpus-hardening validation: 600/600 usable records, 0 format failures, top-1 Gemini 0.920 / Composer 0.913 / GPT-5.5 0.913 / Claude Opus 4.7 0.840). Headline finding: targeted description rewrites moved context-fundamentals top-1 by +23.4pp and project-development top-1 to 1.000; corpus-wide hardening did not cause broad routing collapse.

Repository Operating Defaults

• Deterministic checks before model judges. Always run validate_repo.py --strict before claiming a change is complete.
• Adversarial benchmarks before declaring the harness safe. Add a scenario when a new failure mode is discovered.
• Append-only ledgers for accepted and rejected mechanisms so future agents do not rediscover failed paths.
• Atomic writes (tempfile + os.replace) and fcntl locks for any shared file the loop touches.
• Live execution is the highest-signal validation for orchestration code; smoke-test changes against the actual loop before declaring them safe.
• Cursor SDK is the only paid-API surface allowed for benchmarks. Privacy Mode required, apiKey passed explicitly per call, never settingSources: "all" in benchmarks (use [] for control, ["project"] with a curated .cursor/skills/ for ablation). Cost gates (--max-runs, --max-budget-usd, or --dry-run) must be set before any SDK call.
• Description quality is measurable. When changing skill activation descriptions, re-run the router benchmark with the same seed and fixture and publish the delta. Aggregate accuracy is a misleading unit; per-skill effect sizes and the confusion matrix are the right view.
• A skill is a multi-surface artifact. Changing the frontmatter description is not enough; the SKILL.md body When to Activate and Integration sections must be audited the same day so the body does not contradict the description that routed the agent to it. The router benchmark only sees descriptions (settingSources: []) and cannot catch body inconsistencies; only Stage 3 effectiveness benchmarks (which actually load skill bodies) measure body-alignment impact.
• Any runner that calls a paid API in a loop must have three features before execution: bounded parallelism via --concurrency, resume capability via results-folder scan, and per-run progress logging that surfaces stalls inside one call's duration.
• API keys provided in chat should be considered exposed; rotate immediately after use. Runner enforces this via apiKeyFingerprint() which only logs the last 4 characters.

Changelog

All notable changes to this project are documented here. Versions follow semantic versioning where practical, with skill content treated as data.

[2.3.0] - 2026-05-15

First release with measured benchmark results across four frontier models, closing the loop from "we wrote skill descriptions" to "we proved they route correctly."

Added

Stage 2 router benchmark, executed end-to-end

• 600 of 600 runs completed across composer-2, claude-opus-4-7, gpt-5.5, gemini-3.1-pro at 3 replications per (prompt, model). Initial v2.2.0 baseline at researcher/benchmarks/router/results-published/2026-05-15.md (566 of 600 due to the v1 runner dying); updated run after the description fixes at researcher/benchmarks/router/results-published/2026-05-15-v2.md with full delta-vs-baseline table.
• 50 ground-truth router prompts at researcher/benchmarks/router/prompts.jsonl covering positive controls, adversarial boundary pairs, combined-skill prompts, and negative controls.
• researcher/scripts/render_router_report.py with --baseline flag for delta reports.
• researcher/benchmarks/router/results-published/README.md explains the committed-summary vs gitignored-raw split.

Hardened SDK runner (researcher/benchmarks/sdk-runner/src/)

• Resume: scans the destination directory on startup and skips plan items that already have a per-run JSON. A killed sweep can be picked up exactly where it stopped; no wasted credits, no duplicate runs.
• Bounded parallelism: --concurrency N runs N agent calls simultaneously. Cuts the 600-run sweep from ~60 minutes (sequential) to ~15 minutes (concurrency=4) with identical correctness.
• Per-run progress logging: every completed run prints [N/total] model prompt rep=R status durationMs T1 ETA=duration. The v1 sweep silently stalled at 566 of 600 with no signal; the v2 sweep would have surfaced the cause immediately.
• Format-failure retry: transient empty or unparsable SDK responses are retried once before being recorded as format failures. This was added after the May 19 sweep produced transient blank outputs that succeeded on rerun.
• runConcurrently helper in common.ts, reusable by future runners.

Skill description rewrites (data-driven)

Targeted at the two routing failures the v2.2.0 baseline benchmark surfaced:

• context-fundamentals: rewrote to be unambiguously about conceptual foundations and explicitly route operational work to the specialized skills. Top-1 rate went from 0.255 to 0.489 (+23.4pp).
• project-development: tightened with explicit cross-references to tool-design. Top-1 rate went from 0.750 to 1.000 (now perfect routing).
• tool-design: tightened with explicit cross-references to project-development. Top-1 rate went from 0.729 to 0.807 (+7.8pp).

Skill body alignment with new descriptions

The router benchmark only sees frontmatter description because settingSources: [] excludes the SKILL.md body. The first description rewrite pass left the bodies (When to Activate, Practical Guidance, Integration) claiming the broader pre-rewrite scope, which would have steered the agent toward operational work the moment the skill actually activated in production. Aligned the bodies in a follow-up pass:

• context-fundamentals body: rewrote When to Activate to list conceptual triggers and explicit do-not-activate routing; removed the operational File-System-Based Access and Context Budgeting practical-guidance sections (owned by filesystem-context and context-optimization respectively); replaced with conceptual application advice plus a reading-order recommendation for new contributors; rewrote Integration as an explicit routing map across all 14 sibling skills. Internal version bump 2.0.0 -> 2.1.0.
• tool-design body: rewrote When to Activate to anchor on the unit of work (single tool or tool catalog) and listed adjacent decisions owned by project-development, multi-agent-patterns, context-optimization; rewrote Integration with explicit routing reasons. Internal version bump 2.0.0 -> 2.1.0.
• project-development body: rewrote When to Activate to anchor on project shape and pipeline; listed adjacent decisions owned by tool-design, context-optimization, multi-agent-patterns, harness-engineering; rewrote Integration with explicit routing reasons. Internal version bump 1.1.0 -> 1.2.0.

The body changes do not affect router-benchmark numbers (the router sees only descriptions) but they do affect what the agent loads when these skills activate. Stage 3 (effectiveness benchmark, which loads full bodies) is the right place to measure the impact of this alignment.

Corpus-wide skill hardening pass

After the targeted three-skill body alignment, every published skill was audited against the same standard: explicit ownership boundary, Do not activate routing, executable practical guidance, examples, gotchas, integration boundaries, mechanism coverage, claim provenance, and activation fixtures.

• Updated all 15 skill bodies with scoped improvements, including structural fixes for bdi-mental-states and hosted-agents, stronger negative routing across older skills, clearer examples for context failure modes, and claim-backed wording for volatile benchmark statements.
• Expanded researcher/mechanisms/registry.jsonl from 5 to 16 accepted mechanisms so every published skill owns at least one machine-readable behavior pattern.
• Expanded claim provenance from 6 to 12 records and replaced vague run-summary sources with concrete repo paths for BrowseComp, RULER/lost-in-middle, compression, d0, Latent Briefing, memory, and tool-output claims.
• Expanded activation regression coverage from 14 to 19 cases so every skill has deterministic routing coverage, including bdi-mental-states, context-degradation, hosted-agents, latent-briefing, and multi-agent-patterns.
• Tightened validate_repo.py --strict so Core Concepts, Practical Guidance, Examples, References, and explicit non-activation boundaries are now enforced rather than optional.
• Updated template/SKILL.md with the new corpus-wide standard: body/frontmatter alignment, mechanism registration, executable guidance, and claim-* provenance for volatile claims.
• Re-ran the no-API gates after the pass: validate_repo.py --strict passed with 0 errors / 0 warnings; skill_health.py --strict --no-history improved from corpus score 0.8111 / 2 flagged skills to 0.9117 / 0 flagged skills; check_activation_cases.py passed 19/19; run_benchmarks.py passed 3 checks and 7 adversarial scenarios.
• Re-ran the paid Stage 2 router benchmark after the corpus-wide pass: 600/600 usable records, 0 format failures after retrying transient format failures, published at researcher/benchmarks/router/results-published/2026-05-19.md. Per-model top-1: Gemini 0.920, Composer 0.913, GPT-5.5 0.913, Claude Opus 4.7 0.840. Remaining failures are concentrated in known ambiguous/negative-control prompts (p046, p048) and the context-fundamentals catch-all boundary.

Eleven new boundary regression cases

researcher/fixtures/activation-cases.jsonl grew from 8 to 19 cases. The first six new cases target specific confusions observed in the v2.2.0 baseline:

• activation-fundamentals-vs-degradation, activation-fundamentals-onboarding, activation-fundamentals-vs-optimization
• activation-tool-vs-project-structured-output, activation-tool-individual-tool, activation-tool-consolidation

These act as a tripwire so any future description change is held accountable.

The corpus-wide pass added five more cases for previously uncovered skills:

• activation-bdi-vs-memory
• activation-degradation-poisoning
• activation-hosted-vs-harness
• activation-latent-briefing-vs-memory
• activation-multi-agent-topology

Stage 1 skill health (still no API cost)

• researcher/scripts/skill_health.py: per-skill structural scoring. Initial corpus baseline: 0.8111 aggregate, 2 of 15 skills flagged (bdi-mental-states for missing required section, hosted-agents for multiple structural issues). After the corpus-wide hardening pass: 0.9117 aggregate, 0 flagged skills.
• Output at researcher/reports/skill-health.json (gitignored runtime artifact) + optional append to skill-health-history.jsonl.

Changed

• Version bumped 2.2.0 -> 2.3.0 across .claude-plugin/marketplace.json, .plugin/plugin.json, root SKILL.md.
• researcher/benchmarks/PLAN.md status table reflects Stage 0/1/2 shipped, Stage 3/4 still scaffolded.

Headline measured results

Per-model top-1 accuracy (baseline -> new descriptions, 600-run sweep at seed=1, fixture sha 8f974d9):

Per-skill top-1 rate change for the three skills targeted by description rewrites:

Format compliance: 99.5% (3 failures, all Gemini). Latency: Gemini ~9.1s median, others 3.3-4.2s. Total sweep cost approximately 7.20 USD against the 15 USD budget cap.

Honest scope caveats

• context-fundamentals improved a lot but is still the weakest skill (0.489 top-1). Remaining failures route to project-development for generic onboarding prompts. One more description pass may push it past 0.75.
• Two prompts remain at 0.00 across all models: p046 (Python reformatting, negative control) and p048 (evaluate KV compaction, genuinely ambiguous). Should be re-labeled or removed from positive-routing tests.
• advanced-evaluation looks regressed (-18.3pp) but is largely an artifact of the v2.2.0 baseline missing 11 attempts when the runner died at 566/600. Absolute correct count: 48 baseline -> 47 new.
• Stage 3 (real agent tasks with and without skills loaded) is still scaffolded but not executed; that is the next investment.
• No LLM-judge adapter for the run state machine. No automated source discovery beyond manual seed.

[2.2.0] - 2026-05-15

Added

Researcher operating system

• Mechanism registry (researcher/mechanisms/registry.jsonl) seeded with five accepted mechanisms (locked-editable-surfaces, durable-research-thread, deterministic-first-validation, structured-novelty-gate, pairwise-skill-revision).
• Mechanism ledgers (researcher/mechanisms/ledgers/accepted.jsonl, rejected.jsonl) for append-only promotion events.
• Claim provenance (researcher/claims/index.jsonl) for six volatile or benchmark-backed claims across evaluation, multi-agent-patterns, context-optimization, memory-systems, advanced-evaluation, and harness-engineering.
• Corpus index (researcher/corpus/index.json) mapping skills to activation scenarios, mechanism IDs, and claim IDs.
• Activation regression fixtures (researcher/fixtures/activation-cases.jsonl) covering high-risk skill-boundary pairs.
• Adversarial benchmark harness (researcher/benchmarks/scenarios/adversarial.jsonl + goldens) with seven scenarios that try to game the loop.
• Benchmark history (researcher/reports/benchmark-history.jsonl) for longitudinal trend tracking.
• Pairwise revision rubric and script (researcher/rubrics/pairwise-skill-revision.md, researcher/scripts/compare_skill_revisions.py).
• Run state machine in run-state.json with explicit transitions: initialized -> retrieved -> evaluated -> proposed -> novelty_checked -> validated -> pr_ready -> closed.

Continuous loop

• Queue infrastructure (researcher/queue/): inbox, parked, done, quarantine.
• Orchestration config (researcher/orchestration/config.json) with daily/active/parked/failure budgets.
• Discovery feeder (researcher/scripts/loop_discover.py) reading from researcher/discovery/manual-seed.jsonl.
• Loop step orchestrator (researcher/scripts/loop_step.py) that reaps closed runs, pulls from inbox, retrieves via stdlib urllib, and parks at human-review gates.
• Daily ops (researcher/scripts/loop_daily.py) running validators, benchmarks, activation cases, and writing dated snapshots.
• Status dashboard (researcher/scripts/loop_status.py) plus parked-review surface.
• launchd service definitions (researcher/orchestration/launchd/) with install/uninstall scripts and per-script wrappers.
• Continuous operation runbook (researcher/runbooks/continuous-operation.md).

Scripts

• researcher/scripts/validate_run.py: per-run publish readiness, skips closed runs.
• researcher/scripts/research_loop.py subcommands: retrieve, evaluate, propose, novelty, validate-run, pr-ready, close, promote-mechanisms.
• researcher/scripts/check_activation_cases.py: deterministic activation regression checker.
• researcher/scripts/run_benchmarks.py: runs deterministic gates, scenarios, optional history recording.
• researcher/scripts/loop_common.py: shared atomic-write helpers and fcntl locks.

CI

• .github/workflows/validate.yml runs validate_repo.py --strict, run_benchmarks.py, check_activation_cases.py, and Python compile checks on every push and PR.

Changed

• Skill activation surface refactored from keyword triggers to task-boundary descriptions in frontmatter and README. Affected: all 14 published skills plus the example skills in examples/.
• validate_repo.py hardened: duplicate JSON keys, exact doc sync, rubric IDs, run artifacts, registry schema, evidence paths, partial-retrieval approvals, root-level raw provenance, claims schema, corpus index consistency, activation cases, benchmark scenarios.
• novelty_check.py now compares mechanism registry overlap as the primary signal, with corpus overlap secondary.
• Mechanism registry evidence may now reference claim IDs from researcher/claims/index.jsonl in addition to URLs and repo paths.

Hardened

• All queue mutations use atomic temp-file os.replace and fcntl exclusive locks scoped per queue family.
• read_jsonl is tolerant: malformed lines quarantine to researcher/reports/jsonl-quarantine/ rather than crashing the loop.
• fetch_url allows only http(s):// and re-checks scheme after redirect.
• Closed runs are automatically reaped from parked.jsonl and recorded in done.jsonl.
• Inbox lock now held through init_run so concurrent loop_step invocations cannot exceed budgets.
• URL deduplication normalizes case before hashing.

Repository policy

• Active research runs under researcher/runs/*/ are runtime state and not committed. The seed run 20260515-035228-executable-autonomous-research-frameworks is kept as a worked-example fixture.
• Runtime queue and report files (researcher/queue/*.jsonl, researcher/reports/{logs,snapshots,loop-events.jsonl,loop-failures.jsonl,status.md,parked-review.md}, researcher/queue/.locks/) are gitignored.

Out of scope for 2.2.0

• LLM-judge adapters for advancing retrieved -> evaluated automatically.
• Automated source discovery beyond the manual seed file (Parallel deep research and web search adapters are placeholders behind config toggles).
• Log rotation; benchmark history pruning.

[2.1.0] - 2026-05-14

Added

• harness-engineering skill: locked/editable surface model, durable threads, novelty gates, rollback, human approval boundaries.
• researcher/ directory v1: source registry, content/skill/harness rubrics, source-evaluation JSON template, skill-proposal template, autonomous research loop runbook, PR readiness runbook.

[2.0.0] - earlier

Baseline corpus of 13 skills distributed as a single Claude Code plugin.

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

Agent Skills for Context Engineering: an open collection of 15 Agent Skills teaching context engineering and harness engineering principles for production AI agent systems. Skills are platform-agnostic (Claude Code, Cursor, GitHub Copilot, any Open Plugins-conformant tool). v2.2.0 ships a file-based researcher operating system with deterministic gates and a continuous loop.

Context engineering is the discipline of curating everything that enters a model's context window (system prompts, tool definitions, retrieved documents, message history, tool outputs) to maximize signal within limited attention budget.

Repository Structure

• skills/ - 15 skill directories, each containing a SKILL.md with YAML frontmatter (name, description) and optional references/ and scripts/ subdirectories
• examples/ - 5 complete demonstration projects (digital-brain-skill, llm-as-judge-skills, book-sft-pipeline, x-to-book-system, interleaved-thinking)
• docs/ - Research materials and reference documentation
• researcher/ - File-based research-to-skill operating system: rubrics, mechanism registry, claim provenance, corpus index, run state machine, adversarial benchmarks, continuous loop, launchd service definitions
• template/SKILL.md - Canonical skill template (use when creating new skills)
• SKILL.md (root) - Collection-level metadata and skill map
• .claude-plugin/marketplace.json - Claude Code marketplace manifest (single bundled plugin, v2.2.0)
• .plugin/plugin.json - Open Plugins format manifest (v2.2.0)

Build & Test Commands

No top-level build system. Repo-level gates and per-project tooling below.

Top-level deterministic gates (run on every PR via CI)

python3 researcher/scripts/validate_repo.py --strict       # corpus structure, manifests, rubric math, mechanism registry, claims, corpus index, activation cases, benchmark scenarios, run artifacts
python3 researcher/scripts/skill_health.py --strict --no-history  # deterministic skill-body quality gate
python3 researcher/scripts/run_benchmarks.py               # adversarial benchmark harness + repo + activation gates
python3 researcher/scripts/check_activation_cases.py       # skill-boundary regression fixtures

Per-run readiness (active runs only)

python3 researcher/scripts/validate_run.py --run-dir researcher/runs/<run-id>

Continuous loop (manual or launchd)

python3 researcher/scripts/loop_discover.py
python3 researcher/scripts/loop_step.py --allow-fetch
python3 researcher/scripts/loop_daily.py
python3 researcher/scripts/loop_status.py

researcher/orchestration/launchd/install.sh    # macOS daemon
researcher/orchestration/launchd/uninstall.sh

Example projects

examples/llm-as-judge-skills (TypeScript, Node >= 18)

cd examples/llm-as-judge-skills
npm install
npm run build        # tsc
npm test             # vitest (19 tests)
npm run lint         # eslint
npm run format       # prettier
npm run typecheck    # tsc --noEmit

examples/interleaved-thinking (Python >= 3.10)

cd examples/interleaved-thinking
pip install -e ".[dev]"
pytest               # pytest + pytest-asyncio
ruff check .         # linting (100 char line length)

examples/digital-brain-skill (Node.js)

cd examples/digital-brain-skill
npm run setup
npm run weekly-review
npm run content-ideas
npm run stale-contacts

Skill Authoring Rules

When creating or editing skills:

1. SKILL.md must stay under 500 lines: move detailed content to references/ directory 2. YAML frontmatter is required: must include name and description fields 3. Folder naming: lowercase with hyphens (e.g., context-fundamentals) 4. Write in third person: descriptions are injected into system prompts; inconsistent POV causes discovery issues 5. Platform-agnostic: no vendor-locked examples or platform-specific tool names without abstraction 6. Token-conscious: challenge each paragraph and assume an advanced audience 7. Body standard: include When to Activate, Core Concepts, Practical Guidance, Examples, Guidelines, Gotchas, Integration, and References 8. Explicit boundaries: every When to Activate section needs positive triggers plus a Do not activate block routing adjacent work to the right skill 9. Include a Gotchas section: experience-derived failure modes are the highest-signal content in any skill 10. Update root README.md when adding new skills 11. Update marketplace/plugin manifests when adding skills (.claude-plugin/marketplace.json, .plugin/plugin.json) 12. Update the corpus index (researcher/corpus/index.json) to map the new skill to activation scenarios, mechanism IDs, and claim IDs 13. Update mechanisms and claims: add registry entries for reusable behavior changes and claim-* provenance for numeric, benchmark, volatile, or vendor-performance claims 14. Run `validate_repo.py --strict`, `skill_health.py --strict --no-history`, `check_activation_cases.py`, and `run_benchmarks.py` before committing skill changes

Researcher OS Rules

When working through the researcher operating system:

1. Initialize runs via `research_loop.py init`: it creates run-state.json, queue entry, thread log, source evaluation scaffold, and mechanism proposal template 2. Advance state explicitly: use retrieve, evaluate, propose, novelty, validate-run, pr-ready, close subcommands; do not edit run-state.json by hand 3. Promote mechanisms only after run readiness: research_loop.py promote-mechanisms requires --reviewed-by and a passing run-readiness check 4. Add claim provenance to researcher/claims/index.jsonl for any numeric, benchmark, or volatile claim added to a skill 5. Never invoke paid LLMs from the continuous loop: HTTP retrieval is stdlib-only, judge adapters are explicitly out of scope until budget-gated 6. Never commit runtime queue/report files: .gitignore covers researcher/queue/*.jsonl, researcher/reports/{logs,snapshots,loop-events.jsonl,loop-failures.jsonl,status.md,parked-review.md}, and researcher/runs/*/ except the seed run

Plugin Architecture

All 15 skills are distributed as a single plugin (context-engineering) in the marketplace manifest. This avoids cache duplication: Claude Code caches each plugin's source directory separately, so multiple plugins pointing to source: "./" would each cache a full copy of the repo.

Progressive disclosure pattern: only skill names/descriptions load at startup; full content loads on activation.

Key Design Principles

• Context quality over quantity: attention scarcity and lost-in-middle behavior mean more context is not always better
• Sub-agents isolate context: they exist to manage attention budget, not simulate org roles
• Skills reference each other: use plain text skill names (not links) in Integration sections to avoid cross-directory reference issues
• Examples use Python pseudocode: conceptual demonstrations that work across environments, not production-ready implementations
• Deterministic first, model-judged second: structure, schema, rubric math, manifest sync, retrieval status, and registry shape must pass before any LLM judge is invoked
• Human-controlled merge: agents may prepare PRs and pass gates, but push and merge always require explicit human approval

Contributing to Agent Skills for Context Engineering

Thank you for your interest in contributing to this collection of Agent Skills for Context Engineering. This document provides guidelines and instructions for contributing.

How to Contribute

Reporting Issues

If you find errors, unclear explanations, or missing topics, please open an issue with:

• A clear description of the problem
• The skill and section where the issue was found
• Suggested improvements if you have them

Submitting Changes

For substantive changes, please:

1. Fork the repository 2. Create a feature branch for your changes 3. Make changes following the skill template structure 4. Ensure SKILL.md files remain under 500 lines 5. Add references or scripts as appropriate 6. Submit a pull request with a clear description of changes

Adding New Skills

When adding new skills:

1. Use the template in template/SKILL.md 2. Follow naming conventions (lowercase with hyphens) 3. Include both SKILL.md and appropriate references/scripts 4. Update the root README.md to include the new skill 5. Update root SKILL.md, .claude-plugin/marketplace.json, and .plugin/plugin.json when publishing the skill 6. Update researcher/corpus/index.json with the new skill's name, activation scenarios, mechanism IDs, and claim IDs 7. Add at least one entry to researcher/fixtures/activation-cases.jsonl; include rejected or adjacent skills when the boundary is easy to confuse 8. Ensure content is platform-agnostic (works across Cursor, Claude Code, etc.) 9. Run python3 researcher/scripts/validate_repo.py --strict, python3 researcher/scripts/skill_health.py --strict --no-history, python3 researcher/scripts/check_activation_cases.py, and python3 researcher/scripts/run_benchmarks.py before opening a PR

Researcher Operating System Contributions

The repository ships with a file-based research-to-skill operating system in researcher/. Contributions that introduce skill changes derived from external research should flow through it.

Run lifecycle

initialized -> retrieved -> evaluated -> proposed -> novelty_checked -> validated -> pr_ready -> closed

Use researcher/scripts/research_loop.py subcommands rather than editing run-state.json by hand. Each transition appends to the run's thread log and updates the state machine atomically.

Mechanism promotion

New behavior changes proposed for the corpus go through researcher/mechanisms/registry.jsonl. The promotion path is gated:

1. Author the proposal in the run's proposals/mechanism-proposal.jsonl. 2. Pass validate_run.py --run-dir <run>. 3. Run research_loop.py promote-mechanisms --run-dir <run> --reviewed-by <handle>. This appends to the registry and to researcher/mechanisms/ledgers/accepted.jsonl.

Rejected mechanisms append to ledgers/rejected.jsonl so future agents do not rediscover them.

Claim provenance

Any numeric, benchmark, or volatile claim added to a published skill should also receive an entry in researcher/claims/index.jsonl with claim_id, owning_skill, section, source_url, retrieved_at, evidence_strength, volatility, and last_reviewed. The validator checks ownership and source paths.

Parked runs

Runs that hit human-review gates land in researcher/queue/parked.jsonl and the dashboard at researcher/reports/parked-review.md. Reviewers should:

1. Read researcher/runs/<run-id>/THREAD.md and sources/evidence/. 2. Complete the next required step (retrieve, evaluate, propose, novelty, validate-run, or pr-ready). 3. Close the run with research_loop.py close --status accepted|rejected|reference-only|abandoned --reason <text> --reviewed-by <handle>.

The continuous loop will reap closed runs into researcher/queue/done.jsonl on the next iteration.

Runtime state is not committed

researcher/runs/*/ (except the seed run), researcher/queue/*.jsonl, and researcher/reports/{logs,snapshots,loop-events.jsonl,loop-failures.jsonl,status.md,parked-review.md} are gitignored. PRs should not introduce new committed runs; bug fixtures belong in researcher/fixtures/ instead.

Skill Structure Requirements

Each skill must include:

• YAML frontmatter with name and description fields
• ## When to Activate with positive triggers and an explicit Do not activate boundary for adjacent skills
• ## Core Concepts focused on behavior-changing mechanisms, not generic background
• ## Practical Guidance with an executable workflow, checklist, decision table, or operating rule
• ## Examples with at least one worked artifact, before/after, or boundary example
• ## Guidelines, ## Gotchas, ## Integration, and ## References
• Integration notes that explain routing and composition boundaries, not only topical relationships

Any numeric, benchmark, volatile, or vendor-performance claim in a published skill must either reference a claim-* ID from researcher/claims/index.jsonl or be softened and moved to dated reference material. Any reusable behavior pattern should be represented in researcher/mechanisms/registry.jsonl and linked from researcher/corpus/index.json.

Optional additions:

• references/ directory with additional documentation
• scripts/ directory with executable examples
• Multiple markdown files for complex skills

Content Guidelines

Writing Style

• Be direct and precise
• Use technical terminology appropriately
• Include specific guidance, not vague recommendations
• Provide concrete examples
• Point out complexity and trade-offs

Avoiding Platform Specificity

Skills should work across agent platforms. Avoid:

• Platform-specific tool names without abstraction
• Vendor-locked examples
• Features specific to one agent product

Keeping Skills Focused

Each skill should have a single focus. If a topic grows too large, consider splitting into multiple skills with clear dependencies.

Code of Conduct

This project follows a professional, technical collaboration model. Be respectful of different perspectives and focus on improving the collective knowledge base.

Questions

For questions about contributing, please open an issue for discussion.

Overview

Copy page

A simple, open format for giving agents new capabilities and expertise.

Agent Skills are folders of instructions, scripts, and resources that agents can discover and use to do things more accurately and efficiently. ​ Why Agent Skills? Agents are increasingly capable, but often don’t have the context they need to do real work reliably. Skills solve this by giving agents access to procedural knowledge and company-, team-, and user-specific context they can load on demand. Agents with access to a set of skills can extend their capabilities based on the task they’re working on. For skill authors: Build capabilities once and deploy them across multiple agent products. For compatible agents: Support for skills lets end users give agents new capabilities out of the box. For teams and enterprises: Capture organizational knowledge in portable, version-controlled packages. ​ What can Agent Skills enable? Domain expertise: Package specialized knowledge into reusable instructions, from legal review processes to data analysis pipelines. New capabilities: Give agents new capabilities (e.g. creating presentations, building MCP servers, analyzing datasets). Repeatable workflows: Turn multi-step tasks into consistent and auditable workflows. Interoperability: Reuse the same skill across different skills-compatible agent products. ​ Adoption Agent Skills are supported by leading AI development tools. OpenCode Cursor Amp Letta Goose GitHub VS Code Claude Code Claude OpenAI Codex ​ Open development The Agent Skills format was originally developed by Anthropic, released as an open standard, and has been adopted by a growing number of agent products. The standard is open to contributions from the broader ecosystem.

What are skills?

Copy page

Agent Skills are a lightweight, open format for extending AI agent capabilities with specialized knowledge and workflows.

At its core, a skill is a folder containing a SKILL.md file. This file includes metadata (name and description, at minimum) and instructions that tell an agent how to perform a specific task. Skills can also bundle scripts, templates, and reference materials. my-skill/ ├── SKILL.md # Required: instructions + metadata ├── scripts/ # Optional: executable code ├── references/ # Optional: documentation └── assets/ # Optional: templates, resources ​ How skills work Skills use progressive disclosure to manage context efficiently: Discovery: At startup, agents load only the name and description of each available skill, just enough to know when it might be relevant. Activation: When a task matches a skill’s description, the agent reads the full SKILL.md instructions into context. Execution: The agent follows the instructions, optionally loading referenced files or executing bundled code as needed. This approach keeps agents fast while giving them access to more context on demand. ​ The SKILL.md file Every skill starts with a SKILL.md file containing YAML frontmatter and Markdown instructions: --- name: pdf-processing description: Extract text and tables from PDF files, fill forms, merge documents. ---

PDF Processing

When to use this skill

Use this skill when the user needs to work with PDF files...

How to extract text

1. Use pdfplumber for text extraction...

How to fill forms

... The following frontmatter is required at the top of SKILL.md: name: A short identifier description: When to use this skill The Markdown body contains the actual instructions and has no specific restrictions on structure or content. This simple format has some key advantages: Self-documenting: A skill author or user can read a SKILL.md and understand what it does, making skills easy to audit and improve. Extensible: Skills can range in complexity from just text instructions to executable code, assets, and templates. Portable: Skills are just files, so they’re easy to edit, version, and share. ​ Next steps View the specification to understand the full format. Add skills support to your agent to build a compatible client. See example skills on GitHub. Read authoring best practices for writing effective skills. Use the reference library to validate skills and generate prompt XML.

Specification

Copy page

The complete format specification for Agent Skills.

This document defines the Agent Skills format. ​ Directory structure A skill is a directory containing at minimum a SKILL.md file: skill-name/ └── SKILL.md # Required You can optionally include additional directories such as scripts/, references/, and assets/ to support your skill. ​ SKILL.md format The SKILL.md file must contain YAML frontmatter followed by Markdown content. ​ Frontmatter (required) --- name: skill-name description: A description of what this skill does and when to use it. --- With optional fields: --- name: pdf-processing description: Extract text and tables from PDF files, fill forms, merge documents. license: Apache-2.0 metadata: author: example-org version: "1.0" --- Field Required Constraints name Yes Max 64 characters. Lowercase letters, numbers, and hyphens only. Must not start or end with a hyphen. description Yes Max 1024 characters. Non-empty. Describes what the skill does and when to use it. license No License name or reference to a bundled license file. compatibility No Max 500 characters. Indicates environment requirements (intended product, system packages, network access, etc.). metadata No Arbitrary key-value mapping for additional metadata. allowed-tools No Space-delimited list of pre-approved tools the skill may use. (Experimental) ​ name field The required name field: Must be 1-64 characters May only contain unicode lowercase alphanumeric characters and hyphens (a-z and -) Must not start or end with - Must not contain consecutive hyphens (--) Must match the parent directory name Valid examples: name: pdf-processing name: data-analysis name: code-review Invalid examples: name: PDF-Processing # uppercase not allowed name: -pdf # cannot start with hyphen name: pdf--processing # consecutive hyphens not allowed ​ description field The required description field: Must be 1-1024 characters Should describe both what the skill does and when to use it Should include specific keywords that help agents identify relevant tasks Good example: description: Extracts text and tables from PDF files, fills PDF forms, and merges multiple PDFs. Use when working with PDF documents or when the user mentions PDFs, forms, or document extraction. Poor example: description: Helps with PDFs. ​ license field The optional license field: Specifies the license applied to the skill We recommend keeping it short (either the name of a license or the name of a bundled license file) Example: license: Proprietary. LICENSE.txt has complete terms ​ compatibility field The optional compatibility field: Must be 1-500 characters if provided Should only be included if your skill has specific environment requirements Can indicate intended product, required system packages, network access needs, etc. Examples: compatibility: Designed for Claude Code (or similar products) compatibility: Requires git, docker, jq, and access to the internet Most skills do not need the compatibility field. ​ metadata field The optional metadata field: A map from string keys to string values Clients can use this to store additional properties not defined by the Agent Skills spec We recommend making your key names reasonably unique to avoid accidental conflicts Example: metadata: author: example-org version: "1.0" ​ allowed-tools field The optional allowed-tools field: A space-delimited list of tools that are pre-approved to run Experimental. Support for this field may vary between agent implementations Example: allowed-tools: Bash(git:) Bash(jq:) Read ​ Body content The Markdown body after the frontmatter contains the skill instructions. There are no format restrictions. Write whatever helps agents perform the task effectively. Recommended sections: Step-by-step instructions Examples of inputs and outputs Common edge cases Note that the agent will load this entire file once it’s decided to activate a skill. Consider splitting longer SKILL.md content into referenced files. ​ Optional directories ​ scripts/ Contains executable code that agents can run. Scripts should: Be self-contained or clearly document dependencies Include helpful error messages Handle edge cases gracefully Supported languages depend on the agent implementation. Common options include Python, Bash, and JavaScript. ​ references/ Contains additional documentation that agents can read when needed: REFERENCE.md - Detailed technical reference FORMS.md - Form templates or structured data formats Domain-specific files (finance.md, legal.md, etc.) Keep individual reference files focused. Agents load these on demand, so smaller files mean less use of context. ​ assets/ Contains static resources: Templates (document templates, configuration templates) Images (diagrams, examples) Data files (lookup tables, schemas) ​ Progressive disclosure Skills should be structured for efficient use of context: Metadata (~100 tokens): The name and description fields are loaded at startup for all skills Instructions (< 5000 tokens recommended): The full SKILL.md body is loaded when the skill is activated Resources (as needed): Files (e.g. those in scripts/, references/, or assets/) are loaded only when required Keep your main SKILL.md under 500 lines. Move detailed reference material to separate files. ​ File references When referencing other files in your skill, use relative paths from the skill root: See the reference guide for details.

Run the extraction script: scripts/extract.py Keep file references one level deep from SKILL.md. Avoid deeply nested reference chains. ​ Validation Use the skills-ref reference library to validate your skills: skills-ref validate ./my-skill This checks that your SKILL.md frontmatter is valid and follows all naming conventions.

Integrate skills into your agent

Copy page

How to add Agent Skills support to your agent or tool.

This guide explains how to add skills support to an AI agent or development tool. ​ Integration approaches The two main approaches to integrating skills are: Filesystem-based agents operate within a computer environment (bash/unix) and represent the most capable option. Skills are activated when models issue shell commands like cat /path/to/my-skill/SKILL.md. Bundled resources are accessed through shell commands. Tool-based agents function without a dedicated computer environment. Instead, they implement tools allowing models to trigger skills and access bundled assets. The specific tool implementation is up to the developer. ​ Overview A skills-compatible agent needs to: Discover skills in configured directories Load metadata (name and description) at startup Match user tasks to relevant skills Activate skills by loading full instructions Execute scripts and access resources as needed ​ Skill discovery Skills are folders containing a SKILL.md file. Your agent should scan configured directories for valid skills. ​ Loading metadata At startup, parse only the frontmatter of each SKILL.md file. This keeps initial context usage low. ​ Parsing frontmatter function parseMetadata(skillPath): content = readFile(skillPath + "/SKILL.md") frontmatter = extractYAMLFrontmatter(content)

return { name: frontmatter.name, description: frontmatter.description, path: skillPath } ​ Injecting into context Include skill metadata in the system prompt so the model knows what skills are available. Follow your platform’s guidance for system prompt updates. For example, for Claude models, the recommended format uses XML: <available_skills> <skill> <name>pdf-processing</name> <description>Extracts text and tables from PDF files, fills forms, merges documents.</description> <location>/path/to/skills/pdf-processing/SKILL.md</location> </skill> <skill> <name>data-analysis</name> <description>Analyzes datasets, generates charts, and creates summary reports.</description> <location>/path/to/skills/data-analysis/SKILL.md</location> </skill> </available_skills> For filesystem-based agents, include the location field with the absolute path to the SKILL.md file. For tool-based agents, the location can be omitted. Keep metadata concise. Each skill should add roughly 50-100 tokens to the context. ​ Security considerations Script execution introduces security risks. Consider: Sandboxing: Run scripts in isolated environments Allowlisting: Only execute scripts from trusted skills Confirmation: Ask users before running potentially dangerous operations Logging: Record all script executions for auditing ​ Reference implementation The skills-ref library provides Python utilities and a CLI for working with skills. For example: Validate a skill directory: skills-ref validate <path> Generate <available_skills> XML for agent prompts: skills-ref to-prompt <path>... Use the library source code as a reference implementation.

Skill authoring best practices

Copy page

Learn how to write effective Skills that Claude can discover and use successfully. Good Skills are concise, well-structured, and tested with real usage. This guide provides practical authoring decisions to help you write Skills that Claude can discover and use effectively.

For conceptual background on how Skills work, see the Skills overview.

Core principles Concise is key The context window is a public good. Your Skill shares the context window with everything else Claude needs to know, including:

The system prompt Conversation history Other Skills' metadata Your actual request Not every token in your Skill has an immediate cost. At startup, only the metadata (name and description) from all Skills is pre-loaded. Claude reads SKILL.md only when the Skill becomes relevant, and reads additional files only as needed. However, being concise in SKILL.md still matters: once Claude loads it, every token competes with conversation history and other context.

Default assumption: Claude is already very smart

Only add context Claude doesn't already have. Challenge each piece of information:

"Does Claude really need this explanation?" "Can I assume Claude knows this?" "Does this paragraph justify its token cost?" Good example: Concise (approximately 50 tokens):

Extract PDF text

Use pdfplumber for text extraction:

import pdfplumber

with pdfplumber.open("file.pdf") as pdf:
    text = pdf.pages[0].extract_text()

Bad example: Too verbose (approximately 150 tokens):

Extract PDF text

PDF (Portable Document Format) files are a common file format that contains text, images, and other content. To extract text from a PDF, you'll need to use a library. There are many libraries available for PDF processing, but we recommend pdfplumber because it's easy to use and handles most cases well. First, you'll need to install it using pip. Then you can use the code below... The concise version assumes Claude knows what PDFs are and how libraries work.

Set appropriate degrees of freedom Match the level of specificity to the task's fragility and variability.

High freedom (text-based instructions):

Use when:

Multiple approaches are valid Decisions depend on context Heuristics guide the approach Example:

Code review process

1. Analyze the code structure and organization 2. Check for potential bugs or edge cases 3. Suggest improvements for readability and maintainability 4. Verify adherence to project conventions Medium freedom (pseudocode or scripts with parameters):

Use when:

A preferred pattern exists Some variation is acceptable Configuration affects behavior Example:

Generate report

Use this template and customize as needed:

def generate_report(data, format="markdown", include_charts=True):
    # Process data
    # Generate output in specified format
    # Optionally include visualizations

Low freedom (specific scripts, few or no parameters):

Use when:

Operations are fragile and error-prone Consistency is critical A specific sequence must be followed Example:

Database migration

Run exactly this script:

python scripts/migrate.py --verify --backup

Do not modify the command or add additional flags. Analogy: Think of Claude as a robot exploring a path:

Narrow bridge with cliffs on both sides: There's only one safe way forward. Provide specific guardrails and exact instructions (low freedom). Example: database migrations that must run in exact sequence. Open field with no hazards: Many paths lead to success. Give general direction and trust Claude to find the best route (high freedom). Example: code reviews where context determines the best approach. Test with all models you plan to use Skills act as additions to models, so effectiveness depends on the underlying model. Test your Skill with all the models you plan to use it with.

Testing considerations by model:

Claude Haiku (fast, economical): Does the Skill provide enough guidance? Claude Sonnet (balanced): Is the Skill clear and efficient? Claude Opus (powerful reasoning): Does the Skill avoid over-explaining? What works perfectly for Opus might need more detail for Haiku. If you plan to use your Skill across multiple models, aim for instructions that work well with all of them.

Skill structure YAML Frontmatter: The SKILL.md frontmatter requires two fields:

name:

Maximum 64 characters Must contain only lowercase letters, numbers, and hyphens Cannot contain XML tags Cannot contain reserved words: "anthropic", "claude" description:

Must be non-empty Maximum 1024 characters Cannot contain XML tags Should describe what the Skill does and when to use it For complete Skill structure details, see the Skills overview.

Naming conventions Use consistent naming patterns to make Skills easier to reference and discuss. We recommend using gerund form (verb + -ing) for Skill names, as this clearly describes the activity or capability the Skill provides.

Remember that the name field must use lowercase letters, numbers, and hyphens only.

Good naming examples (gerund form):

processing-pdfs analyzing-spreadsheets managing-databases testing-code writing-documentation Acceptable alternatives:

Noun phrases: pdf-processing, spreadsheet-analysis Action-oriented: process-pdfs, analyze-spreadsheets Avoid:

Vague names: helper, utils, tools Overly generic: documents, data, files Reserved words: anthropic-helper, claude-tools Inconsistent patterns within your skill collection Consistent naming makes it easier to:

Reference Skills in documentation and conversations Understand what a Skill does at a glance Organize and search through multiple Skills Maintain a professional, cohesive skill library Writing effective descriptions The description field enables Skill discovery and should include both what the Skill does and when to use it.

Always write in third person. The description is injected into the system prompt, and inconsistent point-of-view can cause discovery problems.

Good: "Processes Excel files and generates reports" Avoid: "I can help you process Excel files" Avoid: "You can use this to process Excel files" Be specific and include key terms. Include both what the Skill does and specific triggers/contexts for when to use it.

Each Skill has exactly one description field. The description is critical for skill selection: Claude uses it to choose the right Skill from potentially 100+ available Skills. Your description must provide enough detail for Claude to know when to select this Skill, while the rest of SKILL.md provides the implementation details.

Effective examples:

PDF Processing skill:

description: Extract text and tables from PDF files, fill forms, merge documents. Use when working with PDF files or when the user mentions PDFs, forms, or document extraction. Excel Analysis skill:

description: Analyze Excel spreadsheets, create pivot tables, generate charts. Use when analyzing Excel files, spreadsheets, tabular data, or .xlsx files. Git Commit Helper skill:

description: Generate descriptive commit messages by analyzing git diffs. Use when the user asks for help writing commit messages or reviewing staged changes. Avoid vague descriptions like these:

description: Helps with documents description: Processes data description: Does stuff with files Progressive disclosure patterns SKILL.md serves as an overview that points Claude to detailed materials as needed, like a table of contents in an onboarding guide. For an explanation of how progressive disclosure works, see How Skills work in the overview.

Practical guidance:

Keep SKILL.md body under 500 lines for optimal performance Split content into separate files when approaching this limit Use the patterns below to organize instructions, code, and resources effectively Visual overview: From simple to complex A basic Skill starts with just a SKILL.md file containing metadata and instructions:

Simple SKILL.md file showing YAML frontmatter and markdown body

As your Skill grows, you can bundle additional content that Claude loads only when needed:

Bundling additional reference files like reference.md and forms.md.

The complete Skill directory structure might look like this:

pdf/ ├── SKILL.md # Main instructions (loaded when triggered) ├── FORMS.md # Form-filling guide (loaded as needed) ├── reference.md # API reference (loaded as needed) ├── examples.md # Usage examples (loaded as needed) └── scripts/ ├── analyze_form.py # Utility script (executed, not loaded) ├── fill_form.py # Form filling script └── validate.py # Validation script Pattern 1: High-level guide with references --- name: pdf-processing description: Extracts text and tables from PDF files, fills forms, and merges documents. Use when working with PDF files or when the user mentions PDFs, forms, or document extraction. ---

PDF Processing

Quick start

Extract text with pdfplumber:

import pdfplumber
with pdfplumber.open("file.pdf") as pdf:
    text = pdf.pages[0].extract_text()

Advanced features

Form filling: See FORMS.md for complete guide API reference: See REFERENCE.md for all methods Examples: See EXAMPLES.md for common patterns Claude loads FORMS.md, REFERENCE.md, or EXAMPLES.md only when needed.

Pattern 2: Domain-specific organization For Skills with multiple domains, organize content by domain to avoid loading irrelevant context. When a user asks about sales metrics, Claude only needs to read sales-related schemas, not finance or marketing data. This keeps token usage low and context focused.

bigquery-skill/ ├── SKILL.md (overview and navigation) └── reference/ ├── finance.md (revenue, billing metrics) ├── sales.md (opportunities, pipeline) ├── product.md (API usage, features) └── marketing.md (campaigns, attribution) SKILL.md

BigQuery Data Analysis

Available datasets

Finance: Revenue, ARR, billing → See reference/finance.md Sales: Opportunities, pipeline, accounts → See reference/sales.md Product: API usage, features, adoption → See reference/product.md Marketing: Campaigns, attribution, email → See reference/marketing.md

Quick search

Find specific metrics using grep:

grep -i "revenue" reference/finance.md
grep -i "pipeline" reference/sales.md
grep -i "api usage" reference/product.md

Pattern 3: Conditional details Show basic content, link to advanced content:

DOCX Processing

Creating documents

Use docx-js for new documents. See DOCX-JS.md.

Editing documents

For simple edits, modify the XML directly.

For tracked changes: See REDLINING.md For OOXML details: See OOXML.md Claude reads REDLINING.md or OOXML.md only when the user needs those features.

Avoid deeply nested references Claude may partially read files when they're referenced from other referenced files. When encountering nested references, Claude might use commands like head -100 to preview content rather than reading entire files, resulting in incomplete information.

Keep references one level deep from SKILL.md. All reference files should link directly from SKILL.md to ensure Claude reads complete files when needed.

Bad example: Too deep:

SKILL.md

See advanced.md...

advanced.md

See details.md...

details.md

Here's the actual information... Good example: One level deep:

SKILL.md

Basic usage: [instructions in SKILL.md] Advanced features: See advanced.md API reference: See reference.md Examples: See examples.md Structure longer reference files with table of contents For reference files longer than 100 lines, include a table of contents at the top. This ensures Claude can see the full scope of available information even when previewing with partial reads.

Example:

API Reference

Contents

• Authentication and setup
• Core methods (create, read, update, delete)
• Advanced features (batch operations, webhooks)
• Error handling patterns
• Code examples

Authentication and setup

...

Core methods

... Claude can then read the complete file or jump to specific sections as needed.

For details on how this filesystem-based architecture enables progressive disclosure, see the Runtime environment section in the Advanced section below.

Workflows and feedback loops Use workflows for complex tasks Break complex operations into clear, sequential steps. For particularly complex workflows, provide a checklist that Claude can copy into its response and check off as it progresses.

Example 1: Research synthesis workflow (for Skills without code):

Research synthesis workflow

Copy this checklist and track your progress:

Research Progress:
- [ ] Step 1: Read all source documents
- [ ] Step 2: Identify key themes
- [ ] Step 3: Cross-reference claims
- [ ] Step 4: Create structured summary
- [ ] Step 5: Verify citations

Step 1: Read all source documents

Review each document in the sources/ directory. Note the main arguments and supporting evidence.

Step 2: Identify key themes

Look for patterns across sources. What themes appear repeatedly? Where do sources agree or disagree?

Step 3: Cross-reference claims

For each major claim, verify it appears in the source material. Note which source supports each point.

Step 4: Create structured summary

Organize findings by theme. Include:

• Main claim
• Supporting evidence from sources
• Conflicting viewpoints (if any)

Step 5: Verify citations

Check that every claim references the correct source document. If citations are incomplete, return to Step 3. This example shows how workflows apply to analysis tasks that don't require code. The checklist pattern works for any complex, multi-step process.

Example 2: PDF form filling workflow (for Skills with code):

PDF form filling workflow

Copy this checklist and check off items as you complete them:

Task Progress:
- [ ] Step 1: Analyze the form (run analyze_form.py)
- [ ] Step 2: Create field mapping (edit fields.json)
- [ ] Step 3: Validate mapping (run validate_fields.py)
- [ ] Step 4: Fill the form (run fill_form.py)
- [ ] Step 5: Verify output (run verify_output.py)

Step 1: Analyze the form

Run: python scripts/analyze_form.py input.pdf

This extracts form fields and their locations, saving to fields.json.

Step 2: Create field mapping

Edit fields.json to add values for each field.

Step 3: Validate mapping

Run: python scripts/validate_fields.py fields.json

Fix any validation errors before continuing.

Step 4: Fill the form

Run: python scripts/fill_form.py input.pdf fields.json output.pdf

Step 5: Verify output

Run: python scripts/verify_output.py output.pdf

If verification fails, return to Step 2. Clear steps prevent Claude from skipping critical validation. The checklist helps both Claude and you track progress through multi-step workflows.

Implement feedback loops Common pattern: Run validator → fix errors → repeat

This pattern greatly improves output quality.

Example 1: Style guide compliance (for Skills without code):

Content review process

1. Draft your content following the guidelines in STYLE_GUIDE.md 2. Review against the checklist:

• Check terminology consistency
• Verify examples follow the standard format
• Confirm all required sections are present

3. If issues found:

• Note each issue with specific section reference
• Revise the content
• Review the checklist again

4. Only proceed when all requirements are met 5. Finalize and save the document This shows the validation loop pattern using reference documents instead of scripts. The "validator" is STYLE_GUIDE.md, and Claude performs the check by reading and comparing.

Example 2: Document editing process (for Skills with code):

Document editing process

1. Make your edits to word/document.xml 2. Validate immediately: python ooxml/scripts/validate.py unpacked_dir/ 3. If validation fails:

• Review the error message carefully
• Fix the issues in the XML
• Run validation again

4. Only proceed when validation passes 5. Rebuild: python ooxml/scripts/pack.py unpacked_dir/ output.docx 6. Test the output document The validation loop catches errors early.

Content guidelines Avoid time-sensitive information Don't include information that will become outdated:

Bad example: Time-sensitive (will become wrong):

If you're doing this before August 2025, use the old API. After August 2025, use the new API. Good example (use "old patterns" section):

Current method

Use the v2 API endpoint: api.example.com/v2/messages

Old patterns

<details> <summary>Legacy v1 API (deprecated 2025-08)</summary>

The v1 API used: api.example.com/v1/messages

This endpoint is no longer supported. </details> The old patterns section provides historical context without cluttering the main content.

Use consistent terminology Choose one term and use it throughout the Skill:

Good - Consistent:

Always "API endpoint" Always "field" Always "extract" Bad - Inconsistent:

Mix "API endpoint", "URL", "API route", "path" Mix "field", "box", "element", "control" Mix "extract", "pull", "get", "retrieve" Consistency helps Claude understand and follow instructions.

Common patterns Template pattern Provide templates for output format. Match the level of strictness to your needs.

For strict requirements (like API responses or data formats):

Report structure

ALWAYS use this exact template structure:

# [Analysis Title]

## Executive summary
[One-paragraph overview of key findings]

## Key findings
- Finding 1 with supporting data
- Finding 2 with supporting data
- Finding 3 with supporting data

## Recommendations
1. Specific actionable recommendation
2. Specific actionable recommendation

For flexible guidance (when adaptation is useful):

Report structure

Here is a sensible default format, but use your best judgment based on the analysis:

# [Analysis Title]

## Executive summary
[Overview]

## Key findings
[Adapt sections based on what you discover]

## Recommendations
[Tailor to the specific context]

Adjust sections as needed for the specific analysis type. Examples pattern For Skills where output quality depends on seeing examples, provide input/output pairs just like in regular prompting:

Commit message format

Generate commit messages following these examples:

Example 1: Input: Added user authentication with JWT tokens Output:

feat(auth): implement JWT-based authentication

Add login endpoint and token validation middleware

Example 2: Input: Fixed bug where dates displayed incorrectly in reports Output:

fix(reports): correct date formatting in timezone conversion

Use UTC timestamps consistently across report generation

Example 3: Input: Updated dependencies and refactored error handling Output:

chore: update dependencies and refactor error handling

- Upgrade lodash to 4.17.21
- Standardize error response format across endpoints

Follow this style: type(scope): brief description, then detailed explanation. Examples help Claude understand the desired style and level of detail more clearly than descriptions alone.

Conditional workflow pattern Guide Claude through decision points:

Document modification workflow

1. Determine the modification type:

Creating new content? → Follow "Creation workflow" below Editing existing content? → Follow "Editing workflow" below

2. Creation workflow:

• Use docx-js library
• Build document from scratch
• Export to .docx format

3. Editing workflow:

• Unpack existing document
• Modify XML directly
• Validate after each change
• Repack when complete

If workflows become large or complicated with many steps, consider pushing them into separate files and tell Claude to read the appropriate file based on the task at hand.

Evaluation and iteration Build evaluations first Create evaluations BEFORE writing extensive documentation. This ensures your Skill solves real problems rather than documenting imagined ones.

Evaluation-driven development:

Identify gaps: Run Claude on representative tasks without a Skill. Document specific failures or missing context Create evaluations: Build three scenarios that test these gaps Establish baseline: Measure Claude's performance without the Skill Write minimal instructions: Create just enough content to address the gaps and pass evaluations Iterate: Execute evaluations, compare against baseline, and refine This approach ensures you're solving actual problems rather than anticipating requirements that may never materialize.

Evaluation structure:

{ "skills": ["pdf-processing"], "query": "Extract all text from this PDF file and save it to output.txt", "files": ["test-files/document.pdf"], "expected_behavior": [ "Successfully reads the PDF file using an appropriate PDF processing library or command-line tool", "Extracts text content from all pages in the document without missing any pages", "Saves the extracted text to a file named output.txt in a clear, readable format" ] } This example demonstrates a data-driven evaluation with a simple testing rubric. We do not currently provide a built-in way to run these evaluations. Users can create their own evaluation system. Evaluations are your source of truth for measuring Skill effectiveness.

Develop Skills iteratively with Claude The most effective Skill development process involves Claude itself. Work with one instance of Claude ("Claude A") to create a Skill that will be used by other instances ("Claude B"). Claude A helps you design and refine instructions, while Claude B tests them in real tasks. This works because Claude models understand both how to write effective agent instructions and what information agents need.

Creating a new Skill:

Complete a task without a Skill: Work through a problem with Claude A using normal prompting. As you work, you'll naturally provide context, explain preferences, and share procedural knowledge. Notice what information you repeatedly provide.

Identify the reusable pattern: After completing the task, identify what context you provided that would be useful for similar future tasks.

Example: If you worked through a BigQuery analysis, you might have provided table names, field definitions, filtering rules (like "always exclude test accounts"), and common query patterns.

Ask Claude A to create a Skill: "Create a Skill that captures this BigQuery analysis pattern we just used. Include the table schemas, naming conventions, and the rule about filtering test accounts."

Claude models understand the Skill format and structure natively. You don't need special system prompts or a "writing skills" skill to get Claude to help create Skills. Simply ask Claude to create a Skill and it will generate properly structured SKILL.md content with appropriate frontmatter and body content.

Review for conciseness: Check that Claude A hasn't added unnecessary explanations. Ask: "Remove the explanation about what win rate means - Claude already knows that."

Improve information architecture: Ask Claude A to organize the content more effectively. For example: "Organize this so the table schema is in a separate reference file. We might add more tables later."

Test on similar tasks: Use the Skill with Claude B (a fresh instance with the Skill loaded) on related use cases. Observe whether Claude B finds the right information, applies rules correctly, and handles the task successfully.

Iterate based on observation: If Claude B struggles or misses something, return to Claude A with specifics: "When Claude used this Skill, it forgot to filter by date for Q4. Should we add a section about date filtering patterns?"

Iterating on existing Skills:

The same hierarchical pattern continues when improving Skills. You alternate between:

Working with Claude A (the expert who helps refine the Skill) Testing with Claude B (the agent using the Skill to perform real work) Observing Claude B's behavior and bringing insights back to Claude A Use the Skill in real workflows: Give Claude B (with the Skill loaded) actual tasks, not test scenarios

Observe Claude B's behavior: Note where it struggles, succeeds, or makes unexpected choices

Example observation: "When I asked Claude B for a regional sales report, it wrote the query but forgot to filter out test accounts, even though the Skill mentions this rule."

Return to Claude A for improvements: Share the current SKILL.md and describe what you observed. Ask: "I noticed Claude B forgot to filter test accounts when I asked for a regional report. The Skill mentions filtering, but maybe it's not prominent enough?"

Review Claude A's suggestions: Claude A might suggest reorganizing to make rules more prominent, using stronger language like "MUST filter" instead of "always filter", or restructuring the workflow section.

Apply and test changes: Update the Skill with Claude A's refinements, then test again with Claude B on similar requests

Repeat based on usage: Continue this observe-refine-test cycle as you encounter new scenarios. Each iteration improves the Skill based on real agent behavior, not assumptions.

Gathering team feedback:

Share Skills with teammates and observe their usage Ask: Does the Skill activate when expected? Are instructions clear? What's missing? Incorporate feedback to address blind spots in your own usage patterns Why this approach works: Claude A understands agent needs, you provide domain expertise, Claude B reveals gaps through real usage, and iterative refinement improves Skills based on observed behavior rather than assumptions.

Observe how Claude navigates Skills As you iterate on Skills, pay attention to how Claude actually uses them in practice. Watch for:

Unexpected exploration paths: Does Claude read files in an order you didn't anticipate? This might indicate your structure isn't as intuitive as you thought Missed connections: Does Claude fail to follow references to important files? Your links might need to be more explicit or prominent Overreliance on certain sections: If Claude repeatedly reads the same file, consider whether that content should be in the main SKILL.md instead Ignored content: If Claude never accesses a bundled file, it might be unnecessary or poorly signaled in the main instructions Iterate based on these observations rather than assumptions. The 'name' and 'description' in your Skill's metadata are particularly critical. Claude uses these when deciding whether to trigger the Skill in response to the current task. Make sure they clearly describe what the Skill does and when it should be used.

Anti-patterns to avoid Avoid Windows-style paths Always use forward slashes in file paths, even on Windows:

✓ Good: scripts/helper.py, reference/guide.md ✗ Avoid: scripts\helper.py, reference\guide.md Unix-style paths work across all platforms, while Windows-style paths cause errors on Unix systems.

Avoid offering too many options Don't present multiple approaches unless necessary:

Bad example: Too many choices (confusing): "You can use pypdf, or pdfplumber, or PyMuPDF, or pdf2image, or..."

Good example: Provide a default (with escape hatch): "Use pdfplumber for text extraction:

import pdfplumber

For scanned PDFs requiring OCR, use pdf2image with pytesseract instead." Advanced: Skills with executable code The sections below focus on Skills that include executable scripts. If your Skill uses only markdown instructions, skip to Checklist for effective Skills.

Solve, don't punt When writing scripts for Skills, handle error conditions rather than punting to Claude.

Good example: Handle errors explicitly:

def process_file(path): """Process a file, creating it if it doesn't exist.""" try: with open(path) as f: return f.read() except FileNotFoundError:

Create file with default content instead of failing

print(f"File {path} not found, creating default") with open(path, 'w') as f: f.write('') return '' except PermissionError:

Provide alternative instead of failing

print(f"Cannot access {path}, using default") return '' Bad example: Punt to Claude:

def process_file(path):

Just fail and let Claude figure it out

return open(path).read() Configuration parameters should also be justified and documented to avoid "voodoo constants" (Ousterhout's law). If you don't know the right value, how will Claude determine it?

Good example: Self-documenting:

HTTP requests typically complete within 30 seconds

Longer timeout accounts for slow connections

REQUEST_TIMEOUT = 30

Three retries balances reliability vs speed

Most intermittent failures resolve by the second retry

MAX_RETRIES = 3 Bad example: Magic numbers:

TIMEOUT = 47 # Why 47? RETRIES = 5 # Why 5? Provide utility scripts Even if Claude could write a script, pre-made scripts offer advantages:

Benefits of utility scripts:

More reliable than generated code Save tokens (no need to include code in context) Save time (no code generation required) Ensure consistency across uses Bundling executable scripts alongside instruction files

The diagram above shows how executable scripts work alongside instruction files. The instruction file (forms.md) references the script, and Claude can execute it without loading its contents into context.

Important distinction: Make clear in your instructions whether Claude should:

Execute the script (most common): "Run analyze_form.py to extract fields" Read it as reference (for complex logic): "See analyze_form.py for the field extraction algorithm" For most utility scripts, execution is preferred because it's more reliable and efficient. See the Runtime environment section below for details on how script execution works.

Example:

Utility scripts

analyze_form.py: Extract all form fields from PDF

python scripts/analyze_form.py input.pdf > fields.json

Output format:

{
  "field_name": {"type": "text", "x": 100, "y": 200},
  "signature": {"type": "sig", "x": 150, "y": 500}
}

validate_boxes.py: Check for overlapping bounding boxes

python scripts/validate_boxes.py fields.json
# Returns: "OK" or lists conflicts

fill_form.py: Apply field values to PDF

python scripts/fill_form.py input.pdf fields.json output.pdf

Use visual analysis When inputs can be rendered as images, have Claude analyze them:

Form layout analysis

1. Convert PDF to images:

   python scripts/pdf_to_images.py form.pdf

2. Analyze each page image to identify form fields 3. Claude can see field locations and types visually In this example, you'd need to write the pdf_to_images.py script.

Claude's vision capabilities help understand layouts and structures.

Create verifiable intermediate outputs When Claude performs complex, open-ended tasks, it can make mistakes. The "plan-validate-execute" pattern catches errors early by having Claude first create a plan in a structured format, then validate that plan with a script before executing it.

Example: Imagine asking Claude to update 50 form fields in a PDF based on a spreadsheet. Without validation, Claude might reference non-existent fields, create conflicting values, miss required fields, or apply updates incorrectly.

Solution: Use the workflow pattern shown above (PDF form filling), but add an intermediate changes.json file that gets validated before applying changes. The workflow becomes: analyze → create plan file → validate plan → execute → verify.

Why this pattern works:

Catches errors early: Validation finds problems before changes are applied Machine-verifiable: Scripts provide objective verification Reversible planning: Claude can iterate on the plan without touching originals Clear debugging: Error messages point to specific problems When to use: Batch operations, destructive changes, complex validation rules, high-stakes operations.

Implementation tip: Make validation scripts verbose with specific error messages like "Field 'signature_date' not found. Available fields: customer_name, order_total, signature_date_signed" to help Claude fix issues.

Package dependencies Skills run in the code execution environment with platform-specific limitations:

claude.ai: Can install packages from npm and PyPI and pull from GitHub repositories Anthropic API: Has no network access and no runtime package installation List required packages in your SKILL.md and verify they're available in the code execution tool documentation.

Runtime environment Skills run in a code execution environment with filesystem access, bash commands, and code execution capabilities. For the conceptual explanation of this architecture, see The Skills architecture in the overview.

How this affects your authoring:

How Claude accesses Skills:

Metadata pre-loaded: At startup, the name and description from all Skills' YAML frontmatter are loaded into the system prompt Files read on-demand: Claude uses bash Read tools to access SKILL.md and other files from the filesystem when needed Scripts executed efficiently: Utility scripts can be executed via bash without loading their full contents into context. Only the script's output consumes tokens No context penalty for large files: Reference files, data, or documentation don't consume context tokens until actually read File paths matter: Claude navigates your skill directory like a filesystem. Use forward slashes (reference/guide.md), not backslashes Name files descriptively: Use names that indicate content: form_validation_rules.md, not doc2.md Organize for discovery: Structure directories by domain or feature Good: reference/finance.md, reference/sales.md Bad: docs/file1.md, docs/file2.md Bundle comprehensive resources: Include complete API docs, extensive examples, large datasets; no context penalty until accessed Prefer scripts for deterministic operations: Write validate_form.py rather than asking Claude to generate validation code Make execution intent clear: "Run analyze_form.py to extract fields" (execute) "See analyze_form.py for the extraction algorithm" (read as reference) Test file access patterns: Verify Claude can navigate your directory structure by testing with real requests Example:

bigquery-skill/ ├── SKILL.md (overview, points to reference files) └── reference/ ├── finance.md (revenue metrics) ├── sales.md (pipeline data) └── product.md (usage analytics) When the user asks about revenue, Claude reads SKILL.md, sees the reference to reference/finance.md, and invokes bash to read just that file. The sales.md and product.md files remain on the filesystem, consuming zero context tokens until needed. This filesystem-based model is what enables progressive disclosure. Claude can navigate and selectively load exactly what each task requires.

For complete details on the technical architecture, see How Skills work in the Skills overview.

MCP tool references If your Skill uses MCP (Model Context Protocol) tools, always use fully qualified tool names to avoid "tool not found" errors.

Format: ServerName:tool_name

Example:

Use the BigQuery:bigquery_schema tool to retrieve table schemas. Use the GitHub:create_issue tool to create issues. Where:

BigQuery and GitHub are MCP server names bigquery_schema and create_issue are the tool names within those servers Without the server prefix, Claude may fail to locate the tool, especially when multiple MCP servers are available.

Avoid assuming tools are installed Don't assume packages are available:

Bad example: Assumes installation: "Use the pdf library to process the file."

Good example: Explicit about dependencies: "Install required package: pip install pypdf

Then use it:

from pypdf import PdfReader
reader = PdfReader("file.pdf")

Technical notes YAML frontmatter requirements The SKILL.md frontmatter requires name and description fields with specific validation rules:

name: Maximum 64 characters, lowercase letters/numbers/hyphens only, no XML tags, no reserved words description: Maximum 1024 characters, non-empty, no XML tags See the Skills overview for complete structure details.

Token budgets Keep SKILL.md body under 500 lines for optimal performance. If your content exceeds this, split it into separate files using the progressive disclosure patterns described earlier. For architectural details, see the Skills overview.

Checklist for effective Skills Before sharing a Skill, verify:

Core quality Description is specific and includes key terms Description includes both what the Skill does and when to use it SKILL.md body is under 500 lines Additional details are in separate files (if needed) No time-sensitive information (or in "old patterns" section) Consistent terminology throughout Examples are concrete, not abstract File references are one level deep Progressive disclosure used appropriately Workflows have clear steps Code and scripts Scripts solve problems rather than punt to Claude Error handling is explicit and helpful No "voodoo constants" (all values justified) Required packages listed in instructions and verified as available Scripts have clear documentation No Windows-style paths (all forward slashes) Validation/verification steps for critical operations Feedback loops included for quality-critical tasks Testing At least three evaluations created Tested with Haiku, Sonnet, and Opus Tested with real usage scenarios Team feedback incorporated (if applicable)

https://github.com/anthropics/skills

Engineering Production-Grade LLM Agents: A Technical Deep Dive The shift from prompt engineering to context engineering represents the most significant paradigm change in building LLM agents. As Anthropic's research articulates, the challenge isn't writing better prompts—it's curating "the smallest possible set of high-signal tokens that maximize the likelihood of desired outcomes." Inkeepanthropic This report synthesizes technical findings from major AI labs and framework developers on multi-agent architectures, context management, attention degradation, and agent reliability patterns. Multi-agent architectures: From orchestrators to swarms Production multi-agent systems have converged on three dominant patterns, each with distinct tradeoffs. Orchestrator-worker (supervisor) patterns place a central agent in control, delegating to specialists and synthesizing results. LangGraph's benchmarks found this architecture initially performed 50% worse than optimized versions due to the "telephone game" problem—supervisors paraphrasing sub-agent responses incorrectly. The fix: implementing a forward_message tool allowing sub-agents to pass responses directly to users. langchainLangChain Swarm architectures, pioneered by OpenAI's experimental Swarm framework, enable peer-to-peer handoffs where any agent transfers control to any other. LangGraph benchmarks show swarms slightly outperform supervisors because sub-agents respond directly to users, eliminating translation errors. langchainLangChain The core abstraction is elegantly simple: pythondef transfer_to_agent_b(): return agent_b # Handoff via function return

agent_a = Agent( name="Agent A", functions=[transfer_to_agent_b] ) Hierarchical patterns, implemented in CrewAI's Process.hierarchical mode, create management trees where managers decompose goals and delegate to subordinates. Activewizards This mirrors organizational structures and works well for complex, multi-stage tasks. The critical insight from Manus AI's production experience: sub-agents exist primarily to isolate context, not to anthropomorphize role division. Rlancemartin Context isolation prevents KV-cache penalties and avoids context confusion between specialized tasks. Context coordination and the file system as memory How agents share context determines both performance and cost. Manus AI identified KV-cache hit rate as the single most important production metric— Manusthe difference between $0.30/MTok (cached) and $3/MTok (uncached) for Claude Sonnet, a 10× cost differential. manus Three context-sharing patterns emerge from production systems: PatternMechanismUse CaseFull context delegationPlanner shares entire context with sub-agentComplex tasks requiring complete understandingInstruction passingPlanner creates instructions via function callSimple, well-defined subtasksFile system memoryAgents read/write to persistent storageUnlimited size, agent-operable context Claude Code exemplifies file-system-as-memory: rather than stuffing context windows, agents use grep, head, and tail to navigate codebases, storing query results and analyzing large databases without loading full data. AnthropicRlancemartin This "just-in-time" context loading maintains small active context while enabling access to arbitrarily large information. anthropic Manus AI's context engineering principles offer production-tested guidance: use append-only context (never modify previous actions), employ logit masking instead of tool removal to constrain actions, and keep errors in context for implicit belief updates rather than hiding failures. manusManus KV-cache optimization: From PagedAttention to prefix caching The KV-cache stores Key and Value tensors computed during inference, growing linearly with sequence length. Neptune.ai For LLaMA-2 13B, this means approximately 1MB per token per sequence—a 4K context consumes ~4GB, comparable to the model itself. Rohan-paul PagedAttention, introduced by vLLM, revolutionized memory efficiency by applying OS-inspired virtual memory concepts. Medium Instead of pre-allocating contiguous memory, it partitions KV cache into fixed-size blocks (typically 16 tokens), mapping logical blocks to non-contiguous physical memory via block tables. Results: 2-4× throughput improvement arXiv with up to 96% reduction in memory waste. Medium Prefix caching (Automatic Prefix Caching) reuses KV blocks across requests sharing identical prefixes, using hash-based block matching: hash(parent_hash, block_tokens, extra_hashes). Anthropic reports up to 90% cost savings and 85% latency reduction with prefix caching on Claude. Advanced quantization pushes efficiency further. SKVQ achieves 1M token context on 80GB GPUs using 2-bit keys and 1.5-bit values with only <5% accuracy drop. Emergent Mind Layer-Condensed KV caches only top layers for 26× throughput. Emergent Mind RazorAttention identifies "retrieval heads" that need full caches versus those that can use buffers, achieving 40-60% memory reduction. Emergent Mind Context rot: The hidden performance cliff Despite claims of 100K+ token context windows, empirical research reveals significant performance degradation—a phenomenon researchers call context rot. anthropic The "lost in the middle" effect, documented by Liu et al. (TACL 2024), shows a U-shaped performance curve: accuracy drops 10-40% when relevant information sits in the middle of context versus beginning or end. arXivACL Anthology The RULER benchmark delivers a sobering finding: only half of models claiming 32K+ context maintain satisfactory performance at 32K tokens. arXivOpenReview GPT-4 showed the least degradation (15.4 points from 4K to 128K), while most models dropped 30+ points. Medium Near-perfect scores on simple needle-in-haystack tests don't translate to real long-context understanding— trychromaRULER's multi-hop tracing, aggregation, and question-answering tasks expose the gap. arXivOpenReview Chroma's 2025 research across 18 LLMs identified critical patterns: trychroma

Distractor effect: Even a single irrelevant document reduces performance; multiple distractors compound degradation Needle-question similarity: Lower similarity pairs show faster degradation with context length trychroma Counterintuitive haystack structure: Shuffled (incoherent) haystacks produce better performance than logically coherent ones trychroma Model-specific behaviors: Claude shows lowest hallucination rates but high abstention under ambiguity; GPT shows highest hallucination rates with confident-but-incorrect responses trychroma

Four failure modes in production contexts Beyond simple degradation, long-running agents encounter distinct context failure patterns that require different mitigations: Context poisoning occurs when hallucinations or errors enter context and compound through repeated reference. Feluda As Drew Breunig documents, if an agent's "goals" section becomes poisoned, it develops nonsensical strategies that take "very long time to undo." Drew Breunig Symptoms include degraded output quality, tool misalignment, and hallucinations treated as facts. Context distraction emerges when context grows so long that models over-focus on context at the expense of training knowledge. The Gemini 2.5 technical report notes: "While Gemini 2.5 Pro supports 1M+ token context, making effective use of it for agents presents a new research frontier." Drew Breunig Context confusion arises when irrelevant information influences responses. As one practitioner observed: "If you put something in the context, the model has to pay attention to it. It may be irrelevant information or needless tool definitions, but the model will take it into account." Drew Breunig Context clash develops when accumulated information directly conflicts, documented by Microsoft and Salesforce research showing that sharding information across multiple prompts creates conflicting contexts that derail reasoning. Drew Breunig Mitigation strategies that work Effective context management employs four strategies, formalized by LangChain as the "four-bucket" approach: StrategyImplementationExampleWriteSave context outside windowScratchpads, memory stores, file systemSelectPull relevant context inRAG, memory retrieval, tool selectionCompressReduce tokens preserving infoSummarization, observation maskingIsolateSplit context across agentsSub-agents, sandboxes, state schemas Observation masking deserves special attention: replacing old tool outputs with fixed masks like "Previous X lines elided for brevity" often matches or exceeds LLM summarization performance while adding zero token overhead (versus 5-7% for summarization). Research shows observations comprise 83.9% of tokens in typical agent trajectories—masking offers significant efficiency gains. Architectural approaches include Core Context Aware (CCA) Attention, a plug-and-play module achieving 5.7× faster inference at 64K tokens, arXiv and Google's Chain of Agents (CoA), which breaks inputs into chunks processed by worker agents sequentially, reducing time complexity from n² to nk. Google Research Tool design for agent ergonomics Tools are contracts between deterministic systems and non-deterministic agents—design matters critically. anthropic Anthropic's guidance emphasizes minimizing functional overlap: "If a human can't definitively say which tool to use, an AI agent can't either." Anthropic The consolidation principle transforms API design: Instead ofImplementlist_users, list_events, create_eventschedule_event (finds availability + schedules)read_logssearch_logs (returns relevant lines with context)get_customer_by_id, list_transactions, list_notesget_customer_context (compiles all relevant info) Tool descriptions require engineering. Poor descriptions like "Search the database" with cryptic parameter names force agents to guess. Optimized descriptions include usage context ("Use this when the user asks about company policies"), examples ("Example: 'vacation policy remote employees'"), and defaults ("Start with 3-5 for most queries"). Response format options offer significant token savings: implementing a response_format parameter with DETAILED (full JSON, 206 tokens) versus CONCISE (essential info only, 72 tokens) cuts context consumption by 65% when full metadata isn't needed. Reasoning patterns and their measured impact ReAct (Reasoning + Acting) interleaves thinking with tool use: "Thought 1: [reasoning] → Action 1: [tool call] → Observation 1: [result]". Prompt Engineering Guide Performance gains are substantial: +34% absolute success rate on ALFWorld, +10% on WebShop versus imitation learning. React-lm However, 2024 research reveals brittleness—40-90% of generated thoughts lead to invalid actions depending on the model. arXiv Tree of Thoughts (ToT) explores multiple reasoning paths simultaneously. On Game of 24, performance jumps from 4% (Chain-of-Thought) to 74% with GPT-4 using ToT. KDnuggets The approach works by generating multiple candidates at each reasoning step, having the LLM self-evaluate progress, and using tree search (BFS/DFS) for exploration. Dynamic few-shot selection consistently outperforms static examples. LangChain benchmarks show Claude 3 Sonnet jumping from 16% to 52% accuracy with just 3 semantically similar examples—often matching or exceeding 13 static examples. The key is semantic similarity: retrieve examples similar to the current query rather than maintaining fixed lists. Hallucination prevention in agentic contexts Agentic settings amplify hallucination risk since errors compound across tool calls. A critical MIT survey finding: "No prior work demonstrates successful self-correction with feedback from prompted LLMs, except for tasks exceptionally suited for self-correction." What does work for self-correction:

External tool feedback: Code execution results, API verification, calculator outputs Retrieval grounding: Web search for fact verification Fine-tuned correction models: Models specifically trained for correction tasks

RAG-based grounding can decrease hallucination by 60-80% according to industry surveys. Implementation requires explicit constraints: "Answer based ONLY on the provided context. If the context doesn't contain relevant information, respond: 'I cannot find information about this in the provided documents.'" The Chain-of-Verification (CoVe) pattern generates verification questions about claims, answers them independently, compares answers with initial claims, and revises based on inconsistencies. ProCo framework achieves +6.8 EM on QA and +14.1% on arithmetic through systematic condition verification. Evaluation methods for production agents Anthropic's multi-agent evaluation approach uses a structured rubric: factual accuracy (claims match sources), citation accuracy (cited sources match claims), completeness (all aspects covered), source quality (primary versus secondary), and tool efficiency (reasonable usage). Anthropic Key benchmarks reveal capability gaps: BenchmarkFindingRULEROnly 50% of 32K+ models maintain performance at 32K tokens arXiv∞Bench"Existing long-context LLMs require significant advancements for 100K+"LongBench v2Best model achieves 50.1% accuracy; humans achieve 53.7% Longbench2τ-benchTests single/multi-agent cognitive architectures on real-world scenarios The methodology: start with small samples (~20 queries), use LLM-as-judge for scalable evaluation, supplement with human evaluation to catch automation misses, and focus on end-state evaluation for agents that mutate state. Anthropic Conclusion Building production LLM agents requires treating context as the central engineering concern rather than an afterthought. The research converges on several principles: Context quality trumps context length—despite 1M+ token windows, effective performance often degrades past 32K-256K tokens depending on task complexity. Use just-in-time context loading, observation masking, and sub-agent isolation to maintain signal quality. Multi-agent architecture selection depends on coordination needs: swarms for peer-to-peer handoffs with direct user interaction, supervisors for integrating diverse sub-agents with minimal assumptions, hierarchical patterns for complex decomposition tasks. Tool design directly impacts agent capability. Consolidate overlapping tools, return contextual information in error messages, implement response format options, and namespace clearly. anthropic Poor tool descriptions create failure modes no amount of prompt engineering can fix. Verification requires external grounding. Self-correction without external feedback doesn't work reliably. RAG, tool execution results, and multi-agent verification architectures provide the grounding necessary for production reliability. The field is rapidly evolving—KV-cache optimization, attention architectures, and evaluation methods continue advancing. Engineers building agents should monitor production metrics (especially KV-cache hit rates and token efficiency), implement compaction triggers at 80% of effective context limits, and design systems assuming context will degrade rather than hoping it won't.

Book SFT Pipeline

A standalone skill for training language models to write in any author's style. This is a separate plugin from the main Context Engineering collection.

Installation

Claude Code

# Add the marketplace first
/plugin marketplace add muratcankoylan/Agent-Skills-for-Context-Engineering

# Install the book-sft-pipeline plugin
/plugin install book-sft-pipeline@context-engineering-marketplace

Cursor / Codex / IDE

Copy SKILL.md to your .rules or project skills folder.

Manual

Reference the SKILL.md file directly in your agent's context.

What's Included

book-sft-pipeline/
├── README.md                 # This file
├── SKILL.md                  # Complete skill documentation (standalone)
├── examples/
│   └── gertrude-stein/       # Complete case study with real outputs
│       ├── README.md         # Results and analysis
│       ├── sample_outputs.md # Raw model outputs
│       ├── training_config.json
│       ├── dataset_sample.jsonl
│       └── pangram/          # AI detector screenshots
├── scripts/
│   └── pipeline_example.py   # Conceptual implementation
└── references/
    ├── segmentation-strategies.md
    ├── tinker-format.md
    └── tinker.txt

Key Results

Trained Qwen3-8B-Base on Gertrude Stein's "Three Lives" (1909):

Related Context Engineering Skills

This skill applies patterns from the Agent Skills for Context Engineering collection:

Resources

• Dataset on Hugging Face
• Research Paper (Chakrabarty et al. 2025)

License

MIT