Config Skills

Config Skills for LlamaFarm

Specialized patterns and best practices for the LlamaFarm configuration module (config/).

Module Overview

The config module provides YAML/TOML/JSON configuration loading with JSONSchema validation:

Links to Shared Skills

This module follows Python conventions from the shared skills:

Framework-Specific Checklists

Tech Stack

• Python: 3.11+
• Pydantic: v2 with ConfigDict, Field, constrained types
• JSONSchema: Draft-07 with $ref dereferencing via jsonref
• YAML: ruamel.yaml for comment-preserving read/write
• Code Generation: datamodel-codegen for schema-to-Pydantic

Key Patterns

Generated Pydantic Models

The datamodel.py file is auto-generated from JSONSchema:

# Generated by datamodel-codegen from schema.deref.yaml
from pydantic import BaseModel, ConfigDict, Field, conint, constr

class Database(BaseModel):
    model_config = ConfigDict(extra="forbid")
    name: constr(pattern=r"^[a-z][a-z0-9_]*$", min_length=1, max_length=50)
    type: Type
    config: dict[str, Any] | None = Field(None, description="Database-specific configuration")

Custom Validators for Cross-Field Constraints

JSONSchema draft-07 cannot express all constraints. Custom validators extend validation:

def validate_llamafarm_config(config_dict: dict[str, Any]) -> None:
    """Validate constraints beyond JSONSchema (uniqueness, references)."""
    # Check for duplicate prompt names
    prompt_names = [p.get("name") for p in config_dict.get("prompts", [])]
    duplicates = [name for name in prompt_names if prompt_names.count(name) > 1]
    if duplicates:
        raise ValueError(f"Duplicate prompt set names: {', '.join(set(duplicates))}")

Comment-Preserving YAML with ruamel.yaml

Configuration files preserve user comments when modified:

from ruamel.yaml import YAML
from ruamel.yaml.comments import CommentedMap

def _get_ruamel_yaml() -> YAML:
    yaml_instance = YAML()
    yaml_instance.preserve_quotes = True
    yaml_instance.indent(mapping=2, sequence=4, offset=2)
    return yaml_instance

Directory Structure

config/
├── pyproject.toml       # UV-managed dependencies
├── schema.yaml          # Source JSONSchema with $ref
├── schema.deref.yaml    # Dereferenced schema (generated)
├── datamodel.py         # Pydantic models (generated)
├── compile_schema.py    # Schema compilation script
├── generate_types.py    # Type generation script
├── validators.py        # Custom validation beyond JSONSchema
├── validate_config.py   # CLI validation wrapper
├── __init__.py          # Public API exports
├── helpers/
│   ├── loader.py        # Config loading/saving
│   └── generator.py     # Template-based generation
├── templates/
│   └── default.yaml     # Default config template
└── tests/
    ├── conftest.py      # Shared fixtures
    └── test_*.py        # Test modules

Workflow: Schema Changes

When modifying the configuration schema:

1. Edit schema.yaml (or referenced schemas like ../rag/schema.yaml) 2. Run nx run generate-types to compile and generate types 3. Update validators.py if new cross-field constraints are needed 4. Test with uv run pytest config/tests/

Common Commands

# Generate types from schema
nx run generate-types

# Validate a config file
uv run python config/validate_config.py path/to/llamafarm.yaml --verbose

# Run tests
uv run pytest config/tests/ -v

# Lint and format
ruff check config/ --fix
ruff format config/

JSONSchema Generation and Validation Checklist

JSONSchema patterns for LlamaFarm configuration validation.

---

Category: Schema Structure

Use Draft-07 Schema Version

What to check: Schema declares JSON Schema draft-07

Good pattern:

# yaml-language-server: $schema=http://json-schema.org/draft-07/schema#
$schema: http://json-schema.org/draft-07/schema#
title: LlamaFarm Config
type: object

Why: Draft-07 is well-supported by validators and code generators

Search pattern:

rg '\$schema.*draft-07' config/

Severity: Medium

---

Declare Required Fields Explicitly

What to check: Required fields are listed in required array

Good pattern:

type: object
required:
  - version
  - name
  - namespace
  - runtime
properties:
  version:
    type: string
  name:
    type: string

Bad pattern:

properties:
  version:
    type: string
  # No 'required' array - all fields optional

Search pattern:

rg "^required:" config/schema.yaml

Severity: High

---

Use Descriptive Property Descriptions

What to check: All properties have description field

Good pattern:

properties:
  name:
    type: string
    description: Project name
    example: my-project
  namespace:
    type: string
    description: Project namespace
    example: my-namespace

Why: Descriptions appear in generated docs and IDE tooltips

Severity: Low

---

Category: Type Definitions

Use definitions for Reusable Types

What to check: Shared types are defined in definitions section

Good pattern:

definitions:
  Tool:
    type: object
    required:
      - type
      - name
      - description
      - parameters
    properties:
      type:
        type: string
        enum: [function]
      name:
        type: string
      description:
        type: string
      parameters:
        type: object

# Usage via $ref
properties:
  tools:
    type: array
    items:
      $ref: "#/definitions/Tool"

Search pattern:

rg "definitions:" config/schema.yaml

Severity: Medium

---

Use $ref for Cross-File References

What to check: Large schemas are split across files with $ref

Good pattern:

# schema.yaml
properties:
  rag:
    $ref: "../rag/schema.yaml"

Why: Keeps schemas maintainable, allows domain-specific ownership

Search pattern:

rg '\$ref:' config/schema.yaml

Severity: Low

---

Use enum for Fixed Value Sets

What to check: Fields with fixed options use enum

Good pattern:

provider:
  type: string
  enum: [openai, ollama, lemonade, universal]
  description: Runtime provider for this model

transport:
  type: string
  enum: [stdio, http, sse]
  description: Connection transport to the MCP server

Search pattern:

rg "enum:" config/schema.yaml

Severity: Medium

---

Category: String Constraints

Use pattern for String Formats

What to check: Identifier fields use regex patterns

Good pattern:

name:
  type: string
  pattern: "^[a-z][a-z0-9_]*$"
  description: Unique prompt set identifier

file_extensions:
  type: array
  items:
    type: string
    pattern: "^\\.[a-zA-Z0-9]+$"

Why: Catches invalid identifiers at validation time

Search pattern:

rg "pattern:" config/schema.yaml

Severity: Medium

---

Use minLength and maxLength for Bounds

What to check: String fields have reasonable length limits

Good pattern:

name:
  type: string
  pattern: "^[a-z][a-z0-9_]*$"
  minLength: 1
  maxLength: 50

description:
  type: string
  minLength: 10
  maxLength: 500

Why: Prevents empty strings and excessively long values

Severity: Low

---

Category: Numeric Constraints

Use minimum and maximum for Integer Bounds

What to check: Integer fields have appropriate bounds

Good pattern:

priority:
  type: integer
  minimum: 0
  maximum: 1000
  default: 50
  description: Parser priority (lower = try first)

max_length:
  type: integer
  minimum: 1
  description: Maximum sequence length for tokenization

Search pattern:

rg "minimum:|maximum:" config/schema.yaml

Severity: Medium

---

Category: Array Constraints

Use minItems for Required Arrays

What to check: Arrays that must have items use minItems

Good pattern:

databases:
  type: array
  description: Database definitions
  minItems: 1
  items:
    $ref: "#/definitions/Database"

parsers:
  type: array
  description: Document parsers in processing order
  minItems: 1
  items:
    $ref: "#/definitions/Parser"

Why: Ensures arrays have at least one element when required

Search pattern:

rg "minItems:" config/schema.yaml

Severity: Medium

---

Use default for Optional Arrays

What to check: Optional arrays have sensible defaults

Good pattern:

prompts:
  type: array
  description: List of named prompt sets
  items:
    $ref: "#/definitions/PromptSet"
  default: []

tools:
  type: array
  description: List of tools to use
  items:
    $ref: "#/definitions/Tool"
  default: []

Severity: Low

---

Category: Object Constraints

Use additionalProperties Appropriately

What to check: Objects specify whether extra properties are allowed

Good pattern - strict objects:

type: object
additionalProperties: false
properties:
  name:
    type: string

Good pattern - extensible objects:

model_api_parameters:
  type: object
  description: Additional parameters passed to the API
  additionalProperties: true

Search pattern:

rg "additionalProperties:" config/schema.yaml

Severity: Medium

---

Category: Schema Compilation

Dereference All $ref Before Validation

What to check: Schema is fully dereferenced before use

Good pattern (from compile_schema.py):

import jsonref

def load_and_deref_schema(path: Path):
    """Load YAML schema and dereference all $refs."""
    with path.open(encoding="utf-8") as f:
        schema = yaml.safe_load(f)

    deref = jsonref.JsonRef.replace_refs(
        schema,
        base_uri=path.as_uri(),
        loader=yaml_json_loader,
    )
    return jsonref_to_dict(deref, is_root=True)

Why: jsonschema validator doesn't resolve external $ref automatically

Severity: High

---

Strip $schema and $id from Nested Refs

What to check: Dereferenced schema removes metadata from inlined refs

Good pattern (from compile_schema.py):

def jsonref_to_dict(obj, is_root=False):
    """Convert jsonref proxies to plain dicts, stripping nested metadata."""
    if isinstance(obj, dict):
        if not is_root:
            schema_metadata_fields = {"$schema", "$id"}
            obj = {k: v for k, v in obj.items() if k not in schema_metadata_fields}
        return {k: jsonref_to_dict(v, is_root=False) for k, v in obj.items()}

Why: Nested $schema fields can confuse validators

Severity: Medium

---

Validate Dereferenced Schema Output

What to check: Schema compilation validates the result

Good pattern (from compile_schema.py):

if deref is None:
    raise ValueError("Schema dereferencing produced None")

if "type" not in deref and "properties" not in deref:
    raise ValueError("Schema is missing required top-level fields")

file_size = output_file.stat().st_size
if file_size < 100:
    raise ValueError(f"Schema file is suspiciously small ({file_size} bytes)")

Severity: High

---

Category: Code Generation

Use datamodel-codegen for Type Generation

What to check: Pydantic models are generated from schema

Good pattern (from generate_types.py):

run_command([
    "uv", "run", "datamodel-codegen",
    "--input", "schema.deref.yaml",
    "--output", "datamodel.py",
    "--input-file-type=jsonschema",
    "--output-model-type=pydantic_v2.BaseModel",
    "--target-python-version=3.12",
    "--use-standard-collections",
    "--use-title-as-name",
    "--formatters=ruff-format",
    "--class-name=LlamaFarmConfig",
], cwd=config_dir)

Why: Ensures Pydantic models match schema exactly

Severity: High

---

Use --use-standard-collections for Modern Types

What to check: Generated code uses list[T] not List[T]

Good pattern:

datamodel-codegen --use-standard-collections

Result:

# Generated with modern syntax
messages: list[PromptMessage] = Field(...)
config: dict[str, Any] | None = Field(None, ...)

Severity: Low

---

Use --use-title-as-name for Clear Class Names

What to check: Generated classes use schema title as class name

Good pattern:

# In schema.yaml
title: PromptSet
type: object

# Generated class
class PromptSet(BaseModel):
    ...

Severity: Low

---

Category: Validation

Use jsonschema for Runtime Validation

What to check: Config validation uses jsonschema library

Good pattern (from helpers/loader.py):

import jsonschema

def _validate_config(config: dict, schema: dict) -> None:
    """Validate configuration against JSON schema."""
    try:
        jsonschema.validate(config, schema)
    except jsonschema.ValidationError as e:
        path_str = ".".join(str(p) for p in e.path)
        raise ConfigError(
            f"Configuration validation error: {e.message}"
            + (f" at path {path_str}" if path_str else "")
        ) from e

Severity: High

---

Add Custom Validators for Complex Constraints

What to check: Constraints beyond JSONSchema are in validators.py

What JSONSchema draft-07 CANNOT express:

• Uniqueness of properties within arrays
• Cross-field references (e.g., prompt name must exist)
• Case-insensitive uniqueness
• Complex conditional validation

Good pattern (from validators.py):

def validate_llamafarm_config(config_dict: dict[str, Any]) -> None:
    """Validate constraints beyond JSONSchema capabilities."""

    # Check unique prompt names
    prompt_names = [p.get("name") for p in config_dict.get("prompts", [])]
    duplicates = [name for name in prompt_names if prompt_names.count(name) > 1]
    if duplicates:
        raise ValueError(f"Duplicate prompt set names: {', '.join(set(duplicates))}")

    # Check model.prompts reference existing prompt sets
    prompt_names_set = {p.get("name") for p in config_dict.get("prompts", [])}
    for model in config_dict.get("runtime", {}).get("models", []):
        for prompt_ref in model.get("prompts", []):
            if prompt_ref not in prompt_names_set:
                raise ValueError(f"Model references non-existent prompt: {prompt_ref}")

Severity: High

---

Provide Clear Validation Error Messages

What to check: Validation errors include path and context

Good pattern:

raise ValueError(
    f"Model '{model_name}' references non-existent prompt set '{prompt_ref}'. "
    f"Available prompt sets: {', '.join(sorted(prompt_names_set))}"
)

Bad pattern:

raise ValueError("Invalid prompt reference")  # No context

Severity: Medium

---

Category: YAML Processing

Use ruamel.yaml for Comment Preservation

What to check: YAML read/write preserves comments

Good pattern (from helpers/loader.py):

from ruamel.yaml import YAML
from ruamel.yaml.comments import CommentedMap

def _get_ruamel_yaml() -> YAML:
    yaml_instance = YAML()
    yaml_instance.preserve_quotes = True
    yaml_instance.indent(mapping=2, sequence=4, offset=2)
    return yaml_instance

Why: Users add comments to configs; preserving them improves UX

Severity: Medium

---

Convert Between CommentedMap and dict

What to check: Internal processing uses plain dicts

Good pattern:

def _commented_map_to_dict(obj: Any) -> Any:
    """Recursively convert CommentedMap/CommentedSeq to plain dict/list."""
    if isinstance(obj, CommentedMap):
        return {k: _commented_map_to_dict(v) for k, v in obj.items()}
    elif isinstance(obj, CommentedSeq):
        return [_commented_map_to_dict(item) for item in obj]
    return obj

Why: Pydantic and jsonschema work with plain dicts

Severity: Medium

---

Use LiteralScalarString for Multiline Strings

What to check: Multiline strings use YAML block scalar style

Good pattern:

from ruamel.yaml.scalarstring import LiteralScalarString

def _dict_to_commented_map(obj: Any) -> Any:
    if isinstance(obj, str) and "\n" in obj:
        return LiteralScalarString(obj)
    return obj

Result in YAML:

content: |
  This is a multiline
  string that preserves
  formatting nicely.

Severity: Low

Pydantic v2 Configuration Patterns Checklist

Pydantic patterns for LlamaFarm configuration models.

Important: The datamodel.py file is auto-generated from schema.yaml via datamodel-codegen. These patterns describe the expected output of generation and should be verified in the generated code. Custom validators belong in validators.py, not in the generated models.

Workflow: 1. Edit schema.yaml to change model structure 2. Run nx run generate-types to regenerate datamodel.py 3. Add custom validation logic in validators.py (function-based, not decorators)

---

Category: Model Configuration

Use ConfigDict Instead of Inner Config Class

What to check: All Pydantic models use model_config = ConfigDict(...) pattern

Good pattern:

from pydantic import BaseModel, ConfigDict

class Database(BaseModel):
    model_config = ConfigDict(
        extra="forbid",
        str_strip_whitespace=True,
    )
    name: str
    type: str

Bad pattern:

class Database(BaseModel):
    class Config:  # DEPRECATED - Pydantic v1
        extra = "forbid"

Search pattern:

rg "class Config:" --type py config/

Pass criteria: No inner Config classes in Pydantic models

Severity: Medium

Recommendation: Use model_config = ConfigDict(...) for all configuration

---

Use extra="forbid" for Strict Configuration

What to check: Configuration models reject unknown fields

Good pattern:

class EmbeddingStrategy(BaseModel):
    model_config = ConfigDict(extra="forbid")
    name: str
    type: Type1
    config: dict[str, Any] | None = None

Why: Catches typos and misconfigurations early

Search pattern:

rg 'extra="forbid"' --type py config/

Pass criteria: All strict configuration models use extra="forbid"

Severity: Medium

---

Category: Constrained Types

Use constr for String Patterns

What to check: String fields with patterns use constr(pattern=...)

Good pattern:

from pydantic import constr

class PromptSet(BaseModel):
    name: constr(pattern=r"^[a-z][a-z0-9_]*$")

Why: Enforces naming conventions at validation time

Search pattern:

rg "constr\(pattern=" --type py config/

Severity: Medium

---

Use constr for Length Constraints

What to check: String fields with length limits use constr(min_length=..., max_length=...)

Good pattern:

name: constr(pattern=r"^[a-z][a-z0-9_]*$", min_length=1, max_length=50)
description: constr(min_length=10, max_length=500) | None = None

Bad pattern:

name: str  # No length validation - could be empty or extremely long

Severity: Medium

---

Use conint for Integer Ranges

What to check: Integer fields with bounds use conint(ge=..., le=...)

Good pattern:

from pydantic import conint

class Parser(BaseModel):
    priority: conint(ge=0, le=1000) | None = Field(50, description="Parser priority")

class EncoderConfig(BaseModel):
    max_length: conint(ge=1) | None = None

Search pattern:

rg "conint\(" --type py config/

Severity: Medium

---

Category: Field Definitions

Use Field for Documentation

What to check: All fields have descriptions via Field(description=...)

Good pattern:

class Model(BaseModel):
    name: str = Field(..., description="Model identifier (unique name)")
    provider: Provider = Field(..., description="Runtime provider for this model")
    base_url: str | None = Field(None, description="Base URL for the provider")

Bad pattern:

class Model(BaseModel):
    name: str  # No description - unclear purpose
    provider: str

Search pattern:

rg 'Field\([^)]*description=' --type py config/

Pass criteria: All public fields have descriptions

Severity: Low

---

Use Ellipsis for Required Fields

What to check: Required fields use Field(...) or have no default

Good pattern:

class Database(BaseModel):
    name: constr(...) = Field(..., description="Unique database identifier")
    type: Type = Field(..., description="Database type")
    config: dict[str, Any] | None = Field(None, description="Optional config")

Severity: Low

---

Use Field Examples for Schema Docs

What to check: Fields with complex formats include examples

Good pattern:

class Parser(BaseModel):
    file_extensions: list[constr(pattern=r"^\.[a-zA-Z0-9]+$")] | None = Field(
        None,
        description='File extensions this parser handles',
        examples=[[".pdf", ".PDF"], [".csv", ".tsv"], [".md", ".markdown"]],
    )

Search pattern:

rg "examples=\[" --type py config/

Severity: Low

---

Category: Nested Models

Properly Type Nested Models

What to check: Nested configurations use typed Pydantic models

Good pattern:

class Database(BaseModel):
    embedding_strategies: list[EmbeddingStrategy] | None = None
    retrieval_strategies: list[RetrievalStrategy] | None = None

class LlamaFarmConfig(BaseModel):
    rag: RAGStrategyConfigurationSchema | None = None
    runtime: Runtime
    mcp: Mcp | None = None

Bad pattern:

class Database(BaseModel):
    embedding_strategies: list[dict] | None = None  # Loses type safety

Severity: High

---

Use Optional with None Default for Optional Nested Models

What to check: Optional nested models have None as default

Good pattern:

class LlamaFarmConfig(BaseModel):
    rag: RAGStrategyConfigurationSchema | None = Field(None, description="RAG configuration")
    mcp: Mcp | None = Field(None, description="MCP client configuration")

Severity: Medium

---

Category: Enum Types

Use Enum for Fixed Value Sets

What to check: Fields with fixed options use Python Enum

Good pattern:

from enum import Enum

class Provider(Enum):
    openai = "openai"
    ollama = "ollama"
    lemonade = "lemonade"
    universal = "universal"

class Model(BaseModel):
    provider: Provider = Field(..., description="Runtime provider")

Why: Type-safe, self-documenting, and generates proper JSONSchema enum

Search pattern:

rg "class \w+\(Enum\)" --type py config/

Severity: Medium

---

Enum Values Match Schema Strings

What to check: Enum member values are the schema string values

Good pattern:

class Transport(Enum):
    stdio = "stdio"  # Value matches what appears in YAML
    http = "http"
    sse = "sse"

Bad pattern:

class Transport(Enum):
    STDIO = "stdio"  # Member name doesn't match value - confusing

Severity: Low

---

Category: Serialization

Use model_dump for Dict Conversion

What to check: Use model_dump() instead of deprecated dict()

Good pattern:

config_dict = config.model_dump(mode="json", exclude_none=True)

Bad pattern:

config_dict = config.dict()  # DEPRECATED

Search pattern:

rg "\.dict\(\)" --type py config/

Pass criteria: No .dict() calls on Pydantic models

Severity: Medium

---

Use mode="json" for JSON-Safe Output

What to check: When serializing for JSON/YAML, use mode="json"

Good pattern:

# Converts Enums to strings, datetimes to ISO format, etc.
config_dict = config.model_dump(mode="json", exclude_none=True)

Why: Ensures output is JSON/YAML serializable

Severity: Medium

---

Use exclude_none for Clean Output

What to check: Use exclude_none=True when optional fields should be omitted

Good pattern:

config_dict = validated.model_dump(mode="json", exclude_none=True)

Why: Keeps configuration files clean, avoids null values

Severity: Low

---

Category: Validation

Use field_validator for Single Field Validation

What to check: Use @field_validator for field-specific validation

Good pattern (for custom validators in validators.py):

from pydantic import field_validator

class Config(BaseModel):
    url: str

    @field_validator("url")
    @classmethod
    def validate_url(cls, v: str) -> str:
        if not v.startswith(("http://", "https://")):
            raise ValueError("URL must start with http:// or https://")
        return v

Bad pattern:

@validator("url")  # DEPRECATED - Pydantic v1

Note: In the config module, custom validation uses function-based validators in validators.py instead of decorator-based validators in models. This is because datamodel.py is generated and cannot contain custom decorators.

Search pattern:

rg "@validator\(" --type py config/

Pass criteria: Use @field_validator, not @validator

Severity: Medium

---

Use model_validator for Cross-Field Validation

What to check: Use @model_validator for validations involving multiple fields

Decorator pattern (for non-generated models):

from pydantic import model_validator

class Database(BaseModel):
    embedding_strategies: list[EmbeddingStrategy] | None = None
    default_embedding_strategy: str | None = None

    @model_validator(mode="after")
    def validate_default_strategy_exists(self) -> "Database":
        if self.default_embedding_strategy and self.embedding_strategies:
            names = [s.name for s in self.embedding_strategies]
            if self.default_embedding_strategy not in names:
                raise ValueError(f"default_embedding_strategy must reference an existing strategy")
        return self

Function-based pattern (used in config module's validators.py):

def validate_llamafarm_config(config_dict: dict[str, Any]) -> None:
    """Validate constraints that JSONSchema cannot express."""
    # Cross-field validation on plain dicts before Pydantic construction
    if "default_embedding_strategy" in config_dict:
        strategies = config_dict.get("embedding_strategies", [])
        if config_dict["default_embedding_strategy"] not in [s["name"] for s in strategies]:
            raise ValueError("default_embedding_strategy must reference an existing strategy")

Bad pattern:

@root_validator  # DEPRECATED - Pydantic v1

Search pattern:

rg "@root_validator" --type py config/

Pass criteria: Use @model_validator or function-based validators, not @root_validator

Severity: Medium

---

Category: Generated Models

Never Edit datamodel.py Directly

What to check: datamodel.py is generated and should not be manually edited

Good pattern:

• Edit schema.yaml to change model structure
• Run nx run generate-types to regenerate
• Add custom validators in validators.py

Bad pattern:

• Manually editing datamodel.py (changes will be overwritten)

Pass criteria: datamodel.py header shows it's generated

Severity: High

---

Keep Validators Separate from Generated Code

What to check: Custom validation logic lives in validators.py

Good pattern:

# validators.py - separate file for custom validation
def validate_llamafarm_config(config_dict: dict[str, Any]) -> None:
    """Validate constraints that JSONSchema cannot express."""
    # Uniqueness checks, cross-references, naming patterns

Why: Generated code can be regenerated without losing customizations

Severity: High