Axiom Ios Testing

Apple Intelligence & AI

You MUST use this skill for ANY Apple Intelligence or Foundation Models work.

When to Use

Use this router when:

• Implementing Apple Intelligence features
• Using Foundation Models
• Working with LanguageModelSession
• Generating structured output with @Generable
• Debugging AI generation issues
• iOS 26 on-device AI

AI Approach Triage

First, determine which kind of AI the developer needs:

Key boundary: Foundation Models vs ML (custom models)

• Foundation Models = Apple's on-device LLM framework (LanguageModelSession, @Generable)
• ML = Custom model deployment (CoreML conversion, quantization, MLTensor, speech-to-text)
• If developer says "run my own model" → skills/ios-ml.md. If "use Apple Intelligence" → stay here.

Training Path Boundaries

When developers say "I need to train / fine-tune / personalize a model," four distinct paths exist. They are often conflated; each has different output, lifecycle, and runtime compatibility.

Critical distinctions:

• MLX LM output (.safetensors) cannot be loaded into a LanguageModelSession. Different toolchain, different deployment target.
• MLUpdateTask is NN-spec only — does not support ML Program (.mlpackage) models from modern PyTorch / TensorFlow conversion. This is the main reason it's rarely used in new projects.
• FM custom adapters are pinned per-base-model version (per-OS). One adapter does NOT serve every device in your install base — see the Approach Triage section in skills/foundation-models.md for the deflection ladder.

For the full "which path applies to me?" disambiguation (decision tree, the three week-costing mistakes, per-path routing) → skills/training-paths.md.

Cross-Domain Routing

Foundation Models + concurrency (session blocking main thread, UI freezes):

• Foundation Models sessions are async — blocking likely means missing await or running on @MainActor
• Fix here first using async session patterns in foundation-models skill
• If concurrency issue is broader than Foundation Models → also invoke axiom-concurrency

Foundation Models + data (@Generable decoding errors, structured output issues):

• @Generable output problems are Foundation Models-specific, NOT generic Codable issues
• Stay here → foundation-models-diag handles structured output debugging
• If developer also has general Codable/serialization questions → also invoke axiom-data

Foundation Models + security (prompt injection, securing agent tools, confirmation gating):

• Threat modeling and mitigations for agentic features (.onToolCall confirmation, .historyTransform spotlighting/redaction, lock-screen intent policy) → axiom-security (skills/agentic-security.md)
• Stay here for the API surface itself (DynamicProfile, tools, sessions)

Routing Logic

Custom Core ML Work (your own models, not Apple's LLM)

skills/ios-ml.md is the hub (deployment, runtime, speech-to-text). The lifecycle stages have dedicated files:

• Convert a trained PyTorch/TF/Keras model → skills/coreml-conversion.md (coremltools.convert, ML Program vs NN-spec, parity validation)
• Compress it → skills/coreml-compression.md (the PTQ-vs-QAT decision, palettization/quantization/pruning)
• Train from scratch / personalize on-device → skills/coreml-training.md (Create ML; MLUpdateTask and its NN-spec-only limitation)

Core AI — the 27-cycle path for LLM-scale on-device models (OS27)

skills/core-ai.md covers Core AI, the on-device inference framework that powers Apple Intelligence and is now open to your apps. Route here (not skills/ios-ml.md) when the model is LLM-scale / a transformer, or when the developer needs custom Metal kernels, multi-function assets, ahead-of-time compilation, KV-cache states, or the specialization/caching deployment model. Covers the Python toolchain (coreai-torch/coreai-opt), the Swift runtime (import CoreAI → AIModel/InferenceFunction/NDArray), specialization discipline, and the Foundation Models bridge (CoreAILanguageModel from the open-source coreai-models package — not a system-framework type).

Turnkey Apple Intelligence UI — Suggested Actions (OS27)

skills/suggested-actions.md covers the SuggestedActions framework: a drop-in SwiftUI SuggestedActionsView that renders Apple-Intelligence-generated suggested actions for a messaging conversation (iOS/macOS/macCatalyst/visionOS 27). This is a system-provided feature — you describe the message (SuggestedActionsMessage) and add the com.apple.developer.suggested-actions entitlement; there's no LanguageModelSession, prompt, or @Generable. Route here for messaging/chat/email apps that want inline system suggestions. If the developer wants to generate their own structured output, that's Foundation Models, not this. The entitlement/capability half also surfaces via axiom-integration, which cross-points back here.

Foundation Models Work

Implementation patterns → skills/foundation-models.md

• LanguageModelSession basics
• @Generable structured output
• Tool protocol integration
• Streaming with PartiallyGenerated
• Dynamic schemas
• Private Cloud Compute model + multimodal image input (OS27)
• WWDC 2025 + 2026 code examples

API reference → skills/foundation-models-ref.md

• Complete API documentation
• All @Generable examples
• Tool protocol patterns
• Streaming generation patterns
• OS27: Private Cloud Compute, multimodal Attachment + ImageReference tool args, LanguageModel protocol + capabilities, reasoning + token usage, Dynamic Profiles (full modifier surface + @SessionProperty), Dynamic Instructions, custom model providers (LanguageModelExecutor), LanguageModelError migration, built-in system tools, improved Foundation Models Instrument

Diagnostics → skills/foundation-models-diag.md

• AI response blocked
• Generation slow
• Guardrail violations
• Context limits exceeded
• Model unavailable

Guardrails & safety decisions → skills/foundation-models-guardrails.md

• When to use permissiveContentTransformations vs .default
• False-positive triage (correct refusal vs over-restrictive)
• Custom safety eval / red-team methodology
• Adapter × guardrail interaction (safety erosion)

Measuring feature quality (Evaluations framework, `OS27`) → skills/foundation-models-evaluations-ref.md

• Building a regression suite for an AI feature (Evaluation, Metric, Evaluator, run via Swift Testing .evaluates)
• Datasets (ModelSample/ArrayLoader) + synthesizing more (makeSamples/SampleGenerator)
• Model-as-judge for open-ended output (ModelJudgeEvaluator, ScoringScale)
• Agentic tool-call/trajectory evaluation (ToolCallEvaluator, TrajectoryExpectation)
• Hill-climbing a prompt/instruction change against an optimization-target metric

Custom adapter training (after Approach Triage rungs 1-4) → skills/foundation-models-adapters.md

• Decision discipline (when adapter training is justified vs. rungs 1-4)
• Maintenance contract (per-OS retrain burden, four-axis eval)
• Per-OS variant strategy and runtime fallback
• Dataset construction discipline
• HIG disclosure for adapter-enhanced features

Adapter toolkit & runtime API → skills/foundation-models-adapters-ref.md

• Python toolkit setup (3.11, 32 GB Apple silicon Mac or Linux GPU)
• Dataset JSONL schema (chat-turn + tool-calling extension)
• examples.train_adapter, examples.train_draft_model, examples.generate, export.export_fmadapter
• SystemLanguageModel.Adapter runtime API and AssetError cases
• Per-base-model-version compatibility matrix
• com.apple.developer.foundation-model-adapter entitlement

Adapter-specific diagnostics → skills/foundation-models-adapters-diag.md

• compatibleAdapterNotFound, invalidAdapterName, invalidAsset
• Tool calls don't fire from adapter
• Adapter consumes context window with trivial prompts
• Accuracy drops after OS update (FB18924722)
• coremltools.libmilstoragepython missing on export

Automated scanning → Launch foundation-models-auditor agent or /axiom:audit foundation-models

Detects anti-patterns AND architectural gaps:

• Missing availability checks, main-thread respond(), manual JSON parsing, missing specific error catches (guardrail / contextWindow), session created per-tap, no streaming for long output, missing @Guide constraints, nested non-@Generable types, no fallback UI
• Prompt-injection risk from direct user-text interpolation, @Generable enums without @frozen (future-case crash), missing Cancel UX, missing transcript trimming, stale availability cache after Settings toggle, partial-output validation gaps, Tool errors indistinguishable from session errors, no retry on transient errors

Scores: PRODUCTION-READY / NEEDS HARDENING / FRAGILE

Decision Tree

1. Custom ML model / CoreML? → skills/ios-ml.md hub → convert (coreml-conversion.md), compress (coreml-compression.md), or train/personalize (coreml-training.md). LLM-scale / transformer / 27-cycle custom model? → skills/core-ai.md (Core AI) 2. Computer vision / image analysis / OCR? → /skill axiom-vision 3. Cloud AI API integration? → /skill axiom-networking 4. Implementing Foundation Models / @Generable / Tool protocol? → foundation-models 5. Need API reference / code examples? → foundation-models-ref 6. Debugging AI issues (blocked, slow, guardrails)? → foundation-models-diag 7. Foundation Models + UI freezing? → foundation-models (async patterns) + also invoke axiom-concurrency if needed 8. Considering training a custom adapter? → foundation-models Approach Triage (rungs 1-4) FIRST; only after documented rung-1-4 failures → foundation-models-adapters 9. Implementing adapter loading, training pipeline, or runtime selection? → foundation-models-adapters + foundation-models-adapters-ref + axiom-integration (skills/background-assets.md) for delivery 10. Debugging adapter-specific failures (compatibleAdapterNotFound, tool calls don't fire from adapter, accuracy regression after OS update)? → foundation-models-adapters-diag 11. Want automated Foundation Models code scan? → foundation-models-auditor (Agent — detects 10 anti-patterns AND completeness gaps including prompt injection, frozen-enum discipline, transcript trimming, Cancel UX; scores PRODUCTION-READY / NEEDS HARDENING / FRAGILE) 12. Measuring whether an AI feature improved/regressed, or building an eval/regression suite (incl. agentic tool-call eval)? → foundation-models-evaluations-ref (OS27 Evaluations framework) 13. Adding Apple's built-in suggested actions to a messaging/chat/email app (SuggestedActionsView, suggested-actions entitlement)? → skills/suggested-actions.md (OS27 — turnkey, system-provided; NOT Foundation Models)

Anti-Rationalization

External Resources

Cloud Claude integration (`claude-api` skill, ships outside Axiom). Opus 4.7 removed temperature, top_p, top_k, and prefill from the Messages API — code that built successfully on 4.6 returns HTTP 400 at runtime, not compile time. The claude-api skill automates the migration (model ID swap, sampling-param removal, prefill replacement) and enforces prompt caching from day one. Skipping it costs an afternoon of production debugging when the first 400s arrive.

Apple's on-device Foundation Models and Anthropic's cloud Claude are unrelated stacks; use both in parallel when an app needs both, and treat claude-api as mandatory reading before any Claude model-ID change ships.

Critical Patterns

foundation-models:

• LanguageModelSession setup
• @Generable for structured output
• Tool protocol for function calling
• Streaming generation
• Dynamic schema evolution

foundation-models-diag:

• Blocked response handling
• Performance optimization
• Guardrail violations
• Context management

Example Invocations

User: "How do I use Apple Intelligence to generate structured data?" → Read: skills/foundation-models.md

User: "My AI generation is being blocked" → Read: skills/foundation-models-diag.md

User: "Show me @Generable examples" → Read: skills/foundation-models-ref.md

User: "Implement streaming AI generation" → Read: skills/foundation-models.md

User: "I want to add AI to my app" → First ask: Apple Intelligence (Foundation Models) or custom ML model? Route accordingly.

User: "My Foundation Models session is blocking the UI" → Read: skills/foundation-models.md (async patterns) + also invoke axiom-concurrency if needed

User: "Review my Foundation Models code for issues" → Invoke: foundation-models-auditor agent

User: "I want to run my PyTorch model on device" → Read: skills/ios-ml.md (classic Core ML conversion, not Foundation Models)

User: "I want to run my own LLM / SAM segmentation model on device" / "convert a PyTorch transformer with Core AI" / "my Core AI model stalls on first launch" → Read: skills/core-ai.md (Core AI conversion, runtime, specialization & caching)

User: "How do I train a custom adapter for our app's summarization?" → Read: skills/foundation-models.md (Approach Triage rungs 1-4 FIRST), then skills/foundation-models-adapters.md only if rung-1-4 failures are documented

User: "Our adapter loaded fine on iOS 26.0 but throws compatibleAdapterNotFound on 26.1" → Read: skills/foundation-models-adapters-diag.md (Pattern 1)

User: "What's the toolkit setup for adapter training?" → Read: skills/foundation-models-adapters-ref.md (Toolkit Setup)

User: "How do we ship a custom adapter to users?" → Read: skills/foundation-models-adapters.md (runtime lifecycle) + axiom-integration (skills/background-assets.md) (delivery)

User: "How do I measure if my prompt change made the tagging feature better?" / "Write an eval suite for my AI feature" → Read: skills/foundation-models-evaluations-ref.md (Evaluations framework — Metrics, Swift Testing .evaluates, model-as-judge, tool-call eval)

User: "Add Apple's suggested actions to my messaging app" / "Show smart/on-device suggested replies for a message thread" / "What's the com.apple.developer.suggested-actions entitlement for?" → Read: skills/suggested-actions.md (turnkey SuggestedActionsView, system-provided — not Foundation Models)

Core AI

Core AI is the on-device inference framework that powers Apple Intelligence — new in the 27 platform releases and now open to your apps. It is the modern successor path for running your own advanced models — large language models, vision transformers, diarization models — locally across CPU, GPU, and Neural Engine, with no server and no per-token cost. It is a complete set of technologies: a Python authoring/conversion/optimization toolchain, a .aimodel on-device format, a memory-safe Swift runtime, and a developer toolchain (ahead-of-time compilation, Core AI Instruments, the Core AI Debugger).

This page owns the Core AI path. Core ML still exists for classic models — see skills/ios-ml.md for the Core ML lifecycle and the boundary below. For Apple's built-in on-device LLM (you don't ship a model), use Foundation Models in axiom-ai.

When to Use

• Bringing a PyTorch model (LLM, SAM-style segmentation, custom architecture) to device via the Core AI format
• Optimizing/quantizing a large model to fit on-device memory and run fast on Apple silicon
• Loading and running a .aimodel from Swift (AIModel, InferenceFunction, NDArray)
• A transformer decode loop that slows down over time → KV-cache via Core AI states
• First-launch stalls from model specialization; planning model download/caching
• Backing a Foundation Models LanguageModelSession with your own custom model

Boundary — Core AI vs Core ML vs Foundation Models

Rule of thumb: Core ML is the established path for classic models; Core AI is the 27-cycle path built for modern/LLM-scale workloads and deep customization (custom kernels, multi-function assets, ahead-of-time compilation). Both convert from PyTorch; pick Core AI when you need its runtime, its optimization library, or LLM-scale execution.

The Deployment Lifecycle

Core AI spans five stages. Authoring/optimization/debugging happen off-device in Python; integration/deployment happen in your app in Swift.

A model ships as a .aimodel asset — a source representation that runs on any Apple device. Before it can execute, it is specialized for the specific device (see Specialization & Caching).

Python Authoring Toolchain

The authoring side reuses the Python/PyTorch workflow you already know. These are pip packages, not OS-gated framework APIs — they run on your Mac, not on device, so they carry no OS27 marker and are not SDK-verifiable. Signatures below are from WWDC 2026 sessions 324/325; treat the coreai-models package APIs as illustrative and verify against the repository.

`coreai-torch` — conversion (pip install coreai-torch pulls in coreai):

import torch, coreai_torch

exported = torch.export.export(pt_model, args=(example,),
    dynamic_shapes={"features": {1: torch.export.Dim("seq", min=1, max=256)}})
exported = exported.run_decompositions(coreai_torch.get_decomp_table())  # preserve attention etc.

ai_program = coreai_torch.TorchConverter().add_exported_program(
    exported, input_names=["features"], output_names=["logits"]).to_coreai()
ai_program.save_asset("Model.aimodel")

• dynamic_shapes keeps a dimension dynamic (e.g. sequence length) instead of tracing it to the static sample size.
• state_names=[...] on add_exported_program turns PyTorch register_buffer tensors into Core AI states (in-place KV-cache — see below).
• Multiple add_exported_program calls with distinct entrypoint names → one asset, multiple callable functions (e.g. image_encode / text_encode / detect), each runnable at a different cadence and compressed independently.
• Verify converted numerics in Python: load both models, assert a small delta on a sample input.

`coreai-opt` — optimization/compression: config-driven, choose a different scheme per platform (macOS vs iOS). Supports int4/int8/FP4/FP8 weight compression with flexible granularity. Quantizer (calibration or quantization-aware training) and KMeansPalettizer (lookup-table palettization, power-efficient on iOS) take a config + example inputs, then prepare/finalize. ExecutionMode.EAGER for weight compression, GRAPH for activations. Presets like presets.w4 give 4-bit per-channel in one line.

Custom Metal 4 kernels: register a coreai_torch.dsl.TorchMetalKernel (Metal Shading Language source + a PyTorch reference + input/output names + result_shapes) with the converter via register_custom_kernels([...]). The MSL is embedded directly in the .aimodel — the kernel ships with the model. For writing efficient kernels and the MTLTensor side, see axiom-graphics metal-migration-ref Part 6 (TorchMetalKernel, WWDC 330) — this page does not duplicate the shader surface.

Model reauthoring (advanced, especially for iOS): rewrite the PyTorch implementation for the target — convolutional projections instead of linear layers, static tensor shapes, channels-first layouts, explicit in-place KV-cache updates — so Core AI maps to native hardware primitives. Unit- and integration-test each module. The Core AI Models repository ships reusable components, conversion recipes for popular models (LLMs, SAM 3, Qwen families), a Swift runtime package, and Core AI Skills you install into a coding agent to get expert conversion/optimization guidance from day one.

Swift Runtime API (OS27)

import CoreAI re-exports the whole surface (the framework is split into CoreAIRuntime/CoreAIAsset/CoreAIDelegates subframeworks plus empty shells — you never import them directly). All types are available on all Apple platforms at 27 (iOS/iPadOS/macOS/watchOS/tvOS/visionOS). The API uses non-escapable/~Copyable types and lifetime dependence for memory safety without copies.

import CoreAI

@available(anyAppleOS 27, *)
func run(modelURL: URL) async throws {
    let model = try await AIModel(contentsOf: modelURL)        // loads the .aimodel
    guard let fn = try model.loadFunction(named: "main") else { return }  // throws -> InferenceFunction?

    var input = NDArray(shape: [seqLen, hiddenDim], scalarType: .float32)
    writeFeatures(into: input.mutableView(as: Float.self))     // MutableView<Float> is ~Escapable

    var outputs = try await fn.run(inputs: ["features": input])
    guard let logits = outputs.remove("logits")?.ndArray else { throw ModelError.missingOutput }
    use(logits.view(as: Float.self))
}

Core types (all SDK-verified against Xcode 27.0 beta, compile-checked):

• `AIModel` — init(contentsOf:options:) async throws, functionNames: [String], functionDescriptor(for:) -> InferenceFunctionDescriptor?, loadFunction(named:) throws -> InferenceFunction?, static var deviceArchitectureName. (AIModel, InferenceFunction are Sendable.)
• `InferenceFunction` — descriptor, run(inputs:states:outputViews:) async throws -> Outputs (overloads accept [String: NDArray] or a built Inputs). For Metal command-stream pipelining there is encode(inputs:states:outputViews:to:) + ComputeStream.
• `InferenceFunctionDescriptor` — name, inputCount/outputCount, inputNames/stateNames/outputNames, inputDescriptor(of:)/stateDescriptor(of:)/outputDescriptor(of:).
• `NDArray` (@unchecked Sendable) — init(shape:scalarType:) (+ strides:/interleaveLayout:/scalars:shape:/descriptor: overloads), scalarType/shape/strides/interleaveLayout. Access via mutableView(as:) (MutableView<Element>), view(as:) (View<Element>), or raw rawView()/mutableRawView(). The mutable views (MutableView, MutableRawView, MutableViews) are ~Escapable, ~Copyable; the read-only View/RawView are ~Escapable (still copyable). Write through them in place; don't store or return them.
• `NDArray.ScalarType` (CaseIterable) — covers low-bit and modern ML dtypes: bool, int2…int128, uint1…uint128, float8e5m2/float8e4m3fn/float8e8m0fn/float4e2m1fn, float16/float32/float64, bfloat16, cfloat16/32/64.
• `AIModelAsset` (inspection, no run) — init(contentsOf:), static isValid(at:), metadata (author/license/description + typed creatorDefinedMetadata), summary(includingStatistics:) (functions, storage types, compute types, operation distribution), updateMetadata(_:). AssetError.Kind: unsupportedVersion/invalidFeatureType/corruptedMetadata/invalidName/duplicateName.

States (KV-cache). A transformer that re-feeds full history is O(n²) per step — latency grows with sequence length. Declare key/value caches as states (PyTorch register_buffer → state_names at conversion). At runtime they are read and updated in-place each inference, so you pass only the newest input:

var keyCache = NDArray(shape: [layers, maxContext, hiddenDim], scalarType: .float32)
var valueCache = NDArray(shape: [layers, maxContext, hiddenDim], scalarType: .float32)

var states = InferenceFunction.MutableViews()
states.insert(&keyCache, for: "keyCache")
states.insert(&valueCache, for: "valueCache")
let outputs = try await fn.run(inputs: ["features": input], states: states)

Tight-loop / pipeline optimizations (reach for only when profiling demands): allocate NDArrays in the function's optimal memory layout to avoid layout conversions; pre-allocate output values (outputViews:) so the framework writes into them; use AsyncValue / ComputeStream to pipeline multiple inference functions. The higher-level run(inputs:) is correct for most apps.

Specialization & Caching (OS27)

A .aimodel is a portable source representation. To run, it must be specialized for the device: (1) a core set of compilation steps (segment/plan/optimize compute — the expensive part), then (2) executable-artifact generation tied to that device + OS version. First specialization of a large model can take a long time; subsequent loads come from cache and are fast.

Discipline: never let specialization happen inside an interactive flow. Apple's explicit guidance. Move it to a dedicated first-run experience, a feature opt-in, or right after asset download — with progress UI — so the user never waits mid-task.

// Gate a feature: is the model already specialized & cached?
let cache = AIModelCache.default
guard let model = try cache.model(for: modelURL, options: .default) else {
    informUser("Preparing AI features…")          // specialize ahead of time instead
    return
}

// Explicitly specialize ahead of time (after download / on opt-in); returns a ready-to-run AIModel
let prepared = try await AIModel.specialize(contentsOf: modelURL,
    options: .default, cache: .default, cachePolicy: .persistent)
_ = prepared

• `AIModelCache` — .default; init?(appGroup:) to share one cache across apps in an app group; model(for:options:) throws -> AIModel? (nil = not specialized yet); deleteEntry(for:options:)/deleteEntries(for:)/deleteAll(); Policy (.default/.persistent) with PurgeConditions (.storagePressure, .sourceAssetChangedOrDeleted).
• `SpecializationOptions` — .default, .cpuOnly, init(preferredComputeUnitKind:), expectFrequentReshapes. ComputeUnitKind is .cpu/.gpu/.neuralEngine with static var availableKinds.
• Ahead-of-time compilation — move the expensive compilation step to your dev machine with the coreai-build CLI: xcrun coreai-build compile MyModel.aimodel --platform iOS (emit per-architecture compiled models). The device still specializes, but with far less work, so it finishes much faster. See the Compiling Core AI models ahead of time article. Detect the device architecture (AIModel.deviceArchitectureName) and fetch the matching compiled asset.
• Large models (>1 GB) — don't bundle them into the app download (it taxes every user, including those who never use the feature). Deliver on demand with Background Assets, triggered when the user opts in — see axiom-integration (skills/background-assets.md).

Foundation Models Bridge

You can back a Foundation Models LanguageModelSession with your own model, reusing respond / @Generable / tools / streaming. This is done via the open-source `coreai-models` Swift package (CoreAILanguageModel), not a system-framework type — CoreAILanguageModel is not in the CoreAI SDK. The package's type conforms to FoundationModels' LanguageModel protocol.

import FoundationModels
import CoreAILanguageModels   // module CoreAILanguageModels, type CoreAILanguageModel — open-source coreai-models package (WWDC 326 code sample); verify names vs repo

let model = try await CoreAILanguageModel(resourcesAt: modelURL)
let session = LanguageModelSession(model: model)

@Generable struct VocabCard { let word: String; let translation: String; let example: String }
let card = try await session.respond(to: "Create a vocab card for flower",
                                     generating: VocabCard.self).content

Same session API, your model underneath — guided generation, streaming, and structured output all work. For the LanguageModel / LanguageModelExecutor protocol surface and the MLX provider, see foundation-models-ref (Custom Model Providers + Ecosystem). The package also ships task libraries (e.g. an image segmenter wrapping SAM 3) that abstract tensor pre/post-processing behind clean Swift APIs.

Developer Tools (WWDC 2026)

• Core AI Instruments — a new Xcode instrument to profile inference intervals in your app (e.g. spot latency growing with sequence length → add states; spot a specialization event blocking launch).
• Core AI debug gauge — streaming Core AI activity in Xcode while the app runs; a quick first look before opening Instruments.
• Core AI Debugger — a standalone Mac app: visualize the model as a graph grouped by PyTorch module, ground every op in its original Python source line, run on real hardware to inspect intermediate tensors, and compare a specialized run against a PyTorch reference (save intermediates API) at automatically-identified sync points scored by PSNR — turning "which layer did quantization break?" from hours into minutes.

Anti-Rationalization

API Quick Reference (OS27)

import CoreAI                                            // re-exports everything

AIModel(contentsOf:options:) async throws                // load .aimodel
  .functionNames / .functionDescriptor(for:) / .loadFunction(named:) throws -> InferenceFunction?
  static .specialize(contentsOf:options:cache:cachePolicy:) async throws -> AIModel
  static .deviceArchitectureName
InferenceFunction.run(inputs:states:outputViews:) async throws -> Outputs   // inputs: [String:NDArray] or Inputs
  .descriptor : InferenceFunctionDescriptor (inputNames/stateNames/outputNames…)
  InferenceFunction.MutableViews().insert(&ndArray, for:)                    // states / output views
NDArray(shape:scalarType:[strides:][interleaveLayout:]) ; .scalarType/.shape/.strides
  .mutableView(as:) -> MutableView<T> ; .view(as:) -> View<T> ; .rawView()/.mutableRawView()
  NDArray.ScalarType: .float32/.float16/.bfloat16/.int4/.int8/.float8e4m3fn/… (CaseIterable)
AIModelAsset(contentsOf:) ; AIModelAsset.isValid(at:) ; .metadata ; .summary(includingStatistics:)
AIModelCache.default ; init?(appGroup:) ; .model(for:options:) ; .deleteAll() ; .Policy(.default/.persistent)
SpecializationOptions(.default/.cpuOnly/init(preferredComputeUnitKind:)) ; ComputeUnitKind(.cpu/.gpu/.neuralEngine)

# Python (off-device, pip — not OS-gated):
coreai_torch.TorchConverter().add_exported_program(…, input_names=, output_names=, state_names=).to_coreai()
coreai_torch.get_decomp_table() ; .register_custom_kernels([TorchMetalKernel(…)])
coreai-opt: Quantizer / KMeansPalettizer / presets.w4 ; int4/int8/FP4/FP8 ; EAGER|GRAPH
xcrun coreai-build compile Model.aimodel --platform iOS    # ahead-of-time compilation

Resources

WWDC: 2026-324, 2026-325, 2026-326, 2026-330

Docs: /CoreAI, /CoreAI/compiling-core-ai-models-ahead-of-time, /CoreAI/integrating-on-device-ai-models-in-your-app-with-core-ai, /CoreAI/managing-model-specialization-and-caching

Skills: skills/ios-ml.md (Core ML lifecycle + boundary), foundation-models-ref (LanguageModel bridge + Ecosystem), axiom-graphics metal-migration-ref (custom Metal kernels / MTLTensor), axiom-integration background-assets (model delivery)

Core ML Compression (QAT vs PTQ)

Shrinking a custom Core ML model so it fits in memory and runs fast on device — and the one decision that dominates the outcome: post-training quantization (PTQ) vs quantization-aware training (QAT). Compression happens after conversion (coreml-conversion.md) and before deployment (skills/ios-ml.md).

When to Use

• A converted model is too large or too slow, and you need to quantize / palettize / prune it.
• You compressed a model and accuracy dropped more than you can accept.
• You're deciding whether a cheap post-training pass is enough or whether you have to retrain.

The core decision: PTQ vs QAT

Decision rubric: 1. Start with PTQ — it's free. Measure accuracy. 2. If PTQ accuracy holds at your target bit-width → ship it. Done. 3. If PTQ degrades too much (common at int4 / sub-4-bit, or on small models) → calibration-based PTQ first (uses a data sample), then QAT if that's still not enough. 4. If even QAT can't hold accuracy at the bit-width → the bit-width is too aggressive; back off.

Authority: Apple ships its own on-device foundation model at 2 bits per weight using QAT (arXiv 2507.13575), with a balanced 2-bit weight set {-1.5, -0.5, 0.5, 1.5} chosen because it trained more smoothly than the unbalanced {-2, -1, 0, 1}. The lesson isn't "use 2-bit" — it's that Apple reached for QAT, not PTQ, to survive aggressive quantization. PTQ at 2-bit on a custom model will almost always fall apart; that regime requires QAT.

The three compression families

Two modules, three method tiers. `optimize.coreml` runs post-training on the `.mlpackage` (data-free). `optimize.torch` is PyTorch-side and holds both QAT (training-time) and calibration-based PTQ — so "calibration PTQ" lives in optimize.torch, not optimize.coreml. Don't conflate "post-training" with "optimize.coreml."

• Palettization — cluster weights into an N-bit lookup table. Usually the best size/accuracy trade-off. Bit-widths {1,2,3,4,6,8}.
• PTQ, data-free (optimize.coreml): palettize_weights() / OpPalettizerConfig
• PTQ, calibration (optimize.torch): SKMPalettizer (sensitive k-means), PostTrainingPalettizer
• QAT (optimize.torch): DKMPalettizer (differentiable k-means)
• Quantization — linear weight (and optionally activation) quantization. int8 / int4 weights, int8 activations; W8A8 runs on the Neural Engine (A17 Pro+, M4).
• PTQ, data-free (optimize.coreml): linear_quantize_weights() / OpLinearQuantizerConfig
• PTQ, calibration (optimize.torch): linear_quantize_activations(), PostTrainingQuantizer, LayerwiseCompressor (GPTQ-style)
• QAT (optimize.torch): LinearQuantizer
• Pruning — zero out low-magnitude weights (magnitude threshold, target sparsity, block-structured, N:M). Orthogonal to the above and composable — coremltools supports joint sparse-palettization and sparse-quantization.
• PTQ, data-free (optimize.coreml): prune_weights() / OpMagnitudePrunerConfig
• PTQ, calibration (optimize.torch): SparseGPT
• QAT (optimize.torch): MagnitudePruner

Always re-measure after compressing

Compression is lossy by definition. Never assume accuracy held — run your eval set before and after every compression pass, on a realistic input distribution. A model that looks fine on synthetic inputs can collapse on real ones. This is the single most-skipped step and the most expensive to discover in production.

Boundary

• QAT/PTQ are deployment-stage decisions for custom Core ML models you train or convert yourself.
• Apple's Foundation Models are already 2-bit-quantized at the framework level — you do not quantize them. FM adapters are LoRA deltas that inherit the base model's quantization. If you're working with Apple's on-device LLM, this page does not apply — see axiom-ai.

Anti-Rationalization

Resources

WWDC: 2024-10159

Docs: coremltools 9.0 guide at apple.github.io/coremltools (opt-overview, opt-quantization-api, opt-palettization-api, opt-pruning-api); Apple Intelligence Foundation Language Models Tech Report 2025 (arXiv 2507.13575) for the 2-bit QAT specifics

Skills: coreml-conversion (produce the model first), skills/ios-ml.md (deploy the compressed model), coreml-training (train from scratch / personalize), axiom-ai (Foundation Models — already quantized)

Core ML Conversion (coremltools)

Converting an already-trained PyTorch / TensorFlow / Keras model into Core ML format with the Python coremltools package. This is the bridge step between "I have a trained model" and "I can run it on device" — it does not train, compress (see coreml-compression.md), or update (see coreml-training.md) anything.

When to Use

• You have a .pt / TorchScript / torch.export PyTorch model, or a TensorFlow/Keras SavedModel, and need a .mlpackage to ship.
• A conversion "succeeded" but the Core ML model's outputs don't match the source.
• You hit coremltools import or op-support errors and need to know whether it's a version mismatch or an unsupported layer.

Boundary: conversion vs everything adjacent

Rule of thumb: coremltools converts a model you already trained elsewhere. If you're producing the weights, that's training, not conversion.

The unified converter

One entry point: `coremltools.convert(model, ...)`. The conversion target is decided by convert_to / minimum_deployment_target:

• `.mlpackage` (ML Program) — the modern format and the default. Use this for anything new.
• `.mlmodel` (NeuralNetwork) — legacy NN-spec. Only relevant if you specifically need NN-spec (e.g. you intend on-device personalization via MLUpdateTask, which is NN-spec-only — see coreml-training.md).

import coremltools as ct

mlmodel = ct.convert(
    traced_model,                              # source model (see capture modes below)
    convert_to="mlprogram",                    # default; "neuralnetwork" only for NN-spec needs
    minimum_deployment_target=ct.target.iOS26, # pin to the OS you actually ship
    compute_precision=ct.precision.FLOAT16,    # FP16 is the default; set FLOAT32 only if accuracy demands
    inputs=[ct.TensorType(name="x", shape=(1, 3, 224, 224))],
)
mlmodel.save("Model.mlpackage")

Supported source formats

PyTorch capture: trace vs export

• `torch.jit.trace` — the stable, well-supported path. Traces one forward pass, so control flow that depends on input values won't be captured. This is still the right default for production conversions.
• `torch.export.export` — newer (coremltools 8+), still maturing/beta. Preferred long-term, but verify parity carefully before relying on it.

Either way, pass concrete inputs (ct.TensorType / ct.ImageType); for variable sizes use ct.RangeDim or ct.EnumeratedShapes.

Always validate parity

Conversion can succeed and still produce a subtly different model. Before you trust it, run the same representative inputs through the source model and the Core ML model and compare outputs (max abs/relative error, and — for classifiers — top-k agreement). Do this on a real input distribution, not random noise.

Common failure modes

Anti-Rationalization

Resources

WWDC: 2024-10159

Docs: /coreml (runtime side); coremltools 9.0 guide at apple.github.io/coremltools (Unified Conversion API, convert-pytorch-workflow)

Skills: coreml-compression (shrink the converted model), coreml-training (Create ML / MLUpdateTask), skills/ios-ml.md (deploy + run), axiom-ai (Foundation Models), axiom-apple-docs

Core ML Training & Personalization

Two distinct on-device/on-Mac training paths that developers constantly conflate:

• Create ML — train a new Core ML model from scratch (or via transfer learning).
• `MLUpdateTask` — personalize an already-deployed model on the user's own data, at runtime.

Both produce or update a Core ML model. Neither has anything to do with fine-tuning Apple's Foundation Models (that's foundation-models-adapters.md) or converting an existing model (that's coreml-conversion.md).

When to Use

• You want to train an image/sound/text/tabular model without writing a training loop → Create ML.
• You want each user's copy of a model to adapt to their data on-device → MLUpdateTask.
• You started building on-device personalization and hit a wall because your model is an .mlpackage → read the NN-spec limitation below before going further.

---

Create ML — train from scratch

Two surfaces, same engine:

• Create ML app (macOS) — no-code GUI. Drag in training data, pick a template, train, export.
• CreateML framework (import CreateML) — programmatic training.

Availability is per-type, not blanket-macOS. MLImageClassifier and MLSoundClassifier train on iOS 15+/iPadOS/visionOS as well as macOS; others (e.g. MLRecommender) remain macOS-only. Don't assume "Create ML = Mac-only" — check the specific type.

Model types (CreateML framework)

Image (MLImageClassifier, MLObjectDetector), sound (MLSoundClassifier), text (MLTextClassifier, MLWordTagger), pose/action (MLHandPoseClassifier, MLActionClassifier, MLHandActionClassifier), style (MLStyleTransfer), tabular (MLRegressor, MLClassifier — backed by boosted-tree / linear / random-forest), and MLRecommender (macOS-only).

The Create ML app also offers a Tabular/Time-Series flow, but there is no confirmed MLTimeSeriesForecaster type in the framework — use the app for that workflow.

Programmatic shape

import CreateML

let data = try MLImageClassifier.DataSource.labeledDirectories(at: trainingURL)
let model = try MLImageClassifier(trainingData: data)      // synchronous — BLOCKS the calling thread
try model.write(to: URL(filePath: "Classifier.mlmodel"))   // exports a .mlmodel

• Training data: directory-of-label-folders for image classifiers; MLDataTable(contentsOf:) from CSV/JSON for tabular.
• The throwing init(trainingData:parameters:) is synchronous and blocking. For UI apps use makeTrainingSession(...) / the async train(...) API and report progress; resume from a checkpoint with init(checkpoint:).
• Export produces a `.mlmodel` (attach MLModelMetadata for author/version/description).
• Training is GPU-accelerated on Apple silicon (Metal).

---

MLUpdateTask — on-device personalization

MLUpdateTask retrains the last few updatable layers of an already-deployed model on user-specific data, then saves an updated .mlmodelc to disk. This is per-user personalization at runtime — not training from scratch.

let task = try MLUpdateTask(
    forModelAt: compiledModelURL,
    trainingData: userBatchProvider,
    configuration: config,
    completionHandler: { context in
        try? context.model.write(to: updatedModelURL)   // persist the personalized model
    }
)
task.resume()

• Initializers come in completionHandler: and progressHandlers: variants (with/without configuration:); MLUpdateContext carries the updated model.
• Loss functions: categorical cross-entropy, MSE. Optimizers: SGD, Adam.
• Available since iOS 13.

⚠️ The NN-spec-only limitation (read before building)

`MLUpdateTask` only works with NeuralNetwork-spec models — NOT ML Program (`.mlpackage`) models from modern PyTorch/TensorFlow conversion. You make a model updatable with coremltools NeuralNetworkBuilder.make_updatable(layer_names), which supports only `innerProduct` (fully-connected) and `convolution` layers, and exists only for the NN-spec format.

This is the single most important fact about on-device personalization, and the reason it's rarely used in new projects: if your pipeline converts a PyTorch/TF model the modern way, you get an .mlpackage that cannot be personalized with MLUpdateTask. Developers routinely discover this after building most of the pipeline. Surface it on day one:

• Modern conversion (coremltools.convert → .mlpackage) → no `MLUpdateTask`.
• On-device personalization required → you must build/convert to an NN-spec model and call make_updatable() — accepting NN-spec's constraints and legacy status.

When MLUpdateTask is still the right tool

• A legacy NN-spec model you already ship.
• Very constrained per-user personalization (last-layer adaptation) where the NN-spec limitation matches the task.
• The personalization must stay on-device for privacy and run without a server round-trip.

If you need richer adaptation than last-layer updates, MLUpdateTask is the wrong tool — retrain with Create ML (server-side or on-Mac) and ship updated models, or rethink the architecture.

---

Boundary recap

Anti-Rationalization

Resources

WWDC: 2018-703, 2019-430, 2019-704 (MLUpdateTask / on-device personalization), 2021-10037, 2024-10183

Docs: /createml, /createml/mlimageclassifier, /coreml/mlupdatetask, /coreml/mlupdatecontext — plus coremltools NeuralNetworkBuilder.make_updatable (apple.github.io/coremltools)

Skills: coreml-conversion (NN-spec vs ML Program), coreml-compression (shrink trained models), skills/ios-ml.md (deploy), foundation-models-adapters (fine-tune Apple's LLM, not a Core ML model), axiom-concurrency (off-main-thread training)

Foundation Models Custom Adapter Diagnostics

First diagnostic on a 27 build: the adapter runtime is obsoleted in 27.0. If the failure is a compile error ('Adapter' was obsoleted in iOS 27.0, 'init(name:)' is unavailable) or the adapter silently never loads on a 27 device, the cause is the obsoletion, not a bug — SystemLanguageModel.Adapter and friends are deprecated: 26.4, obsoleted: 27.0 (iOS/iPadOS/macOS/visionOS) with no replacement in the 27 SDK. The runtime patterns below apply only to 26.x deployments. See axiom-ai (skills/foundation-models-adapters.md) for the 27 pivot.

Overview

Adapter-specific failure modes — distinct from base Foundation Models failures covered in axiom-ai (skills/foundation-models-diag.md). Core principle: most adapter failures are toolkit setup mismatches, per-base-model-version compatibility breakage, or training-data schema gaps — not framework bugs. For decision discipline see axiom-ai (skills/foundation-models-adapters.md); for the API and toolkit reference see axiom-ai (skills/foundation-models-adapters-ref.md).

---

Red Flags

If any of these appear, treat as adapter-specific, not generic Foundation Models:

• SystemLanguageModel.Adapter.AssetError.compatibleAdapterNotFound at runtime
• SystemLanguageModel.Adapter.AssetError.invalidAdapterName at load
• Adapter accuracy regression after an OS minor update
• Tool calls work for the base model but never fire from the adapter
• Trivial user prompts consume disproportionate context window
• ModuleNotFoundError: coremltools.libmilstoragepython during export
• BAErrorCode.downloadBackgroundActivityProhibited during adapter download
• Entitlement-related load failure in production (works in development)

---

Mandatory First Steps

Before changing any code, capture:

// 1. Adapter compatibility for current device
let name = "my_adapter"
let ids = SystemLanguageModel.Adapter.compatibleAdapterIdentifiers(name: name)
print("Compatible variant count: \(ids.count)")
print("Variants: \(ids)")
// Record: empty array? non-empty? which IDs?

// 2. Asset pack state for the expected variant
if let preferredID = ids.first {
    let pack = try await AssetPackManager.shared.assetPack(withID: preferredID)
    let status = try await AssetPackManager.shared.status(relativeTo: pack)
    print("Pack status: \(status)")
}
// status(ofAssetPackWithID:) is deprecated as of iOS 26.4 in favor of
// status(relativeTo:), which takes an AssetPack instead of an ID string.
// Record: downloadAvailable / downloading / downloaded / upToDate / outOfDate / obsolete?

// 3. Base model availability (rule out non-adapter issue)
let availability = SystemLanguageModel.default.availability
print("Base availability: \(availability)")
// Record: available / unavailable(reason)?

For toolkit-setup failures (export errors, missing modules) on the developer Mac:

python --version          # MUST be 3.11.x — record exact
which python              # MUST be inside the active conda/venv env
python -c "import coremltools; print(coremltools.__version__)"
                          # Must succeed; record version
uname -m                  # arm64 expected for Apple silicon Mac export

---

Decision Tree

Adapter problem?
│
├─ Adapter won't load
│  ├─ AssetError.compatibleAdapterNotFound → Pattern 1
│  ├─ AssetError.invalidAdapterName → Pattern 2
│  ├─ AssetError.invalidAsset → Pattern 3
│  └─ Entitlement-related crash on production build → Pattern 4
│
├─ Background Assets download fails
│  └─ Pattern 5 (cross-references axiom-integration)
│
├─ Adapter loads but behaves wrong
│  ├─ Tool calls never fire → Pattern 6
│  ├─ Context window over-consumed by trivial prompts → Pattern 7
│  └─ Accuracy regressed after OS update → Pattern 8
│
├─ Draft model stops speeding up inference (iOS only, never macOS)
│  └─ Pattern 10 (draft-model compilation rate-limited: 3/app/day)
│
├─ Toolkit / export fails on developer Mac
│  └─ Pattern 9 (coremltools / Python version / Linux export)
│
└─ Generic @Generable schema issues (recursive types, Playgrounds macro)
   └─ Cross-reference to axiom-ai (skills/foundation-models-diag.md) Patterns 6a-6c

---

Diagnostic Patterns

Pattern 1: compatibleAdapterNotFound at Runtime

Symptom:

SystemLanguageModel.Adapter.AssetError.compatibleAdapterNotFound

compatibleAdapterIdentifiers(name:) returns an empty array even though an adapter is expected.

Causes (most common first):

1. Device's base-model version has no matching adapter variant uploaded yet (typical after an OS minor update where the team hasn't shipped a retrained adapter) 2. Asset pack containing the matching variant has not yet downloaded (AssetPack.Status == .downloadAvailable) 3. The adapter was trained against a different OS minor than the device runs 4. Apple-hosted asset pack still in App Store review

Diagnosis:

let ids = SystemLanguageModel.Adapter.compatibleAdapterIdentifiers(name: name)
if ids.isEmpty {
    // Case 1, 3, or 4 — no compatible variant uploaded for this OS
    print("No compatible adapter variant; current OS may lack a trained adapter")
} else {
    // Case 2 — variant exists but may not be local
    let preferredID = ids[0]
    let pack = try await AssetPackManager.shared.assetPack(withID: preferredID)
    let status = try await AssetPackManager.shared.status(relativeTo: pack)
    // status(ofAssetPackWithID:) is deprecated as of iOS 26.4 in favor of status(relativeTo:).
    print("Variant exists but status: \(status)")
}

Fix:

• For an empty-array result: train a new adapter against the toolkit version matching the device's OS; upload the resulting asset pack; ship. Until then, the runtime fallback to the base model must keep the feature functional.
• For a pending download: ensure local availability before consuming.

guard let preferredID = ids.first else {
    // Fall back to base model
    let session = LanguageModelSession()
    return session
}

let pack = try await AssetPackManager.shared.assetPack(withID: preferredID)
// ensureLocalAvailability(of:) (single-arg) is deprecated as of iOS 26.4 in favor of
// ensureLocalAvailability(of:requireLatestVersion:).
try await AssetPackManager.shared.ensureLocalAvailability(of: pack, requireLatestVersion: false)
let adapter = try SystemLanguageModel.Adapter(name: name)

Time cost: 10 minutes to add fallback path; days/weeks to retrain and ship a new variant.

---

Pattern 2: invalidAdapterName (Hyphen in Adapter Name)

Symptom:

SystemLanguageModel.Adapter.AssetError.invalidAdapterName

Adapter fails to load at SystemLanguageModel.Adapter(name:) despite a present, downloaded asset pack.

Cause:

The runtime identifier regex is /fmadapter-\w+-\w+/. \w matches word characters (alphanumerics + underscore) but not hyphens. The framework constructs the full identifier as fmadapter-{name}-{variant}; if name contains a hyphen, the identifier has three hyphens and the regex matches only the first segment.

Diagnosis:

Inspect the adapter name passed to the toolkit's --adapter-name flag and at the Swift call site:

let adapter = try SystemLanguageModel.Adapter(name: "my-summarizer")
// ❌ Hyphen will fail the regex

Fix:

Re-export with underscores:

python -m export.export_fmadapter \
    --checkpoint checkpoints/run_001/step_5000.pt \
    --adapter-name my_summarizer \
    --output-dir exports/

Re-upload the asset pack with the new ID. Update Swift call sites to use the underscored name.

✅ Valid: my_summarizer, restaurant_summary_v2 ❌ Invalid: my-summarizer, restaurant-summary

Time cost: 30 minutes (re-export + re-upload + Swift edit). Not a retrain.

---

Pattern 3: invalidAsset (Corrupted or Schema-Incompatible Pack)

Symptom:

SystemLanguageModel.Adapter.AssetError.invalidAsset

The asset pack downloaded successfully but the framework rejects it at load time.

Causes:

1. Toolkit export/ folder was modified (most common — see axiom-ai (skills/foundation-models-adapters-ref.md)) 2. Toolkit version mismatch between training and the target OS 3. Asset pack files corrupted in upload pipeline 4. Adapter package missing required metadata files

Diagnosis:

# Check the export folder is unmodified
diff -r toolkit-26.0.0/export/ working-toolkit/export/

# Verify toolkit version against target OS
cat toolkit-26.0.0/VERSION
# Should match the device's system-model OS line

Fix:

1. Restore unmodified export/ from the toolkit archive 2. Re-export the adapter 3. Re-upload the asset pack 4. If the toolkit version doesn't match the target OS, switch to the matching toolkit and retrain

Time cost: 1-4 hours depending on cause (re-export only) or weeks (retrain against correct toolkit).

---

Pattern 4: Entitlement Missing (Production Load Failure)

Symptom:

Adapter loads in development builds but fails in production / TestFlight / App Store with an entitlement-related error.

Cause:

com.apple.developer.foundation-model-adapter is required for deployment but is not required for local training or development testing. The entitlement must be:

1. Requested by the Account Holder via Apple's developer portal 2. Granted by Apple 3. Included in the provisioning profile used to sign the production build

Diagnosis:

# Inspect the entitlements in the signed production .ipa or .xcarchive
codesign -d --entitlements - /path/to/YourApp.app
# Look for com.apple.developer.foundation-model-adapter

If the key is absent, the entitlement is missing from the profile.

Fix:

1. Account Holder opens Apple Developer Account → Account → Membership → request the Foundation Models Framework Adapter Entitlement 2. Wait for Apple to grant (timeline varies) 3. Regenerate provisioning profiles after the entitlement is granted 4. Re-sign the build with the updated profile

Time cost: Apple's review (hours to days) + minutes to re-sign.

---

Pattern 5: Background Assets Download Fails

Symptom:

The adapter asset pack never downloads, downloads partially, or surfaces a BAErrorCode during ensureLocalAvailability or in statusUpdates.

Cross-reference: this lives in axiom-integration (skills/background-assets.md) — full diagnostic patterns there. Adapter-specific notes:

Fix: see axiom-integration (skills/background-assets.md) "Pressure Scenarios" and "Audit Checklists" sections.

---

Pattern 6: Tool Calls Never Fire From Trained Adapter

Symptom:

A LanguageModelSession(model:) initialized with an adapter loads successfully, but the adapter never invokes attached Tool implementations even when the prompt clearly requires them. The base-model session (no adapter) calls the tools as expected.

Cause:

Training data schema is incomplete. The toolkit's training JSONL must encode:

1. A system message that describes available tools (mirroring how the runtime presents tools to the model) 2. Assistant turns with the full tool_calls array structure

Common missing pieces:

Diagnosis:

Inspect training JSONL for assistant turns:

jq -c '.messages[] | select(.role == "assistant" and .tool_calls != null)' train.jsonl | head -5

Each match should have the full shape:

{
  "role": "assistant",
  "tool_calls": [
    {
      "id": "call_1",
      "type": "function",
      "function": {
        "name": "getRestaurants",
        "arguments": "{\"cuisine\":\"Italian\",\"openNow\":true}"
      }
    }
  ]
}

Fix:

Regenerate training JSONL with complete tool-call schema (see axiom-ai (skills/foundation-models-adapters-ref.md) "Tool-calling schema extension"). Retrain. Re-evaluate. Re-export. Re-upload.

Time cost: days (full retrain cycle).

---

Pattern 7: Adapter Over-Consumes Context Window

Symptom:

Trivial user prompts (a few words) consume 30-90% of the 4096-token context window. Multi-turn conversations exceed exceededContextWindowSize after only 2-3 turns.

Cause:

Training data used multi-paragraph system prompts. The adapter learns to expect verbose preamble at inference time and behaves as if it's present even when the runtime caller omits it. The internal tokenizer state effectively reserves space for the learned verbose context.

Diagnosis:

# Check system-message length distribution across training samples
jq '.messages[] | select(.role == "system") | .content | length' train.jsonl | sort -n | uniq -c

If median system-message length exceeds ~200 characters, this pattern is likely.

Fix:

1. Rewrite training JSONL with short, consistent system messages (≤100 characters, ideally a single sentence) 2. Retrain 3. Re-evaluate token efficiency: Transcript is a RandomAccessCollection of Transcript.Entry (iterate it directly — there is no .entries property). On iOS 26.4+ measure its size with SystemLanguageModel.default.tokenCount(for: Array(transcript)) (tokenCount(for:) is a SystemLanguageModel method, not a LanguageModelSession one), which accepts some Collection<Transcript.Entry>; capture this for a representative single-turn baseline and compare against the previous adapter's measurements

Time cost: days (dataset rewrite + retrain).

---

Pattern 8: Adapter Accuracy Drops After OS Update

Symptom:

An adapter that passed evaluation at ship-time produces noticeably worse outputs after an OS minor update (e.g., 26.0 → 26.1). No code changed. Telemetry shows quality regression across user metrics.

Cause:

The base model changed silently with the OS update. Apple does not provide a public version-pinning API; the runtime always uses the system-model version installed on the device. Apple Developer Forums radar FB18924722 tracks the request for explicit version pinning; as of 2026-05-16, no public API exists.

Diagnosis:

// Check whether the adapter's expected base model still matches
let ids = SystemLanguageModel.Adapter.compatibleAdapterIdentifiers(name: name)
// If empty → adapter is now incompatible (Pattern 1)
// If non-empty → adapter loads but trained against a stale base model

If compatibleAdapterIdentifiers(name:) is non-empty but eval metrics dropped:

# Re-run the eval suite against the production adapter on the new OS
python -m examples.generate \
    --checkpoint exports/my_summarizer.fmadapter \
    --input eval_set.jsonl \
    --output predictions_after_os_update.jsonl
# Compare to predictions captured at ship time

Fix:

1. Train a fresh adapter against the new toolkit version matching the updated OS 2. Re-run the four-axis eval suite 3. Ship the new adapter as a new asset pack variant 4. Old variant remains available for devices not yet on the new OS

Treat as recurring engineering work, not a one-time incident. Plan the next OS update's retrain as a known calendar item.

Time cost: 1-2 weeks per retrain cycle once the training pipeline is automated.

---

Pattern 9: coremltools.libmilstoragepython Missing on Export

Symptom:

ModuleNotFoundError: No module named 'coremltools.libmilstoragepython'

Or other coremltools-related import errors during python -m export.export_fmadapter.

Causes:

1. Python version is 3.12 or 3.13 (toolkit export/ pins coremltools versions only available on Python 3.11) 2. Export running on Linux (the export/ step requires Apple silicon Mac for the coremltools compilation path) 3. Active virtual environment is not the one used to pip install -r requirements.txt

Diagnosis:

python --version
# MUST report 3.11.x

uname -m
# arm64 on Apple silicon

which python
# Should be inside the active conda/venv environment

python -c "import coremltools; print(coremltools.__version__)"
# Should succeed and report a version matching the toolkit's pin

Fix:

# Recreate the environment with Python 3.11 on an Apple silicon Mac
conda create -n fm-adapter python=3.11
conda activate fm-adapter
pip install -r requirements.txt

Training and evaluation can run on Linux GPU machines; export must run on Apple silicon Mac.

Time cost: 15-30 minutes (recreate environment, re-run export only — no retrain).

---

Pattern 10: Draft Model Compilation Rate-Limited

Symptom:

An adapter that uses a draft model loads and works initially, then after several launches (or several adapter switches) on a device, the draft model stops compiling — inference falls back to the slower non-speculative path, or draft-model setup fails outright. Reproduces on iPhone/iPad/Vision Pro but never on macOS.

Cause:

Apple rate-limits draft-model compilation to three compilations per app, per day, on all platforms except macOS (/foundationmodels/loading-and-using-a-custom-adapter-with-foundation-models). Something is triggering recompilation instead of reusing a cached compiled draft model. Common triggers:

1. The compiled draft model isn't cached/persisted, so it recompiles every launch (3 launches → quota exhausted for the day) 2. The app switches the active adapter repeatedly in one session; each switch can force a new draft-model compilation 3. Defensive "recompile to be safe" logic on a code path that runs more than three times a day

macOS never reproduces because the limit excludes it — which is exactly why this slips through Mac-based testing.

Diagnosis:

• Count how many times your code path that loads/compiles the draft model runs per day on device. If it exceeds three, that's the bug.
• Check whether the compiled draft model is written to and read back from disk across launches, or regenerated each time.
• Reproduce on a real iOS device (not the Mac), launching the app 4+ times in a 24-hour window.

Fix:

• Cache the compiled draft model and reuse it across launches. Compile once; persist; load the cached form thereafter.
• Remove any speculative/defensive recompilation. Recompile only when the adapter (and therefore its draft model) actually changes.
• If the feature legitimately needs more than three distinct draft models per day (e.g. frequent adapter switching), reconsider whether a draft model is the right latency strategy for that flow — see foundation-models-adapters.md Pattern 5.

Time cost: hours (add caching, remove redundant compilation). The 24-hour window resets on its own; no retrain or re-export needed.

---

Cross-Referenced @Generable Issues

The following appear in adapter contexts but are general Foundation Models macro/schema issues. Solutions live in axiom-ai (skills/foundation-models-diag.md):

These are not adapter-specific — they affect any @Generable usage. Apply the foundation-models-diag patterns directly.

---

Quick Reference

---

Cross-References

• axiom-ai (skills/foundation-models-adapters.md) — discipline file (decision to train, pressure scenarios, audit checklists)
• axiom-ai (skills/foundation-models-adapters-ref.md) — toolkit CLIs, runtime API, compatibility matrix
• axiom-ai (skills/foundation-models-diag.md) — base Foundation Models diagnostics (@Generable macro issues, context overflow, guardrails)
• axiom-ai (skills/foundation-models.md) — Approach Triage (rungs 1-4 before adapter training)
• axiom-ai (skills/foundation-models-ref.md) — base Foundation Models API (LanguageModelSession, @Generable, Tool protocol)
• axiom-integration (skills/background-assets.md) — asset pack delivery, BAErrorCode patterns
• axiom-integration (skills/background-assets-ref.md) — AssetPackManager API surface

---

Resources

WWDC: 2024-10159, 2025-286, 2025-301, 2025-325

Docs: /foundationmodels/loading-and-using-a-custom-adapter-with-foundation-models, /foundationmodels/systemlanguagemodel/adapter, /bundleresources/entitlements/com.apple.developer.foundation-model-adapter, /backgroundassets

Skills: axiom-ai (skills/foundation-models-adapters.md), axiom-ai (skills/foundation-models-adapters-ref.md), axiom-ai (skills/foundation-models-diag.md), axiom-integration (skills/background-assets.md)

---

Last Updated: 2026-05-16 Toolkit Version: 26.0.0 Skill Type: Diagnostic

Foundation Models Custom Adapter Reference

Status — the custom-adapter runtime is a 26-cycle-only capability, obsoleted in 27.0. In the Xcode 27 SDK, SystemLanguageModel.Adapter, SystemLanguageModel(adapter:), and the entire init(name:)/init(fileURL:)/compile()/compatibleAdapterIdentifiers(name:)/removeObsoleteAdapters() surface are annotated deprecated: 26.4, obsoleted: 27.0 on iOS, iPadOS, macOS, and visionOS (never available on watchOS or tvOS). Code that uses them does not compile when the deployment target is 27.0 or later — the compiler reports 'Adapter' was obsoleted in iOS 27.0. It still builds when you deploy back to 26.0–26.x. The 27 SDK (beta 1) ships no replacement adapter-loading API and no renamed:/message: migration hint; Apple's direction for on-device specialization is Core AI (ahead-of-time model authoring) and bring-your-own-model custom providers (LanguageModelExecutor, see axiom-ai (skills/foundation-models-ref.md)), neither of which is a drop-in replacement. If any deployment target you support is 27.0 or later, custom adapters are off the table — work the Approach Triage (rungs 1-4) in axiom-ai (skills/foundation-models-adapters.md) or a custom provider instead. Everything below remains accurate for 26-cycle deployments.

Overview

This reference documents the Foundation Models Adapter Training Toolkit (Python) and the runtime API (SystemLanguageModel.Adapter) for loading custom-trained adapters in Swift. For when-and-why decisions, see axiom-ai (skills/foundation-models-adapters.md). For delivery API (AssetPackManager, StoreDownloaderExtension), see axiom-integration (skills/background-assets-ref.md) — this file owns training and runtime selection, the background-assets reference owns asset pack delivery.

Two halves of the workflow

• Build-time (Python toolkit): dataset preparation, training, evaluation, export. Runs on a developer Mac (Apple silicon, ≥32 GB) or Linux GPU machine. Produces .fmadapter packages.
• Runtime (Swift framework): adapter loading, compatibility checking, asset pack lookup, session creation. Runs on the user's device under FoundationModels.

Toolkit version

• Current: 26.0.0 (matches iOS / iPadOS / macOS / visionOS 26 — the platforms with the adapter runtime; never watchOS/tvOS)
• Cadence: a new toolkit ships per system-model OS release; adapters trained against an older toolkit are not guaranteed compatible with a newer base model

---

When to Use This Reference

Use this reference when:

• Setting up the Foundation Models Adapter Training Toolkit Python environment
• Authoring the training dataset JSONL (chat-turn or tool-calling schema)
• Looking up examples.train_adapter, examples.train_draft_model, examples.generate, or export.export_fmadapter CLI signatures
• Looking up SystemLanguageModel.Adapter method signatures
• Looking up SystemLanguageModel.Adapter.AssetError cases
• Wiring an adapter into a LanguageModelSession
• Implementing the per-base-model lifecycle (removeObsoleteAdapters(), compatibleAdapterIdentifiers(name:)) — 26-cycle deployments only
• Configuring the com.apple.developer.foundation-model-adapter entitlement

Related skills:

• axiom-ai (skills/foundation-models-adapters.md) — discipline file with decision tree, when-not-to-train, pressure scenarios, eval discipline
• axiom-ai (skills/foundation-models-adapters-diag.md) — diagnostic patterns for adapter-specific failures
• axiom-integration (skills/background-assets-ref.md) — AssetPackManager, StoreDownloaderExtension, manifest schema (the delivery half)
• axiom-ai (skills/foundation-models-ref.md) — base Foundation Models API (LanguageModelSession, @Generable, Tool protocol)

---

Toolkit Setup

Hardware requirements

• Mac: Apple silicon (M1 or later) with ≥32 GB unified memory. Mac Studio and Mac Pro recommended for longer training runs.
• Linux GPU: CUDA-capable machine; specific GPU memory requirements depend on adapter rank and batch size. Apple's docs do not pin a minimum.
• Storage: ≥100 GB free for toolkit assets, dataset, checkpoints, and exported adapter packs.

Software requirements

• Python: exactly 3.11. The toolkit's export/ folder pins coremltools versions that are not available for Python 3.12 / 3.13. Using a newer Python silently fails at export time with ModuleNotFoundError: coremltools.libmilstoragepython.
• Apple Developer Program membership: required for toolkit download. Sign in to the developer site, accept the toolkit license, then download.

Environment setup

# Create a clean 3.11 environment (conda or venv equivalent)
conda create -n fm-adapter python=3.11
conda activate fm-adapter

# Install toolkit dependencies
pip install -r requirements.txt

The toolkit ships a requirements.txt against pinned versions; do not loosen pins without understanding the export folder's expectations.

License constraint

The toolkit ships model assets used during training. The license is explicit: "You are only permitted to use these model assets for training adapters." These assets are not redistributable, not usable for analysis beyond training, and not usable for other ML projects.

Folder layout

foundation-models-adapter-toolkit-26.0.0/
├── examples/              # User-editable training scripts
│   ├── train_adapter.py
│   ├── train_draft_model.py
│   ├── generate.py
│   └── end_to_end_example.ipynb
├── export/                # SEALED — do not modify
│   └── export_fmadapter.py
├── requirements.txt
└── README.md

Critical: the toolkit's export/ folder is sealed. Apple's warning: "Code in the `export` folder should not be modified, since the export logic must match exactly to make your adapter compatible with the system model and Xcode." Modifications break runtime compatibility.

---

Dataset Schema

The toolkit consumes JSONL files where each line is one training conversation.

Basic chat-turn schema

{"messages": [{"role": "system", "content": "You summarize restaurant reviews."}, {"role": "user", "content": "The pasta was bland but the tiramisu was incredible."}, {"role": "assistant", "content": "Mixed dinner — pasta underwhelmed, tiramisu standout."}]}
{"messages": [{"role": "user", "content": "Service was slow but the views from the patio were worth it."}, {"role": "assistant", "content": "Slow service, scenic patio worth the wait."}]}

Roles:

• system (optional): role / persona / task description. Keep short and consistent across samples.
• user: the prompt the adapter must learn to handle.
• assistant: the desired output for this prompt.

Tool-calling schema extension

For adapters that must learn to invoke Tool protocol implementations, the assistant turn carries a tool_calls array:

{
  "messages": [
    {"role": "system", "content": "You help the user find restaurants. Use the getRestaurants tool for live data."},
    {"role": "user", "content": "Italian near me, open now"},
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "call_1",
          "type": "function",
          "function": {
            "name": "getRestaurants",
            "arguments": "{\"cuisine\":\"Italian\",\"openNow\":true,\"radius\":2000}"
          }
        }
      ]
    },
    {"role": "tool", "tool_call_id": "call_1", "content": "[{\"name\":\"Bella\",\"distance\":300},{\"name\":\"Trattoria\",\"distance\":900}]"},
    {"role": "assistant", "content": "Bella (300m) and Trattoria (900m) are open Italian options nearby."}
  ]
}

Required fields on each tool call:

• id: unique identifier for matching the subsequent tool response (tool_call_id)
• type: literal string "function"
• function.name: the Tool.name value the adapter should produce at inference time
• function.arguments: a JSON-encoded string containing the structured arguments matching the tool's @Generable Arguments type

Sample volumes

Apple's explicit framing: "Focus on quality over quantity. A smaller dataset of clear, consistent, and well-structured samples may be more effective than a larger dataset of noisy, low-quality samples."

Dataset file conventions

• One conversation per line (.jsonl)
• UTF-8 encoded
• No empty lines
• Split into train.jsonl and eval.jsonl before training; the toolkit does not auto-split

---

Training

CLI signature

python -m examples.train_adapter \
    --train-data path/to/train.jsonl \
    --eval-data path/to/eval.jsonl \
    --epochs 3 \
    --learning-rate 1e-4 \
    --batch-size 8 \
    --checkpoint-dir checkpoints/run_001

Hyperparameters

LoRA architecture

The toolkit trains rank-32 LoRA adapters against Apple's frozen on-device 3B-parameter base model. LoRA decomposes weight updates as ΔW = BA where B ∈ ℝ^(d×r) and A ∈ ℝ^(r×k) with rank r = 32. B is initialized to zero so the adapter starts as identity (ΔW = 0 at step 0), ensuring training begins from the base model's exact behavior.

Target modules (per Apple's 2024 tech report): attention W_q, W_v, W_k, W_o, plus feed-forward and projection layers. The toolkit does not expose per-module rank tuning; rank 32 is fixed.

Trainable parameter count: ~0.1% of base-model parameters, which is why the resulting adapter pack is ~160 MB rather than the multi-gigabyte size of full fine-tuning.

Checkpoint discipline

The trainer writes checkpoints periodically to --checkpoint-dir. Conventions:

• Retain every checkpoint for shipped-adapter training runs; required for rollback and ablation
• Tag checkpoints with run config in the filename or a sibling JSON (run_001_lr1e4_e3_b8.pt)
• Do not delete intermediate checkpoints until the final adapter passes all four eval axes; earlier checkpoints sometimes generalize better

Per-base-model-version targeting

Each toolkit version targets exactly one base-model version. You cannot train a single adapter that works across OS versions — you train one adapter per supported OS, each with the matching toolkit version.

---

Optional: Draft Model for Speculative Decoding

For latency-sensitive features, the toolkit can also train a smaller draft model used in speculative decoding to accelerate inference.

python -m examples.train_draft_model \
    --train-data path/to/train.jsonl \
    --epochs 3 \
    --learning-rate 1e-4 \
    --checkpoint-dir checkpoints/draft_001

Compilation rate limit

When loaded at runtime, the draft model is compiled into a device-specific form. On non-macOS platforms, the system enforces three draft model compilations per app per day. Hitting this limit returns an error on subsequent compilation attempts; the previously compiled draft model continues to work.

Implication: do not regenerate / recompile draft models on every app launch. Cache the compiled form and reuse it across sessions.

---

Evaluation

CLI signature

python -m examples.generate \
    --checkpoint checkpoints/run_001/step_5000.pt \
    --draft-checkpoint checkpoints/draft_001/step_3000.pt \
    --input path/to/eval_prompts.jsonl \
    --output predictions.jsonl

Four-axis eval requirement

Apple's docs: "Evaluation needs to be a custom process that makes sense for your specific use case." The toolkit provides examples.generate but does not provide eval metrics — you compute them against predictions.jsonl.

A complete eval covers:

1. Quantitative

• Task-appropriate metric (accuracy, F1, ROUGE, BLEU, custom)
• Defined before training, not after seeing results
• Compared against the base-model baseline (no adapter)

2. Qualitative — human grading

• Stratified sample of predictions
• Grader sees pairs (base vs adapter) blind
• Pick a small but defensible sample size (~100 pairs minimum)

3. Qualitative — larger-model grading

• Server LLM (Claude, GPT-4o-class) grades the full eval set
• Useful for catching regressions human graders miss at scale
• Not a substitute for human grading

4. Safety

• Re-run an internal red-team prompt set against the trained adapter
• Task-specific training can erode the base model's guardrails on adjacent topics
• You own safety eval for your task — Apple's base-model guardrails are necessary but not sufficient

Locale-specific eval groupings

Per Apple's 2025 tech report, locale eval is grouped as:

If the app ships in any non-US locale, run eval against the corresponding group. The base model supports 16 languages; the adapter's interaction with each is untested unless explicitly evaluated.

Hard rule

If any axis regresses against the base-model baseline, do not ship the adapter. A "+5% on task quality, -8% on safety eval" adapter is a net loss — the safety regression is paid for by user trust.

---

Export

CLI signature

python -m export.export_fmadapter \
    --checkpoint checkpoints/run_001/step_5000.pt \
    --draft-checkpoint checkpoints/draft_001/step_3000.pt \
    --adapter-name my_summarizer \
    --output-dir exports/

Adapter name regex

The runtime identifier regex is /fmadapter-\w+-\w+/ — the \w+ matches word characters (alphanumeric + underscore) but not hyphens. The framework constructs the full identifier as fmadapter-{name}-{variant}; if your --adapter-name contains a hyphen, the resulting identifier has three hyphens, the regex matches only the first \w+ between hyphens, and the adapter fails to load with SystemLanguageModel.Adapter.AssetError.invalidAdapterName.

✅ Valid: my_summarizer, restaurant_summary_v2, tagger_v1 ❌ Invalid: my-summarizer, restaurant-summary, tagger-v1

Output

The export step produces a {adapter-name}.fmadapter package — a structured directory containing the LoRA weight deltas, optional draft model, and metadata pinning the base-model version. This package is what you upload to Background Assets (Apple-hosted) or your CDN (server-hosted).

---

Entitlement

com.apple.developer.foundation-model-adapter

Required for deployment of apps that load custom adapters. Training and local testing do not require the entitlement.

Acquisition flow: 1. Account Holder (not just any team member) requests the entitlement from Apple via the developer portal 2. Apple reviews the request and grants the entitlement to the Account Holder's team 3. Provisioning profiles for apps that load adapters must include the entitlement 4. App Review verifies the entitlement is wired correctly

Entitlement key:

<key>com.apple.developer.foundation-model-adapter</key>
<true/>

Without the entitlement, the runtime SystemLanguageModel.Adapter initializers throw at app launch on production builds.

---

Runtime API

The entire runtime API below is deprecated: 26.4, obsoleted: 27.0 (iOS/iPadOS/macOS/visionOS; never on watchOS/tvOS). It compiles only when your deployment target is 26.x — the compile gate is the deployment-target ceiling, not a runtime check. At runtime, use if #available and keep a base-model fallback for every device whose installed OS has reached 27.

SystemLanguageModel.Adapter

import FoundationModels

// All members: @available(iOS/macOS/visionOS, deprecated: 26.4, obsoleted: 27.0)
public struct SystemLanguageModel.Adapter {
    public var creatorDefinedMetadata: [String : Any] { get }

    public init(name: String) throws
    public init(fileURL: URL) throws

    public func compile() async throws

    public static func removeObsoleteAdapters() throws
    public static func compatibleAdapterIdentifiers(name: String) -> [String]
}

There is no public isCompatible(_:) on Adapter. The .swiftinterface carries it only as an internal @usableFromInline symbol (ABI signature isCompatible(BackgroundAssets.AssetPack) -> Bool), not as public API, so it does not compile from source — do not call it. To gate an adapter asset-pack download to compatible variants, match the pack identifier against compatibleAdapterIdentifiers(name:) instead (see axiom-integration (skills/background-assets-ref.md) "Foundation Models Adapter Bridge").

init(name:)

let adapter = try SystemLanguageModel.Adapter(name: "my_summarizer")

Loads the adapter by name from a Background Assets-delivered asset pack. The framework picks the variant that matches the current base-model version using compatibleAdapterIdentifiers(name:) semantics internally.

Throws:

• AssetError.compatibleAdapterNotFound — no variant matches the device's base-model version
• AssetError.invalidAdapterName — name violates the /fmadapter-\w+-\w+/ regex
• AssetError.invalidAsset — asset pack files are corrupted or malformed
• Underlying I/O errors if the asset pack is not locally available

init(fileURL:)

let url = bundleURL.appendingPathComponent("my_summarizer.fmadapter")
let adapter = try SystemLanguageModel.Adapter(fileURL: url)

Loads from a direct file URL. Used primarily for testing — production adapters ship via Background Assets, not bundled file URLs.

compile() async throws

try await adapter.compile()

Compiles the adapter to the device-specific form and caches the result across launches (a later compile() returns the saved compiled draft model immediately). Called automatically on first use; can be invoked early to warm the cache. The rate limit applies to this method: three per app per day on non-macOS, counted when the adapter includes a draft model — see foundation-models-adapters.md "The compilation rate limit".

@concurrent per the function signature — runs off the calling actor's executor.

removeObsoleteAdapters()

try SystemLanguageModel.Adapter.removeObsoleteAdapters()

Removes adapter asset packs that no longer match any current base-model version. Call at app launch and after OS upgrades. Without this, obsolete adapter packs occupy storage indefinitely at ~160 MB per pack (three abandoned variants = ~480 MB).

compatibleAdapterIdentifiers(name:)

let ids = SystemLanguageModel.Adapter
    .compatibleAdapterIdentifiers(name: "my_summarizer")

Returns asset pack identifiers whose adapter variants match the current device's base-model version, in descending preference order. The first element is the recommended variant.

Return value:

• Non-empty array → at least one compatible variant exists and has been uploaded for this app
• Empty array → no compatible variant has been uploaded (or the device is not Apple Intelligence-capable)

Use this for the runtime selection contract:

let ids = SystemLanguageModel.Adapter
    .compatibleAdapterIdentifiers(name: "my_summarizer")

guard let preferredID = ids.first else {
    // No compatible adapter — fall back to base model.
    let session = LanguageModelSession()
    return session
}

Gating adapter downloads to compatible variants

To download only the adapter variants that match the device's current base-model version, match the asset-pack identifier against compatibleAdapterIdentifiers(name:) inside the download extension — there is no isCompatible(AssetPack) to call (see the Runtime API note above):

@main
struct AdapterDownloader: StoreDownloaderExtension {
    func shouldDownload(_ assetPack: AssetPack) -> Bool {
        guard assetPack.id.hasPrefix("fmadapter-") else { return true }
        let compatible = SystemLanguageModel.Adapter
            .compatibleAdapterIdentifiers(name: "my_summarizer")
        return compatible.contains(assetPack.id)
    }
}

See axiom-integration (skills/background-assets-ref.md) "Foundation Models Adapter Bridge" for full context on the extension pattern.

---

SystemLanguageModel.Adapter.AssetError

public enum SystemLanguageModel.Adapter.AssetError: Error, LocalizedError {
    case compatibleAdapterNotFound(Context)
    case invalidAdapterName(Context)
    case invalidAsset(Context)
}

Each case carries a Context value with diagnostic detail; check errorDescription for a human-readable message and recoverySuggestion for a suggested fix (both are String?, exposed via LocalizedError).

For the broader error space (ManagedBackgroundAssetsError, BAErrorCode), see axiom-integration (skills/background-assets-ref.md). For diagnostic flows that combine adapter errors with their root causes, see axiom-ai (skills/foundation-models-adapters-diag.md).

---

SystemLanguageModel Initializer for Adapters

SystemLanguageModel(adapter:) / SystemLanguageModel(adapter:guardrails:) are themselves obsoleted: 27.0 — 26-cycle deployments only (see the Runtime API note).

let adapter = try SystemLanguageModel.Adapter(name: "my_summarizer")

// Default guardrails
let model = SystemLanguageModel(adapter: adapter)

// Or override guardrails
let permissive = SystemLanguageModel.Guardrails.permissiveContentTransformations
let model = SystemLanguageModel(adapter: adapter, guardrails: permissive)

The adapter-aware initializer composes the trained adapter on top of the base model. Guardrails default to the base model's settings; override only with permissiveContentTransformations when your task requires looser content controls and you've completed the safety eval axis. See axiom-ai (skills/foundation-models-ref.md) for the full Guardrails API.

let session = LanguageModelSession(model: model)
let response = try await session.respond(to: "Summarize this restaurant review: ...")

---

Compatibility Matrix

Per-base-model-version pinning

Each adapter is bound to exactly one base-model version. The mapping is approximately:

Lookup rule: pick the toolkit version that matches the lowest system-model OS version you plan to support. Adapters trained against an older toolkit may or may not load on a newer OS — compatibleAdapterIdentifiers(name:) is the authoritative runtime answer.

App install base strategy

Adapter asset pack naming convention

Apple recommends asset pack IDs of the form fmadapter-{name}-{variant} where variant encodes the base-model version (e.g., fmadapter-my_summarizer-base26_0). The framework uses the variant suffix to disambiguate at lookup time.

Concrete example with three adapter variants for the same logical adapter:

The runtime resolves compatibleAdapterIdentifiers(name: "my_summarizer") to the appropriate ID for the device's current base model.

---

Complete End-to-End Pattern

Build-time

# 1. Author dataset
mkdir -p data
# write data/train.jsonl, data/eval.jsonl (see Dataset Schema)

# 2. Train
python -m examples.train_adapter \
    --train-data data/train.jsonl \
    --eval-data data/eval.jsonl \
    --epochs 3 \
    --learning-rate 1e-4 \
    --batch-size 8 \
    --checkpoint-dir checkpoints/summarizer_v1

# 3. (Optional) Train draft model for speculative decoding
python -m examples.train_draft_model \
    --train-data data/train.jsonl \
    --epochs 3 \
    --learning-rate 1e-4 \
    --checkpoint-dir checkpoints/summarizer_v1_draft

# 4. Evaluate
python -m examples.generate \
    --checkpoint checkpoints/summarizer_v1/step_5000.pt \
    --draft-checkpoint checkpoints/summarizer_v1_draft/step_3000.pt \
    --input data/eval.jsonl \
    --output predictions.jsonl
# Compute quantitative metrics, human grading, larger-model grading, safety,
# locale-specific eval against predictions.jsonl. See "Evaluation" section.

# 5. Export
python -m export.export_fmadapter \
    --checkpoint checkpoints/summarizer_v1/step_5000.pt \
    --draft-checkpoint checkpoints/summarizer_v1_draft/step_3000.pt \
    --adapter-name my_summarizer \
    --output-dir exports/

# 6. Package for Background Assets delivery
# See axiom-integration (skills/background-assets.md) for xcrun ba-package usage:
xcrun ba-package template -o Manifest.json
# Edit Manifest.json:
# {
#   "assetPackID": "fmadapter-my_summarizer-base26_0",
#   "downloadPolicy": {"onDemand": {}},
#   "fileSelectors": [{"file": "exports/my_summarizer.fmadapter"}],
#   "platforms": []
# }
xcrun ba-package Manifest.json -o my_summarizer.aar

# 7. Upload to App Store Connect (Apple-hosted) or push to CDN (server-hosted)

Runtime

import FoundationModels
import BackgroundAssets

@MainActor
final class AdapterLifecycle {
    func session(forAdapter name: String) async throws -> LanguageModelSession {
        // Clean up adapters that don't match this OS.
        try SystemLanguageModel.Adapter.removeObsoleteAdapters()

        // Pick the compatible variant.
        let ids = SystemLanguageModel.Adapter
            .compatibleAdapterIdentifiers(name: name)

        guard let preferredID = ids.first else {
            // No compatible adapter — degrade to base model.
            return LanguageModelSession()
        }

        // Ensure the asset pack is local.
        let pack = try await AssetPackManager.shared.assetPack(withID: preferredID)
        try await AssetPackManager.shared.ensureLocalAvailability(of: pack)

        // Load and use.
        let adapter = try SystemLanguageModel.Adapter(name: name)
        try await adapter.compile()
        let model = SystemLanguageModel(adapter: adapter)
        return LanguageModelSession(model: model)
    }

    func handleOSUpgrade() async throws {
        try? SystemLanguageModel.Adapter.removeObsoleteAdapters()
        try await AssetPackManager.shared.checkForUpdates()
    }
}

Extension (Apple-hosted)

import BackgroundAssets
import ExtensionFoundation
import StoreKit
import FoundationModels

@main
struct AdapterDownloader: StoreDownloaderExtension {
    func shouldDownload(_ assetPack: AssetPack) -> Bool {
        // For FM adapter packs, gate on compatibility with current base model.
        guard assetPack.id.hasPrefix("fmadapter-") else { return true }
        let compatible = SystemLanguageModel.Adapter
            .compatibleAdapterIdentifiers(name: "my_summarizer")
        return compatible.contains(assetPack.id)
    }
}

For server-hosted delivery (BADownloaderExtension), see axiom-integration (skills/background-assets-ref.md).

---

API Quick Reference

• Toolkit CLI: examples.train_adapter, examples.train_draft_model, examples.generate, export.export_fmadapter
• Toolkit Python entry points: documented in toolkit README.md; do not modify export/
• Runtime types: SystemLanguageModel.Adapter (struct), SystemLanguageModel.Adapter.AssetError (enum)
• Runtime initializers: init(name:), init(fileURL:)
• Runtime instance methods: compile() async throws
• Runtime static methods: removeObsoleteAdapters() throws, compatibleAdapterIdentifiers(name:) -> [String] (no public isCompatible(_:) — see Runtime API note)
• Runtime status: the Adapter type and its loading surface (init(name:)/init(fileURL:)/compile()/removeObsoleteAdapters()/compatibleAdapterIdentifiers(name:)) are deprecated: 26.4, obsoleted: 27.0; SystemLanguageModel(adapter:guardrails:) is obsoleted: 27.0 (not separately deprecated); AssetError is deprecated: 26.4 (not obsoleted). All iOS/iPadOS/macOS/visionOS; never watchOS/tvOS — 26-cycle deployments only, no 27 replacement
• Error cases: .compatibleAdapterNotFound(_), .invalidAdapterName(_), .invalidAsset(_)
• Composition: SystemLanguageModel(adapter:), SystemLanguageModel(adapter:guardrails:), LanguageModelSession(model:)
• Entitlement: com.apple.developer.foundation-model-adapter (deployment only)
• Rate limits: 3 draft-model compilations per app per day on non-macOS platforms
• Sizing: ~160 MB per adapter pack; 200 GB / 100-pack Apple-hosted quota per app (shared with all asset packs); see axiom-integration (skills/background-assets-ref.md)

---

Resources

WWDC: 2024-10159, 2024-10160, 2025-248, 2025-286, 2025-301, 2025-325

Docs: /apple-intelligence/foundation-models-adapter (toolkit hub article), /foundationmodels, /foundationmodels/loading-and-using-a-custom-adapter-with-foundation-models, /foundationmodels/systemlanguagemodel/adapter, /bundleresources/entitlements/com.apple.developer.foundation-model-adapter, /backgroundassets

Apple ML Research: apple-foundation-models-tech-report-2025 (arXiv 2507.13575), Apple Intelligence Foundation Language Models (arXiv 2407.21075)

Background: LoRA paper (Hu et al., arXiv 2106.09685)

Skills: axiom-ai (skills/foundation-models-adapters.md), axiom-ai (skills/foundation-models-adapters-diag.md), axiom-ai (skills/foundation-models.md), axiom-ai (skills/foundation-models-ref.md), axiom-integration (skills/background-assets.md), axiom-integration (skills/background-assets-ref.md)

---

Last Updated: 2026-06-11 Toolkit Version: 26.0.0 Platforms: iOS / iPadOS / macOS / visionOS 26.0–26.x only (runtime deprecated 26.4, obsoleted 27.0; never watchOS/tvOS); macOS 14+ Apple silicon ≥32 GB or Linux GPU (training) Skill Type: Reference

Foundation Models Evaluations Reference

Overview

The Evaluations framework (import Evaluations, OS27 — all Apple platforms except tvOS) is a Swift-native harness for measuring the quality of a generative-AI feature as you iterate on its prompts, instructions, schema, or model. You define an Evaluation — a dataset of inputs with expected outputs, plus Evaluators that score each result into named Metrics — and run it from a Swift Testing test or directly. It works with any LanguageModel (the on-device SystemLanguageModel, PrivateCloudComputeLanguageModel, or a custom provider), so it is the measurement half of the Foundation Models workflow: axiom-ai (skills/foundation-models.md) builds the feature, this skill proves it got better (or didn't regress).

This is the disciplined alternative to "eyeball a few outputs and ship." The custom-adapter four-axis eval discipline in axiom-ai (skills/foundation-models-adapters.md) predates this framework; for 27-cycle features, express those axes as Metrics here.

When to Use This Reference

Use when:

• Measuring whether a prompt/instruction/schema change improved or regressed a Foundation Models feature
• Building a regression suite for an AI feature (run it in CI via Swift Testing)
• Scoring open-ended output where pass/fail isn't mechanical — use a model-as-judge
• Evaluating an agentic feature's tool-calling trajectory (did it call the right tools, in the right order, with the right arguments?)
• Synthesizing a larger evaluation dataset from a handful of seed examples

The shape of an Evaluation

An Evaluation is a protocol with four moving parts:

@available(anyAppleOS 27, *)
public protocol Evaluation: Sendable {
    var dataset: SampleLoader { get }                          // inputs + expected outputs
    func subject(from sample: Sample) async throws -> Subject  // run your feature on one input
    @EvaluatorsBuilder var evaluators: Evaluators { get }      // score the result into Metrics
    func aggregateMetrics(using aggregator: inout MetricsAggregator)  // required — no default; implement it (may aggregate nothing)
}

subject(from:) is where you invoke the feature under test (e.g. start a LanguageModelSession and call respond) and return its output as the Subject. The framework runs every sample through it, feeds each result to every Evaluator, and aggregates the Metrics.

A complete example

import Evaluations
import FoundationModels

@Generable
struct BookTags: Codable {
    @Guide(description: "Themes, genres, moods, and topics", .count(3...8))
    var tags: [String]
}

@available(anyAppleOS 27, *)
struct BookTaggingEvaluation: Evaluation {
    let tagCount = Metric("TagCount")

    var dataset: ArrayLoader<ModelSample<BookTags>> {
        ArrayLoader(samples: Book.sampleBooks.map { book in
            ModelSample(prompt: book.review, expected: BookTags(tags: book.tags))
        })
    }

    func subject(from sample: ModelSample<BookTags>) async throws -> ModelSubject<BookTags> {
        let session = LanguageModelSession(instructions: "Tag this book review.")
        let tags = try await session.respond(to: sample.prompt, generating: BookTags.self).content
        return ModelSubject(value: tags)
    }

    @EvaluatorsBuilder<ModelSample<BookTags>, ModelSubject<BookTags>>
    var evaluators: Evaluators {
        Evaluator { _, subject in
            let count = subject.value.tags.count
            return (3...8).contains(count)
                ? tagCount.passing(rationale: "\(count) tags")
                : tagCount.failing(rationale: "Got \(count) tags, expected 3–8")
        }
    }

    // Required by Evaluation (no default) — even a one-liner satisfies it.
    func aggregateMetrics(using aggregator: inout MetricsAggregator) {
        aggregator.computeMean(of: tagCount)
    }
}

Metrics & Evaluators

A Metric is a named score channel. An Evaluator's closure receives the original (input, subject) and returns a Metric carrying the outcome:

public struct Metric {
    public init(_ name: String)
    public func passing(rationale: String? = nil) -> Metric
    public func failing(rationale: String? = nil) -> Metric
    public func scoring(_ value: Double, rationale: String? = nil) -> Metric   // numeric outcome
    public func ignore(rationale: String? = nil) -> Metric                     // exclude this sample
}

// Evaluator's closure: (Input, ModelSubject<ExpectedValue>) async throws -> Metric
Evaluator { input, subject in
    subject.value.tags.allSatisfy { !$0.contains(" ") }
        ? wordCount.passing()
        : wordCount.failing(rationale: "a tag had multiple words")
}

The closure is async throws, so an evaluator can call a service, look up a reference set, or run another model. Use .passing()/.failing() for boolean checks, .scoring(_:) for a numeric judgment (model-as-judge produces these), and .ignore() to drop a sample from a metric's aggregate. Each distinct Metric name becomes a column in the result. subject.value is the model output you returned from subject(from:).

Datasets & Loaders

ModelSample pairs a prompt with the expected output; ArrayLoader is the in-memory Loader:

ModelSample(prompt: "okay I am OBSESSED…", expected: BookTags(tags: ["classic", "romance"]))
ArrayLoader(samples: [sample1, sample2, /* … */])

ModelSample(prompt:expected:instructions:generationSchema:expectations:) — expected is optional (omit it for unsupervised judging), instructions/generationSchema override per-sample, and expectations: carries a TrajectoryExpectation for tool-call evaluation (below). For large or streamed corpora use JSONLoader / StreamLoader, or conform your own type to Loader.

Synthesizing more samples

Grow a seed dataset with the model itself. makeSamples is the convenience — it's an extension on the array of seed samples (not on the Loader), so call it on [ModelSample], not on ArrayLoader. SampleGenerator is the configurable form:

let prompt = Prompt("Generate diverse book reviews and matching tags across genres and eras.")
let seeds = Book.sampleBooks.map { ModelSample(prompt: $0.review, expected: BookTags(tags: $0.tags)) }

var expanded = seeds
for try await sample in seeds.makeSamples(prompt, targetCount: 100) {
    expanded.append(sample)
}

// Full control over the generating session, sampling strategy, and a validator:
let generator = SampleGenerator<ModelSample<BookTags>>(
    prompt, samples: seeds, targetCount: 100,
    sessionProvider: { LanguageModelSession(model: PrivateCloudComputeLanguageModel(),
                                            instructions: "Generate realistic, diverse book reviews…") },
    samplingStrategy: .random(),
    validator: { sample in sample.promptDescription.count >= 100 }   // promptDescription, not prompt.description
)
for try await sample in generator.run() { expanded.append(sample) }

The validator rejects samples that don't meet your bar (e.g. minimum length); a larger model via PrivateCloudComputeLanguageModel makes a stronger generator. SampleGenerator is an actor — iterate its run() async sequence.

Running an evaluation

From Swift Testing (regression suite)

The .evaluates(_:info:) trait runs the evaluation; read the result from EvaluationContext.current and assert on an optimization target — the metric you're trying to move:

@Test("Book tagging quality", .evaluates(BookTaggingEvaluation()))
func bookTagging() async throws {
    let result = EvaluationContext.current.result
    #expect(result.aggregateValue(.mean(of: BookTaggingEvaluation().tagCount)) >= 0.8)
}

result.aggregateValue(_ operation: AggregationOperation) -> Double reads an aggregate back out. This turns "is the feature good enough?" into a CI gate.

Directly

let result = try await BookTaggingEvaluation().run(info: ["build": "1234"])

Aggregating metrics

Implement aggregateMetrics(using:) (a required Evaluation member with no default) to compute statistics across all samples. MetricsAggregator offers computeMean/Median/Mode/Minimum/Maximum/StandardDeviation/Variance(of:) and group(_:_:) for nested sections:

func aggregateMetrics(using aggregator: inout MetricsAggregator) {
    aggregator.computeMean(of: tagCount)
    aggregator.group("Tag totals") { a in
        a.computeStandardDeviation(of: tagTotal)
        a.computeVariance(of: tagTotal)
    }
}

AggregationOperation mirrors these (.mean(of:), .median(of:), .mode(of:), .minimum(of:), .maximum(of:), .standardDeviation(of:), .variance(of:), .custom(label:)) for aggregateValue.

Model-as-judge (open-ended output)

When correctness isn't mechanical, score with another model. ModelJudgeEvaluator runs a judge LanguageModel against a ScoringScale:

ModelJudgeEvaluator(
    "Helpfulness",
    scale: .numeric([1.0: "unhelpful", 3.0: "adequate", 5.0: "excellent"]),
    judge: SystemLanguageModel(),         // or PrivateCloudComputeLanguageModel() for a tougher judge
    scoringMode: .discrete                 // .discrete or .continuous
)

ScoringScale factories: .numeric([Double: String]), .passFail(passDescription:failDescription:), .custom(SomeScoreLevel.self) (your ScoreLevel-conforming enum). For multi-axis judging pass dimensions: [ScoreDimension(_ name:description:scale:)] instead of a single scale. A judge produces a numeric Metric.scoring(_:rationale:) rather than pass/fail. To customize the rubric, use the prompt:-taking init — ModelJudgeEvaluator(_:scale:judge:scoringMode:prompt:) (the prompt: overloads drop the default judge, so name the judge explicitly) — passing ModelJudgePrompt(instructions:evaluationTarget:reference:). Judge alignment matters: validate the judge against human grades before trusting it, and re-check for drift each model release (WWDC 335).

Agentic / tool-call evaluation

For a feature that calls tools, evaluate the trajectory, not just the final text. Attach a TrajectoryExpectation to each sample and score with ToolCallEvaluator:

let sample = ModelSample(
    prompt: "What hikes have I gone on near Big Sur?",
    expected: nil,
    expectations: TrajectoryExpectation(
        unordered: [ ToolExpectation("searchSpotlight",
                                     arguments: [.keyOnly(argumentName: "query")]) ]
    )
)

// In the evaluation's evaluators:
ToolCallEvaluator(allPass: Metric("AllToolsMatched"),
                  percentagePass: Metric("ToolMatchRate"))

TrajectoryExpectation inits: (ordered:unordered:allowsAdditionalToolCalls:), (ordered:unordered:disallowed:), or (unordered:). ToolExpectation(_ name: String, arguments: [ArgumentMatcher]) declares one expected call. ArgumentMatcher has nine cases for matching a tool argument: .exact(argumentName:value:), .keyOnly(argumentName:), .oneOf(argumentName:allowedValues:), .range(argumentName:minimum:maximum:) (both bounds Double?), .pattern(argumentName:regex:), .contains(argumentName:substring:), .hasPrefix(argumentName:prefix:), .hasSuffix(argumentName:suffix:), and .naturalLanguage(argumentName:criteria:). ToolCallEvaluator(allPass:percentagePass:argumentMatchModel:) takes a model to judge .naturalLanguage argument matches.

Hill-climbing workflow

WWDC 335's loop: pick one optimization-target metric, change one thing (instructions, prompt, schema, or model), re-run the suite, keep the change only if the target moved up without regressing the guardrail metrics. The Evaluations report makes each round measurable instead of vibes-based. Persist each round's result to compare across runs — EvaluationResult.saveJSON(to:) / EvaluationResult.loadJSON(from:) (and, for an appended history, saveJSONLines(to:) — which is a method on a [EvaluationResult] collection, not on a single result — plus the static EvaluationResult.loadJSONLines(from:)). Watch for judge drift when a model-as-judge is part of the loop — a judge that shifts between releases silently moves your baseline.

API Quick Reference

• `Evaluation` (protocol) — dataset: some Loader, subject(from:) async throws -> Subject, @EvaluatorsBuilder var evaluators, aggregateMetrics(using:); run(info:) async throws -> EvaluationResult.
• `Metric` — init(_:), .passing(rationale:), .failing(rationale:), .scoring(_:rationale:), .ignore(rationale:).
• `Evaluator { (input, subject) async throws -> Metric }`; subject.value is the output.
• `ModelSample(prompt:expected:instructions:generationSchema:expectations:)`, .promptDescription; loaders ArrayLoader, JSONLoader, StreamLoader, Loader.
• `[ModelSample].makeSamples(_:targetCount:sessionProvider:validator:)` (on the array) / `SampleGenerator(_:samples:targetCount:sessionProvider:samplingStrategy:validator:)` (actor; iterate run()).
• Swift Testing: .evaluates(_:info:) trait, EvaluationContext.current.result, result.aggregateValue(.mean(of:)); persist via EvaluationResult.saveJSON(to:)/loadJSON(from:) (and [EvaluationResult].saveJSONLines(to:) on a results array / static EvaluationResult.loadJSONLines(from:)).
• `MetricsAggregator` — computeMean/Median/Mode/Minimum/Maximum/StandardDeviation/Variance(of:), group(_:_:); AggregationOperation cases mirror these + .custom(label:).
• `ModelJudgeEvaluator(_:scale:judge:scoringMode:)` / (judge:dimensions:scoringMode:) / prompt:-taking overloads; ScoringScale.numeric/.passFail/.custom; ScoreDimension; ScoringMode.discrete/.continuous; ModelJudgePrompt.
• `ToolCallEvaluator(allPass:percentagePass:argumentMatchModel:)`; TrajectoryExpectation, ToolExpectation(_:arguments:), ArgumentMatcher (9 cases: .exact/.keyOnly/.oneOf/.range/.pattern/.contains/.hasPrefix/.hasSuffix/.naturalLanguage).

Resources

WWDC: 2026-298, 2026-299, 2026-335, 2026-246

Docs: /Evaluations, /Evaluations/designing-effective-evaluations, /Evaluations/generating-synthetic-evaluation-datasets, /foundationmodels

Skills: axiom-ai (skills/foundation-models.md), axiom-ai (skills/foundation-models-ref.md), axiom-ai (skills/foundation-models-adapters.md)

---

Last Updated: 2026-06-11 Platforms: iOS / iPadOS / macOS / watchOS / visionOS 27+ (not tvOS) Skill Type: Reference