Coreml

Core ML Swift Integration

Load, configure, and run Core ML models in iOS apps. This skill covers the Swift side: model loading, prediction, MLTensor, profiling, and deployment. Target iOS 26+ with Swift 6.3, backward-compatible to iOS 14 unless noted.

Scope boundary: Python-side model conversion, optimization (quantization,

palettization, pruning), and framework selection live in the apple-on-device-ai

skill. This skill owns Swift integration only.

See references/coreml-swift-integration.md for complete code patterns including actor-based caching, batch inference, image preprocessing, and testing.

Contents

• Loading Models
• Model Configuration
• Making Predictions
• MLTensor (iOS 18+)
• Working with MLMultiArray
• Image Preprocessing
• Multi-Model Pipelines
• Vision Integration
• Performance Profiling
• Model Deployment
• Memory Management
• Common Mistakes
• Review Checklist
• References

Loading Models

Auto-Generated Classes

When you add a .mlmodel or .mlpackage to an app target, Xcode generates a Swift class with typed input/output. Use this whenever possible.

import CoreML

let config = MLModelConfiguration()
config.computeUnits = .all

let model = try MyImageClassifier(configuration: config)

Manual Loading

Load from a URL when the model is downloaded at runtime or stored outside the bundle.

let modelURL = Bundle.main.url(
    forResource: "MyModel", withExtension: "mlmodelc"
)!
let model = try MLModel(contentsOf: modelURL, configuration: config)

Async Loading (iOS 15+)

Load models without blocking the main thread. Prefer this for large models.

let model = try await MLModel.load(
    contentsOf: modelURL,
    configuration: config
)

Compile at Runtime (iOS 16+)

Compile a .mlpackage or .mlmodel to .mlmodelc on device. Useful for models downloaded from a server. Do this once per model version, not on every launch.

let compiledURL = try await MLModel.compileModel(at: packageURL)
let model = try await MLModel.load(contentsOf: compiledURL, configuration: config)

Cache the compiled URL -- recompiling on every launch is a bug. Copy compiledURL to a persistent location (e.g., Application Support). When reviewing runtime-loaded models, call out both facts together: async MLModel.compileModel(at:) is iOS 16+, and compiled models must be cached so the app does not recompile on every launch.

Model Configuration

MLModelConfiguration controls compute units, GPU access, and model parameters.

Compute Units Decision Table

let config = MLModelConfiguration()
config.computeUnits = .cpuAndNeuralEngine

// Optional fallback for constrained work after profiling and policy review
config.computeUnits = .cpuOnly

Configuration Properties

let config = MLModelConfiguration()
config.computeUnits = .all
config.allowLowPrecisionAccumulationOnGPU = true // faster, slight precision loss

Making Predictions

With Auto-Generated Classes

The generated class provides typed input/output structs.

let model = try MyImageClassifier(configuration: config)
let input = MyImageClassifierInput(image: pixelBuffer)
let output = try model.prediction(input: input)
print(output.classLabel)        // "golden_retriever"
print(output.classLabelProbs)   // ["golden_retriever": 0.95, ...]

With MLDictionaryFeatureProvider

Use when inputs are dynamic or not known at compile time.

let inputFeatures = try MLDictionaryFeatureProvider(dictionary: [
    "image": MLFeatureValue(pixelBuffer: pixelBuffer),
    "confidence_threshold": MLFeatureValue(double: 0.5),
])
let output = try model.prediction(from: inputFeatures)
let label = output.featureValue(for: "classLabel")?.stringValue

Prediction Inside Async Workflows

MLModel.prediction(...) is synchronous. In async pipelines, keep model loading async, then run prediction from an actor or non-main task without adding await to the prediction call.

let output = try model.prediction(from: inputFeatures)

Batch Prediction

Process multiple inputs in one call for better throughput.

let batchInputs = try MLArrayBatchProvider(array: inputs.map { input in
    try MLDictionaryFeatureProvider(dictionary: ["image": MLFeatureValue(pixelBuffer: input)])
})
let batchOutput = try model.predictions(fromBatch: batchInputs)
for i in 0..<batchOutput.count {
    let result = batchOutput.features(at: i)
    print(result.featureValue(for: "classLabel")?.stringValue ?? "unknown")
}

Use predictions(fromBatch:) when batching without explicit MLPredictionOptions. Use predictions(from:options:) only when passing both an MLBatchProvider and MLPredictionOptions; predictions(from:) by itself is not the no-options batch API.

Stateful Prediction (iOS 18+)

Use MLState for models that maintain state across predictions (sequence models, LLMs, audio accumulators). Create state once and pass it to each prediction call.

let state = model.makeState()

// Each synchronous prediction carries forward the internal model state
for frame in audioFrames {
    let input = try MLDictionaryFeatureProvider(dictionary: [
        "audio_features": MLFeatureValue(multiArray: frame)
    ])
    let output = try model.prediction(from: input, using: state)
    let classification = output.featureValue(for: "label")?.stringValue
}

MLState is Sendable, but Sendable does not make one state safe for concurrent inference. Predictions using the same state must be serialized; do not read or write state buffers while a prediction is in flight. Call model.makeState() for each independent concurrent stream. If you need MLPredictionOptions, iOS 18+ also provides the async prediction(from:using:options:) overload; the same one-in-flight-per-state rule still applies.

MLTensor (iOS 18+)

MLTensor is a Swift-native multidimensional array for pre/post-processing. Operations run lazily -- call await tensor.shapedArray(of:) to materialize results.

import CoreML

// Creation
let tensor = MLTensor([1.0, 2.0, 3.0, 4.0])
let zeros = MLTensor(zeros: [3, 224, 224], scalarType: Float.self)

// Reshaping
let reshaped = tensor.reshaped(to: [2, 2])

// Math operations
let softmaxed = tensor.softmax(alongAxis: -1)
let centered = tensor - tensor.mean()

// Interop with MLShapedArray / MLMultiArray
let shaped = await tensor.shapedArray(of: Float.self)
let multiArray = try MLMultiArray(shaped)
let shapedAgain = MLShapedArray<Float>(multiArray)

Do not invent MLTensor APIs for statistics or bridging. Avoid examples such as MLTensor(multiArray), tensor.std(), tensor.standardDeviation(), direct lazy-buffer access, or synchronous extraction; perform unsupported DSP/statistics outside the tensor pipeline or with source-confirmed tensor operations.

Working with MLMultiArray

MLMultiArray is the primary data exchange type for non-image model inputs and outputs. Use it when the auto-generated class expects array-type features.

// Create a 3D array: [batch, sequence, features]
let array = try MLMultiArray(shape: [1, 128, 768], dataType: .float32)

// Write values
for i in 0..<128 {
    array[[0, i, 0] as [NSNumber]] = NSNumber(value: Float(i))
}

// Read values
let value = array[[0, 0, 0] as [NSNumber]].floatValue

let data: [Float] = [1.0, 2.0, 3.0]
let shaped = MLShapedArray(scalars: data, shape: [3])
let fromShaped = try MLMultiArray(shaped)

See references/coreml-swift-integration.md for advanced MLMultiArray patterns including NLP tokenization and audio feature extraction.

Image Preprocessing

Image models expect CVPixelBuffer input. Use CGImage conversion for photos from the camera or photo library. Vision's VNCoreMLRequest handles this automatically; manual conversion is needed only for direct MLModel prediction.

import CoreVideo

func createPixelBuffer(from cgImage: CGImage, width: Int, height: Int) -> CVPixelBuffer? {
    var pixelBuffer: CVPixelBuffer?
    let attrs: [CFString: Any] = [
        kCVPixelBufferCGImageCompatibilityKey: true,
        kCVPixelBufferCGBitmapContextCompatibilityKey: true,
    ]
    CVPixelBufferCreate(kCFAllocatorDefault, width, height,
                        kCVPixelFormatType_32ARGB, attrs as CFDictionary, &pixelBuffer)

    guard let buffer = pixelBuffer else { return nil }
    CVPixelBufferLockBaseAddress(buffer, [])
    let context = CGContext(
        data: CVPixelBufferGetBaseAddress(buffer),
        width: width, height: height,
        bitsPerComponent: 8, bytesPerRow: CVPixelBufferGetBytesPerRow(buffer),
        space: CGColorSpaceCreateDeviceRGB(),
        bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue
    )
    context?.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height))
    CVPixelBufferUnlockBaseAddress(buffer, [])
    return buffer
}

For additional preprocessing patterns (normalization, center-cropping), see references/coreml-swift-integration.md.

Multi-Model Pipelines

Chain models when preprocessing or postprocessing requires a separate model.

// Sequential inference: preprocessor -> main model -> postprocessor
let preprocessed = try preprocessor.prediction(from: rawInput)
let mainOutput = try mainModel.prediction(from: preprocessed)
let finalOutput = try postprocessor.prediction(from: mainOutput)

For Xcode-managed pipelines, use the pipeline model type in the .mlpackage. Each sub-model runs on its optimal compute unit.

Vision Integration

Use Vision to run Core ML image models with automatic image preprocessing (resizing, normalization, color space, orientation).

Modern: CoreMLRequest (iOS 18+)

import Vision
import CoreML

let model = try MLModel(contentsOf: modelURL, configuration: config)
let request = CoreMLRequest(model: .init(model))
let results = try await request.perform(on: cgImage)

if let classification = results.first as? ClassificationObservation {
    print("\(classification.identifier): \(classification.confidence)")
}

Legacy: VNCoreMLRequest

let vnModel = try VNCoreMLModel(for: model)
let request = VNCoreMLRequest(model: vnModel) { request, error in
    guard let results = request.results as? [VNRecognizedObjectObservation] else { return }
    for observation in results {
        let label = observation.labels.first?.identifier ?? "unknown"
        let confidence = observation.labels.first?.confidence ?? 0
        let boundingBox = observation.boundingBox // normalized coordinates
        print("\(label): \(confidence) at \(boundingBox)")
    }
}
request.imageCropAndScaleOption = .scaleFill

let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer)
try handler.perform([request])

For complete Vision framework patterns (text recognition, barcode detection,

document scanning), see the vision-framework skill.

Performance Profiling

MLComputePlan (iOS 17.4+)

Inspect which compute device each operation will use before running predictions.

let computePlan = try await MLComputePlan.load(
    contentsOf: modelURL, configuration: config
)
guard case let .program(program) = computePlan.modelStructure else { return }
guard let mainFunction = program.functions["main"] else { return }

for operation in mainFunction.block.operations {
    let deviceUsage = computePlan.deviceUsage(for: operation)
    let estimatedCost = computePlan.estimatedCost(of: operation)
    print("\(operation.operatorName): \(String(describing: deviceUsage?.preferred))")
}

Instruments

Use the Core ML instrument template in Instruments to profile:

• Model load time
• Prediction latency (per-operation breakdown)
• Compute device dispatch (CPU/GPU/ANE per operation)
• Memory allocation

Run outside the debugger for accurate results (Xcode: Product > Profile).

Model Deployment

Bundle vs Downloaded Assets

Size Considerations

• For iOS/iPadOS 18+, App Store Connect lists a 4 GB thinned app bundle limit

and 8 GB thinned ODR asset-pack limit.

• Prefer Background Assets for new large or updateable model assets; keep ODR

guidance for existing projects that already use it.

• Pre-compile to .mlmodelc to skip on-device compilation
• For downloaded .mlmodel or .mlpackage files, compile once with

MLModel.compileModel(at:), move the resulting .mlmodelc out of Core ML's temporary location, and cache it by model version.

• Validate memory and performance on physical target devices, especially the

lowest-memory supported device. Check model load, first prediction, repeated predictions, background/foreground transitions, and low-memory behavior.

For Background Assets, make the asset pack locally available, resolve the model URL, then load the compiled model with MLModel.load(contentsOf:configuration:).

// Existing On-Demand Resources project
let request = NSBundleResourceRequest(tags: ["ml-model-v2"])
try await request.beginAccessingResources()
let modelURL = Bundle.main.url(forResource: "LargeModel", withExtension: "mlmodelc")!
let model = try await MLModel.load(contentsOf: modelURL, configuration: config)
// Call request.endAccessingResources() when done

Memory Management

• Unload on background: Release model references when the app enters background

to free GPU/ANE memory. Reload on foreground return.

• Choose compute units by context: use .all by default. Consider .cpuOnly

only when profiling or app policy shows accelerator contention, thermal state, energy budget, deterministic testing, or a legitimate background execution constraint makes CPU the right tradeoff.

• Share model instances: Never create multiple MLModel instances from the same

compiled model. Use an actor to provide shared access.

• Monitor memory pressure: Large models (>100 MB) can trigger memory warnings.

Register for UIApplication.didReceiveMemoryWarningNotification and release cached models when under pressure.

See references/coreml-swift-integration.md for an actor-based model manager with lifecycle-aware loading and cache eviction.

Common Mistakes

DON'T: Load models on the main thread. DO: Use MLModel.load(contentsOf:configuration:) async API or load on a background actor. Why: Large models can take seconds to load, freezing the UI.

DON'T: Recompile .mlpackage to .mlmodelc on every app launch. DO: Compile once with MLModel.compileModel(at:) and cache the compiled URL persistently. Why: Compilation is expensive. Cache the .mlmodelc in Application Support.

DON'T: Hardcode .cpuOnly unless you have a specific reason. DO: Use .all and let the system choose the optimal compute unit. Why: .all enables Neural Engine and GPU, which are faster and more energy-efficient.

DON'T: Claim GPU or Neural Engine are categorically unavailable for all background-adjacent work. DO: Treat background execution as policy-, mode-, contention-, thermal-, and energy-dependent, and profile the actual workload on device. Why: Apps may be suspended, throttled, or limited by their background mode; .cpuOnly is a tradeoff, not a universal requirement.

DON'T: Ignore MLFeatureValue type mismatches between input and model expectations. DO: Match types exactly -- use MLFeatureValue(pixelBuffer:) for images, not raw data. Why: Type mismatches cause cryptic runtime crashes or silent incorrect results.

DON'T: Create a new MLModel instance for every prediction. DO: Load once and reuse. Use an actor to manage the model lifecycle. Why: Model loading allocates significant memory and compute resources.

DON'T: Skip error handling for model loading and prediction. DO: Catch errors and provide fallback behavior when the model fails. Why: Models can fail to load on older devices or when resources are constrained.

DON'T: Assume all operations run on the Neural Engine. DO: Use MLComputePlan (iOS 17.4+) to verify device dispatch per operation. Why: Unsupported operations fall back to CPU, which may bottleneck the pipeline.

DON'T: Process images manually before passing to Vision + Core ML. DO: Use CoreMLRequest (iOS 18+) or VNCoreMLRequest (legacy) to let Vision handle preprocessing. Why: Vision handles orientation, scaling, and pixel format conversion correctly.

Review Checklist

• [ ] Model loaded asynchronously (not blocking main thread)
• [ ] MLModelConfiguration.computeUnits set appropriately for use case
• [ ] Model instance reused across predictions (not recreated each time)
• [ ] Auto-generated class used when available (typed inputs/outputs)
• [ ] Error handling for model loading and prediction failures
• [ ] Compiled model cached persistently if compiled at runtime
• [ ] Image inputs use Vision pipeline (CoreMLRequest iOS 18+ or VNCoreMLRequest) for correct preprocessing
• [ ] MLComputePlan checked to verify compute device dispatch (iOS 17.4+)
• [ ] Batch predictions used when processing multiple inputs
• [ ] Model size appropriate for deployment strategy (bundle, Background Assets, ODR)
• [ ] Memory tested on target devices (especially older devices with less RAM)
• [ ] Predictions run outside debugger for accurate performance measurement

References

• Patterns and code: references/coreml-swift-integration.md
• Model conversion and optimization (Python-side): covered in the apple-on-device-ai skill
• Apple docs: Core ML |

MLModel | MLTensor | MLComputePlan | Background Assets

Core ML Swift Integration Reference

Complete implementation patterns for loading, configuring, and running Core ML models in Swift. All patterns target iOS 26+ with Swift 6.3, backward-compatible to iOS 14 unless noted.

Contents

• Actor-Based Model Loading and Caching
• Auto-Generated Class Usage
• Manual MLFeatureProvider
• Prediction in Async Workflows
• MLBatchProvider for Batch Inference
• Stateful Predictions with MLState (iOS 18+)
• Image Preprocessing
• MLMultiArray Creation and Manipulation
• MLTensor Advanced Operations
• Vision + Core ML Pipelines
• NaturalLanguage Integration
• MLComputePlan Detailed Usage (iOS 17.4+)
• Background Loading and Memory Management
• Error Handling Patterns
• Testing Patterns

Actor-Based Model Loading and Caching

Use an actor to manage model lifecycle, prevent concurrent loading, and cache compiled models persistently.

import CoreML

actor ModelManager {
    private var loadedModels: [String: MLModel] = [:]
    private let cacheDirectory: URL

    init() {
        let appSupport = FileManager.default.urls(
            for: .applicationSupportDirectory, in: .userDomainMask
        ).first!
        cacheDirectory = appSupport.appendingPathComponent("CompiledModels", isDirectory: true)
        try? FileManager.default.createDirectory(at: cacheDirectory, withIntermediateDirectories: true)
    }

    func model(named name: String, configuration: MLModelConfiguration = .init()) async throws -> MLModel {
        if let cached = loadedModels[name] {
            return cached
        }

        let compiledURL = try await compiledModelURL(for: name)
        let model = try await MLModel.load(contentsOf: compiledURL, configuration: configuration)
        loadedModels[name] = model
        return model
    }

    func unloadModel(named name: String) {
        loadedModels.removeValue(forKey: name)
    }

    func unloadAll() {
        loadedModels.removeAll()
    }

    private func compiledModelURL(for name: String) async throws -> URL {
        // Check for pre-compiled model in cache
        let cachedURL = cacheDirectory.appendingPathComponent("\(name).mlmodelc")
        if FileManager.default.fileExists(atPath: cachedURL.path) {
            return cachedURL
        }

        // Check for pre-compiled model in bundle
        if let bundledCompiledURL = Bundle.main.url(forResource: name, withExtension: "mlmodelc") {
            return bundledCompiledURL
        }

        // Compile from .mlpackage and cache
        guard let packageURL = Bundle.main.url(forResource: name, withExtension: "mlpackage") else {
            throw ModelManagerError.modelNotFound(name)
        }
        let tempCompiledURL = try await MLModel.compileModel(at: packageURL)

        // Move compiled model to persistent cache
        if FileManager.default.fileExists(atPath: cachedURL.path) {
            try FileManager.default.removeItem(at: cachedURL)
        }
        try FileManager.default.moveItem(at: tempCompiledURL, to: cachedURL)
        return cachedURL
    }
}

enum ModelManagerError: Error {
    case modelNotFound(String)
    case predictionFailed(String)
}

Usage with SwiftUI

@MainActor
@Observable
final class ClassifierViewModel {
    var classLabel: String = ""
    var confidence: Double = 0
    var isLoading = false
    var errorMessage: String?

    private let modelManager = ModelManager()

    func classify(image: CGImage) async {
        isLoading = true
        defer { isLoading = false }

        do {
            let config = MLModelConfiguration()
            config.computeUnits = .all

            let model = try await modelManager.model(named: "ImageClassifier", configuration: config)

            let pixelBuffer = try createPixelBuffer(from: image, width: 224, height: 224)
            let input = try MLDictionaryFeatureProvider(dictionary: [
                "image": MLFeatureValue(pixelBuffer: pixelBuffer),
            ])
            let output = try model.prediction(from: input)

            classLabel = output.featureValue(for: "classLabel")?.stringValue ?? "Unknown"
            confidence = output.featureValue(for: "classLabelProbs")?
                .dictionaryValue[classLabel]?
                .doubleValue ?? 0
        } catch {
            errorMessage = "Classification failed: \(error.localizedDescription)"
        }
    }
}

Auto-Generated Class Usage

When you add a .mlmodel or .mlpackage to your Xcode project, Xcode generates a Swift class with typed inputs and outputs.

import CoreML

// Xcode generates: MyImageClassifier, MyImageClassifierInput, MyImageClassifierOutput

// Synchronous prediction with generated types
func classifyWithGeneratedClass(pixelBuffer: CVPixelBuffer) throws -> (label: String, confidence: Double) {
    let config = MLModelConfiguration()
    config.computeUnits = .all

    let classifier = try MyImageClassifier(configuration: config)
    let input = MyImageClassifierInput(image: pixelBuffer)
    let output = try classifier.prediction(input: input)

    let topLabel = output.classLabel
    let topConfidence = output.classLabelProbs[topLabel] ?? 0
    return (topLabel, topConfidence)
}

Accessing the Underlying MLModel

// Get the underlying MLModel from a generated class
let classifier = try MyImageClassifier(configuration: config)
let mlModel = classifier.model

// Useful for Vision integration
let vnModel = try VNCoreMLModel(for: mlModel)

Manual MLFeatureProvider

Implement MLFeatureProvider when you need custom input construction.

import CoreML

final class CustomImageInput: MLFeatureProvider {
    let image: CVPixelBuffer
    let confidenceThreshold: Double

    var featureNames: Set<String> {
        ["image", "confidence_threshold"]
    }

    func featureValue(for featureName: String) -> MLFeatureValue? {
        switch featureName {
        case "image":
            return MLFeatureValue(pixelBuffer: image)
        case "confidence_threshold":
            return MLFeatureValue(double: confidenceThreshold)
        default:
            return nil
        }
    }

    init(image: CVPixelBuffer, confidenceThreshold: Double = 0.5) {
        self.image = image
        self.confidenceThreshold = confidenceThreshold
    }
}

// Usage
let input = CustomImageInput(image: pixelBuffer, confidenceThreshold: 0.7)
let output = try model.prediction(from: input)

Prediction in Async Workflows

MLModel.prediction(...) is synchronous. Use Swift concurrency to keep loading, preprocessing, and caller coordination off the main actor, then call prediction without await.

Single Prediction from an Actor

actor PredictionService {
    private let model: MLModel

    init(model: MLModel) {
        self.model = model
    }

    func predict(input: any MLFeatureProvider) async throws -> any MLFeatureProvider {
        try model.prediction(from: input)
    }
}

Streaming Predictions

func classifyFrames(_ frames: AsyncStream<CVPixelBuffer>) async throws -> AsyncThrowingStream<String, Error> {
    let model = try await ModelManager().model(named: "Classifier")

    return AsyncThrowingStream { continuation in
        Task {
            do {
                for await frame in frames {
                    let input = try MLDictionaryFeatureProvider(dictionary: [
                        "image": MLFeatureValue(pixelBuffer: frame),
                    ])
                    let output = try model.prediction(from: input)
                    let label = output.featureValue(for: "classLabel")?.stringValue ?? "unknown"
                    continuation.yield(label)
                }
                continuation.finish()
            } catch {
                continuation.finish(throwing: error)
            }
        }
    }
}

MLBatchProvider for Batch Inference

import CoreML

func batchClassify(images: [CVPixelBuffer], model: MLModel) throws -> [(label: String, confidence: Double)] {
    let batchInputs = try MLArrayBatchProvider(array: images.map { buffer in
        try MLDictionaryFeatureProvider(dictionary: [
            "image": MLFeatureValue(pixelBuffer: buffer),
        ])
    })

    let batchOutput = try model.predictions(fromBatch: batchInputs)

    var results: [(String, Double)] = []
    for i in 0..<batchOutput.count {
        let features = batchOutput.features(at: i)
        let label = features.featureValue(for: "classLabel")?.stringValue ?? "unknown"
        let probs = features.featureValue(for: "classLabelProbs")?.dictionaryValue ?? [:]
        let confidence = (probs[label] as? NSNumber)?.doubleValue ?? 0
        results.append((label, confidence))
    }
    return results
}

Batch prediction has two valid API labels:

// No explicit prediction options
let output = try model.predictions(fromBatch: batchInputs)

// Explicit prediction options
let options = MLPredictionOptions()
let outputWithOptions = try model.predictions(from: batchInputs, options: options)

Do not write predictions(from:) for the no-options batch path; the from: label belongs to the overload that also takes options:.

Stateful Predictions with MLState (iOS 18+)

MLState enables models that maintain internal state across predictions. This is essential for sequence models (text generation, audio classification, time-series) where each prediction depends on previous context.

import CoreML

/// Audio classification that accumulates context over time
actor AudioClassifier {
    private let model: MLModel
    private var state: MLState?

    init(model: MLModel) {
        self.model = model
    }

    /// Start a new classification session
    func beginSession() {
        state = model.makeState()
    }

    /// Classify the next audio frame using accumulated state
    func classify(audioFeatures: MLMultiArray) async throws -> String {
        guard let state else {
            throw ClassifierError.noActiveSession
        }

        let input = try MLDictionaryFeatureProvider(dictionary: [
            "audio_features": MLFeatureValue(multiArray: audioFeatures)
        ])

        let output = try model.prediction(from: input, using: state)
        return output.featureValue(for: "label")?.stringValue ?? "unknown"
    }

    /// End the session and release state
    func endSession() {
        state = nil
    }
}

enum ClassifierError: Error {
    case noActiveSession
}

Key Rules for MLState

• Serialized use: MLState is Sendable, but predictions that use the same

state must be serialized. Sendable permits transfer across concurrency domains; it does not permit concurrent predictions on one state. Do not read or write state buffers while a prediction is in flight.

• Async options overload: Use the synchronous prediction(from:using:)

overload for simple serialized loops. If you need MLPredictionOptions, iOS 18+ also provides async prediction(from:using:options:); keep one in-flight prediction per state.

• Independent streams: Call model.makeState() per stream when processing

multiple concurrent sequences (e.g., multiple audio channels).

• Resettable: Create a new state to reset accumulated context. There is no

explicit reset method -- just discard the old state and create fresh.

• Memory: State holds model-specific internal buffers. Release it when the

session ends to free memory.

Image Preprocessing

CVPixelBuffer from CGImage

import CoreVideo
import CoreGraphics

func createPixelBuffer(from cgImage: CGImage, width: Int, height: Int) throws -> CVPixelBuffer {
    let attrs: [CFString: Any] = [
        kCVPixelBufferCGImageCompatibilityKey: true,
        kCVPixelBufferCGBitmapContextCompatibilityKey: true,
    ]

    var pixelBuffer: CVPixelBuffer?
    let status = CVPixelBufferCreate(
        kCFAllocatorDefault,
        width, height,
        kCVPixelFormatType_32ARGB,
        attrs as CFDictionary,
        &pixelBuffer
    )
    guard status == kCVReturnSuccess, let buffer = pixelBuffer else {
        throw ImageError.pixelBufferCreationFailed
    }

    CVPixelBufferLockBaseAddress(buffer, [])
    defer { CVPixelBufferUnlockBaseAddress(buffer, []) }

    guard let context = CGContext(
        data: CVPixelBufferGetBaseAddress(buffer),
        width: width,
        height: height,
        bitsPerComponent: 8,
        bytesPerRow: CVPixelBufferGetBytesPerRow(buffer),
        space: CGColorSpaceCreateDeviceRGB(),
        bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue
    ) else {
        throw ImageError.contextCreationFailed
    }

    context.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height))
    return buffer
}

enum ImageError: Error {
    case pixelBufferCreationFailed
    case contextCreationFailed
}

For CIImage sources, use CIContext.render(_:to:) into a CVPixelBuffer created with kCVPixelFormatType_32BGRA.

MLMultiArray Creation and Manipulation

import CoreML

let array = try MLMultiArray(shape: [1, 3, 224, 224], dataType: .float32)
for i in 0..<array.count { array[i] = NSNumber(value: Float.random(in: 0...1)) }

let values: [Float] = [1.0, 2.0, 3.0, 4.0, 5.0]
let mlArray = try MLMultiArray(values)

// Access elements
let element = array[[0, 0, 112, 112] as [NSNumber]].floatValue

// Convert to Swift array
func toFloatArray(_ multiArray: MLMultiArray) -> [Float] {
    let pointer = multiArray.dataPointer.assumingMemoryBound(to: Float.self)
    return Array(UnsafeBufferPointer(start: pointer, count: multiArray.count))
}

let featureValue = MLFeatureValue(multiArray: array)

MLTensor Advanced Operations (iOS 18+)

import CoreML

// Creation patterns
let tensor1D = MLTensor([1.0, 2.0, 3.0, 4.0])
let zeros = MLTensor(zeros: [3, 224, 224], scalarType: Float.self)
let ones = MLTensor(ones: [2, 2], scalarType: Float.self)

// Reshaping
let reshaped = tensor1D.reshaped(to: [2, 2])
let expanded = tensor1D.expandingShape(at: 0)   // [1, 4]

// Arithmetic and reduction
let sum = tensor1D + ones.reshaped(to: [4])
let mean = tensor1D.mean()
let argmax = tensor1D.argmax()

// Activation functions
let softmaxed = tensor1D.softmax(alongAxis: -1)

// Interop with MLShapedArray / MLMultiArray
let shaped = await tensor1D.shapedArray(of: Float.self) // MLShapedArray<Float>
let multiArray = try MLMultiArray(shaped)
let shapedAgain = MLShapedArray<Float>(multiArray)

// Concatenation
let a = MLTensor([1.0, 2.0])
let b = MLTensor([3.0, 4.0])
let concatenated = MLTensor(concatenating: [a, b], alongAxis: 0)

// Normalization pattern (e.g., ImageNet preprocessing)
func normalize(_ tensor: MLTensor, mean: [Float], std: [Float]) -> MLTensor {
    let meanTensor = MLTensor(mean)
    let stdTensor = MLTensor(std)
    return (tensor - meanTensor) / stdTensor
}

Vision + Core ML Pipelines

Modern API (iOS 18+)

import Vision
import CoreML

@MainActor
@Observable
final class ObjectDetectionViewModel {
    var detections: [Detection] = []
    var isProcessing = false

    struct Detection: Identifiable {
        let id = UUID()
        let label: String
        let confidence: Float
        let boundingBox: CGRect
    }

    func detect(in image: CGImage) async {
        isProcessing = true
        defer { isProcessing = false }

        do {
            let config = MLModelConfiguration()
            config.computeUnits = .all

            let detector = try MyObjectDetector(configuration: config)
            let request = CoreMLRequest(model: .init(detector.model))

            let results = try await request.perform(on: image)
            detections = results.compactMap { observation in
                guard let object = observation as? RecognizedObjectObservation,
                      let topLabel = object.labels.first else { return nil }
                return Detection(
                    label: topLabel.identifier,
                    confidence: topLabel.confidence,
                    boundingBox: object.boundingBox
                )
            }
        } catch {
            detections = []
        }
    }
}

func classifyImage(_ image: CGImage) async throws -> [(label: String, confidence: Float)] {
    let classifier = try MyImageClassifier(configuration: .init())
    let request = CoreMLRequest(model: .init(classifier.model))

    let results = try await request.perform(on: image)
    return results.compactMap { observation in
        guard let classification = observation as? ClassificationObservation else { return nil }
        return (classification.identifier, classification.confidence)
    }
}

Legacy API (Pre-iOS 18)

import Vision
import CoreML

func detectLegacy(in image: CGImage) async throws -> [VNRecognizedObjectObservation] {
    let config = MLModelConfiguration()
    config.computeUnits = .all

    let detector = try MyObjectDetector(configuration: config)
    let vnModel = try VNCoreMLModel(for: detector.model)

    let request = VNCoreMLRequest(model: vnModel)
    request.imageCropAndScaleOption = .scaleFill

    let handler = VNImageRequestHandler(cgImage: image)
    return try await Task.detached {
        try handler.perform([request])
        return request.results as? [VNRecognizedObjectObservation] ?? []
    }.value
}

func classifyImageLegacy(_ image: CGImage) async throws -> [(label: String, confidence: Float)] {
    let classifier = try MyImageClassifier(configuration: .init())
    let vnModel = try VNCoreMLModel(for: classifier.model)
    let request = VNCoreMLRequest(model: vnModel)
    request.imageCropAndScaleOption = .centerCrop

    let handler = VNImageRequestHandler(cgImage: image)
    return try await Task.detached {
        try handler.perform([request])
        guard let results = request.results as? [VNClassificationObservation] else { return [] }
        return results.prefix(5).map { ($0.identifier, $0.confidence) }
    }.value
}

NaturalLanguage Integration

Use NLModel to load Core ML models trained for NLP tasks.

import NaturalLanguage

func analyzeSentiment(text: String) throws -> (label: String, confidence: Double)? {
    let modelURL = Bundle.main.url(forResource: "SentimentClassifier", withExtension: "mlmodelc")!
    let nlModel = try NLModel(contentsOf: modelURL)

    guard let label = nlModel.predictedLabel(for: text) else { return nil }
    let hypotheses = nlModel.predictedLabelHypotheses(for: text, maximumCount: 1)
    let confidence = hypotheses[label] ?? 0
    return (label, confidence)
}

MLComputePlan Detailed Usage (iOS 17.4+)

import CoreML

func profileModel(at url: URL) async throws {
    let config = MLModelConfiguration()
    config.computeUnits = .all
    let computePlan = try await MLComputePlan.load(contentsOf: url, configuration: config)

    guard case let .program(program) = computePlan.modelStructure,
          let mainFunction = program.functions["main"] else {
        print("Model is not an ML program or has no main function")
        return
    }

    for operation in mainFunction.block.operations {
        let opName = operation.operatorName
        if let deviceUsage = computePlan.deviceUsage(for: operation) {
            print("  \(opName): \(deviceUsage.preferred)")
        }
        if let cost = computePlan.estimatedCost(of: operation) {
            print("    Estimated weight: \(cost.weight)")
        }
    }
}

Interpreting MLComputePlan Results

If many critical operations fall back to CPU, try .cpuAndGPU compute units, check for unsupported ANE operations, or re-convert with a different deployment target.

Background Model Loading and App Lifecycle

Manage model loading and memory across app lifecycle transitions.

@MainActor
@Observable
final class AppModelState {
    var isModelReady = false
    private let modelManager = ModelManager()

    func warmup() async {
        do {
            let config = MLModelConfiguration()
            config.computeUnits = .all
            _ = try await modelManager.model(named: "MainClassifier", configuration: config)
            isModelReady = true
        } catch {
            isModelReady = false
        }
    }

    func handleBackground() async {
        await modelManager.unloadAll()
        isModelReady = false
    }
}

// SwiftUI integration
@main
struct MyApp: App {
    @State private var modelState = AppModelState()
    @Environment(\.scenePhase) private var scenePhase

    var body: some Scene {
        WindowGroup {
            ContentView()
                .environment(modelState)
                .task { await modelState.warmup() }
                .onChange(of: scenePhase) { _, newPhase in
                    Task {
                        if newPhase == .background {
                            await modelState.handleBackground()
                        } else if newPhase == .active {
                            await modelState.warmup()
                        }
                    }
                }
        }
    }
}

Error Handling

import CoreML

func loadAndPredict(modelName: String, input: any MLFeatureProvider) async -> (any MLFeatureProvider)? {
    let config = MLModelConfiguration()
    config.computeUnits = .all

    do {
        guard let url = Bundle.main.url(forResource: modelName, withExtension: "mlmodelc") else {
            print("Model \(modelName) not found in bundle")
            return nil
        }
        let model = try await MLModel.load(contentsOf: url, configuration: config)
        return try model.prediction(from: input)
    } catch {
        print("Model error: \(error)")
        return nil
    }
}

Common Error Types

For MLTensor preprocessing, keep examples to source-confirmed operations. Do not use MLTensor(multiArray), tensor.std(), tensor.standardDeviation(), direct lazy-buffer access, or synchronous extraction unless Apple documents that exact API and availability.

Testing Patterns

import Testing
import CoreML

struct ModelLoadingTests {
    @Test func loadModelSucceeds() async throws {
        let config = MLModelConfiguration()
        config.computeUnits = .cpuOnly // CPU for test stability
        let model = try MyImageClassifier(configuration: config)
        #expect(model.model.modelDescription.inputDescriptionsByName.count > 0)
    }

    @Test func predictionReturnsValidOutput() async throws {
        let config = MLModelConfiguration()
        config.computeUnits = .cpuOnly

        let model = try MyImageClassifier(configuration: config)
        let input = try createTestInput(width: 224, height: 224)
        let output = try model.prediction(input: input)

        #expect(!output.classLabel.isEmpty)
        #expect(output.classLabelProbs.values.allSatisfy { $0 >= 0 && $0 <= 1 })
    }

    @Test func predictionLatencyUnderThreshold() async throws {
        let config = MLModelConfiguration()
        config.computeUnits = .all

        let model = try MyImageClassifier(configuration: config)
        let input = try createTestInput(width: 224, height: 224)

        _ = try model.prediction(input: input) // Warm up

        let start = ContinuousClock.now
        for _ in 0..<10 {
            _ = try model.prediction(input: input)
        }
        let avgMs = (ContinuousClock.now - start) / 10

        #expect(avgMs < .milliseconds(50), "Average prediction time \(avgMs) exceeds 50ms")
    }
}

private func createTestPixelBuffer(width: Int, height: Int) throws -> CVPixelBuffer {
    var pixelBuffer: CVPixelBuffer?
    let status = CVPixelBufferCreate(
        kCFAllocatorDefault, width, height,
        kCVPixelFormatType_32ARGB, nil, &pixelBuffer
    )
    guard status == kCVReturnSuccess, let buffer = pixelBuffer else {
        throw ImageError.pixelBufferCreationFailed
    }
    return buffer
}

Memory Management Best Practices

1. Unload on background. Unload models when scenePhase == .background and reload on return to foreground. iOS reclaims memory aggressively. 2. Choose compute units by context. Use .all by default. Consider .cpuOnly only when profiling or app policy shows accelerator contention, thermal state, energy budget, deterministic testing, or a legitimate background execution constraint makes CPU the right tradeoff. Do not claim GPU or Neural Engine are categorically unavailable for every background-adjacent task; background behavior depends on app mode, suspension, system policy, thermal state, energy, and contention. 3. Prefer compiled models. .mlmodelc loads faster and uses less transient memory than compiling .mlpackage at runtime. If a model is downloaded as .mlmodel or .mlpackage, compile once with MLModel.compileModel(at:), move the .mlmodelc out of Core ML's temporary location, and cache it by model version. Do not call compileModel(at:) on every launch for the same model. 4. Validate on physical devices. Measure model load, first prediction, repeated predictions, background/foreground transitions, and low-memory behavior on the lowest-memory supported device. 5. Share model instances. Use an actor (like ModelManager above) to ensure only one instance of each model exists. 6. Release batch providers promptly. Large MLArrayBatchProvider instances hold references to all input data.