
Core Ml
- 288 installs
- 591 repo stars
- Updated July 24, 2026
- rshankras/claude-code-apple-skills
Integrate, convert, and optimize Core ML models for on-device inference in Apple apps using Vision, NLP, and custom ML pipeline patterns.
About
Apple Core ML skill for Claude Code: convert models, integrate on-device inference into Swift iOS/macOS/watchOS apps, apply Vision and NLP patterns, and tune quantization and performance for production mobile releases.
- On-device model conversion
- Vision and NLP framework patterns
- Swift Core ML inference wiring
- Quantization and performance tuning
- Apple Silicon optimization guidance
Core Ml by the numbers
- 288 all-time installs (skills.sh)
- +20 installs in the week ending Jul 26, 2026 (Skillselion tracking)
- Ranked #596 of 2,064 Data Science & ML skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/rshankras/claude-code-apple-skills --skill core-mlAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 288 |
|---|---|
| repo stars | ★ 591 |
| Last updated | July 24, 2026 |
| Repository | rshankras/claude-code-apple-skills ↗ |
What it does
Integrate, convert, and optimize Core ML models for on-device inference in Apple apps using Vision, NLP, and custom ML pipeline patterns.
Files
Core ML Skills
Combined advisory, generator, and workflow skill for integrating machine learning into Apple platform apps. Covers Core ML model integration, Vision framework image analysis, NaturalLanguage framework text processing, Create ML training, and on-device model optimization.
When This Skill Activates
Use this skill when the user:
- Wants to add ML capabilities to their app
- Needs to integrate a Core ML model (.mlmodel) into an Xcode project
- Wants to use the Vision framework for image analysis (faces, text recognition, body pose, object detection)
- Wants to use the NaturalLanguage framework for text processing (sentiment, entities, language detection)
- Needs to train a custom model with Create ML
- Wants to optimize a model for on-device use (quantization, pruning, palettization)
- Needs to choose between Core ML and Foundation Models (Apple Intelligence)
- Asks about image classification, object detection, sound classification, or tabular data prediction
- Wants real-time camera + ML processing
Decision Guide: Core ML vs Foundation Models
Before generating code, determine which framework is appropriate.
Use Foundation Models (Apple Intelligence) When:
- You need general-purpose text generation, summarization, or conversational AI
- Target is iOS 26+ / macOS 26+ (Foundation Models requires Apple Silicon + latest OS)
- The task is open-ended language understanding or generation
- You want
@Generablestructured output from natural language - See
apple-intelligence/foundation-models/skill for implementation
Use Core ML When:
- You need specialized ML: image classification, object detection, sound classification, custom regression/classification
- You have a trained model (.mlmodel, .mlpackage) or plan to train one
- You need broad device support (iOS 14+ / macOS 11+)
- The task requires domain-specific predictions (medical imaging, product recognition, custom NLP)
- Performance-critical inference on Neural Engine or GPU
Use Vision Framework When (No Custom Model Needed):
- Image classification using Apple's built-in models
- Face detection and facial landmark analysis
- Text recognition (OCR) with
VNRecognizeTextRequest - Body and hand pose detection
- Barcode and QR code scanning
- Image similarity and saliency detection
- Horizon detection, rectangle detection
Use NaturalLanguage Framework When (No Custom Model Needed):
- Sentiment analysis on text
- Language identification
- Tokenization (word, sentence, paragraph boundaries)
- Named entity recognition (people, places, organizations)
- Word and sentence embeddings for similarity comparison
- Lemmatization and part-of-speech tagging
Pre-Generation Checks
1. Project Context Detection
- [ ] Check deployment target (Core ML requires iOS 11+ / macOS 10.13+; Vision requires iOS 11+; NaturalLanguage requires iOS 12+)
- [ ] Check for existing ML code or models
- [ ] Identify project structure and source file locations
- [ ] Determine if SwiftUI or UIKit/AppKit
2. Conflict Detection
Search for existing ML integration:
Glob: **/*Model*.swift, **/*Classifier*.swift, **/*Predictor*.swift, **/*.mlmodel, **/*.mlmodelc, **/*.mlpackage
Grep: "import CoreML" or "import Vision" or "import NaturalLanguage"If found, ask user:
- Extend existing ML setup?
- Replace with new implementation?
- Add additional model/capability?
Configuration Questions
Ask user via AskUserQuestion:
1. What ML capability do you need?
- Image classification (identify objects in photos)
- Object detection (locate objects with bounding boxes)
- Text analysis (sentiment, entities, language)
- Custom Core ML model integration
- Vision framework (OCR, faces, poses)
- Sound classification
- Tabular data prediction
2. Do you have a trained model, or need to train one?
- I have a .mlmodel / .mlpackage file
- I want to train with Create ML
- I want to use Apple's built-in models (Vision / NaturalLanguage)
3. Performance requirements?
- Real-time (camera feed, < 33ms per prediction)
- Interactive (user-initiated, < 500ms acceptable)
- Background processing (batch, latency not critical)
Core ML Model Integration
Adding a Model to Xcode
1. Drag .mlmodel or .mlpackage into Xcode project navigator 2. Xcode auto-generates a Swift class with the model name 3. The generated class provides type-safe input/output interfaces 4. Xcode compiles to .mlmodelc at build time (optimized for device)
Loading Models
// Option 1: Auto-generated class (simplest)
let model = try MyImageClassifier(configuration: MLModelConfiguration())
// Option 2: Generic MLModel loading (flexible)
let url = Bundle.main.url(forResource: "MyModel", withExtension: "mlmodelc")!
let config = MLModelConfiguration()
config.computeUnits = .all // CPU + GPU + Neural Engine
let model = try MLModel(contentsOf: url, configuration: config)
// Option 3: Async loading (recommended for large models)
let model = try await MLModel.load(contentsOf: url, configuration: config)Making Predictions
// Type-safe prediction with auto-generated class
let input = MyImageClassifierInput(image: pixelBuffer)
let output = try model.prediction(input: input)
print(output.classLabel) // "cat"
print(output.classLabelProbs) // ["cat": 0.95, "dog": 0.04, ...]
// Batch predictions
let batch = MLArrayBatchProvider(array: inputs)
let results = try model.predictions(from: batch)Create ML Training Overview
Image Classification
- Minimum: 10 images per category; Recommended: 40+ per category
- Organize images in folders named by category
- Supports JPEG, PNG, HEIC formats
- Data augmentation applied automatically (rotation, flip, crop)
- Transfer learning from Apple's base models
Text Classification
- Training data: text samples with labels (CSV or JSON)
- Use cases: sentiment analysis, spam detection, topic classification, intent recognition
- Minimum 10 samples per class; 100+ recommended for accuracy
Tabular Classification / Regression
- Structured data in CSV or JSON
- Automatic feature engineering
- Supports: Boosted Tree, Random Forest, Linear Regression, Decision Tree
Sound Classification
- Audio files organized by category
- Environmental sounds, speech detection, music genre
- Minimum 10 samples per category at 15+ seconds each
Object Detection
- Images with bounding box annotations (JSON format)
- Outputs bounding boxes + class labels + confidence
- Minimum 30 annotated images per class; 300+ recommended
Training Approach
- Xcode Create ML App: Visual interface, drag-and-drop, no code required
- CreateML Framework: Programmatic training in Swift Playgrounds or macOS apps
- coremltools (Python): Convert models from TensorFlow, PyTorch, ONNX to Core ML format
Vision Framework Capabilities
| Capability | Request Class | Custom Model Needed? |
|---|---|---|
| Image classification | VNClassifyImageRequest | No (built-in) |
| Object detection | VNDetectObjectsRequest (custom model) | Yes |
| Face detection | VNDetectFaceRectanglesRequest | No |
| Face landmarks | VNDetectFaceLandmarksRequest | No |
| Text recognition (OCR) | VNRecognizeTextRequest | No |
| Body pose | VNDetectHumanBodyPoseRequest | No |
| Hand pose | VNDetectHumanHandPoseRequest | No |
| Barcode detection | VNDetectBarcodesRequest | No |
| Image saliency | VNGenerateAttentionBasedSaliencyImageRequest | No |
| Horizon detection | VNDetectHorizonRequest | No |
| Rectangle detection | VNDetectRectanglesRequest | No |
| Image similarity | VNGenerateImageFeaturePrintRequest | No |
Vision Request Pipeline
// Multiple requests on the same image
let handler = VNImageRequestHandler(cgImage: cgImage, options: [:])
try handler.perform([
textRequest, // OCR
faceRequest, // Face detection
barcodeRequest // Barcode scanning
])
// Each request's results are populated independentlyNaturalLanguage Framework
Sentiment Analysis
Returns a score from -1.0 (negative) to +1.0 (positive):
let tagger = NLTagger(tagSchemes: [.sentimentScore])
tagger.string = "This app is amazing!"
let (tag, _) = tagger.tag(at: text.startIndex, unit: .paragraph, scheme: .sentimentScore)
// tag?.rawValue == "0.9" (positive)Language Detection
let language = NLLanguageRecognizer.dominantLanguage(for: "Bonjour le monde")
// language == .frenchTokenization
let tokenizer = NLTokenizer(unit: .word)
tokenizer.string = "Hello, world!"
tokenizer.enumerateTokens(in: text.startIndex..<text.endIndex) { range, _ in
print(text[range]) // "Hello" then "world"
return true
}Named Entity Recognition
let tagger = NLTagger(tagSchemes: [.nameType])
tagger.string = "Tim Cook visited Apple Park in Cupertino."
tagger.enumerateTags(in: text.startIndex..<text.endIndex, unit: .word, scheme: .nameType) { tag, range in
if let tag, tag != .other {
print("\(text[range]): \(tag.rawValue)")
// "Tim": PersonalName, "Cook": PersonalName
// "Apple Park": OrganizationName, "Cupertino": PlaceName
}
return true
}Model Optimization
Quantization (coremltools Python)
Reduces model size by lowering numerical precision:
- Float32 to Float16: ~50% size reduction, minimal accuracy loss
- Float16 to Int8: ~50% further reduction, test accuracy carefully
import coremltools as ct
from coremltools.models.neural_network import quantization_utils
model = ct.models.MLModel("MyModel.mlmodel")
# Float16 quantization (safe default)
model_fp16 = quantization_utils.quantize_weights(model, nbits=16)
model_fp16.save("MyModel_fp16.mlmodel")
# Int8 quantization (aggressive, test accuracy)
model_int8 = quantization_utils.quantize_weights(model, nbits=8)
model_int8.save("MyModel_int8.mlmodel")Palettization
Reduces unique weight values using k-means clustering:
from coremltools.optimize.coreml import palettize_weights, OpPalettizerConfig
config = OpPalettizerConfig(nbits=4) # 16 unique values per tensor
model_palettized = palettize_weights(model, config)Pruning
Removes near-zero weights (sparse model):
from coremltools.optimize.torch.pruning import MagnitudePruner, MagnitudePrunerConfig
config = MagnitudePrunerConfig(target_sparsity=0.75) # Remove 75% of weights
pruner = MagnitudePruner(model, config)Optimization Guidelines
- Always benchmark accuracy after optimization
- Start with Float16 (safest, best effort-to-reward ratio)
- Test on target device (Neural Engine behavior differs from GPU)
- Profile with Xcode Instruments > Core ML Performance
Performance Patterns
Compute Unit Selection
let config = MLModelConfiguration()
// Best performance — let system choose CPU, GPU, or Neural Engine
config.computeUnits = .all
// CPU only — predictable latency, no GPU/NE contention
config.computeUnits = .cpuOnly
// CPU + Neural Engine — good balance, avoids GPU contention with UI
config.computeUnits = .cpuAndNeuralEngine
// CPU + GPU — when Neural Engine unavailable
config.computeUnits = .cpuAndGPUAsync Prediction for UI Responsiveness
func classify(_ image: UIImage) async throws -> String {
let model = try await MLModelManager.shared.model(named: "Classifier")
// Prediction runs off main thread via structured concurrency
let input = try MLDictionaryFeatureProvider(dictionary: ["image": image.pixelBuffer!])
let result = try await Task.detached {
try model.prediction(from: input)
}.value
return result.featureValue(for: "classLabel")?.stringValue ?? "unknown"
}Batch Processing
// Process multiple images efficiently
let inputs = images.map { MyModelInput(image: $0.pixelBuffer!) }
let batch = MLArrayBatchProvider(array: inputs)
let results = try model.predictions(from: batch)
for i in 0..<results.count {
let output = results.features(at: i)
print(output.featureValue(for: "classLabel")?.stringValue ?? "")
}Compile Model at Install Time
// Compile .mlmodel to .mlmodelc at install (not runtime)
// This is done automatically when you add .mlmodel to Xcode target
// For downloaded models, compile once and cache:
let compiledURL = try MLModel.compileModel(at: downloadedModelURL)
let permanentURL = appSupportDir.appendingPathComponent("MyModel.mlmodelc")
try FileManager.default.copyItem(at: compiledURL, to: permanentURL)Generation Process
Step 1: Determine Capability
Based on user's answer to configuration questions, select the appropriate template(s) from templates.md.
Step 2: Generate Core Files
| Capability | Files Generated |
|---|---|
| Any Core ML | MLModelManager.swift |
| Image classification | ImageClassifier.swift |
| Text analysis | TextAnalyzer.swift |
| Vision requests | VisionService.swift |
| Custom model | ModelConfig.swift + model-specific predictor |
| Camera + ML | CameraMLPipeline.swift |
Step 3: Determine File Location
Check project structure:
- If
Sources/exists ->Sources/ML/ - If
App/Services/exists ->App/Services/ML/ - If
App/exists ->App/ML/ - Otherwise ->
ML/
Output Format
After generation, provide:
Files Created
ML/
├── MLModelManager.swift # Central model lifecycle management
├── ImageClassifier.swift # Vision-based image classification (if needed)
├── TextAnalyzer.swift # NaturalLanguage wrapper (if needed)
├── ModelConfig.swift # Compute unit configuration
└── VisionService.swift # Vision request pipeline (if needed)Integration Steps
1. Add .mlmodel file to Xcode project (if using custom model) 2. Import the generated ML service files 3. Initialize the service in your app's dependency injection 4. Call prediction methods from your views/view models 5. Handle errors and display results
Testing
- Use known test inputs with expected outputs
- Verify confidence thresholds
- Profile prediction latency on target device
- Test graceful degradation when model unavailable
References
- patterns.md — Architecture patterns, model manager, Vision pipeline, camera + ML, testing
- templates.md — Production-ready Swift code templates for all ML capabilities
- Apple Docs: Core ML Documentation
- Apple Docs: Vision Documentation
- Apple Docs: NaturalLanguage Documentation
- Apple Docs: Create ML Documentation
Core ML Architecture Patterns and Best Practices
Model Manager Pattern
Central service for model lifecycle management. Prevents redundant loading, manages memory, and provides a clean API for the rest of the app.
Singleton or Injected Service
// Option 1: Actor-based singleton (simplest)
actor MLModelManager {
static let shared = MLModelManager()
private var loadedModels: [String: MLModel] = [:]
func model(named name: String) async throws -> MLModel {
if let cached = loadedModels[name] { return cached }
guard let url = Bundle.main.url(forResource: name, withExtension: "mlmodelc") else {
throw MLModelError.modelNotFound(name)
}
let config = MLModelConfiguration()
config.computeUnits = .all
let model = try await MLModel.load(contentsOf: url, configuration: config)
loadedModels[name] = model
return model
}
func unloadModel(named name: String) {
loadedModels.removeValue(forKey: name)
}
func unloadAll() {
loadedModels.removeAll()
}
}
// Option 2: Protocol-based for dependency injection
protocol ModelProviding: Sendable {
func model(named name: String) async throws -> MLModel
func unloadModel(named name: String) async
}Lazy Loading
Do not load models at app launch. Load on first prediction request:
// Wrong - loads at init, delays app launch
class AppDelegate: NSObject, UIApplicationDelegate {
let model = try! MyClassifier(configuration: .init()) // Blocks launch
}
// Right - loads on first use
struct ClassifierView: View {
@State private var result: String?
var body: some View {
Button("Classify") {
Task {
let model = try await MLModelManager.shared.model(named: "MyClassifier")
// First call loads; subsequent calls return cached
}
}
}
}Memory Management
Unload models when no longer needed, especially on memory warnings:
// Respond to memory pressure
actor MLModelManager {
func handleMemoryWarning() {
// Keep most-used model, unload others
let keep = mostRecentlyUsedModelName
for name in loadedModels.keys where name != keep {
loadedModels.removeValue(forKey: name)
}
}
}
// In SwiftUI
.onReceive(NotificationCenter.default.publisher(for: UIApplication.didReceiveMemoryWarningNotification)) { _ in
Task { await MLModelManager.shared.handleMemoryWarning() }
}Error Handling with Graceful Fallback
enum MLModelError: Error, LocalizedError {
case modelNotFound(String)
case predictionFailed(Error)
case lowConfidence(best: String, confidence: Float)
case invalidInput(String)
case modelCorrupted
var errorDescription: String? {
switch self {
case .modelNotFound(let name):
return "ML model '\(name)' not found in bundle"
case .predictionFailed(let error):
return "Prediction failed: \(error.localizedDescription)"
case .lowConfidence(let best, let confidence):
return "Low confidence result: \(best) (\(String(format: "%.1f", confidence * 100))%)"
case .invalidInput(let reason):
return "Invalid input: \(reason)"
case .modelCorrupted:
return "ML model file is corrupted"
}
}
}Vision Request Pipeline Pattern
Structured approach for executing Vision framework requests with proper error handling and image orientation support.
Basic Pipeline
struct VisionService {
func performRequests(
on image: CGImage,
orientation: CGImagePropertyOrientation = .up,
requests: [VNRequest]
) async throws {
let handler = VNImageRequestHandler(
cgImage: image,
orientation: orientation,
options: [:]
)
try handler.perform(requests)
}
}Handling Image Orientation (Critical for Camera Input)
Camera images come with EXIF orientation metadata. Ignoring this produces incorrect results:
// Convert UIImage orientation to Vision orientation
extension CGImagePropertyOrientation {
init(_ uiOrientation: UIImage.Orientation) {
switch uiOrientation {
case .up: self = .up
case .upMirrored: self = .upMirrored
case .down: self = .down
case .downMirrored: self = .downMirrored
case .left: self = .left
case .leftMirrored: self = .leftMirrored
case .right: self = .right
case .rightMirrored: self = .rightMirrored
@unknown default: self = .up
}
}
}
// Always pass orientation
func classify(_ image: UIImage) throws -> [VNClassificationObservation] {
guard let cgImage = image.cgImage else { throw MLModelError.invalidInput("No CGImage") }
let request = VNClassifyImageRequest()
let handler = VNImageRequestHandler(
cgImage: cgImage,
orientation: CGImagePropertyOrientation(image.imageOrientation),
options: [:]
)
try handler.perform([request])
return request.results ?? []
}Pipeline Multiple Requests on Same Image
func analyzeImage(_ image: CGImage) async throws -> ImageAnalysis {
let classifyRequest = VNClassifyImageRequest()
let textRequest = VNRecognizeTextRequest()
textRequest.recognitionLevel = .accurate
let faceRequest = VNDetectFaceRectanglesRequest()
let handler = VNImageRequestHandler(cgImage: image, options: [:])
try handler.perform([classifyRequest, textRequest, faceRequest])
return ImageAnalysis(
classifications: classifyRequest.results ?? [],
recognizedText: textRequest.results?.compactMap { $0.topCandidates(1).first?.string } ?? [],
faces: faceRequest.results ?? []
)
}
struct ImageAnalysis {
let classifications: [VNClassificationObservation]
let recognizedText: [String]
let faces: [VNFaceObservation]
}Camera + ML Real-Time Pattern
Process camera frames with ML in real-time while maintaining smooth UI.
AVCaptureSession to Vision Pipeline
final class CameraMLPipeline: NSObject, AVCaptureVideoDataOutputSampleBufferDelegate {
private let captureSession = AVCaptureSession()
private let processingQueue = DispatchQueue(label: "ml.processing", qos: .userInitiated)
private var lastProcessingTime: Date = .distantPast
private let minimumInterval: TimeInterval = 0.5 // Process every 500ms
var onResult: ((String, Float) -> Void)?
func startCapture() throws {
guard let device = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) else {
throw MLModelError.invalidInput("No camera available")
}
let input = try AVCaptureDeviceInput(device: device)
captureSession.addInput(input)
let output = AVCaptureVideoDataOutput()
output.setSampleBufferDelegate(self, queue: processingQueue)
output.alwaysDiscardsLateVideoFrames = true
captureSession.addOutput(output)
captureSession.startRunning()
}
func captureOutput(
_ output: AVCaptureOutput,
didOutput sampleBuffer: CMSampleBuffer,
from connection: AVCaptureConnection
) {
// Throttle: skip frames to avoid overwhelming ML pipeline
let now = Date()
guard now.timeIntervalSince(lastProcessingTime) >= minimumInterval else { return }
lastProcessingTime = now
guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
let request = VNClassifyImageRequest { [weak self] request, error in
guard let results = request.results as? [VNClassificationObservation],
let top = results.first else { return }
DispatchQueue.main.async {
self?.onResult?(top.identifier, top.confidence)
}
}
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:])
try? handler.perform([request])
}
}Key Rules for Camera + ML
1. Throttle predictions — not every frame. Every 5th frame or 500ms minimum 2. Use `.alwaysDiscardsLateVideoFrames = true` — prevents buffer queue buildup 3. Process on background queue — never block the camera pipeline 4. Use `MLComputeUnits.all` — let the system choose the fastest hardware 5. Update UI on main thread — always dispatch results back to @MainActor
Model Versioning Pattern
Support model updates over time without breaking the app.
Versioned Model Files
struct ModelVersion {
let name: String
let version: Int
let url: URL
var filename: String { "\(name)_v\(version)" }
static func latestBundled(name: String) -> ModelVersion? {
// Find highest version in bundle
var version = 1
var latest: ModelVersion?
while let url = Bundle.main.url(forResource: "\(name)_v\(version)", withExtension: "mlmodelc") {
latest = ModelVersion(name: name, version: version, url: url)
version += 1
}
return latest
}
}Remote Model Download and Update
actor ModelUpdater {
private let fileManager = FileManager.default
func downloadModel(from remoteURL: URL, name: String, version: Int) async throws -> URL {
let (tempURL, _) = try await URLSession.shared.download(from: remoteURL)
// Compile the downloaded .mlmodel
let compiledURL = try MLModel.compileModel(at: tempURL)
// Move to permanent location
let modelsDir = try modelsDirectory()
let destination = modelsDir.appendingPathComponent("\(name)_v\(version).mlmodelc")
if fileManager.fileExists(atPath: destination.path) {
try fileManager.removeItem(at: destination)
}
try fileManager.moveItem(at: compiledURL, to: destination)
return destination
}
func checkForUpdate(name: String, currentVersion: Int) async throws -> Bool {
// Check your server for newer model version
// Return true if update available
let metadata = try await fetchModelMetadata(name: name)
return metadata.latestVersion > currentVersion
}
private func modelsDirectory() throws -> URL {
let appSupport = try fileManager.url(
for: .applicationSupportDirectory,
in: .userDomainMask,
appropriateFor: nil,
create: true
)
let modelsDir = appSupport.appendingPathComponent("MLModels")
if !fileManager.fileExists(atPath: modelsDir.path) {
try fileManager.createDirectory(at: modelsDir, withIntermediateDirectories: true)
}
return modelsDir
}
}Graceful Degradation
func loadBestAvailableModel(named name: String) async throws -> MLModel {
// Try downloaded (newer) model first
if let downloadedURL = try? latestDownloadedModel(named: name) {
do {
return try await MLModel.load(contentsOf: downloadedURL)
} catch {
// Downloaded model corrupted — fall back to bundled
try? FileManager.default.removeItem(at: downloadedURL)
}
}
// Fall back to bundled model
guard let bundledURL = Bundle.main.url(forResource: name, withExtension: "mlmodelc") else {
throw MLModelError.modelNotFound(name)
}
return try await MLModel.load(contentsOf: bundledURL)
}Testing ML Code Pattern
Test with Known Inputs and Expected Outputs
import Testing
import CoreML
@Test
func imageClassifierRecognizesCat() async throws {
let classifier = ImageClassifier()
let catImage = UIImage(named: "test_cat", in: .test, with: nil)!
let results = try await classifier.classify(catImage, maxResults: 3)
#expect(!results.isEmpty)
#expect(results[0].label == "cat" || results[0].label == "tabby")
#expect(results[0].confidence > 0.7)
}Confidence Threshold Testing
@Test
func lowConfidenceResultsFiltered() async throws {
let classifier = ImageClassifier()
let ambiguousImage = UIImage(named: "test_ambiguous", in: .test, with: nil)!
let results = try await classifier.classify(ambiguousImage, maxResults: 5)
let filtered = results.filter { $0.confidence > 0.5 }
// Ambiguous images should have no high-confidence results
#expect(filtered.count <= 1)
}Performance Testing
@Test
func predictionLatencyUnder100ms() async throws {
let classifier = ImageClassifier()
let image = UIImage(named: "test_standard", in: .test, with: nil)!
let start = ContinuousClock.now
_ = try await classifier.classify(image)
let elapsed = ContinuousClock.now - start
#expect(elapsed < .milliseconds(100))
}Mock Model for Unit Tests
protocol ImageClassifying: Sendable {
func classify(_ image: UIImage, maxResults: Int) async throws -> [Classification]
}
struct ImageClassifier: ImageClassifying {
func classify(_ image: UIImage, maxResults: Int = 5) async throws -> [Classification] {
// Real implementation using Vision
}
}
struct MockImageClassifier: ImageClassifying {
var stubbedResults: [Classification] = []
var stubbedError: Error?
func classify(_ image: UIImage, maxResults: Int) async throws -> [Classification] {
if let error = stubbedError { throw error }
return Array(stubbedResults.prefix(maxResults))
}
}
// In tests
@Test
func viewModelUpdatesOnClassification() async {
var mock = MockImageClassifier()
mock.stubbedResults = [Classification(label: "dog", confidence: 0.95)]
let viewModel = ClassifierViewModel(classifier: mock)
await viewModel.classify(testImage)
#expect(viewModel.topLabel == "dog")
#expect(viewModel.confidence == 0.95)
}Edge Case Testing
@Test
func handlesEmptyImage() async {
let classifier = ImageClassifier()
let emptyImage = UIImage()
await #expect(throws: MLModelError.self) {
try await classifier.classify(emptyImage)
}
}
@Test
func handlesVeryLargeImage() async throws {
let classifier = ImageClassifier()
// 4000x4000 image - should still work (Vision framework resizes internally)
let largeImage = createTestImage(width: 4000, height: 4000)
let results = try await classifier.classify(largeImage)
#expect(!results.isEmpty)
}
@Test
func handlesModelNotFound() async {
await #expect(throws: MLModelError.self) {
try await MLModelManager.shared.model(named: "NonexistentModel")
}
}Anti-Patterns to Avoid
Don't Load Models Synchronously on Main Thread
// Wrong - blocks UI for seconds on large models
let model = try! MLModel(contentsOf: url)
// Right - async loading
let model = try await MLModel.load(contentsOf: url, configuration: config)Don't Process Every Camera Frame
// Wrong - overwhelms ML pipeline, drops frames, drains battery
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, ...) {
processWithML(sampleBuffer) // Called 30-60 times per second
}
// Right - throttle to reasonable interval
guard Date().timeIntervalSince(lastProcessing) > 0.5 else { return }Don't Ignore Confidence Scores
// Wrong - trusts any result
let label = results.first!.identifier
// Right - filter by confidence
let label = results.first(where: { $0.confidence > 0.7 })?.identifier ?? "Unknown"Don't Hardcode Model Compute Units for All Devices
// Wrong - Neural Engine not available on all devices
config.computeUnits = .cpuAndNeuralEngine
// Right - let system choose
config.computeUnits = .allDon't Skip Error Handling for Predictions
// Wrong
let result = try! model.prediction(from: input)
// Right
do {
let result = try model.prediction(from: input)
handleResult(result)
} catch {
handlePredictionError(error)
}Core ML Swift Code Templates
Production-ready Swift templates for Core ML, Vision, and NaturalLanguage integration. All code uses modern Swift patterns: actors, async/await, structured concurrency, and protocol-based architecture.
1. MLModelManager.swift — Central Model Management
import CoreML
import os
/// Actor-based model manager for safe concurrent model loading and caching.
/// Provides lazy loading, memory management, and error handling for Core ML models.
actor MLModelManager {
static let shared = MLModelManager()
private let logger = Logger(subsystem: Bundle.main.bundleIdentifier ?? "app", category: "MLModelManager")
private var loadedModels: [String: MLModel] = [:]
/// Load a compiled Core ML model from the app bundle, caching it for reuse.
/// - Parameters:
/// - name: The model filename without extension (e.g., "ImageClassifier")
/// - computeUnits: Hardware to use for inference. Defaults to `.all` (system chooses best).
/// - Returns: The loaded MLModel instance.
func model(named name: String, computeUnits: MLComputeUnits = .all) async throws -> MLModel {
if let cached = loadedModels[name] {
return cached
}
guard let url = Bundle.main.url(forResource: name, withExtension: "mlmodelc") else {
logger.error("Model not found in bundle: \(name)")
throw MLModelManagerError.modelNotFound(name)
}
let config = MLModelConfiguration()
config.computeUnits = computeUnits
logger.info("Loading model: \(name) with compute units: \(String(describing: computeUnits))")
do {
let model = try await MLModel.load(contentsOf: url, configuration: config)
loadedModels[name] = model
logger.info("Model loaded successfully: \(name)")
return model
} catch {
logger.error("Failed to load model \(name): \(error.localizedDescription)")
throw MLModelManagerError.loadFailed(name, error)
}
}
/// Load a model from an arbitrary URL (e.g., downloaded model in Application Support).
func model(at url: URL, name: String, computeUnits: MLComputeUnits = .all) async throws -> MLModel {
if let cached = loadedModels[name] {
return cached
}
let config = MLModelConfiguration()
config.computeUnits = computeUnits
let model = try await MLModel.load(contentsOf: url, configuration: config)
loadedModels[name] = model
return model
}
/// Unload a specific model to free memory.
func unloadModel(named name: String) {
loadedModels.removeValue(forKey: name)
logger.info("Model unloaded: \(name)")
}
/// Unload all models. Call on memory warning.
func unloadAll() {
let count = loadedModels.count
loadedModels.removeAll()
logger.info("All models unloaded (\(count) models)")
}
/// Compile a raw .mlmodel file and return the compiled URL.
/// Use this for models downloaded at runtime.
func compileAndCache(modelAt sourceURL: URL, name: String) async throws -> URL {
let compiledURL = try MLModel.compileModel(at: sourceURL)
let cacheDir = try modelsCacheDirectory()
let destination = cacheDir.appendingPathComponent("\(name).mlmodelc")
let fm = FileManager.default
if fm.fileExists(atPath: destination.path) {
try fm.removeItem(at: destination)
}
try fm.moveItem(at: compiledURL, to: destination)
return destination
}
private func modelsCacheDirectory() throws -> URL {
let appSupport = try FileManager.default.url(
for: .applicationSupportDirectory,
in: .userDomainMask,
appropriateFor: nil,
create: true
)
let dir = appSupport.appendingPathComponent("MLModels")
if !FileManager.default.fileExists(atPath: dir.path) {
try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true)
}
return dir
}
}
enum MLModelManagerError: Error, LocalizedError {
case modelNotFound(String)
case loadFailed(String, Error)
case compileFailed(Error)
case predictionFailed(Error)
var errorDescription: String? {
switch self {
case .modelNotFound(let name):
return "ML model '\(name)' not found in app bundle"
case .loadFailed(let name, let error):
return "Failed to load model '\(name)': \(error.localizedDescription)"
case .compileFailed(let error):
return "Model compilation failed: \(error.localizedDescription)"
case .predictionFailed(let error):
return "Prediction failed: \(error.localizedDescription)"
}
}
}2. ImageClassifier.swift — Vision-Based Image Classification
import Vision
import UIKit
import os
/// Classifies images using Vision framework's built-in model.
/// No custom Core ML model required — uses Apple's on-device classification.
struct ImageClassifier {
private let logger = Logger(subsystem: Bundle.main.bundleIdentifier ?? "app", category: "ImageClassifier")
/// Classification result with label and confidence score.
struct Classification: Sendable {
let label: String
let confidence: Float
/// Confidence as a percentage string (e.g., "94.2%").
var confidencePercent: String {
String(format: "%.1f%%", confidence * 100)
}
}
/// Classify an image using Vision's built-in image classifier.
/// - Parameters:
/// - image: The UIImage to classify.
/// - maxResults: Maximum number of classifications to return (sorted by confidence).
/// - minimumConfidence: Minimum confidence threshold. Results below this are filtered out.
/// - Returns: Array of classifications sorted by confidence (highest first).
func classify(
_ image: UIImage,
maxResults: Int = 5,
minimumConfidence: Float = 0.1
) async throws -> [Classification] {
guard let cgImage = image.cgImage else {
throw ImageClassifierError.invalidImage
}
let orientation = CGImagePropertyOrientation(image.imageOrientation)
return try await withCheckedThrowingContinuation { continuation in
let request = VNClassifyImageRequest { request, error in
if let error {
continuation.resume(throwing: ImageClassifierError.visionError(error))
return
}
let results = (request.results as? [VNClassificationObservation] ?? [])
.filter { $0.confidence >= minimumConfidence }
.prefix(maxResults)
.map { Classification(label: $0.identifier, confidence: $0.confidence) }
continuation.resume(returning: Array(results))
}
let handler = VNImageRequestHandler(
cgImage: cgImage,
orientation: orientation,
options: [:]
)
do {
try handler.perform([request])
} catch {
continuation.resume(throwing: ImageClassifierError.visionError(error))
}
}
}
/// Classify an image using a custom Core ML model via Vision.
/// - Parameters:
/// - image: The UIImage to classify.
/// - modelURL: URL to the compiled .mlmodelc file.
/// - maxResults: Maximum number of results.
/// - Returns: Array of classifications.
func classify(
_ image: UIImage,
using modelURL: URL,
maxResults: Int = 5
) async throws -> [Classification] {
guard let cgImage = image.cgImage else {
throw ImageClassifierError.invalidImage
}
let orientation = CGImagePropertyOrientation(image.imageOrientation)
let vnModel = try VNCoreMLModel(for: MLModel(contentsOf: modelURL))
return try await withCheckedThrowingContinuation { continuation in
let request = VNCoreMLRequest(model: vnModel) { request, error in
if let error {
continuation.resume(throwing: ImageClassifierError.visionError(error))
return
}
let results = (request.results as? [VNClassificationObservation] ?? [])
.prefix(maxResults)
.map { Classification(label: $0.identifier, confidence: $0.confidence) }
continuation.resume(returning: Array(results))
}
// Let Vision handle image scaling
request.imageCropAndScaleOption = .centerCrop
let handler = VNImageRequestHandler(
cgImage: cgImage,
orientation: orientation,
options: [:]
)
do {
try handler.perform([request])
} catch {
continuation.resume(throwing: ImageClassifierError.visionError(error))
}
}
}
}
enum ImageClassifierError: Error, LocalizedError {
case invalidImage
case visionError(Error)
case noResults
var errorDescription: String? {
switch self {
case .invalidImage:
return "Could not extract image data for classification"
case .visionError(let error):
return "Vision framework error: \(error.localizedDescription)"
case .noResults:
return "No classification results returned"
}
}
}
// MARK: - CGImagePropertyOrientation Helper
extension CGImagePropertyOrientation {
init(_ uiOrientation: UIImage.Orientation) {
switch uiOrientation {
case .up: self = .up
case .upMirrored: self = .upMirrored
case .down: self = .down
case .downMirrored: self = .downMirrored
case .left: self = .left
case .leftMirrored: self = .leftMirrored
case .right: self = .right
case .rightMirrored: self = .rightMirrored
@unknown default: self = .up
}
}
}3. TextAnalyzer.swift — NaturalLanguage Framework Wrapper
import NaturalLanguage
import os
/// Wrapper around Apple's NaturalLanguage framework for text analysis.
/// Provides sentiment analysis, language detection, tokenization, and entity recognition.
struct TextAnalyzer {
private let logger = Logger(subsystem: Bundle.main.bundleIdentifier ?? "app", category: "TextAnalyzer")
// MARK: - Sentiment Analysis
/// Analyze sentiment of text.
/// - Parameter text: The text to analyze.
/// - Returns: Score from -1.0 (very negative) to +1.0 (very positive). 0.0 is neutral.
func detectSentiment(_ text: String) -> Double {
guard !text.isEmpty else { return 0.0 }
let tagger = NLTagger(tagSchemes: [.sentimentScore])
tagger.string = text
let (tag, _) = tagger.tag(at: text.startIndex, unit: .paragraph, scheme: .sentimentScore)
return Double(tag?.rawValue ?? "0") ?? 0.0
}
/// Sentiment category based on score thresholds.
enum Sentiment: String, Sendable {
case positive, neutral, negative
init(score: Double) {
if score > 0.1 { self = .positive }
else if score < -0.1 { self = .negative }
else { self = .neutral }
}
}
/// Detect sentiment as a category.
func sentimentCategory(_ text: String) -> Sentiment {
Sentiment(score: detectSentiment(text))
}
// MARK: - Language Detection
/// Detect the dominant language of text.
/// - Parameter text: The text to analyze (best results with 20+ characters).
/// - Returns: The detected language, or nil if undetermined.
func detectLanguage(_ text: String) -> NLLanguage? {
NLLanguageRecognizer.dominantLanguage(for: text)
}
/// Detect multiple possible languages with confidence scores.
/// - Parameters:
/// - text: The text to analyze.
/// - maxResults: Maximum number of language hypotheses.
/// - Returns: Dictionary mapping languages to confidence scores (0.0 to 1.0).
func detectLanguages(_ text: String, maxResults: Int = 5) -> [(NLLanguage, Double)] {
let recognizer = NLLanguageRecognizer()
recognizer.processString(text)
return recognizer.languageHypotheses(withMaximum: maxResults)
.sorted { $0.value > $1.value }
.map { ($0.key, $0.value) }
}
// MARK: - Tokenization
/// Tokenize text into units (words, sentences, or paragraphs).
/// - Parameters:
/// - text: The text to tokenize.
/// - unit: The tokenization unit (.word, .sentence, .paragraph).
/// - Returns: Array of token strings.
func tokenize(_ text: String, unit: NLTokenUnit = .word) -> [String] {
guard !text.isEmpty else { return [] }
let tokenizer = NLTokenizer(unit: unit)
tokenizer.string = text
return tokenizer.tokens(for: text.startIndex..<text.endIndex).map { range in
String(text[range])
}
}
/// Count words in text (language-aware, handles CJK and other scripts correctly).
func wordCount(_ text: String) -> Int {
tokenize(text, unit: .word).count
}
// MARK: - Named Entity Recognition
/// Recognized entity with its text and type.
struct Entity: Sendable {
let text: String
let type: EntityType
}
enum EntityType: String, Sendable {
case person
case place
case organization
case other
init(tag: NLTag) {
switch tag {
case .personalName: self = .person
case .placeName: self = .place
case .organizationName: self = .organization
default: self = .other
}
}
}
/// Recognize named entities (people, places, organizations) in text.
/// - Parameter text: The text to analyze.
/// - Returns: Array of recognized entities with their types.
func recognizeEntities(_ text: String) -> [Entity] {
guard !text.isEmpty else { return [] }
let tagger = NLTagger(tagSchemes: [.nameType])
tagger.string = text
var entities: [Entity] = []
let range = text.startIndex..<text.endIndex
tagger.enumerateTags(in: range, unit: .word, scheme: .nameType, options: [.omitWhitespace, .omitPunctuation, .joinNames]) { tag, tokenRange in
if let tag, tag != .other {
entities.append(Entity(
text: String(text[tokenRange]),
type: EntityType(tag: tag)
))
}
return true
}
return entities
}
// MARK: - Part of Speech
/// Tagged word with its part-of-speech tag.
struct TaggedWord: Sendable {
let word: String
let tag: NLTag
}
/// Tag each word with its part of speech (noun, verb, adjective, etc.).
func tagPartsOfSpeech(_ text: String) -> [TaggedWord] {
guard !text.isEmpty else { return [] }
let tagger = NLTagger(tagSchemes: [.lexicalClass])
tagger.string = text
var tagged: [TaggedWord] = []
let range = text.startIndex..<text.endIndex
tagger.enumerateTags(in: range, unit: .word, scheme: .lexicalClass, options: [.omitWhitespace, .omitPunctuation]) { tag, tokenRange in
if let tag {
tagged.append(TaggedWord(word: String(text[tokenRange]), tag: tag))
}
return true
}
return tagged
}
// MARK: - Word Embedding / Similarity
/// Compute cosine distance between two words using Apple's word embedding.
/// - Parameters:
/// - word1: First word.
/// - word2: Second word.
/// - language: Language for the embedding model.
/// - Returns: Distance from 0.0 (identical) to 2.0 (maximally different), or nil if embedding unavailable.
func wordDistance(_ word1: String, _ word2: String, language: NLLanguage = .english) -> Double? {
guard let embedding = NLEmbedding.wordEmbedding(for: language) else { return nil }
return embedding.distance(between: word1, and: word2)
}
/// Find words similar to the given word.
/// - Parameters:
/// - word: The reference word.
/// - maxResults: Maximum similar words to return.
/// - language: Language for the embedding model.
/// - Returns: Array of (word, distance) tuples sorted by similarity.
func similarWords(to word: String, maxResults: Int = 10, language: NLLanguage = .english) -> [(String, Double)] {
guard let embedding = NLEmbedding.wordEmbedding(for: language) else { return [] }
var results: [(String, Double)] = []
embedding.enumerateNeighbors(for: word, maximumCount: maxResults) { neighbor, distance in
results.append((neighbor, distance))
return true
}
return results
}
}4. ModelConfiguration.swift — Configuration and Compute Unit Selection
import CoreML
/// Configuration presets for Core ML model inference.
/// Choose based on your app's performance and power requirements.
struct ModelConfig {
let computeUnits: MLComputeUnits
let maxPredictionBatchSize: Int
let allowLowPrecision: Bool
/// Maximum performance. Uses CPU, GPU, and Neural Engine.
/// Best for: real-time camera processing, time-critical predictions.
static let highPerformance = ModelConfig(
computeUnits: .all,
maxPredictionBatchSize: 10,
allowLowPrecision: true
)
/// CPU only. Predictable latency, no GPU contention with UI rendering.
/// Best for: background processing, when GPU is busy with rendering.
static let lowPower = ModelConfig(
computeUnits: .cpuOnly,
maxPredictionBatchSize: 1,
allowLowPrecision: false
)
/// CPU + Neural Engine. Good balance of speed and efficiency.
/// Best for: most apps. Avoids GPU contention while using Neural Engine acceleration.
static let balanced = ModelConfig(
computeUnits: .cpuAndNeuralEngine,
maxPredictionBatchSize: 5,
allowLowPrecision: true
)
/// CPU + GPU. For devices without Neural Engine or when NE is unavailable.
/// Best for: older devices, GPU-optimized models.
static let gpuAccelerated = ModelConfig(
computeUnits: .cpuAndGPU,
maxPredictionBatchSize: 5,
allowLowPrecision: true
)
/// Create an MLModelConfiguration from this config.
func mlConfiguration() -> MLModelConfiguration {
let config = MLModelConfiguration()
config.computeUnits = computeUnits
config.allowLowPrecisionAccumulationOnGPU = allowLowPrecision
return config
}
}5. VisionService.swift — Vision Request Pipeline
import Vision
import UIKit
import os
/// Service for executing Vision framework requests with proper error handling
/// and image orientation support.
struct VisionService {
private let logger = Logger(subsystem: Bundle.main.bundleIdentifier ?? "app", category: "VisionService")
// MARK: - Text Recognition (OCR)
/// Recognize text in an image.
/// - Parameters:
/// - image: The image to scan.
/// - level: Recognition accuracy level. `.accurate` is slower but better.
/// - languages: Languages to recognize (e.g., ["en-US", "fr-FR"]). Nil for automatic.
/// - Returns: Array of recognized text strings, ordered top-to-bottom.
func recognizeText(
in image: UIImage,
level: VNRequestTextRecognitionLevel = .accurate,
languages: [String]? = nil
) async throws -> [String] {
guard let cgImage = image.cgImage else {
throw VisionServiceError.invalidImage
}
let orientation = CGImagePropertyOrientation(image.imageOrientation)
return try await withCheckedThrowingContinuation { continuation in
let request = VNRecognizeTextRequest { request, error in
if let error {
continuation.resume(throwing: VisionServiceError.requestFailed(error))
return
}
let results = (request.results ?? [])
.compactMap { $0.topCandidates(1).first?.string }
continuation.resume(returning: results)
}
request.recognitionLevel = level
if let languages {
request.recognitionLanguages = languages
}
performRequest(request, on: cgImage, orientation: orientation, continuation: continuation)
}
}
// MARK: - Face Detection
/// Detected face with bounding box and optional landmarks.
struct DetectedFace: Sendable {
/// Bounding box in normalized coordinates (0.0 to 1.0, origin at bottom-left).
let boundingBox: CGRect
let landmarks: VNFaceLandmarks2D?
}
/// Detect faces in an image.
/// - Parameters:
/// - image: The image to scan.
/// - includeLandmarks: Whether to detect facial landmarks (eyes, nose, mouth, etc.).
/// - Returns: Array of detected faces.
func detectFaces(
in image: UIImage,
includeLandmarks: Bool = false
) async throws -> [DetectedFace] {
guard let cgImage = image.cgImage else {
throw VisionServiceError.invalidImage
}
let orientation = CGImagePropertyOrientation(image.imageOrientation)
return try await withCheckedThrowingContinuation { continuation in
if includeLandmarks {
let request = VNDetectFaceLandmarksRequest { request, error in
if let error {
continuation.resume(throwing: VisionServiceError.requestFailed(error))
return
}
let results = (request.results ?? []).map { face in
DetectedFace(boundingBox: face.boundingBox, landmarks: face.landmarks)
}
continuation.resume(returning: results)
}
performRequest(request, on: cgImage, orientation: orientation, continuation: continuation)
} else {
let request = VNDetectFaceRectanglesRequest { request, error in
if let error {
continuation.resume(throwing: VisionServiceError.requestFailed(error))
return
}
let results = (request.results ?? []).map { face in
DetectedFace(boundingBox: face.boundingBox, landmarks: nil)
}
continuation.resume(returning: results)
}
performRequest(request, on: cgImage, orientation: orientation, continuation: continuation)
}
}
}
// MARK: - Barcode Detection
/// Detected barcode with payload and symbology.
struct DetectedBarcode: Sendable {
let payload: String
let symbology: VNBarcodeSymbology
let boundingBox: CGRect
}
/// Detect barcodes and QR codes in an image.
/// - Parameters:
/// - image: The image to scan.
/// - symbologies: Specific barcode types to look for. Nil for all supported types.
/// - Returns: Array of detected barcodes.
func detectBarcodes(
in image: UIImage,
symbologies: [VNBarcodeSymbology]? = nil
) async throws -> [DetectedBarcode] {
guard let cgImage = image.cgImage else {
throw VisionServiceError.invalidImage
}
let orientation = CGImagePropertyOrientation(image.imageOrientation)
return try await withCheckedThrowingContinuation { continuation in
let request = VNDetectBarcodesRequest { request, error in
if let error {
continuation.resume(throwing: VisionServiceError.requestFailed(error))
return
}
let results = (request.results ?? []).compactMap { barcode -> DetectedBarcode? in
guard let payload = barcode.payloadStringValue else { return nil }
return DetectedBarcode(
payload: payload,
symbology: barcode.symbology,
boundingBox: barcode.boundingBox
)
}
continuation.resume(returning: results)
}
if let symbologies {
request.symbologies = symbologies
}
performRequest(request, on: cgImage, orientation: orientation, continuation: continuation)
}
}
// MARK: - Body Pose Detection
/// Detected body pose with joint positions.
struct BodyPose: Sendable {
let joints: [VNHumanBodyPoseObservation.JointName: CGPoint]
let confidence: Float
}
/// Detect human body poses in an image.
/// - Parameter image: The image to analyze.
/// - Returns: Array of detected body poses with joint positions.
func detectBodyPose(in image: UIImage) async throws -> [BodyPose] {
guard let cgImage = image.cgImage else {
throw VisionServiceError.invalidImage
}
let orientation = CGImagePropertyOrientation(image.imageOrientation)
return try await withCheckedThrowingContinuation { continuation in
let request = VNDetectHumanBodyPoseRequest { request, error in
if let error {
continuation.resume(throwing: VisionServiceError.requestFailed(error))
return
}
let results = (request.results ?? []).compactMap { observation -> BodyPose? in
guard let points = try? observation.recognizedPoints(.all) else { return nil }
let joints = points.reduce(into: [VNHumanBodyPoseObservation.JointName: CGPoint]()) { dict, pair in
if pair.value.confidence > 0.3 {
dict[pair.key] = pair.value.location
}
}
return BodyPose(joints: joints, confidence: observation.confidence)
}
continuation.resume(returning: results)
}
performRequest(request, on: cgImage, orientation: orientation, continuation: continuation)
}
}
// MARK: - Private Helpers
private func performRequest<T>(
_ request: VNRequest,
on cgImage: CGImage,
orientation: CGImagePropertyOrientation,
continuation: CheckedContinuation<T, Error>
) {
let handler = VNImageRequestHandler(
cgImage: cgImage,
orientation: orientation,
options: [:]
)
do {
try handler.perform([request])
} catch {
continuation.resume(throwing: VisionServiceError.requestFailed(error))
}
}
}
enum VisionServiceError: Error, LocalizedError {
case invalidImage
case requestFailed(Error)
var errorDescription: String? {
switch self {
case .invalidImage:
return "Could not extract image data for Vision processing"
case .requestFailed(let error):
return "Vision request failed: \(error.localizedDescription)"
}
}
}6. SwiftUI Integration Examples
Image Classification View
import SwiftUI
struct ClassifierView: View {
@State private var selectedImage: UIImage?
@State private var classifications: [ImageClassifier.Classification] = []
@State private var isClassifying = false
@State private var error: String?
@State private var showingImagePicker = false
private let classifier = ImageClassifier()
var body: some View {
NavigationStack {
VStack(spacing: 20) {
if let image = selectedImage {
Image(uiImage: image)
.resizable()
.scaledToFit()
.frame(maxHeight: 300)
.clipShape(RoundedRectangle(cornerRadius: 12))
}
if isClassifying {
ProgressView("Classifying...")
}
if !classifications.isEmpty {
List(classifications, id: \.label) { item in
HStack {
Text(item.label)
Spacer()
Text(item.confidencePercent)
.foregroundStyle(.secondary)
}
}
.listStyle(.plain)
}
if let error {
Text(error)
.foregroundStyle(.red)
.font(.caption)
}
}
.padding()
.navigationTitle("Image Classifier")
.toolbar {
Button("Choose Photo") { showingImagePicker = true }
}
.sheet(isPresented: $showingImagePicker) {
// Use your image picker implementation
}
.onChange(of: selectedImage) { _, newImage in
guard let newImage else { return }
Task { await classifyImage(newImage) }
}
}
}
private func classifyImage(_ image: UIImage) async {
isClassifying = true
error = nil
defer { isClassifying = false }
do {
classifications = try await classifier.classify(image, maxResults: 5, minimumConfidence: 0.05)
} catch {
self.error = error.localizedDescription
}
}
}Text Analysis View
import SwiftUI
struct TextAnalysisView: View {
@State private var inputText = ""
@State private var sentiment: Double = 0
@State private var language: String = ""
@State private var entities: [TextAnalyzer.Entity] = []
@State private var wordCount: Int = 0
private let analyzer = TextAnalyzer()
var body: some View {
NavigationStack {
Form {
Section("Input") {
TextEditor(text: $inputText)
.frame(minHeight: 100)
}
if !inputText.isEmpty {
Section("Analysis") {
LabeledContent("Sentiment") {
HStack {
Text(sentimentEmoji)
Text(String(format: "%.2f", sentiment))
.foregroundStyle(.secondary)
}
}
LabeledContent("Language", value: language)
LabeledContent("Word Count", value: "\(wordCount)")
}
if !entities.isEmpty {
Section("Entities") {
ForEach(entities, id: \.text) { entity in
LabeledContent(entity.text, value: entity.type.rawValue)
}
}
}
}
}
.navigationTitle("Text Analysis")
.onChange(of: inputText) { _, newText in
analyzeText(newText)
}
}
}
private func analyzeText(_ text: String) {
sentiment = analyzer.detectSentiment(text)
language = analyzer.detectLanguage(text)?.rawValue ?? "Unknown"
entities = analyzer.recognizeEntities(text)
wordCount = analyzer.wordCount(text)
}
private var sentimentEmoji: String {
if sentiment > 0.1 { return "+" }
else if sentiment < -0.1 { return "-" }
else { return "~" }
}
}