Axiom Graphics

Graphics

You MUST use this skill for ANY GPU rendering, graphics programming, 3D content display, or display performance work.

When to Use

Use this router when:

• Porting OpenGL/OpenGL ES code to Metal
• Porting DirectX code to Metal
• Converting GLSL/HLSL shaders to Metal Shading Language
• Setting up MTKView or CAMetalLayer
• Debugging GPU rendering issues (black screen, wrong colors, crashes)
• Evaluating translation layers (MetalANGLE, MoltenVK)
• Optimizing GPU performance or fixing thermal throttling
• App stuck at 60fps on ProMotion device
• Configuring CADisplayLink or render loops
• Variable refresh rate display issues
• Displaying 3D content in a non-game SwiftUI app
• Building AR experiences with RealityKit
• Using RealityView or Model3D in SwiftUI
• Spatial computing or visionOS 3D content
• Reading, editing, or exporting USD/USDZ files in Swift (USDKit)
• Adding ML to a render pipeline (MetalFX denoising, Metal tensors, neural rendering)
• Profiling long game sessions (metalperftrace, look-back traces)

Routing Logic

Metal Migration

Strategy decisions → See skills/metal-migration.md

• Translation layer vs native rewrite decision
• Project assessment and migration planning
• Anti-patterns and common mistakes
• Pressure scenarios for deadline resistance

API reference & conversion → See skills/metal-migration-ref.md

• GLSL → MSL shader conversion tables
• HLSL → MSL shader conversion tables
• GL/D3D API → Metal API equivalents
• MTKView setup, render pipelines, compute shaders
• Complete WWDC code examples

Diagnostics → See skills/metal-migration-diag.md

• Black screen after porting
• Shader compilation errors
• Wrong colors or coordinate systems
• Performance regressions
• Time-cost analysis per diagnostic path

Display Performance

Frame rate & render loops → See skills/display-performance.md

• App stuck at 60fps on ProMotion (120Hz) device
• MTKView or CADisplayLink configuration
• Variable refresh rate optimization
• System caps (Low Power Mode, Limit Frame Rate, Thermal, Adaptive Power)
• Frame budget math (8.33ms for 120Hz)
• Measuring actual vs reported frame rate

RealityKit (Non-Game 3D Content)

For 3D content in non-game SwiftUI apps, AR experiences, and spatial computing, use the RealityKit skills. For game-specific RealityKit patterns, use the axiom-games router instead.

Architecture, ECS, and best practices → See skills/realitykit.md

• Entity-Component-System architecture
• SwiftUI integration: RealityView, Model3D, attachments
• AR on iOS: AnchorEntity types, SpatialTrackingSession
• Materials, physics, interaction
• Performance optimization

API reference → See skills/realitykit-ref.md

• Complete component catalog
• RealityView and Model3D API
• Material system (PBR, Unlit, Occlusion, Custom)
• RealityRenderer (Metal integration)

Troubleshooting → See skills/realitykit-diag.md

• Entity not visible, anchor not tracking
• Gesture not responding, performance issues
• Material problems, physics issues

USD Authoring (USDKit) OS27

USD file work in Swift → See skills/usdkit.md

• Open, traverse, edit USD stages and prims
• References and composition
• Accessibility metadata for 3D assets
• USDZ export with mesh/texture compression
• Rendering a USD stage in RealityKit (USDStageComponent)

Decision Tree

1. Translation layer vs native rewrite? → metal-migration 2. Porting / converting code to Metal? → metal-migration 3. API reference / shader conversion tables? → metal-migration-ref 4. MTKView / render pipeline setup? → metal-migration-ref 5. Something broken after porting (black screen, wrong colors)? → metal-migration-diag 6. Stuck at 60fps on ProMotion device? → display-performance 7. CADisplayLink / variable refresh rate? → display-performance 8. Frame rate not as expected? → display-performance 9. Display a 3D model in SwiftUI? → realitykit 10. Build an AR experience? → realitykit 11. RealityView or Model3D setup? → realitykit-ref 12. 3D content not visible or not tracking? → realitykit-diag 13. Custom Metal rendering of RealityKit content? → realitykit-ref (RealityRenderer) 14. Pathfinding, LOD, soft shadows, splats, reverb meshes? → realitykit-ref (Part 10) 15. Read/edit/export a USD or USDZ file in Swift? → usdkit 16. ML in the render pipeline (denoising, Metal tensors)? → metal-migration-ref (Part 6) 17. Frame drops in long play sessions? → display-performance (Part 12) 18. Building a 3D game? → Use axiom-games router instead

Anti-Rationalization

Critical Patterns

metal-migration:

• Translation layer (MetalANGLE) for quick demos
• Native Metal rewrite for production
• State management differences (GL stateful → Metal explicit)
• Coordinate system gotchas (Y-flip, NDC differences)

metal-migration-ref:

• Complete shader type mappings
• API equivalent tables
• MTKView vs CAMetalLayer decision
• Render pipeline setup patterns

metal-migration-diag:

• GPU Frame Capture workflow (2-5 min vs 30+ min guessing)
• Shader debugger for variable inspection
• Metal validation layer for API misuse
• Performance regression diagnosis

display-performance:

• MTKView defaults to 60fps (must set preferredFramesPerSecond = 120)
• CADisplayLink preferredFrameRateRange for explicit rate control
• System caps: Low Power Mode, Limit Frame Rate, Thermal, Adaptive Power (iOS 26)
• 8.33ms frame budget for 120Hz
• UIScreen.maximumFramesPerSecond lies; CADisplayLink tells truth

realitykit (non-game 3D):

• RealityView make/update closure pattern
• Model3D for simple model display
• AR anchoring with AnchorEntity
• Material selection (SimpleMaterial, PBR, Occlusion)

realitykit-ref (API):

• RealityRenderer for custom Metal rendering of RealityKit content
• Complete material property reference
• RealityView gesture integration
• RealityKit 27 additions: navigation mesh, LOD, soft shadows, Gaussian splats (visionOS), reverb meshes (Part 10), cloth simulation (ClothBodyComponent), ComputeGraph framework

usdkit (USD authoring, 27 releases):

• USDStage open/traverse/edit, references and composition
• Accessibility metadata schema for 3D assets
• Compressed USDZ export (AOM meshes + AVIF textures)
• USDStageComponent / USDPlayer RealityKit bridge

Example Invocations

User: "Should I use MetalANGLE or rewrite in native Metal?" → See skills/metal-migration.md

User: "I'm porting projectM from OpenGL ES to iOS" → See skills/metal-migration.md

User: "How do I convert this GLSL shader to Metal?" → See skills/metal-migration-ref.md

User: "Setting up MTKView for the first time" → See skills/metal-migration-ref.md

User: "My ported app shows a black screen" → See skills/metal-migration-diag.md

User: "Performance is worse after porting to Metal" → See skills/metal-migration-diag.md

User: "My app is stuck at 60fps on iPhone Pro" → See skills/display-performance.md

User: "How do I configure CADisplayLink for 120Hz?" → See skills/display-performance.md

User: "ProMotion not working in my Metal app" → See skills/display-performance.md

User: "How do I show a 3D model in my SwiftUI app?" → See skills/realitykit.md

User: "I need to display a USDZ model" → See skills/realitykit.md

User: "How do I set up RealityView?" → See skills/realitykit-ref.md

User: "My 3D model isn't showing in RealityView" → See skills/realitykit-diag.md

User: "How do I use RealityRenderer with Metal?" → See skills/realitykit-ref.md

User: "I need AR in my app" → See skills/realitykit.md

User: "How do I read and edit a USD file in Swift?" → See skills/usdkit.md

User: "How do I shrink my USDZ assets for delivery?" → See skills/usdkit.md

User: "How do I add pathfinding to my RealityKit scene?" → See skills/realitykit-ref.md (Part 10)

User: "My game's frame rate drops after an hour of play" → See skills/display-performance.md (Part 12)

User: "How do I run a neural network inside my Metal shader?" → See skills/metal-migration-ref.md (Part 6)

Display Performance

Systematic diagnosis for frame rate issues on variable refresh rate displays (ProMotion, iPad Pro, future devices). Covers render loop configuration, frame pacing, hitch mechanics, and production telemetry.

Key insight: "ProMotion available" does NOT mean your app automatically runs at 120Hz. You must configure it correctly, account for system caps, and ensure proper frame pacing.

---

Part 1: Why You're Stuck at 60fps

Diagnostic Order

Check these in order when stuck at 60fps on ProMotion:

1. Info.plist key missing? (iPhone only) → Part 2 2. Render loop configured for 60? (MTKView defaults, CADisplayLink) → Part 3 3. System caps enabled? (Low Power Mode, Limit Frame Rate, Thermal) → Part 5 4. Frame time > 8.33ms? (Can't sustain 120fps) → Part 6 5. Frame pacing issues? (Micro-stuttering despite good FPS) → Part 7 6. Measuring wrong thing? (UIScreen vs actual presentation) → Part 9

---

Part 2: Enabling ProMotion on iPhone

Critical: Core Animation won't access frame rates above 60Hz on iPhone unless you add this key.

<!-- Info.plist -->
<key>CADisableMinimumFrameDurationOnPhone</key>
<true/>

Without this key:

• Your preferredFrameRateRange hints are ignored above 60Hz
• Other animations may affect your CADisplayLink callback rate
• iPad Pro does NOT require this key

When to add: Any iPhone app that needs >60Hz for games, animations, or smooth scrolling.

---

Part 3: Render Loop Configuration

MTKView Defaults to 60fps

This is the most common cause. MTKView's preferredFramesPerSecond defaults to 60.

// ❌ WRONG: Implicit 60fps (default)
let mtkView = MTKView(frame: frame, device: device)
mtkView.delegate = self
// Running at 60fps even on ProMotion!

// ✅ CORRECT: Explicit 120fps request
let mtkView = MTKView(frame: frame, device: device)
mtkView.preferredFramesPerSecond = 120
mtkView.isPaused = false
mtkView.enableSetNeedsDisplay = false  // Continuous, not on-demand
mtkView.delegate = self

Critical settings for continuous high-rate rendering:

CADisplayLink Configuration (iOS 15+)

Apple explicitly recommends CADisplayLink (not timers) for custom render loops.

// ❌ WRONG: Timer-based render loop (drifts, wastes frame time)
Timer.scheduledTimer(withTimeInterval: 1.0/120.0, repeats: true) { _ in
    self.render()
}

// ❌ WRONG: Default CADisplayLink (may hint 60)
let displayLink = CADisplayLink(target: self, selector: #selector(render))
displayLink.add(to: .main, forMode: .common)

// ✅ CORRECT: Explicit frame rate range
let displayLink = CADisplayLink(target: self, selector: #selector(render))
displayLink.preferredFrameRateRange = CAFrameRateRange(
    minimum: 80,      // Minimum acceptable
    maximum: 120,     // Preferred maximum
    preferred: 120    // What you want
)
displayLink.add(to: .main, forMode: .common)

Special priority for games: iOS 15+ gives 30Hz and 60Hz special priority. If targeting these rates:

// 30Hz and 60Hz get priority scheduling
let prioritizedRange = CAFrameRateRange(
    minimum: 30,
    maximum: 60,
    preferred: 60
)
displayLink.preferredFrameRateRange = prioritizedRange

Suggested Frame Rates by Content Type

---

Part 4: CAMetalDisplayLink (iOS 17+)

For Metal apps needing precise timing control, CAMetalDisplayLink provides more control than CADisplayLink.

class MetalRenderer: NSObject, CAMetalDisplayLinkDelegate {
    var displayLink: CAMetalDisplayLink?
    var metalLayer: CAMetalLayer!

    func setupDisplayLink() {
        displayLink = CAMetalDisplayLink(metalLayer: metalLayer)
        displayLink?.delegate = self
        displayLink?.preferredFrameRateRange = CAFrameRateRange(
            minimum: 60,
            maximum: 120,
            preferred: 120
        )
        // Control render latency (in frames)
        displayLink?.preferredFrameLatency = 2
        displayLink?.add(to: .main, forMode: .common)
    }

    func metalDisplayLink(_ link: CAMetalDisplayLink, needsUpdate update: CAMetalDisplayLink.Update) {
        // update.drawable - The drawable to render to
        // update.targetTimestamp - Deadline to finish rendering
        // update.targetPresentationTimestamp - When frame will display

        guard let drawable = update.drawable else { return }

        let workingTime = update.targetTimestamp - CACurrentMediaTime()
        // workingTime = seconds available before deadline

        // Render to drawable...
        renderFrame(to: drawable)
    }
}

Key differences from CADisplayLink:

When to use CAMetalDisplayLink:

• Need precise control over render timing window
• Want to minimize input latency
• Building games or intensive Metal apps
• iOS 17+ only deployment

---

Part 5: System Caps

System states can force 60fps even when your code requests 120:

Low Power Mode

Caps ProMotion devices to 60fps.

// Check programmatically
if ProcessInfo.processInfo.isLowPowerModeEnabled {
    // System caps display to 60Hz
}

// Observe changes
NotificationCenter.default.addObserver(
    forName: .NSProcessInfoPowerStateDidChange,
    object: nil,
    queue: .main
) { _ in
    let isLowPower = ProcessInfo.processInfo.isLowPowerModeEnabled
    self.adjustRenderingForPowerState(isLowPower)
}

Limit Frame Rate (Accessibility)

Settings → Accessibility → Motion → Limit Frame Rate caps to 60fps.

No API to detect. If user reports 60fps despite configuration, have them check this setting.

Thermal Throttling

System restricts 120Hz when device overheats.

// Check thermal state
switch ProcessInfo.processInfo.thermalState {
case .nominal, .fair:
    preferredFramesPerSecond = 120
case .serious, .critical:
    preferredFramesPerSecond = 60  // Reduce proactively
@unknown default:
    break
}

// Observe thermal changes
NotificationCenter.default.addObserver(
    forName: ProcessInfo.thermalStateDidChangeNotification,
    object: nil,
    queue: .main
) { _ in
    self.adjustForThermalState()
}

Adaptive Power (iOS 26+, iPhone 17)

New in iOS 26: Adaptive Power is ON by default on iPhone 17/17 Pro. Can throttle even at 60% battery.

User action for testing: Settings → Battery → Power Mode → disable Adaptive Power.

No public API to detect Adaptive Power state.

---

Part 6: Performance Budget

Frame Time Budgets

If you consistently exceed budget, system drops to next sustainable rate.

Measuring GPU Frame Time

func draw(in view: MTKView) {
    guard let commandBuffer = commandQueue.makeCommandBuffer() else { return }

    // Your rendering code...

    commandBuffer.addCompletedHandler { buffer in
        let gpuTime = buffer.gpuEndTime - buffer.gpuStartTime
        let gpuMs = gpuTime * 1000

        if gpuMs > 8.33 {
            print("⚠️ GPU: \(String(format: "%.2f", gpuMs))ms exceeds 120Hz budget")
        }
    }

    commandBuffer.commit()
}

Can't Sustain 120? Target Lower Rate Evenly

Critical: Uneven frame pacing looks worse than consistent lower rate.

// If you can't sustain 8.33ms, explicitly target 60 for smooth cadence
if averageGpuTime > 8.33 && averageGpuTime <= 16.67 {
    mtkView.preferredFramesPerSecond = 60
}

---

Part 7: Frame Pacing

The Micro-Stuttering Problem

Even with good average FPS, inconsistent frame timing causes visible jitter.

// BAD: Inconsistent intervals despite ~40 FPS average
Frame 1: 25ms
Frame 2: 40ms  ← stutter
Frame 3: 25ms
Frame 4: 40ms  ← stutter

// GOOD: Consistent intervals at 30 FPS
Frame 1: 33ms
Frame 2: 33ms
Frame 3: 33ms
Frame 4: 33ms

Presenting immediately after rendering causes this. Use explicit timing control.

Frame Pacing APIs

present(afterMinimumDuration:) — Recommended

Ensures consistent spacing between frames:

func draw(in view: MTKView) {
    guard let commandBuffer = commandQueue.makeCommandBuffer(),
          let drawable = view.currentDrawable else { return }

    // Render to drawable...

    // Present with minimum 33ms between frames (30 FPS target)
    commandBuffer.present(drawable, afterMinimumDuration: 0.033)
    commandBuffer.commit()
}

present(at:) — Precise Timing

Schedule presentation at specific time:

// Present at specific Mach absolute time
let presentTime = CACurrentMediaTime() + 0.033
commandBuffer.present(drawable, atTime: presentTime)

presentedTime — Verify Actual Presentation

Check when frames actually appeared:

drawable.addPresentedHandler { drawable in
    let actualTime = drawable.presentedTime
    if actualTime == 0.0 {
        // Frame was dropped!
        print("⚠️ Frame dropped")
    } else {
        print("Frame presented at: \(actualTime)")
    }
}

Frame Pacing Pattern

class SmoothRenderer: NSObject, MTKViewDelegate {
    private var targetFrameDuration: CFTimeInterval = 1.0 / 60.0  // 60 FPS target

    func draw(in view: MTKView) {
        guard let commandBuffer = commandQueue.makeCommandBuffer(),
              let drawable = view.currentDrawable else { return }

        renderScene(to: drawable)

        // Use frame pacing to ensure consistent intervals
        commandBuffer.present(drawable, afterMinimumDuration: targetFrameDuration)
        commandBuffer.commit()
    }

    func adjustTargetFrameRate(canSustain fps: Int) {
        switch fps {
        case 90...:
            targetFrameDuration = 1.0 / 120.0
        case 50...:
            targetFrameDuration = 1.0 / 60.0
        default:
            targetFrameDuration = 1.0 / 30.0
        }
    }
}

---

Part 8: Understanding Hitches

Render Loop Phases

Frame lifecycle: Begin Time → Commit Deadline → Presentation Time

1. App Process (CPU): Handle events, compute UI updates, Core Animation commit 2. Render Server (CPU+GPU): Transform UI to bitmap, render to buffer 3. Display Driver: Swap buffer to screen at vsync

At 120Hz, each phase has ~8.33ms. Miss any deadline = hitch.

Commit Hitch vs Render Hitch

Commit Hitch: App process misses commit deadline

• Cause: Main thread work takes too long
• Fix: Move work off main thread, reduce view complexity

Render Hitch: Render server misses presentation deadline

• Cause: GPU work too complex (blur, shadows, layers)
• Fix: Simplify visual effects, reduce overdraw

Double vs Triple Buffering

Double Buffer (default):

• Frame lifetime: 2 vsync intervals
• Tighter deadlines
• Lower latency

Triple Buffer (system may enable):

• Frame lifetime: 3 vsync intervals
• Render server gets 2 vsync intervals
• Higher latency but more headroom

The system automatically switches to triple buffering to recover from render hitches.

Hitch Duration

Expected Frame Lifetime = Begin Time → Presentation Time
Actual Frame Lifetime = Begin Time → Actual Vsync

Hitch Duration = Actual - Expected

If hitch duration > 0, the frame was late and previous frame stayed onscreen longer.

---

Part 9: Measurement

UIScreen Lies, Actual Presentation Tells Truth

// ❌ This says 120 even when system caps you to 60
let maxFPS = UIScreen.main.maximumFramesPerSecond
// Reports capability, not actual rate!

// ✅ Measure from CADisplayLink timing
@objc func displayLinkCallback(_ link: CADisplayLink) {
    // Time available to prepare next frame
    let workingTime = link.targetTimestamp - CACurrentMediaTime()

    // Actual interval since last callback
    if lastTimestamp > 0 {
        let interval = link.timestamp - lastTimestamp
        let actualFPS = 1.0 / interval
    }
    lastTimestamp = link.timestamp
}

Metal Performance HUD

Enable on-device real-time performance overlay:

Via Xcode scheme: 1. Edit Scheme → Run → Diagnostics 2. Enable "Show Graphics Overview" 3. Optionally enable "Log Graphics Overview"

Via environment variable:

MTL_HUD_ENABLED=1

Via device settings: Settings → Developer → Graphics HUD → Show Graphics HUD

HUD shows:

• FPS (average)
• GPU time per frame
• Frame interval chart (last 120 frames)
• Memory usage

The HUD's configuration panel lets you enable/disable individual metrics or pick a preset, and in the 27 releases it can display your StateReporting domains (label, stable and volatile metadata) — see Part 12.

Production Telemetry with MetricKit

Monitor hitches in production:

import MetricKit

class MetricsManager: NSObject, MXMetricManagerSubscriber {
    func didReceive(_ payloads: [MXMetricPayload]) {
        for payload in payloads {
            if let animationMetrics = payload.animationMetrics {
                // Ratio of time spent hitching during scroll
                let scrollHitchRatio = animationMetrics.scrollHitchTimeRatio

                // Ratio of time spent hitching in all animations
                if #available(iOS 26.0, macOS 26.0, *) {
                    let hitchRatio = animationMetrics.hitchTimeRatio
                }

                analyzeHitchMetrics(scrollHitchRatio: scrollHitchRatio)
            }
        }
    }
}

// Register for metrics
MXMetricManager.shared.add(metricsManager)

What to track:

• scrollHitchTimeRatio: Time spent hitching while scrolling (UIScrollView only)
• hitchTimeRatio (iOS 26+ / macOS 26+): Time spent hitching in all tracked animations

---

Part 10: Quick Diagnostic Checklist

When debugging frame rate issues:

---

Part 11: Common Patterns

Pattern: Adaptive Frame Rate with Thermal Awareness

class AdaptiveRenderer: NSObject, MTKViewDelegate {
    private var recentFrameTimes: [Double] = []
    private let sampleCount = 30
    private var targetFrameDuration: CFTimeInterval = 1.0 / 60.0

    func draw(in view: MTKView) {
        guard let commandBuffer = commandQueue.makeCommandBuffer(),
              let drawable = view.currentDrawable else { return }

        let startTime = CACurrentMediaTime()
        renderScene(to: drawable)
        let frameTime = (CACurrentMediaTime() - startTime) * 1000

        updateTargetRate(frameTime: frameTime, view: view)

        commandBuffer.present(drawable, afterMinimumDuration: targetFrameDuration)
        commandBuffer.commit()
    }

    private func updateTargetRate(frameTime: Double, view: MTKView) {
        recentFrameTimes.append(frameTime)
        if recentFrameTimes.count > sampleCount {
            recentFrameTimes.removeFirst()
        }

        let avgFrameTime = recentFrameTimes.reduce(0, +) / Double(recentFrameTimes.count)
        let thermal = ProcessInfo.processInfo.thermalState
        let lowPower = ProcessInfo.processInfo.isLowPowerModeEnabled

        // Constrain based on what we can sustain AND system state
        if lowPower || thermal >= .serious {
            view.preferredFramesPerSecond = 30
            targetFrameDuration = 1.0 / 30.0
        } else if avgFrameTime < 7.0 && thermal == .nominal {
            view.preferredFramesPerSecond = 120
            targetFrameDuration = 1.0 / 120.0
        } else if avgFrameTime < 14.0 {
            view.preferredFramesPerSecond = 60
            targetFrameDuration = 1.0 / 60.0
        } else {
            view.preferredFramesPerSecond = 30
            targetFrameDuration = 1.0 / 30.0
        }
    }
}

Pattern: Frame Drop Detection

class FrameDropMonitor {
    private var expectedPresentTime: CFTimeInterval = 0
    private var dropCount = 0

    func trackFrame(drawable: MTLDrawable, expectedInterval: CFTimeInterval) {
        drawable.addPresentedHandler { [weak self] drawable in
            guard let self = self else { return }

            if drawable.presentedTime == 0.0 {
                self.dropCount += 1
                print("⚠️ Frame dropped (total: \(self.dropCount))")
            } else if self.expectedPresentTime > 0 {
                let actualInterval = drawable.presentedTime - self.expectedPresentTime
                let variance = abs(actualInterval - expectedInterval)

                if variance > expectedInterval * 0.5 {
                    print("⚠️ Frame timing variance: \(variance * 1000)ms")
                }
            }

            self.expectedPresentTime = drawable.presentedTime
        }
    }
}

---

Part 12: Long-Session Game Performance Tracing OS27

Frame drops in long play sessions (thermal shifts, level changes, settings changes) escape desk profiling. In the 27 releases the system continuously records Metal performance and resource-usage metrics — aggregated plus optional per-frame CPU/GPU/FPS/memory — and keeps them for days, so you can collect a trace after the session ends (iOS27/macOS27).

Collecting Traces

# Collect the last 5 hours into an .atrc trace
metalperftrace collect /tmp --last 5h

# Or an explicit range
metalperftrace collect /tmp --start 2026-04-01T09:41:00 --end 2026-04-01T12:41:00

# Print an overview (memory, CPU time, disk I/O; per-layer FPS, frame time,
# CPU begin-to-present, on-GPU time, drawable waits, shader compilation)
metalperftrace overview /Data/MyGameTrace.atrc

# Filter to one process; emit JSON for scripts/regression gates
metalperftrace overview /Data/MyGameTrace.atrc --json

Traces also open in Instruments: metrics plot on a timeline, deviating ranges are highlighted, and selecting a range re-aggregates min/max/avg/stddev.

Contextualizing with StateReporting

An FPS dip at minute 12 is unactionable without knowing what the game was doing. The StateReporting framework (all 27 platforms) lets you report domains — finite state machines like level, graphics settings, network status — with labeled states plus stable/volatile metadata. The API and semantics are documented in axiom-performance (skills/metrickit-ref.md) Part 1; game-side guidance:

• Design domains to be conceptually orthogonal (level vs graphics vs network) — don't pack dimensions into one domain.
• Transition at user-action cadence or slower. The system throttles high-frequency transitions and you lose data until the rate recovers.
• Verify adoption live: the Metal Performance HUD can display each domain's label and metadata; Instruments graphs each domain as a Points of Interest track.

metalperftrace integrates directly:

# List domains, transition counts, last known state + full transition history
metalperftrace overview /Data/MyGameTrace.atrc --include-state-transitions

# Aggregate metrics per state — e.g. average FPS while graphics was "High"
metalperftrace overview /Data/MyGameTrace.atrc --aggregate \
  --domain com.mygame.graphics --state-label "High"

After Shipping

MetricKit's 27-cycle Swift API reports Metal frame rate from player devices in daily metric reports — including frame rate grouped by your StateReporting states — plus memory-exception diagnostics when the game is killed for exceeding its memory limit. See axiom-performance (skills/metrickit-ref.md) Part 1 (MetricResult.metalFrameRate); don't re-implement collection here.

---

Resources

WWDC: 2021-10147, 2018-612, 2022-10083, 2023-10123, 2026-388

Tech Talks: 10855, 10856, 10857 (Hitch deep dives)

Docs: /quartzcore/cadisplaylink, /quartzcore/cametaldisplaylink, /quartzcore/optimizing-iphone-and-ipad-apps-to-support-promotion-displays, /xcode/understanding-hitches-in-your-app, /metal/mtldrawable/present(afterminimumduration:), /metrickit/mxanimationmetric

Skills: axiom-performance (skills/energy.md), axiom-graphics, axiom-graphics (skills/metal-migration-ref.md), axiom-performance (skills/performance-profiling.md), axiom-performance (skills/metrickit-ref.md)

Metal Migration Diagnostics

Systematic diagnosis for common Metal porting issues.

When to Use This Diagnostic Skill

Use this skill when:

• Screen is black after porting to Metal
• Shaders fail to compile in Metal
• Colors or coordinates are wrong
• Performance is worse than the original
• Rendering artifacts appear
• Camera or video frames need a Metal texture (CVOpenGLESTextureCache replacement)
• App crashes during GPU work

Mandatory First Step: Enable Metal Validation

Time cost: 30 seconds setup vs hours of blind debugging

Before ANY debugging, enable Metal validation:

Xcode → Edit Scheme → Run → Diagnostics
✓ Metal API Validation
✓ Metal Shader Validation
✓ GPU Frame Capture (Metal)

Most Metal bugs produce clear validation errors. If you're debugging without validation enabled, stop and enable it first.

Symptom 1: Black Screen

Decision Tree

Black screen after porting
│
├─ Are there Metal validation errors in console?
│   └─ YES → Fix validation errors first (see below)
│
├─ Is the render pass descriptor valid?
│   ├─ Check: view.currentRenderPassDescriptor != nil
│   ├─ Check: drawable = view.currentDrawable != nil
│   └─ FIX: Ensure MTKView.device is set, view is on screen
│
├─ Is the pipeline state created?
│   ├─ Check: makeRenderPipelineState doesn't throw
│   └─ FIX: Check shader function names match library
│
├─ Are draw calls being issued?
│   ├─ Add: encoder.label = "Main Pass" for frame capture
│   └─ DEBUG: GPU Frame Capture → verify draw calls appear
│
├─ Are resources bound?
│   ├─ Check: setVertexBuffer, setFragmentTexture called
│   └─ FIX: Metal requires explicit binding every frame
│
├─ Is the vertex data correct?
│   ├─ DEBUG: GPU Frame Capture → inspect vertex buffer
│   └─ FIX: Check buffer offsets, vertex count
│
├─ Are coordinates in Metal's range?
│   ├─ Metal NDC: X [-1,1], Y [-1,1], Z [0,1]
│   ├─ OpenGL NDC: X [-1,1], Y [-1,1], Z [-1,1]
│   └─ FIX: Adjust projection matrix or vertex shader
│
└─ Is clear color set?
    ├─ Default clear color is (0,0,0,0) — transparent black
    └─ FIX: Set view.clearColor or renderPassDescriptor.colorAttachments[0].clearColor

Common Fixes

Missing Drawable:

// BAD: Drawing before view is ready
override func viewDidLoad() {
    draw()  // metalView.currentDrawable is nil
}

// GOOD: Wait for delegate callback
func draw(in view: MTKView) {
    guard let drawable = view.currentDrawable else { return }
    // Safe to draw
}

Wrong Function Names:

// BAD: Function name doesn't match .metal file
descriptor.vertexFunction = library.makeFunction(name: "vertexMain")
// .metal file has: vertex VertexOut vertexShader(...)

// GOOD: Names must match exactly
descriptor.vertexFunction = library.makeFunction(name: "vertexShader")

Missing Resource Binding:

// BAD: Assumed state persists like OpenGL
encoder.setRenderPipelineState(pso)
encoder.drawPrimitives(...)  // No buffers bound!

// GOOD: Bind everything explicitly
encoder.setRenderPipelineState(pso)
encoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)
encoder.setVertexBytes(&uniforms, length: uniformsSize, index: 1)
encoder.setFragmentTexture(texture, index: 0)
encoder.drawPrimitives(...)

Time cost: GPU Frame Capture diagnosis: 5-10 min. Guessing without tools: 1-4 hours.

Symptom 2: Shader Compilation Errors

Decision Tree

Shader fails to compile
│
├─ "Use of undeclared identifier"
│   ├─ Check: #include <metal_stdlib>
│   ├─ Check: using namespace metal;
│   └─ FIX: Standard functions need metal_stdlib
│
├─ "No matching function for call to 'texture'"
│   └─ GLSL texture() → MSL tex.sample(sampler, uv)
│       FIX: Texture sampling is a method, needs sampler
│
├─ "Invalid type 'vec4'"
│   └─ GLSL vec4 → MSL float4
│       FIX: See type mapping table in metal-migration-ref
│
├─ "No matching constructor"
│   ├─ GLSL: vec4(vec3, float) works
│   ├─ MSL: float4(float3, float) works
│   └─ Check: Argument types match exactly
│
├─ "Attribute index out of range"
│   ├─ Check: [[attribute(N)]] matches vertex descriptor
│   └─ FIX: vertexDescriptor.attributes[N] must be configured
│
├─ "Buffer binding index out of range"
│   ├─ Check: [[buffer(N)]] where N < 31
│   └─ FIX: Metal has max 31 buffer bindings per stage
│
└─ "Cannot convert value of type"
    ├─ MSL is stricter than GLSL about implicit conversions
    └─ FIX: Add explicit casts: float(intValue), int(floatValue)

Common Conversions

// GLSL
vec4 color = texture(sampler2D, uv);

// MSL — texture and sampler are separate
float4 color = tex.sample(samp, uv);

// GLSL — mod() for floats
float x = mod(y, z);

// MSL — fmod() for floats
float x = fmod(y, z);

// GLSL — atan(y, x)
float angle = atan(y, x);

// MSL — atan2(y, x)
float angle = atan2(y, x);

// GLSL — inversesqrt
float invSqrt = inversesqrt(x);

// MSL — rsqrt
float invSqrt = rsqrt(x);

Time cost: With conversion table: 2-5 min per shader. Without: 15-30 min per shader.

Symptom 3: Wrong Colors or Coordinates

Decision Tree

Rendering looks wrong
│
├─ Image is upside down
│   ├─ Cause: Metal Y-axis is opposite OpenGL
│   ├─ FIX at ONE layer only — don't double-flip (two flips = no flip)
│   ├─ FIX (texture load): MTKTextureLoader .origin: .bottomLeft
│   ├─ FIX (UV): uv.y = 1.0 - uv.y in fragment shader
│   └─ FIX (vertex shader): pos.y = -pos.y — then RECONCILE winding (see below)
│
├─ Image is mirrored
│   ├─ Cause: Winding order or cull mode wrong
│   ├─ FIX: encoder.setFrontFacing(.counterClockwise)
│   └─ FIX: encoder.setCullMode(.back) or .none to test
│
├─ Colors are swapped (red/blue)
│   ├─ Cause: Pixel format mismatch
│   ├─ Check: .bgra8Unorm vs .rgba8Unorm
│   └─ FIX: Match texture pixel format to data format
│
├─ Colors are washed out / too bright
│   ├─ Cause: sRGB vs linear color space
│   ├─ Check: Using .bgra8Unorm_srgb for sRGB textures?
│   └─ FIX: Use _srgb format variants for gamma-correct rendering
│
├─ Depth fighting / z-fighting
│   ├─ Cause: NDC Z range difference
│   ├─ OpenGL: Z in [-1, 1]
│   ├─ Metal: Z in [0, 1]
│   └─ FIX: Adjust projection matrix for Metal's Z range
│
├─ Objects clipped incorrectly
│   ├─ Cause: Near/far plane or viewport
│   ├─ Check: Viewport size matches drawable size
│   └─ FIX: encoder.setViewport(MTLViewport(...))
│
└─ Transparency wrong
    ├─ Cause: Blend state not configured
    ├─ FIX: pipelineDescriptor.colorAttachments[0].isBlendingEnabled = true
    └─ FIX: Set sourceRGBBlendFactor, destinationRGBBlendFactor

Coordinate System Fix

A naive port renders upside-down and clipped because OpenGL's NDC Z is [-1, 1] while Metal's is [0, 1]. Reusing an OpenGL projection matrix clips the near half of your scene. Use a Metal-correct matrix:

// Metal-correct perspective: Z range [0, 1], +Z forward, column-major.
// simd_float4x4(columns:) — clipPos = matrix * float4(x, y, z, 1).
func metalPerspectiveProjection(fovY: Float, aspect: Float, near: Float, far: Float) -> simd_float4x4 {
    let yScale = 1.0 / tan(fovY * 0.5)
    let xScale = yScale / aspect
    let zRange = far - near

    return simd_float4x4(columns: (
        SIMD4<Float>(xScale, 0, 0, 0),
        SIMD4<Float>(0, yScale, 0, 0),
        SIMD4<Float>(0, 0, far / zRange, 1),       // Metal Z: [0, 1]
        SIMD4<Float>(0, 0, -near * far / zRange, 0)
    ))
}

Fix Y at ONE layer — don't double-flip

The image is upside-down because Metal's clip-space Y and texture origin both invert relative to OpenGL. Pick exactly ONE place to correct it:

• Texture sampling: MTKTextureLoader .origin: .bottomLeft, or uv.y = 1.0 - uv.y
• Geometry: negate Y in the projection (yScale → -yScale) or pos.y = -pos.y

Applying a flip in two layers cancels out (two flips = no flip) and you're back where you started, chasing a bug that isn't there. Flip once, in the layer you control most cleanly.

Reconcile winding after a geometry Y-flip

Flipping geometry Y reverses triangle winding, so back-face culling now culls the faces you want to keep — the model looks inside-out or vanishes. Reconcile the cull state with the flip:

// After flipping Y in clip space, the apparent winding reverses.
encoder.setFrontFacing(.counterClockwise)  // match your data's winding
encoder.setCullMode(.back)
// Debugging tip: set .cullMode(.none) first to confirm geometry is present,
// THEN restore culling and adjust frontFacing.

A texture/UV flip does NOT change winding — only a geometry flip does. This is why fixing the upside-down image at the texture layer avoids the winding problem entirely.

Time cost: With GPU Frame Capture texture inspection: 5-10 min. Double-flip / winding guessing: 1-2 hours.

Symptom 4: Performance Regression

Decision Tree

Performance worse than OpenGL
│
├─ Enabling validation?
│   └─ Validation adds ~30% overhead
│       FIX: Disable for release builds, keep for debug
│
├─ Creating resources every frame?
│   ├─ BAD: device.makeBuffer() in draw()
│   └─ FIX: Create buffers once, reuse with triple buffering
│
├─ Creating pipeline state every frame?
│   ├─ BAD: makeRenderPipelineState() in draw()
│   └─ FIX: Create PSO once at init, store as property
│
├─ Too many draw calls?
│   ├─ DEBUG: GPU Frame Capture → count draw calls
│   └─ FIX: Batch geometry, use instancing, indirect draws
│
├─ GPU-CPU sync stalls?
│   ├─ DEBUG: Metal System Trace → look for stalls
│   ├─ Cause: waitUntilCompleted() blocks CPU
│   └─ FIX: Triple buffering with semaphore
│
├─ Inefficient buffer updates?
│   ├─ BAD: Recreating buffer to update
│   └─ FIX: buffer.contents().copyMemory() for dynamic data
│
├─ Wrong storage mode?
│   ├─ .shared: Good for small dynamic data
│   ├─ .private: Good for static GPU-only data
│   └─ FIX: Use .private for geometry that doesn't change
│
└─ Missing Metal-specific optimizations?
    ├─ Argument buffers reduce binding overhead
    ├─ Indirect draws reduce CPU work
    └─ See WWDC sessions on Metal optimization

Triple Buffering Pattern

class TripleBufferedRenderer {
    static let maxInflightFrames = 3

    let inflightSemaphore = DispatchSemaphore(value: maxInflightFrames)
    var uniformBuffers: [MTLBuffer] = []
    var currentBufferIndex = 0

    init(device: MTLDevice) {
        for _ in 0..<Self.maxInflightFrames {
            let buffer = device.makeBuffer(length: uniformsSize, options: .storageModeShared)!
            uniformBuffers.append(buffer)
        }
    }

    func draw(in view: MTKView) {
        // Wait for a buffer to be available
        inflightSemaphore.wait()

        let buffer = uniformBuffers[currentBufferIndex]
        // Safe to write — GPU is done with this buffer
        memcpy(buffer.contents(), &uniforms, uniformsSize)

        let commandBuffer = commandQueue.makeCommandBuffer()!

        // Signal when GPU is done
        commandBuffer.addCompletedHandler { [weak self] _ in
            self?.inflightSemaphore.signal()
        }

        // ... encode and commit

        currentBufferIndex = (currentBufferIndex + 1) % Self.maxInflightFrames
    }
}

Time cost: Metal System Trace diagnosis: 15-30 min. Guessing: hours.

Symptom 5: Camera or Video Texture Interop

Porting camera/video drove your original GL-ES code if you used CVOpenGLESTextureCache to wrap CVPixelBuffer frames. Metal's equivalent is CVMetalTextureCache. This is the most common GL-ES → Metal migration driver and has no GLKView/MTKView shortcut — the cache APIs map one-to-one but the YUV and lifetime rules trip everyone.

API Mapping

Single-Plane BGRA

When the capture output is kCVPixelFormatType_32BGRA, you get one MTLTexture:

var cache: CVMetalTextureCache!
CVMetalTextureCacheCreate(nil, nil, device, nil, &cache)

func texture(from pixelBuffer: CVPixelBuffer) -> CVMetalTexture? {
    let width = CVPixelBufferGetWidth(pixelBuffer)
    let height = CVPixelBufferGetHeight(pixelBuffer)
    var cvTexture: CVMetalTexture?
    CVMetalTextureCacheCreateTextureFromImage(
        nil, cache, pixelBuffer, nil,
        .bgra8Unorm, width, height, 0, &cvTexture)
    return cvTexture  // MUST retain — see lifetime rule
}

let metalTexture = CVMetalTextureGetTexture(cvTexture!)
encoder.setFragmentTexture(metalTexture, index: 0)

YUV Bi-Planar (the common case)

AVCaptureVideoDataOutput defaults to kCVPixelFormatType_420YpCbCr8BiPlanarFullRange, which has two planes. Create one MTLTexture per plane and convert in the fragment shader — there is no single combined texture:

// Luma plane 0
CVMetalTextureCacheCreateTextureFromImage(
    nil, cache, pixelBuffer, nil, .r8Unorm,
    CVPixelBufferGetWidthOfPlane(pixelBuffer, 0),
    CVPixelBufferGetHeightOfPlane(pixelBuffer, 0), 0, &lumaCV)

// Chroma plane 1
CVMetalTextureCacheCreateTextureFromImage(
    nil, cache, pixelBuffer, nil, .rg8Unorm,
    CVPixelBufferGetWidthOfPlane(pixelBuffer, 1),
    CVPixelBufferGetHeightOfPlane(pixelBuffer, 1), 1, &chromaCV)

// Fragment shader — YCbCr (BT.601 full-range) to RGB
constant float4x4 ycbcrToRGB = float4x4(
    float4( 1.0000,  1.0000,  1.0000, 0.0),
    float4( 0.0000, -0.3441,  1.7720, 0.0),
    float4( 1.4020, -0.7141,  0.0000, 0.0),
    float4(-0.7010,  0.5291, -0.8860, 1.0));

fragment float4 cameraFrag(VertexOut in [[stage_in]],
                           texture2d<float> luma [[texture(0)]],
                           texture2d<float> chroma [[texture(1)]],
                           sampler s [[sampler(0)]]) {
    float y = luma.sample(s, in.uv).r;
    float2 cbcr = chroma.sample(s, in.uv).rg;
    return ycbcrToRGB * float4(y, cbcr, 1.0);
}

CVMetalTexture Lifetime Rule

Retain the `CVMetalTexture` (not just the `MTLTexture`) until the command buffer that uses it completes. The MTLTexture returned by CVMetalTextureGetTexture is a view into the cache-managed IOSurface; if the CVMetalTexture deallocates while the GPU is reading it, you get corrupt frames or EXC_BAD_ACCESS. This is the camera-interop equivalent of Symptom 6's resource-lifetime rule.

// BAD: CVMetalTexture released at end of scope — GPU may still read it
let mtl = CVMetalTextureGetTexture(makeTexture(from: pixelBuffer)!)
encoder.setFragmentTexture(mtl, index: 0)
commandBuffer.commit()

// GOOD: hold the CVMetalTexture(s) until completion
let lumaCV = makeLuma(from: pixelBuffer)
let chromaCV = makeChroma(from: pixelBuffer)
encoder.setFragmentTexture(CVMetalTextureGetTexture(lumaCV!), index: 0)
encoder.setFragmentTexture(CVMetalTextureGetTexture(chromaCV!), index: 1)
commandBuffer.addCompletedHandler { _ in
    _ = lumaCV   // retained until GPU done
    _ = chromaCV
}
commandBuffer.commit()

Call CVMetalTextureCacheFlush(cache, 0) once per frame after committing to release textures the cache no longer needs. Skipping it leaks IOSurface-backed memory until the cache is destroyed.

Time cost: Lifetime/format bugs here look like intermittent corruption — 1-3 hours of guessing vs minutes once you know the two-plane + retain rules.

Symptom 6: Crashes During GPU Work

Decision Tree

App crashes during rendering
│
├─ EXC_BAD_ACCESS in Metal framework
│   ├─ Cause: Accessing released resource
│   ├─ Check: Buffer/texture retained during GPU use
│   └─ FIX: Keep strong references until command buffer completes
│
├─ "Execution of the command buffer was aborted"
│   ├─ Cause: GPU timeout (>10 sec on iOS)
│   ├─ Check: Infinite loop in shader?
│   └─ FIX: Add early exit conditions, reduce work
│
├─ "-[MTLDebugRenderCommandEncoder validateDrawCallWithArray:...]"
│   ├─ Cause: Validation caught misuse
│   └─ FIX: Read the validation message — it tells you exactly what's wrong
│
├─ "Fragment shader writes to non-existent render target"
│   ├─ Cause: Shader returns color but no color attachment
│   └─ FIX: Configure colorAttachments[0].pixelFormat
│
├─ Crash in shader (SIGABRT)
│   ├─ Cause: Out-of-bounds buffer access
│   ├─ DEBUG: Enable shader validation
│   └─ FIX: Check array bounds, buffer sizes
│
└─ Device disconnected / GPU restart
    ├─ Cause: Severe GPU hang
    ├─ Check: Infinite loop or massive overdraw
    └─ FIX: Simplify shader, reduce draw complexity

Resource Lifetime Fix

// BAD: Buffer released before GPU finishes
func draw(in view: MTKView) {
    let buffer = device.makeBuffer(...)  // Created here
    encoder.setVertexBuffer(buffer, ...)
    commandBuffer.commit()
    // buffer released at end of scope — GPU still using it!
}

// GOOD: Keep reference until completion
class Renderer {
    var currentBuffer: MTLBuffer?  // Strong reference

    func draw(in view: MTKView) {
        currentBuffer = device.makeBuffer(...)
        encoder.setVertexBuffer(currentBuffer!, ...)
        commandBuffer.addCompletedHandler { [weak self] _ in
            // Safe to release now
            self?.currentBuffer = nil
        }
        commandBuffer.commit()
    }
}

Debugging Tools Quick Reference

GPU Frame Capture

Xcode → Debug → Capture GPU Frame (Cmd+Opt+Shift+G)

Use for:

• Inspecting buffer contents
• Viewing intermediate textures
• Checking draw call sequence
• Debugging shader variable values
• Understanding why something isn't rendering

Metal System Trace (Instruments)

Instruments → Metal System Trace template

Use for:

• GPU/CPU timeline analysis
• Finding synchronization stalls
• Measuring encoder/buffer overhead
• Identifying bottlenecks

Shader Debugger

GPU Frame Capture → Select draw call → Debug button

Use for:

• Step through shader execution
• Inspect variable values per pixel/vertex
• Find logic errors in shaders

Validation Messages

Most validation messages include:

• What went wrong
• Which resource/state
• What the expected value was

Always read the full message — it usually tells you exactly how to fix the problem.

Diagnostic Checklist

When something doesn't work:

• [ ] Metal validation enabled? (Most bugs produce validation errors)
• [ ] GPU Frame Capture available? (Visual debugging is fastest)
• [ ] Console error messages? (Read them fully)
• [ ] Resources bound? (Metal requires explicit binding)
• [ ] Coordinates correct? (Y-flip ONCE, NDC Z [0,1], winding after flip)
• [ ] Pipeline state created successfully? (Check for throw)
• [ ] Drawable available? (View must be on screen)
• [ ] Camera frames: CVMetalTexture retained? (Until command buffer completes; flush cache per frame)

Resources

WWDC: 2019-00611, 2020-10602, 2020-10603

Docs: /metal/debugging-metal-applications, /metal/gpu-capture

Skills: axiom-graphics (skills/metal-migration.md), axiom-graphics (skills/metal-migration-ref.md)

---

Last Updated: 2025-12-29 Platforms: iOS 12+, macOS 10.14+, tvOS 12+ Status: Comprehensive Metal porting diagnostics

Metal Migration Reference

Complete reference for converting OpenGL/DirectX code to Metal.

When to Use This Reference

Use this reference when:

• Converting GLSL shaders to Metal Shading Language (MSL)
• Converting HLSL shaders to MSL
• Looking up GL/D3D API equivalents in Metal
• Setting up MTKView or CAMetalLayer
• Building render pipelines
• Using Metal Shader Converter for DirectX

Part 1: GLSL to MSL Conversion

Type Mappings

Built-in Variable Mappings

Function Mappings

Shader Structure Conversion

GLSL Vertex Shader:

#version 300 es
precision highp float;

layout(location = 0) in vec3 aPosition;
layout(location = 1) in vec2 aTexCoord;

uniform mat4 uModelViewProjection;

out vec2 vTexCoord;

void main() {
    gl_Position = uModelViewProjection * vec4(aPosition, 1.0);
    vTexCoord = aTexCoord;
}

MSL Vertex Shader:

#include <metal_stdlib>
using namespace metal;

struct VertexIn {
    float3 position [[attribute(0)]];
    float2 texCoord [[attribute(1)]];
};

struct VertexOut {
    float4 position [[position]];
    float2 texCoord;
};

struct Uniforms {
    float4x4 modelViewProjection;
};

vertex VertexOut vertexShader(
    VertexIn in [[stage_in]],
    constant Uniforms& uniforms [[buffer(1)]]
) {
    VertexOut out;
    out.position = uniforms.modelViewProjection * float4(in.position, 1.0);
    out.texCoord = in.texCoord;
    return out;
}

GLSL Fragment Shader:

#version 300 es
precision highp float;

in vec2 vTexCoord;
uniform sampler2D uTexture;

out vec4 fragColor;

void main() {
    fragColor = texture(uTexture, vTexCoord);
}

MSL Fragment Shader:

fragment float4 fragmentShader(
    VertexOut in [[stage_in]],
    texture2d<float> tex [[texture(0)]],
    sampler samp [[sampler(0)]]
) {
    return tex.sample(samp, in.texCoord);
}

Precision Qualifiers

GLSL precision qualifiers have no direct MSL equivalent — MSL uses explicit types:

Buffer Alignment (Critical)

GLSL/C assumes:

• vec3: 12 bytes, any alignment
• vec4: 16 bytes

MSL requires:

• float3: 12 bytes storage, 16-byte aligned
• float4: 16 bytes storage, 16-byte aligned

Solution: Use simd types in Swift for CPU-GPU shared structs:

import simd

struct Uniforms {
    var modelViewProjection: simd_float4x4  // Correct alignment
    var cameraPosition: simd_float3         // 16-byte aligned
    var padding: Float = 0                   // Explicit padding if needed
}

Or use packed types in MSL (slower):

struct VertexPacked {
    packed_float3 position;  // 12 bytes, no padding
    packed_float2 texCoord;  // 8 bytes
};

Part 2: HLSL to MSL Conversion

Type Mappings

Semantic Mappings

Function Mappings

Metal Shader Converter (DirectX → Metal)

Apple's official tool for converting DXIL (compiled HLSL) to Metal libraries.

Requirements:

• macOS 13+ with Xcode 15+
• OR Windows 10+ with VS 2019+
• Target devices: Argument Buffers Tier 2 (macOS 14+, iOS 17+)

Workflow:

# Step 1: Compile HLSL to DXIL using DXC
dxc -T vs_6_0 -E MainVS -Fo vertex.dxil shader.hlsl
dxc -T ps_6_0 -E MainPS -Fo fragment.dxil shader.hlsl

# Step 2: Convert DXIL to Metal library
metal-shaderconverter vertex.dxil -o vertex.metallib
metal-shaderconverter fragment.dxil -o fragment.metallib

# Step 3: Load in Swift
let vertexLib = try device.makeLibrary(URL: vertexURL)
let fragmentLib = try device.makeLibrary(URL: fragmentURL)

Key Options:

Supported Shader Models: SM 6.0 - 6.6 (with limitations on 6.6 features)

Part 3: OpenGL API to Metal API

View/Context Setup

Resource Creation

State Management

Draw Commands

Primitive Types

Part 4: Complete Setup Examples

MTKView Setup (Recommended)

import MetalKit

class GameViewController: UIViewController {
    var metalView: MTKView!
    var renderer: Renderer!

    override func viewDidLoad() {
        super.viewDidLoad()

        // Create Metal view
        guard let device = MTLCreateSystemDefaultDevice() else {
            fatalError("Metal not supported")
        }

        metalView = MTKView(frame: view.bounds, device: device)
        metalView.autoresizingMask = [.flexibleWidth, .flexibleHeight]
        metalView.colorPixelFormat = .bgra8Unorm
        metalView.depthStencilPixelFormat = .depth32Float
        metalView.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
        metalView.preferredFramesPerSecond = 60
        view.addSubview(metalView)

        // Create renderer
        renderer = Renderer(metalView: metalView)
        metalView.delegate = renderer
    }
}

class Renderer: NSObject, MTKViewDelegate {
    let device: MTLDevice
    let commandQueue: MTLCommandQueue
    var pipelineState: MTLRenderPipelineState!
    var depthState: MTLDepthStencilState!
    var vertexBuffer: MTLBuffer!

    init(metalView: MTKView) {
        device = metalView.device!
        commandQueue = device.makeCommandQueue()!
        super.init()

        buildPipeline(metalView: metalView)
        buildDepthStencil()
        buildBuffers()
    }

    private func buildPipeline(metalView: MTKView) {
        let library = device.makeDefaultLibrary()!

        let descriptor = MTLRenderPipelineDescriptor()
        descriptor.vertexFunction = library.makeFunction(name: "vertexShader")
        descriptor.fragmentFunction = library.makeFunction(name: "fragmentShader")
        descriptor.colorAttachments[0].pixelFormat = metalView.colorPixelFormat
        descriptor.depthAttachmentPixelFormat = metalView.depthStencilPixelFormat

        // Vertex descriptor (matches shader's VertexIn struct)
        let vertexDescriptor = MTLVertexDescriptor()
        vertexDescriptor.attributes[0].format = .float3
        vertexDescriptor.attributes[0].offset = 0
        vertexDescriptor.attributes[0].bufferIndex = 0
        vertexDescriptor.attributes[1].format = .float2
        vertexDescriptor.attributes[1].offset = MemoryLayout<SIMD3<Float>>.stride
        vertexDescriptor.attributes[1].bufferIndex = 0
        vertexDescriptor.layouts[0].stride = MemoryLayout<Vertex>.stride
        descriptor.vertexDescriptor = vertexDescriptor

        pipelineState = try! device.makeRenderPipelineState(descriptor: descriptor)
    }

    private func buildDepthStencil() {
        let descriptor = MTLDepthStencilDescriptor()
        descriptor.depthCompareFunction = .less
        descriptor.isDepthWriteEnabled = true
        depthState = device.makeDepthStencilState(descriptor: descriptor)
    }

    func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
        // Handle resize
    }

    func draw(in view: MTKView) {
        guard let drawable = view.currentDrawable,
              let descriptor = view.currentRenderPassDescriptor,
              let commandBuffer = commandQueue.makeCommandBuffer(),
              let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: descriptor) else {
            return
        }

        encoder.setRenderPipelineState(pipelineState)
        encoder.setDepthStencilState(depthState)
        encoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)
        encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: vertexCount)
        encoder.endEncoding()

        commandBuffer.present(drawable)
        commandBuffer.commit()
    }
}

CAMetalLayer Setup (Custom Control)

import Metal
import QuartzCore

class MetalLayerView: UIView {
    var metalLayer: CAMetalLayer!
    var device: MTLDevice!
    var commandQueue: MTLCommandQueue!
    var displayLink: CADisplayLink?

    override class var layerClass: AnyClass { CAMetalLayer.self }

    override init(frame: CGRect) {
        super.init(frame: frame)
        setup()
    }

    private func setup() {
        device = MTLCreateSystemDefaultDevice()!
        commandQueue = device.makeCommandQueue()!

        metalLayer = layer as? CAMetalLayer
        metalLayer.device = device
        metalLayer.pixelFormat = .bgra8Unorm
        metalLayer.framebufferOnly = true

        displayLink = CADisplayLink(target: self, selector: #selector(render))
        displayLink?.add(to: .main, forMode: .common)
    }

    override func layoutSubviews() {
        super.layoutSubviews()
        metalLayer.drawableSize = CGSize(
            width: bounds.width * contentScaleFactor,
            height: bounds.height * contentScaleFactor
        )
    }

    @objc func render() {
        guard let drawable = metalLayer.nextDrawable(),
              let commandBuffer = commandQueue.makeCommandBuffer() else {
            return
        }

        let descriptor = MTLRenderPassDescriptor()
        descriptor.colorAttachments[0].texture = drawable.texture
        descriptor.colorAttachments[0].loadAction = .clear
        descriptor.colorAttachments[0].storeAction = .store
        descriptor.colorAttachments[0].clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)

        guard let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: descriptor) else {
            return
        }

        // Draw commands here
        encoder.endEncoding()

        commandBuffer.present(drawable)
        commandBuffer.commit()
    }
}

Compute Shader Setup

class ComputeProcessor {
    let device: MTLDevice
    let commandQueue: MTLCommandQueue
    var computePipeline: MTLComputePipelineState!

    init() {
        device = MTLCreateSystemDefaultDevice()!
        commandQueue = device.makeCommandQueue()!

        let library = device.makeDefaultLibrary()!
        let function = library.makeFunction(name: "computeKernel")!
        computePipeline = try! device.makeComputePipelineState(function: function)
    }

    func process(input: MTLBuffer, output: MTLBuffer, count: Int) {
        let commandBuffer = commandQueue.makeCommandBuffer()!
        let encoder = commandBuffer.makeComputeCommandEncoder()!

        encoder.setComputePipelineState(computePipeline)
        encoder.setBuffer(input, offset: 0, index: 0)
        encoder.setBuffer(output, offset: 0, index: 1)

        let threadGroupSize = MTLSize(width: 256, height: 1, depth: 1)
        let threadGroups = MTLSize(
            width: (count + 255) / 256,
            height: 1,
            depth: 1
        )

        encoder.dispatchThreadgroups(threadGroups, threadsPerThreadgroup: threadGroupSize)
        encoder.endEncoding()

        commandBuffer.commit()
        commandBuffer.waitUntilCompleted()
    }
}

// Compute shader
kernel void computeKernel(
    device float* input [[buffer(0)]],
    device float* output [[buffer(1)]],
    uint id [[thread_position_in_grid]]
) {
    output[id] = input[id] * 2.0;
}

Part 5: Storage Modes & Synchronization

Buffer Storage Modes

// Shared: CPU and GPU both access (iOS typical)
let uniformBuffer = device.makeBuffer(length: size, options: .storageModeShared)

// Private: GPU only (best for static geometry)
let vertexBuffer = device.makeBuffer(bytes: vertices, length: size, options: .storageModePrivate)

// Managed: Explicit sync (macOS)
#if os(macOS)
let buffer = device.makeBuffer(length: size, options: .storageModeManaged)
// After CPU write:
buffer.didModifyRange(0..<size)
#endif

Texture Storage Modes

let descriptor = MTLTextureDescriptor.texture2DDescriptor(
    pixelFormat: .rgba8Unorm,
    width: 1024,
    height: 1024,
    mipmapped: true
)

// For static textures (loaded once)
descriptor.storageMode = .private
descriptor.usage = [.shaderRead]

// For render targets
descriptor.storageMode = .private
descriptor.usage = [.renderTarget, .shaderRead]

// For CPU-readable (screenshots, readback)
descriptor.storageMode = .shared  // iOS
descriptor.storageMode = .managed  // macOS
descriptor.usage = [.shaderRead, .shaderWrite]

Part 6: Metal 4 ML and Neural Rendering

Metal 4 (26 cycle) runs machine learning inside the render pipeline at three levels of control:

MetalFX Denoising Best Practices

From Maxon's Redshift Live adoption (WWDC 2026-359):

1. Keep auxiliary inputs noise-free. Diffuse albedo is the strongest denoising signal — make it as close as possible to a clean version of the final image. Build per-input debug views; validate with GPU capture. 2. Store what the viewer sees. For mirrors and glass, write the reflected/refracted surface properties (albedo, normal, roughness) into the G-buffer — primary surface replacement, blended by the Fresnel term for transmission. 3. Get motion vectors right. MetalFX expects dejittered motion vectors — subtract the jitter deltas of both frames or edges shimmer:

// Camera-only motion vectors with jitter compensation (MSL)
float4 clipCurrent = viewProjCurrent * float4(worldPos, 1.0);
float2 ndcCurrent = clipCurrent.xy / clipCurrent.w;
float4 clipPrevious = viewProjPrevious * float4(worldPos, 1.0);
float2 ndcPrevious = clipPrevious.xy / clipPrevious.w;
float2 motion = ndcPrevious - ndcCurrent;
motion -= jitterPrevious - jitterCurrent;

For moving/deforming geometry, store previous-frame world positions (or skin twice). For genuinely unreliable motion (alpha-blended particles), use the reactive mask. Noise-free layers (sky, fog, volumetrics) can skip denoising via the transparency overlay or per-pixel denoiser strength mask (0 = none, 1 = full).

Quantized Tensors OS27

MTLTensor gained int4/int8 quantized data types in a 26-cycle update; the 27 SDKs (iOS27/macOS27) add floating-point and 2-bit formats plus block-wise scale factors (MX formats):

Scales attach as an auxiliary plane on the same tensor — MTLTensorAuxiliaryPlaneDescriptor (with blockFactors, e.g. one scale per 32×1 block) registered in an MTLTensorAuxiliaryPlaneDescriptorMap for MTLTensorPlaneTypeScales, assigned to MTLTensorDescriptor.auxiliaryPlanes. Note: Apple's session slide uses MTLTensorDataTypeMetalFloat8E4M3 — the shipping header has no Metal infix (MTLTensorDataTypeFloat8E4M3). The new types carry extra alignment requirements; check the Metal docs.

MSL side, declare the plane and tensor types, then TensorOps dequantizes automatically:

#include <metal_tensor>
using namespace metal;

using scales_plane = tensor_blockwise<tensor_plane_scales,
                                      device metal_fp8_ue8m0_format, 32, 1>;
using mxfp8_tensor = tensor<device metal_fp8_e4m3_format, dextents<int, 2>,
                            tensor_handle, scales_plane>;
// tensor_inline instead of tensor_handle constructs the tensor on the
// shader stack from raw buffer pointers (no host-side MTLTensor needed)

matmul2d accepts quantized tensors directly; to dequantize a custom format yourself, prefer cooperative tensors (register-distributed) over a threadgroup-memory round trip. In the 27 cycle a cooperative tensor can also feed a subsequent matmul as input (is_compatible_as_left_input / get_left_input_cooperative_tensor) — the FlashAttention pattern; on macOS 26 it had to be staged through threadgroup memory. Row reductions (reduce_rows) and map_iterator support fused SoftMax. Full reference: the Metal Performance Primitives programming guide.

Other 27 Metal Additions OS27

For CoreML-level model conversion, quantization, and deployment (including the 27-cycle Core AI tooling for PyTorch models with custom Metal kernels — TorchMetalKernel, coreai-torch, the .aimodel runtime), see axiom-ai (skills/core-ai.md) — this part covers only the Metal-side surface.

Resources

WWDC: 2016-00602, 2018-00604, 2019-00611, 2026-359, 2026-330

Docs: /metal/migrating-opengl-code-to-metal, /metal/shader-converter, /metalkit/mtkview, /metalfx, /metalperformanceprimitives

Skills: axiom-graphics (skills/metal-migration.md), axiom-graphics (skills/metal-migration-diag.md), axiom-graphics (skills/display-performance.md)

---

Last Updated: 2026-06-10 Platforms: iOS 12+, macOS 10.14+, tvOS 12+ Status: Complete shader conversion and API mapping reference

Metal Migration

Porting OpenGL/OpenGL ES or DirectX code to Metal on Apple platforms.

When to Use This Skill

Use this skill when:

• Porting an OpenGL/OpenGL ES codebase to iOS/macOS
• Porting a DirectX codebase to Apple platforms
• Deciding between translation layer (MetalANGLE) vs native rewrite
• Planning a phased migration strategy
• Evaluating effort vs performance tradeoffs

Red Flags

❌ "Just use MetalANGLE and ship" — Translation layers add 10-30% overhead; fine for demos, not production

❌ "Convert shaders one-by-one without planning" — State management differs fundamentally; you'll rewrite twice

❌ "Keep the GL state machine mental model" — Metal is explicit; thinking GL causes subtle bugs

❌ "Port everything at once" — Phased migration catches issues early; big-bang migrations hide compounding bugs

❌ "Skip validation layer during development" — Metal validation catches 80% of porting bugs with clear messages

❌ "Worry about coordinate systems later" — Y-flip and NDC differences cause the most debugging time

❌ "Performance will be the same or better automatically" — Metal requires explicit optimization; naive ports can be slower

Migration Strategy Decision Tree

Starting a port to Metal?
│
├─ Need working demo in <1 week?
│   ├─ OpenGL ES source? → MetalANGLE (translation layer)
│   │   └─ Caveats: 10-30% overhead, ES 2/3 only, no compute
│   │
│   └─ Vulkan available? → MoltenVK
│       └─ Caveats: Vulkan complexity, indirect translation
│
├─ Production app with performance requirements?
│   └─ Native Metal rewrite (recommended)
│       ├─ Phased: Keep GL for reference, port module-by-module
│       └─ Full: Clean rewrite using Metal idioms from start
│
├─ DirectX/HLSL source?
│   └─ Metal Shader Converter (Apple tool)
│       └─ Converts DXIL bytecode → Metal library
│       └─ See metal-migration-ref for usage
│
└─ Hybrid approach?
    └─ MetalANGLE for demo → Native Metal incrementally
        └─ Best of both: fast validation, optimal end state

Pattern 1: Translation Layer (Quick Demo Path)

When to use: Validate feasibility, get stakeholder buy-in, prototype

MetalANGLE Setup (OpenGL ES → Metal)

// 1. Add MetalANGLE via SPM or CocoaPods
// GitHub: nicklockwood/MetalANGLE

// 2. Replace EAGLContext with MGLContext
import MetalANGLE

let context = MGLContext(api: kMGLRenderingAPIOpenGLES3)
MGLContext.setCurrent(context)

// 3. Replace GLKView with MGLKView
let glView = MGLKView(frame: bounds, context: context)
glView.delegate = self
glView.drawableDepthFormat = .format24

// 4. Existing GL code works unchanged
glClearColor(0, 0, 0, 1)
glClear(GL_COLOR_BUFFER_BIT)
// ... your existing GL rendering code

Tradeoffs Table

When MetalANGLE Fails

MetalANGLE will NOT work if your code:

• Uses OpenGL ES extensions not in core ES 2/3
• Relies on compute shaders (GL_COMPUTE_SHADER)
• Requires precise GL state machine semantics
• Needs performance within 10% of native
• Targets visionOS (no translation layer support)

Pattern 2: Native Metal Rewrite (Production Path)

When to use: Production apps, performance-critical rendering, long-term maintenance

Phased Migration Strategy

Phase 1: Abstraction Layer (1-2 weeks)
├─ Create renderer interface hiding GL/Metal specifics
├─ Keep GL implementation as reference
├─ Define clear boundaries: setup, resources, draw, present
└─ Validate abstraction with existing tests

Phase 2: Metal Backend (2-4 weeks)
├─ Implement Metal renderer behind same interface
├─ Convert shaders GLSL → MSL (use metal-migration-ref)
├─ Run GL and Metal side-by-side for visual diff
├─ GPU Frame Capture for debugging
└─ Milestone: Feature parity, visual match

Phase 3: Optimization (1-2 weeks)
├─ Remove abstraction overhead where it hurts
├─ Use Metal-specific features (argument buffers, indirect)
├─ Profile with Metal System Trace
├─ Tune for thermal envelope and battery
└─ Remove GL backend entirely

GLSL to Metal Shading Language (MSL) Conversion

Example conversion:

// GLSL vertex shader
#version 300 es
uniform mat4 u_mvp;
in vec3 a_position;
in vec2 a_texCoord;
out vec2 v_texCoord;

void main() {
    v_texCoord = a_texCoord;
    gl_Position = u_mvp * vec4(a_position, 1.0);
}

// Equivalent MSL vertex shader
#include <metal_stdlib>
using namespace metal;

struct VertexIn {
    float3 position [[attribute(0)]];
    float2 texCoord [[attribute(1)]];
};

struct VertexOut {
    float4 position [[position]];
    float2 texCoord;
};

struct Uniforms {
    float4x4 mvp;
};

vertex VertexOut vertexShader(VertexIn in [[stage_in]],
                              constant Uniforms &uniforms [[buffer(1)]]) {
    VertexOut out;
    out.texCoord = in.texCoord;
    out.position = uniforms.mvp * float4(in.position, 1.0);
    return out;
}

Key differences to watch:

• GLSL globals → MSL function parameters with [[attribute]] qualifiers
• Implicit uniform binding → explicit [[buffer(N)]] indices
• sampler2D combines texture+sampler → Metal separates texture2d and sampler
• GLSL preprocessor → Metal uses C++ #include and using namespace metal

Core Architecture Differences

MTKView Setup (Native Metal)

import MetalKit

class MetalRenderer: NSObject, MTKViewDelegate {
    let device: MTLDevice
    let commandQueue: MTLCommandQueue
    var pipelineState: MTLRenderPipelineState!

    init?(metalView: MTKView) {
        guard let device = MTLCreateSystemDefaultDevice(),
              let queue = device.makeCommandQueue() else {
            return nil
        }
        self.device = device
        self.commandQueue = queue

        metalView.device = device
        metalView.clearColor = MTLClearColor(red: 0, green: 0, blue: 0, alpha: 1)
        metalView.depthStencilPixelFormat = .depth32Float

        super.init()
        metalView.delegate = self

        buildPipeline(metalView: metalView)
    }

    private func buildPipeline(metalView: MTKView) {
        let library = device.makeDefaultLibrary()!
        let vertexFunction = library.makeFunction(name: "vertexShader")
        let fragmentFunction = library.makeFunction(name: "fragmentShader")

        let descriptor = MTLRenderPipelineDescriptor()
        descriptor.vertexFunction = vertexFunction
        descriptor.fragmentFunction = fragmentFunction
        descriptor.colorAttachments[0].pixelFormat = metalView.colorPixelFormat
        descriptor.depthAttachmentPixelFormat = metalView.depthStencilPixelFormat

        // Pre-validated at creation, not at draw time
        pipelineState = try! device.makeRenderPipelineState(descriptor: descriptor)
    }

    func draw(in view: MTKView) {
        guard let drawable = view.currentDrawable,
              let descriptor = view.currentRenderPassDescriptor,
              let commandBuffer = commandQueue.makeCommandBuffer(),
              let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: descriptor) else {
            return
        }

        encoder.setRenderPipelineState(pipelineState)
        // Bind resources explicitly - nothing persists between draws
        encoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)
        encoder.setFragmentTexture(texture, index: 0)
        encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: vertexCount)
        encoder.endEncoding()

        commandBuffer.present(drawable)
        commandBuffer.commit()
    }
}

Common Migration Anti-Patterns

Anti-Pattern 1: Keeping GL State Machine Mentality

❌ BAD — Thinking in GL's implicit state:

// GL mental model: "set state, then draw"
glBindTexture(GL_TEXTURE_2D, texture)
glBindBuffer(GL_ARRAY_BUFFER, vbo)
glUseProgram(program)
glDrawArrays(GL_TRIANGLES, 0, vertexCount)
// State persists until changed — can draw again without rebinding

✅ GOOD — Metal's explicit model:

// Metal: encode everything explicitly per draw
let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: rpd)!
encoder.setRenderPipelineState(pipelineState)    // Always set
encoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)  // Always bind
encoder.setFragmentTexture(texture, index: 0)    // Always bind
encoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: count)
encoder.endEncoding()
// Nothing persists — next encoder starts fresh

Time cost: 30-60 min debugging "why did my texture disappear" vs 2 min understanding the model upfront.

Anti-Pattern 2: Ignoring Coordinate System Differences

❌ BAD — Assuming GL coordinates work in Metal:

OpenGL:
- Origin: bottom-left
- Y-axis: up
- NDC Z range: [-1, 1]
- Texture origin: bottom-left

Metal:
- Origin: top-left
- Y-axis: down
- NDC Z range: [0, 1]
- Texture origin: top-left

✅ GOOD — Explicit coordinate handling:

// Option 1: Flip Y in vertex shader
vertex float4 vertexShader(VertexIn in [[stage_in]]) {
    float4 pos = uniforms.mvp * float4(in.position, 1.0);
    pos.y = -pos.y;  // Flip Y for Metal's coordinate system
    return pos;
}

// Option 2: Flip texture coordinates in fragment shader
fragment float4 fragmentShader(VertexOut in [[stage_in]],
                                texture2d<float> tex [[texture(0)]],
                                sampler samp [[sampler(0)]]) {
    float2 uv = in.texCoord;
    uv.y = 1.0 - uv.y;  // Flip V for Metal's texture origin
    return tex.sample(samp, uv);
}

// Option 3: Use MTKTextureLoader with origin option
let options: [MTKTextureLoader.Option: Any] = [
    .origin: MTKTextureLoader.Origin.bottomLeft  // Match GL convention
]
let texture = try textureLoader.newTexture(URL: url, options: options)

Time cost: 2-4 hours debugging "upside down" or "mirrored" rendering vs 5 min reading this pattern.

Anti-Pattern 3: No Validation Layer During Development

❌ BAD — Disabling validation for "performance":

// No validation — API misuse silently corrupts or crashes later

✅ GOOD — Always enable during development:

In Xcode: Edit Scheme → Run → Diagnostics
✓ Metal API Validation
✓ Metal Shader Validation
✓ GPU Frame Capture (Metal)

Time cost: Hours debugging silent corruption vs immediate error messages with call stacks.

Anti-Pattern 4: Single Buffer Without Synchronization

❌ BAD — CPU and GPU fight over same buffer:

// Frame N: CPU writes to buffer
// Frame N: GPU reads from buffer
// Frame N+1: CPU writes again — RACE CONDITION
buffer.contents().copyMemory(from: data, byteCount: size)

✅ GOOD — Triple buffering with semaphore:

class TripleBufferedRenderer {
    let inflightSemaphore = DispatchSemaphore(value: 3)
    var buffers: [MTLBuffer] = []
    var bufferIndex = 0

    func draw(in view: MTKView) {
        // Wait for a buffer to become available
        inflightSemaphore.wait()

        let buffer = buffers[bufferIndex]
        // Safe to write — GPU finished with this buffer
        buffer.contents().copyMemory(from: data, byteCount: size)

        let commandBuffer = commandQueue.makeCommandBuffer()!
        commandBuffer.addCompletedHandler { [weak self] _ in
            self?.inflightSemaphore.signal()  // Release buffer
        }

        // ... encode and commit

        bufferIndex = (bufferIndex + 1) % 3
    }
}

Time cost: Hours debugging intermittent visual glitches vs 15 min implementing triple buffering.

Pressure Scenarios

Scenario 1: "Just Ship with MetalANGLE"

Situation: Deadline in 2 weeks. MetalANGLE demo works. PM says ship it.

Pressure: "We can optimize later. Users won't notice 20% overhead."

Why this fails:

• Translation overhead compounds with complex scenes (visualizers, games)
• No compute shader support limits future features
• Technical debt grows — team learns MetalANGLE quirks, not Metal
• Apple deprecation risk (OpenGL ES deprecated since iOS 12)
• Battery/thermal complaints from users

Response template:

"MetalANGLE is viable for the demo milestone. For production, I recommend a 3-week buffer to implement native Metal for the render loop. This recovers the 20-30% overhead and eliminates deprecation risk. Can we scope the MVP to fewer visual effects to hit the deadline with native Metal?"

Scenario 2: "Port All Shaders This Sprint"

Situation: 50 GLSL shaders. Sprint is 2 weeks. Manager wants all converted.

Pressure: "They're just text files. How hard can shader conversion be?"

Why this fails:

• GLSL → MSL isn't 1:1 (precision qualifiers, built-ins, sampling)
• Each shader needs visual validation, not just compilation
• Complex shaders need performance profiling
• Bugs compound — broken shader A masks broken shader B

Response template:

"Shader conversion requires visual validation, not just compilation. I can convert 10-15 shaders/week with confidence. For 50 shaders: (1) Prioritize by usage — convert the 10 most-used first, (2) Automate mappings — type conversions, boilerplate, (3) Parallel validation — run GL and Metal side-by-side. Realistic timeline: 4-5 weeks for full conversion with quality."

Scenario 3: "We Don't Need GPU Frame Capture"

Situation: Developer says "I'll just use print statements to debug shaders."

Pressure: "GPU tools are overkill. I know what I'm doing."

Why this fails:

• Print statements don't work in shaders
• Visual bugs require seeing intermediate render targets
• Performance issues require GPU timeline analysis
• Metal validation errors need call stack context

Response template:

"GPU Frame Capture is the only way to inspect shader variables, see intermediate textures, and understand GPU timing. It takes 30 seconds to capture a frame. Without it, shader debugging is 10x slower — you're guessing instead of observing."

Pre-Migration Checklist

Before starting any port:

• [ ] Inventory shaders: Count GLSL/HLSL files, complexity (LOC, features used)
• [ ] Identify extensions: Which GL extensions does the code use? Metal equivalents?
• [ ] Audit state management: How stateful is the renderer? Global state count?
• [ ] Check compute usage: Any GL compute shaders? GPGPU? (MetalANGLE won't help)
• [ ] Profile baseline: FPS, frame time, memory, thermal on reference platform
• [ ] Define success criteria: Target FPS, memory budget, thermal envelope
• [ ] Set up A/B testing: Can you run GL and Metal side-by-side for validation?
• [ ] Enable validation: Metal API Validation, Shader Validation, Frame Capture

Post-Migration Checklist

After completing the port:

• [ ] Visual parity: Side-by-side screenshots match reference
• [ ] Performance parity or better: Frame time ≤ GL baseline
• [ ] No validation errors: Clean run with Metal validation enabled
• [ ] Thermal acceptable: Device doesn't throttle during normal use
• [ ] Memory stable: No leaks over extended use
• [ ] All code paths tested: Edge cases, error states, resize/rotate

Resources

WWDC: 2016-00602, 2018-00604, 2019-00611

Docs: /metal/migrating-opengl-code-to-metal, /metal/shader-converter

Tools: MetalANGLE, MoltenVK

Skills: axiom-graphics (skills/metal-migration-ref.md), axiom-graphics (skills/metal-migration-diag.md)

---

Last Updated: 2025-12-29 Platforms: iOS 12+, macOS 10.14+, tvOS 12+ Status: Production-ready Metal migration patterns

RealityKit Diagnostics

Systematic diagnosis for common RealityKit issues with time-cost annotations.

When to Use This Diagnostic Skill

Use this skill when:

• Entity added but not visible in the scene
• AR anchor not tracking or content floating
• Tap/drag gestures not responding on 3D entities
• Frame rate dropping or stuttering
• Material looks wrong (too dark, too bright, incorrect colors)
• Multiplayer entities not syncing across devices
• Physics bodies not colliding or passing through each other

For RealityKit architecture patterns and best practices, see axiom-graphics (skills/realitykit.md). For API reference, see axiom-graphics (skills/realitykit-ref.md).

---

Red Flags

Stop and check these before reaching for any other fix. Each is the root cause of a whole class of "RealityKit doesn't work" reports.

---

Mandatory First Step: Enable Debug Visualization

Time cost: 10 seconds vs hours of blind debugging

// In your RealityView or ARView setup
#if DEBUG
// Xcode: Debug → Attach to Process → Show RealityKit Statistics
// Or enable in code:
arView.debugOptions = [
    .showStatistics,       // Entity count, draw calls, FPS
    .showPhysics,          // Collision shapes
    .showAnchorOrigins,    // Anchor positions
    .showAnchorGeometry    // Detected plane geometry
]
#endif

If you can't see collision shapes with .showPhysics, your CollisionComponent is missing or misconfigured. Fix collision before debugging gestures or physics.

---

Symptom 1: Entity Not Visible

Time saved: 30-60 min → 2-5 min

Entity added but nothing appears
│
├─ Is the entity added to the scene?
│   └─ NO → Add to RealityView content:
│        content.add(entity)
│        ✓ Entities must be in the scene graph to render
│
├─ Does the entity have a ModelComponent?
│   └─ NO → Add mesh and material:
│        entity.components[ModelComponent.self] = ModelComponent(
│            mesh: .generateBox(size: 0.1),
│            materials: [SimpleMaterial(color: .red, isMetallic: false)]
│        )
│        ✓ Bare Entity is invisible — it's just a container
│
├─ Is the entity's scale zero or nearly zero?
│   └─ CHECK → Print: entity.scale
│        USD models may import with unexpected scale.
│        Try: entity.scale = SIMD3(repeating: 0.01) for meter-scale models
│
├─ Is the entity behind the camera?
│   └─ CHECK → Print: entity.position(relativeTo: nil)
│        In RealityKit, -Z is forward (toward screen).
│        Try: entity.position = SIMD3(0, 0, -0.5) (half meter in front)
│
├─ Is the entity inside another object?
│   └─ CHECK → Move to a known visible position:
│        entity.position = SIMD3(0, 0, -1)
│
├─ Is the entity's isEnabled set to false?
│   └─ CHECK → entity.isEnabled = true
│        Also check parent: entity.isEnabledInHierarchy
│
├─ Is the entity on an untracked anchor?
│   └─ CHECK → Verify anchor is tracking:
│        entity.isAnchored (should be true)
│        If using plane anchor, ensure surface is detected first
│
└─ Is the material transparent or OcclusionMaterial?
    └─ CHECK → Inspect material:
         If using PhysicallyBasedMaterial, check baseColor is not black
         If using blending = .transparent, check opacity > 0

Quick Diagnostic

func diagnoseVisibility(_ entity: Entity) {
    print("Name: \(entity.name)")
    print("Is enabled: \(entity.isEnabled)")
    print("In hierarchy: \(entity.isEnabledInHierarchy)")
    print("Is anchored: \(entity.isAnchored)")
    print("Position (world): \(entity.position(relativeTo: nil))")
    print("Scale: \(entity.scale)")
    print("Has model: \(entity.components[ModelComponent.self] != nil)")
    print("Children: \(entity.children.count)")
}

---

Symptom 2: Anchor Not Tracking

Time saved: 20-45 min → 3-5 min

AR content not appearing or floating
│
├─ Is the AR session running?
│   └─ For RealityView on iOS 18+, AR runs automatically
│       For ARView, check: arView.session.isRunning
│
├─ Is SpatialTrackingSession configured? (iOS 18+)
│   └─ CHECK → Ensure tracking modes requested:
│        let config = SpatialTrackingSession.Configuration(
│            tracking: [.plane, .object])
│        let result = await session.run(config)
│        if let notSupported = result {
│            // Handle unsupported modes
│        }
│
├─ Is the anchor type appropriate for the environment?
│   ├─ .plane(.horizontal) → Need a flat surface visible to camera
│   ├─ .plane(.vertical) → Need a wall visible to camera
│   ├─ .image → Image must be in "AR Resources" asset catalog
│   ├─ .face → Front camera required (not rear)
│   └─ .body → Full body must be visible
│
├─ Is minimumBounds too large?
│   └─ CHECK → Reduce minimum bounds:
│        AnchorEntity(.plane(.horizontal, classification: .any,
│            minimumBounds: SIMD2(0.1, 0.1)))  // Smaller = detects sooner
│
├─ Is the device supported?
│   └─ CHECK → Plane detection requires A12+ chip
│       Face tracking requires TrueDepth camera
│       Body tracking requires A12+ chip
│
└─ Is the environment adequate?
    └─ CHECK → AR needs:
         - Adequate lighting (not too dark)
         - Textured surfaces (not blank walls)
         - Stable device position during initial detection

---

Symptom 3: Gesture Not Responding

Time saved: 15-30 min → 2-3 min

Tap/drag on entity does nothing
│
├─ Does the entity have a CollisionComponent?
│   └─ NO → Add collision shapes:
│        entity.generateCollisionShapes(recursive: true)
│        // or manual:
│        entity.components[CollisionComponent.self] = CollisionComponent(
│            shapes: [.generateBox(size: SIMD3(0.1, 0.1, 0.1))])
│        ✓ Collision shapes are REQUIRED for gesture hit testing
│
├─ [visionOS] Does the entity have InputTargetComponent?
│   └─ NO → Add it:
│        entity.components[InputTargetComponent.self] = InputTargetComponent()
│        ✓ Required on visionOS for gesture input
│
├─ Is the gesture attached to the RealityView?
│   └─ CHECK → Gesture must be on the view, not the entity:
│        RealityView { content in ... }
│            .gesture(TapGesture().targetedToAnyEntity().onEnded { ... })
│
├─ Is the collision shape large enough to hit?
│   └─ CHECK → Enable .showPhysics to see shapes
│        Shapes too small = hard to tap.
│        Try: .generateBox(size: SIMD3(repeating: 0.1)) minimum
│
├─ Is the entity behind another entity?
│   └─ CHECK → Front entities may block gestures on back entities
│        Ensure collision is on the intended target
│
└─ Is the entity enabled?
    └─ CHECK → entity.isEnabled must be true
         Disabled entities don't receive input

Quick Diagnostic

func diagnoseGesture(_ entity: Entity) {
    print("Has collision: \(entity.components[CollisionComponent.self] != nil)")
    print("Has input target: \(entity.components[InputTargetComponent.self] != nil)")
    print("Is enabled: \(entity.isEnabled)")
    print("Is anchored: \(entity.isAnchored)")

    if let collision = entity.components[CollisionComponent.self] {
        print("Collision shapes: \(collision.shapes.count)")
    }
}

---

Symptom 4: Performance Problems

Time saved: 1-3 hours → 10-20 min

Frame rate dropping or stuttering
│
├─ How many entities are in the scene?
│   └─ CHECK → Print entity count:
│        var count = 0
│        func countEntities(_ entity: Entity) {
│            count += 1
│            for child in entity.children { countEntities(child) }
│        }
│        Under 100: unlikely to be entity count
│        100-500: review for optimization
│        500+: definitely needs optimization
│
├─ Are mesh/material resources shared?
│   └─ NO → Share resources across identical entities:
│        let sharedMesh = MeshResource.generateBox(size: 0.05)
│        let sharedMaterial = SimpleMaterial(color: .white, isMetallic: false)
│        // Reuse for all instances
│        ✓ RealityKit batches entities with identical resources
│
├─ Is a System creating components every frame?
│   └─ CHECK → Look for allocations in update():
│        Creating ModelComponent, CollisionComponent, or materials
│        every frame causes GC pressure.
│        Cache resources, only update when values change.
│
├─ Are collision shapes mesh-based?
│   └─ CHECK → Replace generateCollisionShapes(recursive: true)
│        with simple shapes (box, sphere, capsule) for dynamic entities
│
├─ Is generateCollisionShapes called repeatedly?
│   └─ CHECK → Call once during setup, not every frame
│
├─ Are there too many physics bodies?
│   └─ CHECK → Dynamic bodies are most expensive.
│        Convert distant/static objects to .static mode.
│        Remove physics from non-interactive entities.
│
└─ Is the model polygon count too high?
    └─ CHECK → Decimate models for real-time use.
         Target: <100K triangles total for mobile AR.
         Use LOD (Level of Detail) for distant objects.

---

Symptom 5: Material Looks Wrong

Time saved: 15-45 min → 5-10 min

Colors, lighting, or textures look incorrect
│
├─ Is the scene too dark, or models solid black?
│   └─ AR vs non-AR is the rule:
│        AR sessions apply real-world lighting automatically.
│        Non-AR (RealityView with no AR, macOS, previews) is UNLIT —
│        PBR has nothing to reflect, so it renders black.
│        Fix: add an ImageBasedLightComponent (IBL) for realistic
│        reflections, or a DirectionalLightComponent for a simple key light.
│
├─ Is the baseColor set?
│   └─ CHECK → PhysicallyBasedMaterial defaults to white
│        material.baseColor = .init(tint: .red)
│        If using a texture, verify it loaded:
│        try TextureResource(named: "albedo")
│
├─ Is metallic set incorrectly?
│   └─ CHECK → metallic = 1.0 makes surfaces mirror-like
│        Most real objects: metallic = 0.0
│        Only metals (gold, silver, chrome): metallic = 1.0
│
├─ Is the texture semantic wrong?
│   └─ CHECK → Use correct semantic:
│        .color for albedo/baseColor textures
│        .raw for data textures (metallic, roughness)
│        .normal for normal maps
│        .hdrColor for HDR textures
│
├─ Is the model upside down or inside out?
│   └─ CHECK → Try:
│        material.faceCulling = .none (shows both sides)
│        If that fixes it, the model normals are flipped
│
└─ Is blending/transparency unexpected?
    └─ CHECK → material.blending
         Default is .opaque
         For transparency: .transparent(opacity: ...)

---

Symptom 6: Physics Not Working

Time saved: 20-40 min → 5-10 min

Objects pass through each other or don't collide
│
├─ Do both entities have CollisionComponent?
│   └─ NO → Both sides of a collision need CollisionComponent
│
├─ Does the moving entity have PhysicsBodyComponent?
│   └─ NO → Add physics body:
│        entity.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
│            mode: .dynamic)
│
├─ Are collision groups/filters configured correctly?
│   └─ CHECK → Entities must be in compatible groups:
│        Default: group = .default, mask = .all
│        If using custom groups, verify mask includes the other group
│
├─ Is the physics mode correct?
│   ├─ Two .static bodies → Never collide (both immovable)
│   ├─ .dynamic + .static → Correct (common setup)
│   ├─ .dynamic + .dynamic → Both move on collision
│   └─ .kinematic + .dynamic → Kinematic pushes dynamic
│
├─ Is the collision shape appropriate?
│   └─ CHECK → .showPhysics debug option
│        Shape may be too small, offset, or wrong type
│
└─ Are entities on different anchors?
    └─ CHECK → "Physics bodies and colliders affect only
         entities that share the same anchor" (Apple docs)
         Move entities under the same anchor for physics interaction

---

Symptom 7: Multiplayer Sync Issues

Time saved: 30-60 min → 10-15 min

Entities not appearing on other devices
│
├─ Does the entity have SynchronizationComponent?
│   └─ NO → Add it:
│        entity.components[SynchronizationComponent.self] =
│            SynchronizationComponent()
│
├─ Is the MultipeerConnectivityService set up?
│   └─ CHECK → Verify MCSession is connected before syncing
│
├─ Are custom components Codable?
│   └─ NO → Non-Codable components don't sync
│        struct MyComponent: Component, Codable { ... }
│
├─ Does the entity have an owner?
│   └─ CHECK → Only the owner can modify synced properties
│        Request ownership before modifying:
│        entity.requestOwnership { result in ... }
│
└─ Is the entity anchored?
    └─ CHECK → Unanchored entities may not sync position correctly
         Use a shared world anchor for reliable positioning

---

Common Mistakes

---

Diagnostic Quick Reference

---

Symptom 8: Works in Simulator, Crashes on Device

Time cost: 15-30 min (often misdiagnosed as model issue)

Q1: Is the crash a Metal error (MTLCommandBuffer, shader compilation)?
├─ YES → Simulator uses software rendering, device uses real GPU
│   Common causes:
│   - Custom Metal shaders with unsupported features
│   - Texture formats not supported on device GPU
│   - Exceeding device texture size limits (max 8192x8192 on older)
│   Fix: Check device GPU family, use supported formats
│
└─ NO → Check next

Q2: Is it an out-of-memory crash?
├─ YES → Simulator has more RAM available
│   Common: Large USDZ files with uncompressed textures
│   Fix: Compress textures, reduce polygon count, use LOD
│   Check: USDZ file size (keep < 50MB for reliable loading)
│
└─ NO → Check next

Q3: Is it an AR-related crash (camera, tracking)?
├─ YES → Simulator has no real camera/sensors
│   Fix: Test AR features on device only, use simulator for UI/layout
│
└─ NO → Check device capabilities
    - A12+ required for RealityKit
    - LiDAR for scene reconstruction
    - TrueDepth for face tracking

---

Resources

WWDC: 2019-603, 2019-605, 2023-10080, 2024-10103, 2026-279

Docs: /realitykit, /realitykit/entity, /realitykit/collisioncomponent, /realitykit/physicsbodycomponent

Skills: axiom-graphics (skills/realitykit.md), axiom-graphics (skills/realitykit-ref.md)

SceneKit API Reference & Migration Mapping

Complete API reference for SceneKit with RealityKit equivalents for every major concept.

When to Use This Reference

Use this reference when:

• Looking up SceneKit → RealityKit API equivalents during migration
• Checking specific SceneKit class properties or methods
• Planning which SceneKit features have direct RealityKit counterparts
• Understanding architectural differences between scene graph and ECS

---

Part 1: SceneKit → RealityKit Concept Mapping

Core Architecture

Geometry & Rendering

Transforms & Hierarchy

Lighting

Camera

Physics

Animation

Interaction

AR Integration

---

Part 2: Scene Graph API

SCNScene

// Loading
let scene = SCNScene(named: "scene.usdz")!
let scene = try SCNScene(url: url, options: [
    .checkConsistency: true,
    .convertToYUp: true
])

// Properties
scene.rootNode                    // Root of node hierarchy
scene.background.contents        // Skybox (UIImage, UIColor, MDLSkyCubeTexture)
scene.lightingEnvironment.contents // IBL environment map
scene.fogStartDistance            // Fog near
scene.fogEndDistance              // Fog far
scene.fogColor                   // Fog color
scene.isPaused                   // Pause simulation

SCNNode

// Creation
let node = SCNNode()
let node = SCNNode(geometry: SCNBox(width: 1, height: 1, length: 1, chamferRadius: 0))

// Transform
node.position = SCNVector3(x, y, z)
node.eulerAngles = SCNVector3(pitch, yaw, roll)
node.scale = SCNVector3(1, 1, 1)
node.simdPosition = SIMD3<Float>(x, y, z)  // SIMD variants available
node.pivot = SCNMatrix4MakeTranslation(0, -0.5, 0) // Offset pivot point

// Visibility
node.isHidden = false
node.opacity = 1.0
node.castsShadow = true
node.renderingOrder = 0   // Lower = rendered first

// Hierarchy
node.addChildNode(child)
node.removeFromParentNode()
node.childNodes
node.childNode(withName: "name", recursively: true)
node.enumerateChildNodes { child, stop in }

---

Part 3: Materials

Lighting Models

Material Properties

let mat = SCNMaterial()
mat.lightingModel = .physicallyBased

// Textures or scalar values
mat.diffuse.contents = UIImage(named: "albedo")    // Base color
mat.metalness.contents = 0.0                        // 0 = dielectric, 1 = metal
mat.roughness.contents = 0.5                        // 0 = mirror, 1 = rough
mat.normal.contents = UIImage(named: "normal")      // Normal map
mat.ambientOcclusion.contents = UIImage(named: "ao") // AO map
mat.emission.contents = UIColor.blue                // Glow
mat.displacement.contents = UIImage(named: "height") // Height map

// Options
mat.isDoubleSided = false        // Render both sides
mat.writesToDepthBuffer = true
mat.readsFromDepthBuffer = true
mat.blendMode = .alpha           // .add, .subtract, .multiply, .screen
mat.transparencyMode = .aOne     // .rgbZero for pre-multiplied alpha

---

Part 4: Physics

Body Types and Properties

// Dynamic body with custom shape
let shape = SCNPhysicsShape(geometry: SCNSphere(radius: 0.5), options: nil)
let body = SCNPhysicsBody(type: .dynamic, shape: shape)
body.mass = 1.0
body.friction = 0.5
body.restitution = 0.3       // Bounciness
body.damping = 0.1            // Linear damping
body.angularDamping = 0.1     // Angular damping
body.isAffectedByGravity = true
body.allowsResting = true     // Sleep optimization
node.physicsBody = body

// Compound shapes
let compound = SCNPhysicsShape(shapes: [shape1, shape2],
    transforms: [transform1, transform2])

// Concave (static only)
let concave = SCNPhysicsShape(geometry: mesh, options: [
    .type: SCNPhysicsShape.ShapeType.concavePolyhedron
])

Joint Types

---

Part 5: Animation API

SCNAction Catalog

Timing Functions

action.timingMode = .linear        // Default
action.timingMode = .easeIn        // Slow start
action.timingMode = .easeOut       // Slow end
action.timingMode = .easeInEaseOut // Slow start and end
action.timingFunction = { t in     // Custom curve
    return t * t  // Quadratic ease-in
}

---

Part 6: Constraints

let lookAt = SCNLookAtConstraint(target: targetNode)
lookAt.isGimbalLockEnabled = true  // Prevent roll
lookAt.influenceFactor = 0.8       // Partial constraint
node.constraints = [lookAt]

In RealityKit: No direct constraint system. Implement with System update logic or entity.look(at:from:relativeTo:).

---

Part 7: Scene Configuration

SCNView Configuration

---

Resources

WWDC: 2014-609, 2014-610, 2017-604, 2019-612

Docs: /scenekit, /scenekit/scnscene, /scenekit/scnnode, /scenekit/scnmaterial, /scenekit/scnphysicsbody, /scenekit/scnaction

Skills: axiom-graphics (skills/scenekit.md), axiom-graphics (skills/realitykit.md), axiom-graphics (skills/realitykit-ref.md)