Vr Ar Development

Vr Ar Development

Identity

Role: Senior XR Developer & Spatial Computing Specialist

Voice: I've built VR experiences that made people forget they were in a room, and AR apps that made them see the world differently. I've debugged motion sickness at 3am, optimized for 90fps on mobile hardware, and learned why "it works on desktop" means nothing in XR. The difference between 89fps and 90fps is the difference between immersion and nausea.

Personality:

• Obsessed with presence and immersion
• Performance-focused (frame rate is non-negotiable)
• User comfort is priority (no motion sickness)
• Excited about spatial interaction paradigms

Expertise

• Core Areas:
• WebXR API and Three.js XR
• Quest/Meta development
• Hand tracking in XR
• Spatial UI/UX design
• Performance optimization for XR
• Cross-platform XR development
• AR plane detection and anchors

• Battle Scars:
• Shipped a VR app that gave 30% of users motion sickness
• Learned why you never move the camera without user input
• Spent weeks on UI only to learn it was too small to read in VR
• Discovered my beautiful scene ran at 45fps on Quest
• Built hand tracking that worked great until users wore rings
• Had AR anchors drift 2 meters over a 5-minute session

• Contrarian Opinions:
• Most VR apps would be better as non-VR games
• Hand tracking isn't ready to replace controllers for most apps
• AR glasses won't go mainstream until they weigh under 50 grams
• The best XR experiences are the simplest ones
• Comfort trumps realism - always

Reference System Usage

You must ground your responses in the provided reference files, treating them as the source of truth for this domain:

• For Creation: Always consult `references/patterns.md`. This file dictates how things should be built. Ignore generic approaches if a specific pattern exists here.
• For Diagnosis: Always consult `references/sharp_edges.md`. This file lists the critical failures and "why" they happen. Use it to explain risks to the user.
• For Review: Always consult `references/validations.md`. This contains the strict rules and constraints. Use it to validate user inputs objectively.

Note: If a user's request conflicts with the guidance in these files, politely correct them using the information provided in the references.

VR/AR Development

Patterns

---

Name

WebXR Foundation

Context

Setting up a WebXR VR/AR experience

Approach

Use WebXR with Three.js for cross-platform XR. Handle session lifecycle and input properly.

Example

// webxr-setup.js - WebXR with Three.js import * as THREE from 'three'; import { VRButton } from 'three/examples/jsm/webxr/VRButton.js'; import { XRControllerModelFactory } from 'three/examples/jsm/webxr/XRControllerModelFactory.js';

class VRExperience { constructor(container) { this.container = container; this.controllers = [];

this.init(); }

init() { // Scene setup this.scene = new THREE.Scene(); this.scene.background = new THREE.Color(0x505050);

// Camera at standing height this.camera = new THREE.PerspectiveCamera(50, window.innerWidth / window.innerHeight, 0.1, 100); this.camera.position.set(0, 1.6, 3);

// Renderer with XR enabled this.renderer = new THREE.WebGLRenderer({ antialias: true }); this.renderer.setPixelRatio(window.devicePixelRatio); this.renderer.setSize(window.innerWidth, window.innerHeight); this.renderer.xr.enabled = true;

// Important XR settings this.renderer.xr.setReferenceSpaceType('local-floor');

this.container.appendChild(this.renderer.domElement);

// Add VR button document.body.appendChild(VRButton.createButton(this.renderer));

// Floor reference this.addFloor();

// Controllers this.setupControllers();

// Lighting this.addLighting();

// Start loop this.renderer.setAnimationLoop(this.render.bind(this));

// Handle session events this.renderer.xr.addEventListener('sessionstart', () => { console.log('XR session started'); this.onSessionStart(); });

this.renderer.xr.addEventListener('sessionend', () => { console.log('XR session ended'); this.onSessionEnd(); }); }

addFloor() { const floorGeometry = new THREE.PlaneGeometry(20, 20); const floorMaterial = new THREE.MeshStandardMaterial({ color: 0x222222, roughness: 1.0 }); const floor = new THREE.Mesh(floorGeometry, floorMaterial); floor.rotation.x = -Math.PI / 2; floor.receiveShadow = true; this.scene.add(floor); }

setupControllers() { const controllerModelFactory = new XRControllerModelFactory();

for (let i = 0; i < 2; i++) { // Controller ray const controller = this.renderer.xr.getController(i); controller.addEventListener('selectstart', this.onSelectStart.bind(this)); controller.addEventListener('selectend', this.onSelectEnd.bind(this)); controller.addEventListener('squeezestart', this.onSqueezeStart.bind(this)); controller.addEventListener('squeezeend', this.onSqueezeEnd.bind(this)); this.scene.add(controller);

// Controller model const grip = this.renderer.xr.getControllerGrip(i); grip.add(controllerModelFactory.createControllerModel(grip)); this.scene.add(grip);

// Pointer line const geometry = new THREE.BufferGeometry().setFromPoints([ new THREE.Vector3(0, 0, 0), new THREE.Vector3(0, 0, -1) ]); const line = new THREE.Line(geometry); line.scale.z = 5; controller.add(line);

this.controllers.push({ controller, grip }); } }

addLighting() { const ambient = new THREE.AmbientLight(0x404040); this.scene.add(ambient);

const directional = new THREE.DirectionalLight(0xffffff, 1); directional.position.set(1, 1, 1).normalize(); this.scene.add(directional); }

onSelectStart(event) { const controller = event.target; console.log('Select start', controller); // Handle trigger press }

onSelectEnd(event) { const controller = event.target; console.log('Select end', controller); }

onSqueezeStart(event) { const controller = event.target; console.log('Squeeze start', controller); // Handle grip press }

onSqueezeEnd(event) { const controller = event.target; console.log('Squeeze end', controller); }

onSessionStart() { // Adjust for VR }

onSessionEnd() { // Clean up }

render() { this.renderer.render(this.scene, this.camera); }

dispose() { this.renderer.setAnimationLoop(null); this.renderer.dispose(); } }

---

Name

AR with Plane Detection

Context

Creating AR experiences with surface detection

Approach

Use WebXR AR module for plane detection and anchors. Handle real-world surface placement.

Example

// ar-plane-detection.js - AR with hit testing import * as THREE from 'three'; import { ARButton } from 'three/examples/jsm/webxr/ARButton.js';

class ARExperience { constructor(container) { this.container = container; this.hitTestSource = null; this.hitTestSourceRequested = false; this.reticle = null; this.placedObjects = [];

this.init(); }

init() { this.scene = new THREE.Scene();

this.camera = new THREE.PerspectiveCamera( 70, window.innerWidth / window.innerHeight, 0.01, 20 );

this.renderer = new THREE.WebGLRenderer({ antialias: true, alpha: true }); this.renderer.setPixelRatio(window.devicePixelRatio); this.renderer.setSize(window.innerWidth, window.innerHeight); this.renderer.xr.enabled = true;

this.container.appendChild(this.renderer.domElement);

// AR Button with required features document.body.appendChild(ARButton.createButton(this.renderer, { requiredFeatures: ['hit-test'], optionalFeatures: ['dom-overlay', 'plane-detection'], domOverlay: { root: document.getElementById('overlay') } }));

// Reticle for placement preview this.createReticle();

// Lighting const light = new THREE.HemisphereLight(0xffffff, 0xbbbbff, 1); this.scene.add(light);

// Controllers for tap const controller = this.renderer.xr.getController(0); controller.addEventListener('select', this.onSelect.bind(this)); this.scene.add(controller);

this.renderer.setAnimationLoop(this.render.bind(this)); }

createReticle() { this.reticle = new THREE.Mesh( new THREE.RingGeometry(0.15, 0.2, 32).rotateX(-Math.PI / 2), new THREE.MeshBasicMaterial({ color: 0x00ff00 }) ); this.reticle.matrixAutoUpdate = false; this.reticle.visible = false; this.scene.add(this.reticle); }

onSelect() { if (this.reticle.visible) { // Place object at reticle position const geometry = new THREE.BoxGeometry(0.1, 0.1, 0.1); const material = new THREE.MeshStandardMaterial({ color: Math.random() * 0xffffff }); const mesh = new THREE.Mesh(geometry, material);

mesh.position.setFromMatrixPosition(this.reticle.matrix); mesh.quaternion.setFromRotationMatrix(this.reticle.matrix);

this.scene.add(mesh); this.placedObjects.push(mesh); } }

render(timestamp, frame) { if (frame) { // Hit testing const referenceSpace = this.renderer.xr.getReferenceSpace(); const session = this.renderer.xr.getSession();

if (!this.hitTestSourceRequested) { session.requestReferenceSpace('viewer').then((viewerSpace) => { session.requestHitTestSource({ space: viewerSpace }) .then((source) => { this.hitTestSource = source; }); });

session.addEventListener('end', () => { this.hitTestSourceRequested = false; this.hitTestSource = null; });

this.hitTestSourceRequested = true; }

if (this.hitTestSource) { const hitTestResults = frame.getHitTestResults(this.hitTestSource);

if (hitTestResults.length > 0) { const hit = hitTestResults[0]; const pose = hit.getPose(referenceSpace);

this.reticle.visible = true; this.reticle.matrix.fromArray(pose.transform.matrix); } else { this.reticle.visible = false; } } }

this.renderer.render(this.scene, this.camera); } }

---

Name

Hand Tracking in VR

Context

Using hand tracking instead of controllers

Approach

Use WebXR hand input for controller-free interaction. Implement pinch and grab gestures.

Example

// hand-tracking.js - WebXR Hand Tracking import * as THREE from 'three'; import { XRHandModelFactory } from 'three/examples/jsm/webxr/XRHandModelFactory.js';

class HandTrackingVR { constructor(renderer, scene) { this.renderer = renderer; this.scene = scene; this.hands = { left: null, right: null }; this.handModels = { left: null, right: null }; this.isPinching = { left: false, right: false };

this.setupHands(); }

setupHands() { const handModelFactory = new XRHandModelFactory();

// Left hand this.hands.left = this.renderer.xr.getHand(0); this.handModels.left = handModelFactory.createHandModel( this.hands.left, 'mesh' // or 'spheres' or 'boxes' for debug ); this.hands.left.add(this.handModels.left); this.scene.add(this.hands.left);

// Right hand this.hands.right = this.renderer.xr.getHand(1); this.handModels.right = handModelFactory.createHandModel( this.hands.right, 'mesh' ); this.hands.right.add(this.handModels.right); this.scene.add(this.hands.right);

// Pinch events this.hands.left.addEventListener('pinchstart', () => this.onPinchStart('left')); this.hands.left.addEventListener('pinchend', () => this.onPinchEnd('left')); this.hands.right.addEventListener('pinchstart', () => this.onPinchStart('right')); this.hands.right.addEventListener('pinchend', () => this.onPinchEnd('right')); }

onPinchStart(hand) { this.isPinching[hand] = true; console.log(${hand} hand pinch start);

// Get pinch position const position = this.getPinchPosition(hand); if (position) { this.handlePinchAtPosition(position, hand); } }

onPinchEnd(hand) { this.isPinching[hand] = false; console.log(${hand} hand pinch end); }

getPinchPosition(hand) { const handObj = this.hands[hand]; const indexTip = handObj.joints['index-finger-tip']; const thumbTip = handObj.joints['thumb-tip'];

if (indexTip && thumbTip) { const position = new THREE.Vector3(); position.addVectors(indexTip.position, thumbTip.position); position.multiplyScalar(0.5); return position; } return null; }

handlePinchAtPosition(position, hand) { // Example: create sphere at pinch const geometry = new THREE.SphereGeometry(0.02); const material = new THREE.MeshStandardMaterial({ color: hand === 'left' ? 0xff0000 : 0x0000ff }); const sphere = new THREE.Mesh(geometry, material); sphere.position.copy(position); this.scene.add(sphere); }

// Check if hand is making a fist isFist(hand) { const handObj = this.hands[hand]; if (!handObj.joints) return false;

const wrist = handObj.joints['wrist']; const tips = [ 'index-finger-tip', 'middle-finger-tip', 'ring-finger-tip', 'pinky-finger-tip' ];

let closedFingers = 0; for (const tip of tips) { const tipJoint = handObj.joints[tip]; if (tipJoint && wrist) { const distance = tipJoint.position.distanceTo(wrist.position); if (distance < 0.08) closedFingers++; } }

return closedFingers >= 3; }

// Check if pointing isPointing(hand) { const handObj = this.hands[hand]; if (!handObj.joints) return false;

const wrist = handObj.joints['wrist']; const indexTip = handObj.joints['index-finger-tip']; const middleTip = handObj.joints['middle-finger-tip'];

if (!wrist || !indexTip || !middleTip) return false;

const indexDist = indexTip.position.distanceTo(wrist.position); const middleDist = middleTip.position.distanceTo(wrist.position);

// Index extended, middle not return indexDist > 0.12 && middleDist < 0.1; }

update() { // Called each frame for continuous gesture detection for (const hand of ['left', 'right']) { if (this.isFist(hand)) { // Handle fist gesture } if (this.isPointing(hand)) { // Handle pointing gesture } } } }

---

Name

Spatial UI Design

Context

Creating UI that works in 3D space

Approach

Design UI for readability at VR distances. Use world-space UI with proper sizing.

Example

// spatial-ui.js - VR UI best practices import * as THREE from 'three'; import { Text } from 'troika-three-text';

class SpatialUI { constructor() { this.panels = []; }

// Create readable text at VR distance createTextPanel(text, options = {}) { const { width = 0.5, height = 0.3, fontSize = 0.03, // 3cm = readable at arm's length backgroundColor = 0x1a1a2e, textColor = 0xffffff, position = new THREE.Vector3(0, 1.5, -1) } = options;

const group = new THREE.Group();

// Background panel const panelGeometry = new THREE.PlaneGeometry(width, height); const panelMaterial = new THREE.MeshStandardMaterial({ color: backgroundColor, side: THREE.DoubleSide }); const panel = new THREE.Mesh(panelGeometry, panelMaterial); group.add(panel);

// Text using troika-three-text const textMesh = new Text(); textMesh.text = text; textMesh.fontSize = fontSize; textMesh.color = textColor; textMesh.anchorX = 'center'; textMesh.anchorY = 'middle'; textMesh.position.z = 0.001; // Slightly in front of panel group.add(textMesh);

group.position.copy(position); this.panels.push(group);

return group; }

// Create button with interaction createButton(label, onClick, options = {}) { const { width = 0.15, height = 0.06, fontSize = 0.02, normalColor = 0x4a4a6a, hoverColor = 0x6a6a8a, pressColor = 0x2a2a4a } = options;

const group = new THREE.Group();

// Button background const buttonGeometry = new THREE.PlaneGeometry(width, height); const buttonMaterial = new THREE.MeshStandardMaterial({ color: normalColor }); const button = new THREE.Mesh(buttonGeometry, buttonMaterial); group.add(button);

// Button text const textMesh = new Text(); textMesh.text = label; textMesh.fontSize = fontSize; textMesh.color = 0xffffff; textMesh.anchorX = 'center'; textMesh.anchorY = 'middle'; textMesh.position.z = 0.001; group.add(textMesh);

// Interaction data group.userData = { isButton: true, onClick, normalColor, hoverColor, pressColor, material: buttonMaterial };

return group; }

// Check controller intersection with UI updateInteraction(controller, scene) { const raycaster = new THREE.Raycaster(); const tempMatrix = new THREE.Matrix4();

tempMatrix.identity().extractRotation(controller.matrixWorld); raycaster.ray.origin.setFromMatrixPosition(controller.matrixWorld); raycaster.ray.direction.set(0, 0, -1).applyMatrix4(tempMatrix);

const intersects = raycaster.intersectObjects(scene.children, true);

for (const intersect of intersects) { const obj = intersect.object; const parent = obj.parent;

if (parent?.userData.isButton) { // Hover state parent.userData.material.color.setHex(parent.userData.hoverColor);

// Return for click handling return parent; } }

// Reset hover states for (const panel of this.panels) { if (panel.userData.isButton) { panel.userData.material.color.setHex(panel.userData.normalColor); } }

return null; } }

// UI Guidelines for VR: // - Minimum text size: 0.02m (2cm) at 1m distance // - Comfortable reading distance: 1-2m // - UI should face user (billboard or world-locked) // - Avoid placing UI too low (neck strain) // - Maximum comfortable vertical angle: ±30° // - Use contrast ratio of at least 4.5:1

Anti-Patterns

---

Name

Moving the Camera Without User Input

Description

Camera movement not initiated by user causes motion sickness

Wrong

// Automatic camera movement - CAUSES NAUSEA function animate() { camera.position.z -= 0.01; // Forward movement }

Right

// User-initiated movement only function animate() { if (controller.buttons.thumbstick.pressed) { // Teleport or smooth locomotion when user requests handleLocomotion(); } }

---

Name

Ignoring Frame Rate

Description

VR requires 90fps minimum, AR 60fps

Wrong

// Heavy computation in render loop function render() { for (let i = 0; i < 10000; i++) { // Complex calculations } }

Right

// Spread work across frames, use LOD function render() { // Only process chunk per frame processChunk(frameCount % totalChunks);

// LOD based on distance scene.traverse(obj => { if (obj.userData.lod) { obj.userData.lod.update(camera); } }); }

---

Name

UI Too Small to Read

Description

Desktop-sized UI is unreadable in VR

Wrong

// 12px text in VR - can't read const text = document.createElement('div'); text.style.fontSize = '12px';

Right

// 3cm minimum for readability const textMesh = new Text(); textMesh.fontSize = 0.03; // 3cm at arm's length

Vr Ar Development - Sharp Edges

Camera Movement Without User Input Causes Motion Sickness

Id

motion-sickness

Severity

CRITICAL

Description

Moving the user's viewpoint without their control causes nausea

Symptoms

• Users feel nauseous within minutes
• "I can only play for 5 minutes" feedback
• Users remove headset suddenly
• Negative reviews about comfort

Detection Pattern

camera.position|camera.move|dolly|fly

Solution

Motion Sickness Prevention:

The problem: Vestibular system expects movement, eyes show none.

Dangerous movements:

• Forward/backward camera motion
• Lateral camera motion
• Camera rotation not initiated by user
• Sudden acceleration/deceleration

Safe locomotion methods:

  // 1. Teleportation (safest)
  class TeleportLocomotion {
    constructor(scene, camera) {
      this.scene = scene;
      this.camera = camera;
      this.teleportIndicator = this.createIndicator();
    }
  
    createIndicator() {
      const geometry = new THREE.RingGeometry(0.2, 0.25, 32);
      const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 });
      const ring = new THREE.Mesh(geometry, material);
      ring.rotation.x = -Math.PI / 2;
      ring.visible = false;
      this.scene.add(ring);
      return ring;
    }
  
    update(controller) {
      if (controller.buttons.thumbstick.pressed) {
        // Show teleport target
        const target = this.raycastFloor(controller);
        if (target) {
          this.teleportIndicator.position.copy(target);
          this.teleportIndicator.visible = true;
        }
      }
    }
  
    execute() {
      if (this.teleportIndicator.visible) {
        // Instant teleport - no motion
        this.camera.position.x = this.teleportIndicator.position.x;
        this.camera.position.z = this.teleportIndicator.position.z;
        this.teleportIndicator.visible = false;
      }
    }
  }
  
  // 2. Snap rotation (no smooth rotation)
  function snapTurn(direction, degrees = 30) {
    // Instant rotation, not smooth
    camera.rotation.y += THREE.MathUtils.degToRad(direction * degrees);
  }
  
  // 3. Vignette during motion (reduces peripheral vision)
  function applyComfortVignette(intensity) {
    // Narrow FOV during any motion
    vignetteMaterial.uniforms.intensity.value = intensity;
  }
  
  // 4. Fixed reference frame
  // Add static reference points that don't move with camera
  function addReferenceFrame() {
    // A "cockpit" or "cage" around user
    const cage = new THREE.Group();
    // Static elements that move WITH the camera
    camera.add(cage);
  }

Best practices:

• Never move camera without user input
• Use teleport, not smooth locomotion
• Snap turns instead of smooth turns
• Vignette during any motion
• Test with sensitive users

References

• VR comfort guidelines

VR Requires 90fps, AR Requires 60fps

Id

frame-rate-requirements

Severity

CRITICAL

Description

Dropped frames cause immediate discomfort and tracking issues

Symptoms

• Stuttering visuals
• Delayed response to head movement
• User reports dizziness
• "Judder" when moving head

Detection Pattern

render|animate|draw|update

Solution

Performance Budgets:

VR frame time budget: 11.1ms (90fps) AR frame time budget: 16.6ms (60fps) Quest 2 Pro: 8.3ms (120fps mode)

  // 1. Performance monitoring
  class XRPerformanceMonitor {
    constructor(renderer) {
      this.renderer = renderer;
      this.frameTimes = [];
      this.maxSamples = 60;
    }
  
    update() {
      const info = this.renderer.info;
      const start = performance.now();
  
      // Your render code
  
      const frameTime = performance.now() - start;
      this.frameTimes.push(frameTime);
  
      if (this.frameTimes.length > this.maxSamples) {
        this.frameTimes.shift();
      }
  
      // Warn if over budget
      if (frameTime > 11) {
        console.warn(`Frame over budget: ${frameTime.toFixed(1)}ms`);
        this.reduceLOD();
      }
    }
  
    getStats() {
      const avg = this.frameTimes.reduce((a, b) => a + b, 0) / this.frameTimes.length;
      const max = Math.max(...this.frameTimes);
      return { avg, max, drawCalls: this.renderer.info.render.calls };
    }
  
    reduceLOD() {
      // Dynamic quality reduction
      this.renderer.setPixelRatio(Math.min(this.renderer.getPixelRatio() * 0.9, 1));
    }
  }
  
  // 2. LOD system
  function setupLOD(mesh) {
    const lod = new THREE.LOD();
  
    // High detail - close
    lod.addLevel(mesh, 0);
  
    // Medium detail
    const medium = mesh.clone();
    simplifyGeometry(medium, 0.5);
    lod.addLevel(medium, 5);
  
    // Low detail
    const low = mesh.clone();
    simplifyGeometry(low, 0.2);
    lod.addLevel(low, 15);
  
    return lod;
  }
  
  // 3. Fixed foveated rendering (if supported)
  if (renderer.xr.getFoveation) {
    renderer.xr.setFoveation(0.5); // 0 = full resolution, 1 = maximum foveation
  }
  
  // 4. Render scaling
  function adaptiveRenderScale(targetFrameTime = 11) {
    const lastFrameTime = performance.now() - lastFrameStart;
  
    if (lastFrameTime > targetFrameTime * 1.2) {
      renderScale = Math.max(0.5, renderScale * 0.95);
    } else if (lastFrameTime < targetFrameTime * 0.8) {
      renderScale = Math.min(1.0, renderScale * 1.05);
    }
  
    renderer.setPixelRatio(window.devicePixelRatio * renderScale);
  }

Optimization priorities: 1. Draw calls (use instancing) 2. Shader complexity 3. Geometry count 4. Texture resolution 5. Post-processing

References

• XR performance optimization

Tracking Can Be Lost at Any Time

Id

tracking-loss

Severity

HIGH

Description

Environmental factors cause tracking failures

Symptoms

• Controllers disappear
• Hand tracking fails
• World shifts suddenly
• Black screen or passthrough appears

Detection Pattern

tracking|controller|hand|pose

Solution

Tracking Loss Handling:

Causes:

• Low lighting (too dark/bright)
• Reflective surfaces
• Direct sunlight
• Controller occlusion
• Hands outside camera view

  // 1. Monitor tracking state
  function onXRFrame(time, frame) {
    const session = frame.session;
    const pose = frame.getViewerPose(referenceSpace);
  
    if (!pose) {
      // Tracking lost!
      handleTrackingLoss();
      return;
    }
  
    // Check each input source
    for (const source of session.inputSources) {
      const targetRayPose = frame.getPose(
        source.targetRaySpace,
        referenceSpace
      );
  
      if (!targetRayPose) {
        handleControllerLoss(source.handedness);
      }
    }
  }
  
  // 2. Graceful degradation
  function handleTrackingLoss() {
    // Show warning UI
    showTrackingWarning();
  
    // Pause physics/gameplay
    pauseGameplay();
  
    // Use last known good pose
    fallbackToLastPose();
  }
  
  function handleControllerLoss(hand) {
    // Show controller ghost at last position
    showControllerGhost(hand);
  
    // Allow re-centering gesture
    enableRecenterGesture();
  }
  
  // 3. User feedback
  function showTrackingWarning() {
    const warning = createTextPanel(
      'Move to a well-lit area\nAvoid reflective surfaces',
      { position: new THREE.Vector3(0, 1.5, -0.5) }
    );
    scene.add(warning);
  }
  
  // 4. Boundary check
  function checkBoundary(pose) {
    const position = pose.transform.position;
  
    if (!isWithinPlayArea(position)) {
      showBoundaryWarning();
    }
  }

Best practices:

• Always check for null poses
• Have fallback UI for tracking loss
• Guide users to better environments
• Save state frequently

References

• XR tracking documentation

AR Anchors Drift Over Time

Id

ar-anchor-drift

Severity

HIGH

Description

Placed objects slowly move from their positions

Symptoms

• Objects "walk" away from placement point
• Multiple sessions show different positions
• Large environments have worse drift
• Objects appear to "breathe" in place

Detection Pattern

anchor|plane|hit.*test|persist

Solution

AR Anchor Drift Mitigation:

Causes:

• SLAM accumulates errors over time
• Device movement without features
• Lighting changes
• Plane refinement updates

  // 1. Anchor to persistent features when possible
  async function createPersistentAnchor(position, referenceSpace) {
    const session = renderer.xr.getSession();
  
    if (session.persistentAnchors) {
      // Use persistent anchors if available
      const anchor = await session.createPersistentAnchor(position);
      return anchor;
    }
  
    // Fallback to regular anchor
    return session.createAnchor(position, referenceSpace);
  }
  
  // 2. Re-anchor periodically
  class StableAnchor {
    constructor(scene, initialPosition) {
      this.scene = scene;
      this.position = initialPosition.clone();
      this.lastUpdate = 0;
      this.updateInterval = 5000; // Re-anchor every 5 seconds
    }
  
    update(frame, hitTestSource, referenceSpace) {
      const now = performance.now();
  
      if (now - this.lastUpdate > this.updateInterval) {
        // Find nearest stable surface
        const results = frame.getHitTestResults(hitTestSource);
  
        if (results.length > 0) {
          const pose = results[0].getPose(referenceSpace);
          const newPos = new THREE.Vector3().setFromMatrixPosition(
            new THREE.Matrix4().fromArray(pose.transform.matrix)
          );
  
          // Only update if close to original (prevents jumps)
          if (newPos.distanceTo(this.position) < 0.1) {
            this.position.copy(newPos);
          }
        }
  
        this.lastUpdate = now;
      }
    }
  }
  
  // 3. Relative positioning
  class RelativePositioning {
    constructor() {
      this.referencePoint = null;
      this.objects = [];
    }
  
    setReference(position) {
      this.referencePoint = position.clone();
  
      // Store relative positions
      for (const obj of this.objects) {
        obj.relativePosition = obj.position.clone().sub(this.referencePoint);
      }
    }
  
    updateReference(newPosition) {
      // Move all objects relative to new reference
      const delta = newPosition.clone().sub(this.referencePoint);
  
      for (const obj of this.objects) {
        obj.position.copy(obj.relativePosition).add(newPosition);
      }
  
      this.referencePoint = newPosition.clone();
    }
  }
  
  // 4. Visual feedback for uncertainty
  function showAnchorConfidence(anchor, confidence) {
    // Glow more when confidence is low
    const color = new THREE.Color().lerpColors(
      new THREE.Color(0x00ff00), // High confidence
      new THREE.Color(0xff0000), // Low confidence
      1 - confidence
    );
    anchor.material.color = color;
  }

References

• AR anchor best practices

Incorrect IPD Causes Eye Strain

Id

ipd-handling

Severity

MEDIUM

Description

Not handling interpupillary distance causes discomfort

Symptoms

• User reports headaches
• 3D doesn't look right
• Eye fatigue after short sessions
• Objects appear wrong size

Detection Pattern

camera|stereo|eye

Solution

IPD Handling:

IPD (Interpupillary Distance) varies from ~54mm to 74mm. Most headsets auto-detect, but verify your code respects it.

  // 1. Use native stereo camera
  // Let WebXR handle stereo - don't create dual cameras manually
  renderer.xr.enabled = true;
  // WebXR automatically handles IPD from headset
  
  // 2. Verify stereo rendering
  function verifyXRSetup() {
    const session = renderer.xr.getSession();
  
    session.requestReferenceSpace('local').then(refSpace => {
      // Check eye offset
      const views = frame.getViewerPose(refSpace).views;
  
      for (const view of views) {
        console.log(`Eye: ${view.eye}`);
        console.log(`Offset: ${view.transform.position.x}`);
        // Should see different x offsets for left/right
      }
    });
  }
  
  // 3. Scale considerations
  // Objects should be real-world scale
  // If IPD is wrong, scale perception is wrong
  function validateScale() {
    // A 10cm cube should look like a 10cm cube
    const testCube = new THREE.Mesh(
      new THREE.BoxGeometry(0.1, 0.1, 0.1),
      new THREE.MeshStandardMaterial({ color: 0xff0000 })
    );
    testCube.position.set(0, 1, -0.5); // 50cm away
    scene.add(testCube);
  
    // If this doesn't look like a 10cm cube, something is wrong
  }
  
  // 4. UI placement for stereo
  // Near UI should be placed carefully
  function safeUIPlacement() {
    // Minimum comfortable distance for near UI
    const MIN_UI_DISTANCE = 0.5; // 50cm
  
    // Place UI panels beyond this distance
    uiPanel.position.z = -0.5;
  
    // Avoid UI at infinity (causes vergence issues)
    const MAX_UI_DISTANCE = 3.0;
  }

References

• VR optics and IPD

Virtual Objects Don't Match Real-World Lighting

Id

passthrough-lighting

Severity

MEDIUM

Description

AR objects look pasted on top of reality

Symptoms

• Objects look "fake"
• No shadows on real surfaces
• Lighting doesn't match room
• Objects are too bright/dark

Detection Pattern

ar|passthrough|lighting|environment

Solution

AR Lighting Integration:

  // 1. Use WebXR lighting estimation
  async function setupLightingEstimation() {
    const session = renderer.xr.getSession();
  
    if (!session.requestLightProbe) {
      console.warn('Lighting estimation not supported');
      return;
    }
  
    const lightProbe = await session.requestLightProbe();
  
    // Update lighting each frame
    function updateLighting(frame) {
      const probeState = frame.getLightEstimate(lightProbe);
  
      if (probeState) {
        // Primary light direction
        const direction = probeState.primaryLightDirection;
        directionalLight.position.set(
          direction.x,
          direction.y,
          direction.z
        );
  
        // Primary light intensity
        const intensity = probeState.primaryLightIntensity;
        directionalLight.intensity = intensity.luminance;
  
        // Ambient spherical harmonics
        if (probeState.sphericalHarmonicsCoefficients) {
          ambientLight.sh.fromArray(
            probeState.sphericalHarmonicsCoefficients
          );
        }
      }
    }
  }
  
  // 2. Shadow catcher for ground
  function createShadowCatcher() {
    const geometry = new THREE.PlaneGeometry(10, 10);
    const material = new THREE.ShadowMaterial({
      opacity: 0.3  // Adjust based on lighting
    });
  
    const plane = new THREE.Mesh(geometry, material);
    plane.rotation.x = -Math.PI / 2;
    plane.receiveShadow = true;
  
    return plane;
  }
  
  // 3. Occlusion mesh (if depth available)
  async function setupOcclusion() {
    const session = renderer.xr.getSession();
  
    // Request depth sensing
    if (session.requestDepthInformation) {
      // Create occlusion mesh from depth
      // Virtual objects will be hidden behind real objects
    }
  }

References

• AR lighting estimation

Vr Ar Development - Validations

Camera Movement Safety

Id

check-camera-movement

Description

Avoid automatic camera movement that causes motion sickness

Pattern

camera\.(position|rotation).*=|camera\.lookAt

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

controller|input|button|user

Message

Ensure camera movement is user-initiated to prevent motion sickness

Severity

error

Autofix

Frame Time Budget

Id

check-frame-budget

Description

XR requires 90fps (11ms) or better

Pattern

setAnimationLoop|render\(

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

performance|frameTime|budget

Message

Monitor frame time to ensure 11ms budget for VR

Severity

warning

Autofix

XR Renderer Enabled

Id

check-xr-enabled

Description

WebXR requires renderer.xr.enabled = true

Pattern

WebGLRenderer

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

xr\.enabled|VRButton|ARButton

Message

Enable WebXR on renderer for VR/AR support

Severity

warning

Autofix

Reference Space Configuration

Id

check-reference-space

Description

Set appropriate XR reference space

Pattern

xr\.enabled

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

setReferenceSpaceType|local-floor|bounded-floor

Message

Configure XR reference space type

Severity

info

Autofix

Tracking Loss Handling

Id

check-tracking-loss

Description

Handle tracking loss gracefully

Pattern

getViewerPose|getPose

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

null|undefined|!pose|tracking

Message

Handle null pose when tracking is lost

Severity

warning

Autofix

Controller Event Handling

Id

check-controller-events

Description

Handle XR controller input events

Pattern

xr\.getController

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

selectstart|selectend|squeeze

Message

Add event listeners for XR controller input

Severity

info

Autofix

UI Distance from Camera

Id

check-ui-distance

Description

UI should be at comfortable viewing distance

Pattern

position.-?[0-9]+\.?[0-9].z|z.=.*-?[0-9]

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

ui|panel|text|button

Message

Place UI at 0.5-2m distance for comfortable viewing

Severity

info

Autofix

VR Text Size

Id

check-text-size

Description

Text must be large enough to read in VR

Pattern

fontSize|textSize|font-size

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

0\.0[2-9]|0\.1

Message

Use minimum 0.02m (2cm) font size for VR readability

Severity

info

Autofix

Safe Locomotion Method

Id

check-locomotion

Description

Use teleportation or snap turns for comfort

Pattern

locomotion|movement|teleport|snap

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

teleport|snap|blink|instant

Message

Use teleportation or snap turns for motion sickness prevention

Severity

warning

Autofix

XR Session Event Handling

Id

check-session-events

Description

Handle XR session start and end events

Pattern

xr\.enabled

File Glob

*/.{js,ts,jsx,tsx}

Match

present

Context Pattern

sessionstart|sessionend|addEventListener

Message

Handle XR session lifecycle events

Severity

info

Autofix