Threejs Agents Scene Builder

threejs-agents-scene-builder

Purpose

This skill is an orchestration guide for building complete Three.js scenes. It provides decision trees that map use cases to the correct combination of lighting, materials, camera, controls, environment, and post-processing. ALWAYS consult this skill first when asked to create a 3D scene or visualization.

---

Master Decision Tree: Use Case to Scene Recipe

Step 1: Identify the Use Case

Step 2: Choose R3F vs Imperative

Cross-reference: threejs-impl-r3f for R3F patterns, threejs-core-scene-graph for imperative patterns.

---

Decision Tree: Lighting Setup

Step 1: Scene Type to Lighting Strategy

Step 2: Lighting Intensity (Physically Correct, r160+)

Step 3: Light Count Budget

Critical rules:

• NEVER use PointLight shadows on mobile — each costs 6 shadow map renders
• ALWAYS add directionalLight.target to the scene when repositioning the target
• ALWAYS call RectAreaLightUniformsLib.init() before creating RectAreaLight

Cross-reference: threejs-impl-lighting-shadows for shadow tuning and artifact fixes.

---

Decision Tree: Material Selection

Critical rules:

• NEVER use MeshPhysicalMaterial when MeshStandardMaterial suffices — it is significantly more expensive
• ALWAYS set material.needsUpdate = true after changing defines or extensions
• ALWAYS dispose materials when removing objects from the scene

Cross-reference: threejs-syntax-materials for material properties and textures.

---

Decision Tree: Camera Selection

Critical rules:

• ALWAYS call camera.updateProjectionMatrix() after changing fov, near, far, or aspect
• NEVER set near to 0 — this causes z-fighting across the entire scene
• ALWAYS keep the near/far ratio as small as possible (ideally < 1:10000)

Cross-reference: threejs-core-renderer-camera for camera API details.

---

Decision Tree: Controls Selection

Critical rules:

• ALWAYS call controls.update() in the animation loop when enableDamping is true
• ALWAYS call controls.dispose() on cleanup
• NEVER combine OrbitControls and TransformControls without disabling OrbitControls during transform interaction

Cross-reference: threejs-syntax-controls for controls API details.

---

Decision Tree: Environment Setup

Critical rules:

• ALWAYS use PMREMGenerator to prefilter environment maps for PBR materials
• NEVER use unprocessed HDR textures as scene.environment — they MUST be PMREM-processed
• Setting scene.environment provides IBL to ALL PBR materials automatically

Cross-reference: threejs-syntax-materials for environment map usage on materials.

---

Decision Tree: Post-Processing

When to Add Post-Processing

Post-Processing Pipeline Order

ALWAYS follow this pass order:

1. RenderPass (beauty render) — ALWAYS first 2. Geometry passes: SSAO, GTAO, SSR 3. Effect passes: Bloom, DoF, Outline 4. Anti-aliasing: FXAA or SMAA 5. Color grading: LUT, Film 6. OutputPass (tone mapping + color space) — ALWAYS last

R3F Post-Processing

In R3F, use @react-three/postprocessing (wraps pmndrs/postprocessing library):

import { EffectComposer, Bloom, SSAO } from '@react-three/postprocessing'

<EffectComposer>
  <SSAO />
  <Bloom luminanceThreshold={0.9} intensity={0.5} />
</EffectComposer>

Critical rules:

• ALWAYS resize the composer when the window resizes: composer.setSize(w, h)
• NEVER forget OutputPass at the end — without it, tone mapping and color space conversion are skipped
• In R3F, @react-three/postprocessing handles pass ordering automatically

Cross-reference: threejs-impl-postprocessing for pass configuration and tuning.

---

Scene Composition Checklist

ALWAYS follow this checklist when building a scene:

1. Renderer — Create WebGLRenderer with antialias: true, set pixel ratio, set size 2. Scene — Create Scene, set background and/or environment 3. Camera — Choose type per decision tree, set position 4. Lighting — Choose recipe per decision tree, respect budget 5. Ground/Environment — Add ground plane, environment map, or sky 6. Content — Add geometries, load models, create materials 7. Controls — Choose per decision tree, configure constraints 8. Shadows — Enable on renderer + lights + objects (3-step opt-in) 9. Post-processing — Add if needed, follow pipeline order 10. Resize handling — ALWAYS handle window resize (camera + renderer + composer) 11. Animation loop — requestAnimationFrame or R3F useFrame 12. Disposal — ALWAYS dispose geometries, materials, textures on cleanup

---

Performance Quick Reference

Cross-reference: threejs-errors-performance for detailed optimization strategies.

---

Reference Links

• references/methods.md — Decision trees with detailed selection criteria
• references/examples.md — Complete scene recipes for each use case
• references/anti-patterns.md — Common scene composition mistakes

Related Skills

Official Sources

• https://threejs.org/docs/
• https://threejs.org/examples/
• https://r3f.docs.pmnd.rs/
• https://drei.docs.pmnd.rs/

threejs-agents-scene-builder — Anti-Patterns

Lighting Anti-Patterns

AP-L01: Using AmbientLight as the Only Light Source

Wrong: Adding only AmbientLight to a scene. Why: AmbientLight illuminates all surfaces equally from all directions. The result is a flat, dimensionless scene with no shadows, no depth cues, and no visual interest. Fix: ALWAYS combine AmbientLight with at least one directional, point, or spot light to create depth through light-dark contrast.

AP-L02: Too Many Shadow-Casting Lights

Wrong: Enabling castShadow = true on every light in the scene. Why: Each shadow-casting light renders the scene from its perspective into a shadow map. PointLight shadows render 6 times (cubemap). Three PointLights with shadows = 18 extra render passes per frame. Fix: NEVER exceed 2-3 shadow-casting lights. Prefer DirectionalLight and SpotLight shadows over PointLight shadows. Use ContactShadows or AccumulativeShadows for soft ground shadows without real-time shadow maps.

AP-L03: Forgetting to Add Light Target to Scene

Wrong: Changing directionalLight.target.position without adding the target to the scene. Why: The target is an Object3D. If it is not in the scene graph, its world matrix is NEVER updated, so the light direction does not change. Fix: ALWAYS call scene.add(directionalLight.target) when repositioning the target.

AP-L04: Not Sizing DirectionalLight Shadow Frustum

Wrong: Using default shadow camera frustum bounds for DirectionalLight. Why: The default orthographic frustum is typically too large (wasting shadow map resolution) or too small (clipping shadows). This produces low-quality or missing shadows. Fix: ALWAYS manually set shadow.camera.left/right/top/bottom/near/far to tightly fit the area where shadows must appear. Use CameraHelper to visualize the frustum during development.

AP-L05: Using RectAreaLight Without Initialization

Wrong: Creating a RectAreaLight without calling RectAreaLightUniformsLib.init(). Why: RectAreaLight requires precomputed LTC (Linearly Transformed Cosine) lookup tables. Without initialization, the light produces incorrect or missing illumination. Fix: ALWAYS call RectAreaLightUniformsLib.init() once before creating any RectAreaLight (WebGL). For WebGPU, use RectAreaLightTexturesLib instead.

---

Material Anti-Patterns

AP-M01: Using MeshPhysicalMaterial Everywhere

Wrong: Defaulting to MeshPhysicalMaterial for all objects. Why: MeshPhysicalMaterial is the most expensive built-in material. It computes clearcoat, sheen, transmission, iridescence, and other effects even when not needed, wasting GPU cycles. Fix: ALWAYS use MeshStandardMaterial as the default. Only upgrade to MeshPhysicalMaterial for objects that specifically need clearcoat, transmission, sheen, or other extended PBR features.

AP-M02: Not Disposing Materials and Textures

Wrong: Removing objects from the scene without calling .dispose() on their geometry, material, and textures. Why: GPU resources (buffers, textures, shader programs) are NEVER automatically freed when objects are removed from the scene graph. This causes memory leaks that degrade performance over time. Fix: ALWAYS dispose geometry, material, and all textures when permanently removing objects. In R3F, auto-disposal handles this unless dispose={null} is set.

AP-M03: Ignoring Environment Maps for PBR Materials

Wrong: Using MeshStandardMaterial or MeshPhysicalMaterial without setting scene.environment. Why: PBR materials rely on environment maps for accurate reflections and ambient lighting. Without an environment map, metallic surfaces appear black and rough surfaces lack ambient contribution. Fix: ALWAYS set scene.environment to a PMREM-processed HDR texture when using PBR materials. In R3F, use Drei's <Environment> component.

---

Camera Anti-Patterns

AP-C01: Setting Near Plane to Zero

Wrong: new PerspectiveCamera(75, aspect, 0, 1000). Why: A near plane of 0 causes division-by-zero in the projection matrix, making the depth buffer completely non-functional. Z-fighting occurs across the entire scene. Fix: ALWAYS set near to the largest value that does not clip visible content. Typical minimum: 0.01 for small scenes, 0.1 for medium, 1 for large.

AP-C02: Excessive Near/Far Ratio

Wrong: new PerspectiveCamera(75, aspect, 0.001, 100000). Why: Depth buffer precision is distributed logarithmically between near and far. A ratio of 1:100,000,000 means almost all precision is consumed near the camera, causing z-fighting on distant objects. Fix: ALWAYS minimize the near/far ratio. Ideal: < 1:10,000. For extreme ranges (CAD/BIM), use logarithmicDepthBuffer: true on the renderer.

AP-C03: Forgetting updateProjectionMatrix

Wrong: Changing camera.fov or camera.aspect without calling camera.updateProjectionMatrix(). Why: The projection matrix is computed from fov/aspect/near/far and cached. Changing properties without updating the matrix has no visual effect. Fix: ALWAYS call camera.updateProjectionMatrix() after modifying any camera frustum property.

---

Controls Anti-Patterns

AP-K01: Forgetting controls.update() with Damping

Wrong: Enabling controls.enableDamping = true without calling controls.update() in the render loop. Why: Damping requires per-frame updates to interpolate camera movement. Without the update call, the camera freezes as soon as the user stops interacting. Fix: ALWAYS call controls.update() in every animation frame when damping or autoRotate is enabled.

AP-K02: Not Disposing Controls

Wrong: Creating new controls without disposing old ones. Why: Controls attach event listeners to the DOM element. Without disposal, old listeners accumulate, causing memory leaks and unexpected behavior (duplicate event handling). Fix: ALWAYS call controls.dispose() before creating replacement controls or when the component unmounts.

AP-K03: Conflicting Controls

Wrong: Using OrbitControls and TransformControls simultaneously without coordination. Why: Both controls respond to pointer events. Dragging a TransformControls gizmo also triggers OrbitControls rotation, producing chaotic camera movement. Fix: ALWAYS disable OrbitControls when TransformControls interaction starts, re-enable on end:

transformControls.addEventListener('dragging-changed', (event) => {
    orbitControls.enabled = !event.value;
});

---

Post-Processing Anti-Patterns

AP-P01: Missing OutputPass

Wrong: Building an EffectComposer pipeline without OutputPass at the end. Why: OutputPass handles tone mapping and color space conversion (linear → sRGB). Without it, the final image has incorrect gamma and no tone mapping, appearing washed out or oversaturated. Fix: ALWAYS add OutputPass as the last pass in the EffectComposer pipeline.

AP-P02: Forgetting to Resize the Composer

Wrong: Handling window resize by updating only the camera and renderer, not the EffectComposer. Why: The composer's internal render targets maintain their own size. Without resizing, post-processing renders at the wrong resolution, producing stretched or pixelated output. Fix: ALWAYS call composer.setSize(width, height) alongside renderer.setSize() in the resize handler.

AP-P03: Adding Post-Processing to XR Scenes

Wrong: Using EffectComposer in WebXR mode. Why: XR requires rendering to the XR session's framebuffer with strict timing constraints. EffectComposer redirects rendering to its own framebuffers, breaking the XR pipeline and causing dropped frames or black screens. Fix: NEVER use standard EffectComposer in XR mode. Use XR-compatible post-processing or skip it entirely to maintain frame rate.

---

Scene Composition Anti-Patterns

AP-S01: Not Handling Window Resize

Wrong: Setting up the renderer and camera once without a resize handler. Why: When the window resizes, the camera aspect ratio becomes incorrect (stretched image) and the renderer canvas size does not match the viewport (blurry or cropped output). Fix: ALWAYS add a resize handler that updates: (1) camera.aspect + updateProjectionMatrix(), (2) renderer.setSize(), (3) composer.setSize() if using post-processing.

AP-S02: Not Setting Pixel Ratio

Wrong: Using the default pixel ratio of 1 on high-DPI displays. Why: The scene appears blurry on Retina/HiDPI screens because the canvas renders at 1x resolution while the display is 2x or 3x. Fix: ALWAYS call renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2)). Cap at 2 to prevent excessive GPU load on 3x displays.

AP-S03: Tone Mapping on Data Visualization

Wrong: Using ACESFilmicToneMapping or other artistic tone mapping for data visualization. Why: Tone mapping remaps colors for cinematic appearance, distorting the precise color values needed for data-color mappings. A data point mapped to #FF0000 will display as a different red after tone mapping. Fix: ALWAYS use renderer.toneMapping = THREE.NoToneMapping for data visualization scenes where color accuracy matters.

AP-S04: Mixing R3F and Imperative Patterns

Wrong: Using useRef to imperatively modify Three.js objects managed by R3F's reconciler alongside declarative props. Why: R3F's reconciler sets properties on every render cycle. Imperative modifications made via refs are overwritten on the next React render, causing flickering or lost state. Fix: ALWAYS use R3F's declarative prop system for persistent state. Use useFrame for per-frame imperative updates. Use refs ONLY for reading values or for properties not managed by JSX props.

AP-S05: Loading Assets Without Suspense (R3F)

Wrong: Using useLoader or useGLTF without wrapping the component in <Suspense>. Why: These hooks use React Suspense internally. Without a Suspense boundary, the component throws an unhandled promise and the entire tree crashes. Fix: ALWAYS wrap components that use loader hooks in <Suspense fallback={<LoadingIndicator />}>.

AP-S06: Creating Geometries and Materials in Render Functions

Wrong: Creating new BoxGeometry() or new MeshStandardMaterial() inside a React component body without memoization. Why: React components re-render frequently. Each render creates new GPU resources without disposing the old ones, causing massive memory leaks and performance degradation. Fix: In R3F, declare geometry and material as JSX children (auto-managed). In imperative code within R3F, ALWAYS use useMemo to create and cache Three.js objects.

threejs-agents-scene-builder — Scene Recipes

Recipe A: Product Viewer

Scene Configuration

Imperative Setup Steps

1. WebGLRenderer({ antialias: true, alpha: true })
2. renderer.toneMapping = ACESFilmicToneMapping
3. renderer.toneMappingExposure = 1.0
4. PerspectiveCamera(40, aspect, 0.01, 100)
5. camera.position.set(0, 1, 3)
6. Load HDR environment → PMREMGenerator → scene.environment
7. SpotLight key (left, 300cd) + SpotLight fill (right, 100cd) + SpotLight rim (back, 200cd)
8. AmbientLight(0xffffff, 0.15)
9. Load GLTF model → center with Box3 → add to scene
10. OrbitControls: enableDamping, maxPolarAngle=PI/2, minDistance=1, maxDistance=10, autoRotate
11. Ground plane for shadow reception (or ContactShadows in R3F)
12. Resize handler for camera + renderer

R3F Setup Structure

<Canvas camera={{ fov: 40, position: [0, 1, 3] }} shadows>
  <Environment preset="studio" />
  <Stage preset="rembrandt" intensity={0.5} shadows="contact">
    <Model />
  </Stage>
  <OrbitControls enableDamping autoRotate enablePan={false} maxPolarAngle={Math.PI/2} />
</Canvas>

---

Recipe B: Architectural Walkthrough

Scene Configuration

Setup Steps

1. WebGLRenderer({ antialias: true })
2. renderer.shadowMap.enabled = true
3. renderer.shadowMap.type = PCFSoftShadowMap
4. renderer.toneMapping = ACESFilmicToneMapping
5. PerspectiveCamera(65, aspect, 0.1, 500)
6. camera.position.set(0, 1.6, 5) — eye height
7. HDR environment for IBL and background
8. DirectionalLight for sun: position high, shadow frustum sized to building footprint
9. PointLights for interior fixtures (no shadows or max 1-2 with shadows)
10. Load GLTF architectural model
11. Enable castShadow/receiveShadow per mesh
12. PointerLockControls for walkthrough OR OrbitControls with target at (0, 1.6, 0)
13. EffectComposer: RenderPass + GTAOPass + UnrealBloomPass(0.3) + SMAAPass + OutputPass
14. Collision detection for walkthrough mode (raycaster downward for ground)

---

Recipe C: Game Level / Interactive 3D

Scene Configuration

Setup Steps

1. WebGLRenderer({ antialias: false }) — use FXAA post-process instead
2. renderer.shadowMap.enabled = true, type = PCFSoftShadowMap
3. PerspectiveCamera(60, aspect, 0.1, 1000)
4. DirectionalLight for sun with optimized shadow frustum
5. AmbientLight or HemisphereLight for fill
6. Load level geometry (GLTF)
7. Set matrixAutoUpdate = false on static objects
8. InstancedMesh for repeated objects (trees, rocks, props)
9. Physics world initialization (Rapier)
10. Player controller with collision
11. EffectComposer: RenderPass + SSAOPass + UnrealBloomPass + FXAAPass + OutputPass
12. Game loop via requestAnimationFrame with fixed timestep for physics

---

Recipe D: Data Visualization

Scene Configuration

Setup Steps

1. WebGLRenderer({ antialias: true })
2. renderer.toneMapping = NoToneMapping — CRITICAL for accurate data colors
3. OrthographicCamera (2D) or PerspectiveCamera(50) for 3D
4. scene.background = solid color
5. AmbientLight(0xffffff, 0.6) + DirectionalLight(0xffffff, 0.8)
6. Generate geometry from data (bars, lines, points, surfaces)
7. MeshBasicMaterial with data-mapped colors (no lighting interference)
8. Drei <Text> or HTML overlays for labels and axes
9. OrbitControls with constrained angles for 3D; none for 2D
10. Resize handler

Critical rule: ALWAYS use NoToneMapping for data visualization — tone mapping distorts colors and breaks data-color mappings.

---

Recipe E: Portfolio / Hero Section

Scene Configuration

R3F Setup Structure

<Canvas camera={{ fov: 50, position: [0, 0, 5] }}>
  <Environment preset="city" background blur={0.5} />
  <Float speed={2} rotationIntensity={0.5} floatIntensity={0.5}>
    <Model />
  </Float>
  <ContactShadows position={[0, -1, 0]} opacity={0.4} blur={2} />
  <EffectComposer>
    <Bloom luminanceThreshold={0.8} intensity={0.5} />
  </EffectComposer>
</Canvas>

---

Recipe F: CAD / BIM / IFC Viewer

Scene Configuration

Setup Steps

1. WebGLRenderer({ antialias: true, logarithmicDepthBuffer: true }) — CRITICAL for CAD z-range
2. OrthographicCamera sized to model bounds
3. AmbientLight(0xffffff, 0.6)
4. DirectionalLight from top-right (0.8) + DirectionalLight from bottom-left (0.3)
5. Load IFC via web-ifc-three or GLTF export
6. Assign layers per IFC category (walls=1, floors=2, etc.)
7. OrbitControls with enableDamping, screenSpacePanning=true
8. Raycaster for element picking → OutlinePass for selection highlight
9. Fit camera to model bounds on load using Box3
10. Section planes via clipping planes (renderer.clippingPlanes)

Critical rule: ALWAYS enable logarithmicDepthBuffer for CAD/BIM scenes — standard depth buffer causes z-fighting at architectural scales.

---

Recipe G: AR / XR Experience

Scene Configuration

Setup Steps

1. WebGLRenderer({ antialias: true, alpha: true }) — alpha for AR passthrough
2. renderer.xr.enabled = true
3. Add XRButton (ARButton or VRButton from addons)
4. scene.environment from XR light estimation (AR) or HDR (VR)
5. Lightweight geometry and materials (mobile GPU)
6. Hit-test for AR placement
7. XR controller events for interaction
8. ALWAYS maintain 72-90 FPS — NEVER add heavy post-processing in XR

Critical rule: NEVER drop below 72 FPS in XR — this causes motion sickness. Budget lighting, shadows, and post-processing accordingly.

threejs-agents-scene-builder — Decision Trees Reference

Lighting Recipe Decision Matrix

Outdoor Scene Lighting

Indoor Scene Lighting

Studio / Product Lighting (Three-Point Setup)

HDR-Only Lighting (Image-Based Lighting)

Stylized / Toon Lighting

---

Material Selection Decision Matrix

By Surface Type

By Performance Tier

---

Camera Configuration Reference

PerspectiveCamera Settings by Use Case

OrthographicCamera Settings

---

Controls Configuration Reference

OrbitControls Recommended Defaults

---

Post-Processing Pipeline Reference

Pipeline by Visual Style

UnrealBloomPass Presets

SSAO/GTAO Presets

---

Environment Map Reference

Drei Environment Presets

HDR Loading Pattern (Imperative)

1. Load HDR with RGBELoader or EXRLoader
2. Create PMREMGenerator from renderer
3. Call pmremGenerator.fromEquirectangular(hdrTexture)
4. Assign result.texture to scene.environment
5. Optionally assign to scene.background
6. Dispose the original HDR texture and PMREMGenerator

NEVER skip step 3 — unprocessed HDR textures produce incorrect reflections on PBR materials.