Pmndrs Viverse

VIVERSE Three.js

Use this skill for VIVERSE-ready Three.js and React Three Fiber apps. Use @react-three/viverse for React apps and @pmndrs/viverse for vanilla Three.js.

Start Here

1. Inspect the app structure and package manager before editing. 2. Identify whether the task needs React Three Fiber or vanilla Three.js. 3. Pick the smallest relevant reference before inventing APIs. 4. Keep VIVERSE apps character-first: prefer the standard character/avatar path unless the requested controls or animation semantics require a custom controller. 5. Build game rules, sensors, UI, level geometry, and validation around normal user inputs and the visible player. 6. Validate live gameplay with the app's browser/runtime behavior, not only static state. 7. Use the bundled skill references and installed package types as the example source. For tutorial assets, use only exact asset URLs or package asset exports named by the references; you may download those binary assets into the app public/ folder, but do not fetch remote example source unless the user explicitly asks for it.

Reference Routing

• references/getting-started.md: installation, first scenes, basic examples.
• references/components-and-hooks.md: exact React component, hook, action, character, physics, and animation APIs.
• references/gameplay-quality.md: game architecture and validation heuristics for playable demos.
• references/tutorials/index.md: tutorial index. Read one focused tutorial file rather than loading all tutorials.
• references/tutorials/custom-character-controller.md: custom humanoid controller with BVH character physics, directional locomotion, camera aim, held items, and layered upper-body actions.
• references/without-react.md: vanilla Three.js usage.
• references/publishing.md: VIVERSE CLI, build output, app creation, publishing.

If unsure which reference applies, search first:

rg -n "SimpleCharacter|BvhPhysicsBody|avatar|actions|XR|publish" path/to/skill/references

Architecture Defaults

• Wrap VIVERSE-aware React scenes in <Viverse>.
• Use <SimpleCharacter /> for ordinary embodied React gameplay and new SimpleCharacter(...) for ordinary vanilla Three.js gameplay.
• Keep the standard character visibly embodied; do not hide it or use model={false} unless the user asked for a placeholder or non-avatar actor.
• Put collidable level geometry in BvhPhysicsBody; dynamic blockers should share the same collision truth used by character movement and game logic.
• Tune movement, camera, physics, animation, and input through supported character options and action bindings before reaching for low-level controller code.
• Choose the custom controller tutorial when a game needs custom model/clip stacks, directional strafe/backpedal clips, camera-relative aiming, held weapon actions, reload/shoot layers, or other animation semantics beyond SimpleCharacter.
• Treat third-person shooter, battle royale, Fortnite-style, and action-combat prompts as custom-controller tasks. Use the architecture from references/tutorials/custom-character-controller.md: VIVERSE character physics, model/provider, bone attachments, and animation actions/layers.
• Do not satisfy those prompts with only <Viverse>, BvhPhysicsBody, and a hand-rolled mesh/capsule player. The controllable player itself must use the VIVERSE character/controller/model/action/animation primitives.
• Use a loaded VIVERSE character model for custom controllers, such as useCharacterModelLoader or loadCharacterModel. Do not fabricate the player model by casting a Group of boxes/cylinders/spheres to CharacterModel.
• A custom combat controller is incomplete if it collapses the animation setup to an idle layer plus counters or weapon-only transforms. Adapt the reference lower-body and upper-body timelines, or an equivalent action-driven character/bone pose system, so movement, aim, attack, and reload visibly affect the character while the action is active.
• For held weapons, tools, lights, or props, attach a loaded asset (<Gltf />, useGLTF, or equivalent) under CharacterModelBone; primitive boxes/cylinders are placeholders, not a finished held item, when an asset or tutorial model exists.
• Crosshair shooting should use the full camera/player-view ray, including pitch; do not flatten the aim ray or choose a nearest target just because it is in front of the player. Validate at least one off-crosshair miss and one crosshair hit.
• Directional combat movement should visibly distinguish side/back/diagonal movement from forward movement through directional clips, animation weights, bone/model pose, or equivalent player-visible feedback.
• For games or interactive demos, read references/gameplay-quality.md before final validation.
• For greenfield apps, ask the package manager for current published versions and install a compatible set for react, react-dom, three, @react-three/fiber, @react-three/drei, @react-three/viverse, and companion packages such as @react-three/timeline; do not copy stale tutorial dependency caps or invent future ranges.
• If an existing app already uses an older compatible React/R3F stack, preserve that stack and add packages that match it.
• For TypeScript apps, include matching type packages such as @types/react, @types/react-dom, and @types/three.

Constraints

• Do not set a clientId during local development unless the user explicitly asks for authenticated VIVERSE behavior.
• Do not ask for VIVERSE passwords, tokens, client secrets, or other credentials in prompts.
• Static BVH physics content inside BvhPhysicsBody and BvhPhysicsSensor should not structurally change after creation; use stable groups and visibility toggles when needed.
• When publishing, build first, then follow references/publishing.md.

Before Finishing

Check the essentials:

• The app uses the intended VIVERSE runtime/library path for its framework.
• The player is visible, embodied, and driven by normal input/action bindings.
• Collision, sensors, shots, pickups, checkpoints, or blockers use one coherent gameplay truth.
• Meaningful mechanics are validated through live browser play, with assertions that would fail if the feature were missing.
• The saved vitexec/play.ts route reuses the same helpers and milestones proven by any disposable probes.
• The recorded route shows gameplay after readiness and can pass again without changing the app or weakening the route.

<a id="doc-getting-started-all-components-and-hooks"></a>

All Components and Hooks

Components

<Viverse>

The main provider component that sets up VIVERSE authentication and physics context. Must wrap your entire application or the parts that use VIVERSE features.

Props:

• children?: ReactNode - Child components
• loginRequired?: boolean - Forces user to login before playing (default: false)
• clientId?: string - VIVERSE app client ID. Typically you pass this from your app’s environment (e.g. a VITE_VIVERSE_APP_ID env var you manage) into this prop.
• domain?: string - Authentication domain (default: 'account.htcvive.com')
• authorizationParams?: object - Additional authorization parameters
• cookieDomain?: string - Cookie domain for authentication
• httpTimeoutInMS?: number - HTTP request timeout in milliseconds

[!WARNING]

Don't set the clientId during local development!

Example:

<Viverse loginRequired={true} clientId="your-app-id">
  <YourGame />
</Viverse>

<SimpleCharacter>

Creates a simple character controller with physics based on three-mesh-bvh, walking, running, jumping animations, and camera controls. Automatically uses the active VIVERSE avatar if authenticated.

Props: See SimpleCharacter Options section below for complete details.

Example:

<SimpleCharacter walk={{ speed: 3 }} run={{ speed: 6 }} jump={{ speed: 10 }}>
  {/* Optional child components */}
</SimpleCharacter>

<BvhPhysicsWorld>

Provides physics context for collision detection. Usually wrapped automatically by <Viverse>, but can be used standalone.

Props:

• children?: ReactNode - Child components

<BvhPhysicsBody>

Adds visible children as static (non-moving) or kinematic (moving) objects as obstacles to the physics world.

[!WARNING]

Content inside the object can not structurally change.

Props:

• children?: ReactNode - Static mesh objects for collision
• kinematic?: boolean - whether the objects world transformation can change - default: false

Example:

<BvhPhysicsBody>
  <mesh>
    <boxGeometry />
    <meshStandardMaterial />
  </mesh>
</BvhPhysicsBody>

<BvhPhysicsSensor>

Adds visible children as sensors that detect player intersection and trigger callbacks (does not add obstacles).

[!WARNING]

Content inside the object can not structurally change; Hiding the sensors content requires to wrap it in <group visible={false}>...</group>.

Props:

• children?: ReactNode - Static mesh objects for collision
• isStatic?: boolean - whether the objects world transformation is static - default: true
• onIntersectedChanged?: (intersected: boolean) => void - callback that get's called when the player starts or stops intersecting with the sensor

Example:

<BvhPhysicsSensor onIntersectedChanged={(intersected) => console.log('currently intersected', intersected)}>
  <mesh visible={false}>
    <boxGeometry />
  </mesh>
</BvhPhysicsSensor>

<PrototypeBox>

A quick prototyping component that renders a textured box with the prototype material.

Props:

• color?: ColorRepresentation - Box color tint
• All standard Three.js Group props (position, rotation, scale, etc.)

Example:

<PrototypeBox position={[0, 1, 0]} scale={[2, 1, 3]} color="red" />

<CharacterModelProvider>

Provides the active character model context so that animation and bone utilities can target the same model instance. Wrap any content that uses <CharacterAnimationAction>, <AdditiveCharacterAnimationAction>, <CharacterAnimationLayer>, or <CharacterModelBone>.

Props:

• model: CharacterModel - Model returned by useCharacterModelLoader
• children?: ReactNode - Child components

Example:

const model = useCharacterModelLoader({
  url: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/avatar.vrm',
  castShadow: true,
})

return (
  <CharacterModelProvider model={model}>
    <RunTimeline>
      <CharacterAnimationAction url="https://raw.githubusercontent.com/pmndrs/viverse/main/packages/viverse/assets/idle.glb" />
    </RunTimeline>
    <primitive object={model.scene} />
  </CharacterModelProvider>
)

<CharacterAnimationLayer>

Defines a logical animation layer (e.g., "lower-body", "upper-body"). All nested animation actions inherit this layer unless they provide their own layer prop. Layers allow to manage animations when managing e.g. additive animations or animations with masks.

Props:

• name: string | Array<string | undefined> | undefined - Layer name
• children?: ReactNode - Nested timeline/animation content

Example:

<RunTimeline>
  <CharacterAnimationLayer name="lower-body">
    <CharacterAnimationAction
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/packages/viverse/assets/walk.glb"
      mask={lowerBodyMask}
    />
  </CharacterAnimationLayer>
  <CharacterAnimationLayer name="upper-body">
    <AdditiveCharacterAnimationAction
      referenceClip={{
        url: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-forward.glb',
      }}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-idle.glb"
      mask={upperBodyMask}
    />
  </CharacterAnimationLayer>
  <primitive object={model.scene} />
  {/* ...lights, environment... */}
  {/* masks can be created with @pmndrs/viverse animation masks */}
</RunTimeline>

<CharacterAnimationAction>

Loads and plays a clip on the active character model, integrating with @react-three/timeline for lifecycle and transitions. Supports masking, cross-fading, syncing, and layering. The ref exposes the underlying Three.js AnimationAction.

Props:

• Clip options (from @pmndrs/viverse):
• url: string | DefaultUrl - Source of the animation
• type?: 'mixamo' | 'gltf' | 'vrma' | 'fbx' | 'bvh'
• removeXZMovement?: boolean
• trimTime?: { start?: number; end?: number }
• boneMap?: Record<string, VRMHumanBoneName>
• scaleTime?: number
• mask?: CharacterAnimationMask - Limit animation to specific bones/regions
• Playback and blending:
• fadeDuration?: number - Cross-fade/fade time (default: 0.1)
• crossFade?: boolean - Whether to cross-fade from current layer action (default: true)
• sync?: boolean - Sync time with current action on same layer (if any)
• paused?: boolean
• loop?: AnimationActionLoopStyles - Defaults to LoopRepeat
• layer?: string | Array<string | undefined> - Overrides the current <CharacterAnimationLayer>
• Timeline control (from @react-three/timeline):
• init?(): void | (() => void) - Called when the action starts; return a cleanup
• update?(state, delta): void - Per-frame update
• until?(): Promise<unknown> - Resolve to stop; defaults to when the clip finishes
• dependencies?: unknown[] - Re-run when any dependency changes
• Advanced:
• additiveReferenceClip?: AnimationClip - Use an additive version of the clip relative to this reference clip (prefer <AdditiveCharacterAnimationAction> for convenience)

Example:

<CharacterAnimationAction url="https://raw.githubusercontent.com/pmndrs/viverse/main/packages/viverse/assets/idle.glb" />

<AdditiveCharacterAnimationAction>

Convenience wrapper around <CharacterAnimationAction> that plays an additive version of the clip, using a provided reference pose/clip (e.g., aim offsets layered over locomotion).

Props:

• All <CharacterAnimationAction> props, except it uses:
• referenceClip: CharacterAnimationOptions - Clip used as the additive reference pose

Example:

<AdditiveCharacterAnimationAction
  referenceClip={{ url: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-forward.glb' }}
  url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-reload.glb"
  mask={upperBodyMask}
/>

<CharacterModelBone>

Component for placing content inside the character model at specific bones.

Props:

• bone: VRMHumanBoneName - The bone name to access

<SimpleCharacter>
  <CharacterModelBone bone="rightHand">
    <SwordModel />
  </CharacterModelBone>
</SimpleCharacter>

Hooks

[!NOTE]

useViverseClient() returns undefined if not within a <Viverse> provider or if no clientId is provided. Also all avatar-related hooks return undefined when the user is not authenticated.

useIsMobile

Lightweight media-query based mobile detection. It subscribes to @media (hover: none) and (pointer: coarse).

import { useIsMobile } from '@react-three/viverse'

function MobileOnlyUI() {
  const isMobile = useIsMobile()
  return isMobile ? <div>Shown on mobile</div> : null
}

Action Binding Hooks

Actions allow to decouple specific user inputs from game/business logic. One action can be connected to multiple inputs via Action bindings (keyboard, mouse/touch, on-screen controls). For background on actions vs. action bindings and how to create custom ones, see the Actions tutorial: Create and use actions. We provide several easy to use hooks to setup default action bindings for general use cases such as pressing a button or specific use case such as locomotion using a keyboard.

useKeyboardActionBinding(action, options)

• Description: Binds a KeyboardEvent or boolean state action to one or more keys.
• Options: { keys: string[]; requiresPointerLock?: boolean }
• Returns: void

useKeyboardActionBinding(jumpAction, { keys: ['Space'] })

usePointerButtonActionBinding(action, options)

• Description: Binds a pointer button (mouse/touch) event or state action.
• Options: { domElement?: HTMLElement | RefObject<HTMLElement>; buttons?: number[]; requiresPointerLock?: boolean }
• Returns: void

usePointerButtonActionBinding(fireAction, { buttons: [0] }) // left mouse / primary touch

usePointerCaptureRotateZoomActionBindings(options)

• Description: Enables rotate/zoom camera controls using Pointer Capture on the canvas.
• Options: { rotationSpeed?: number; zoomSpeed?: number }
• Returns: void

usePointerCaptureRotateZoomActionBindings({ rotationSpeed: 1000, zoomSpeed: 1000 })

usePointerLockRotateZoomActionBindings(options)

• Description: Enables rotate/zoom camera controls using Pointer Lock on the canvas.
• Options: { rotationSpeed?: number; zoomSpeed?: number; lockOnClick?: boolean }
• Returns: void

usePointerLockRotateZoomActionBindings({ lockOnClick: true })

useKeyboardLocomotionActionBindings(options)

• Description: WASD movement, Shift to run, Space to jump.
• Options: { moveForwardKeys?, moveBackwardKeys?, moveLeftKeys?, moveRightKeys?, runKeys?, jumpKeys?, requiresPointerLock? } (arrays of key strings)
• Returns: void

useKeyboardLocomotionActionBindings({ requiresPointerLock: false })

useScreenJoystickLocomotionActionBindings(options)

• Description: On-screen joystick for movement and run on mobile devices.
• Options: { runDistancePx?: number; deadZonePx?: number }
• Returns: void

useScreenJoystickLocomotionActionBindings({ deadZonePx: 8, runDistancePx: 40 })

SimpleCharacter Options

The SimpleCharacter component can be configured with a variety of props but also supports all the default group props, such as position, rotation, and scale.

useViverseAvatar flag

Allows to configure whether the users vrm avatar should be displayed as the character model.

• Default: true

movement Options

• walk: object | boolean - Enable walking (default: true)
• speed: Movement speed in units per second (default: 3)
• Set to false to disable walking

• run: object | boolean - Enable running (default: true)
• speed: Running speed in units per second (default: 6)
• Set to false to disable running

• jump: object | boolean - Enable jumping (default: true)
• delay: Time before jump starts in seconds (default: 0.2)
• bufferTime: Jump input buffer time in seconds (default: 0.1)
• speed: Jump velocity in units per second (default: 8)
• Set to false to disable jumping

actionBindings Options

An array of action binding classes to instantiate for handling controls

• Default: [ScreenJoystickLocomotionActionBindings, ScreenButtonJumpActionBindings, PointerCaptureRotateZoomActionBindings, KeyboardLocomotionActionBindings]
• Configure action bindings with custom action binding classes

Available Action Binding Classes provided by @pmndrs/viverse:

• KeyboardLocomotionActionBindings - WASD movement, Space for jump, Shift for run
• PointerCaptureRotateZoomActionBindings - Mouse look with pointer capture (requires manual setPointerCapture)
• PointerLockRotateZoomActionBindings - Mouse look with pointer lock (requires manual requestPointerLock)
• ScreenJoystickLocomotionActionBindings - On-screen joystick for movement and run (mobile). Options: { screenJoystickDeadZonePx?, screenJoystickRunDistancePx? }
• ScreenButtonJumpActionBindings - On-screen jump button (mobile-only). Visible only on mobile.

actionBindingOptions Options

Fine-tune the default action binding classes created by SimpleCharacter. These options are applied to any active bindings that support them (including your custom actionBindings if they expose the same properties).

• screenJoystickDeadZonePx: number - Inner dead zone radius in pixels for the on-screen joystick (default: 24)
• screenJoystickRunDistancePx: number - Distance from center (px) after which the joystick toggles run (default: 46)
• pointerCaptureRotationSpeed: number - Rotation speed multiplier for Pointer Capture look (default: 0.4)
• pointerCaptureZoomSpeed: number - Zoom speed multiplier for Pointer Capture (default: 0.0001)
• pointerLockRotationSpeed: number - Rotation speed multiplier for Pointer Lock look (default: 0.4)
• pointerLockZoomSpeed: number - Zoom speed multiplier for Pointer Lock (default: 0.0001)
• keyboardRequiresPointerLock: boolean - If true, keyboard input only works while the canvas has pointer lock (default: false)
• keyboardMoveForwardKeys: string[] - KeyboardEvent.code values for moving forward (default: ['KeyW'])
• keyboardMoveBackwardKeys: string[] - Keys for moving backward (default: ['KeyS'])
• keyboardMoveLeftKeys: string[] - Keys for moving left (default: ['KeyA'])
• keyboardMoveRightKeys: string[] - Keys for moving right (default: ['KeyD'])
• keyboardRunKeys: string[] - Keys for run modifier (default: ['ShiftRight','ShiftLeft'])
• keyboardJumpKeys: string[] - Keys for jump (default: ['Space'])

[!NOTE]

Key arrays use KeyboardEvent.code strings (e.g., 'KeyW', 'ArrowUp'), not key values.

Example:

<SimpleCharacter
  actionBindingOptions={{
    keyboardRequiresPointerLock: true,
    keyboardMoveForwardKeys: ['KeyW', 'ArrowUp'],
    keyboardRunKeys: ['ShiftLeft'],
    pointerLockRotationSpeed: 0.5,
    pointerLockZoomSpeed: 0.0002,
    screenJoystickDeadZonePx: 16,
    screenJoystickRunDistancePx: 40,
  }}
/>

model Options

• url: string - URL to VRM or GLTF model file
• type: "gltf" | "vrm" - the type of file to be loaded (optional)
• boneMap: Record<string, VRMHumanBoneName> - Mapping of model bone names to standard VRM bone names
• castShadow: boolean - Enable shadow casting (default: true)
• receiveShadow: boolean - Enable shadow receiving (default: true)
• boneRotationOffset: Quaternion | undefined - Allows to apply an rotation offset when placing objects as children of the character's bones (default: undefined)
• Set to false to disable model loading
• Set to true or omit to use default robot model

physics Options

• capsuleRadius: number - Character collision capsule radius (default: 0.4)
• capsuleHeight: number - Character collision capsule height (default: 1.7)
• gravity: number - Gravity acceleration in m/s² (default: -20)
• linearDamping: number - Air resistance coefficient (default: 0.1)
• maxGroundSlope: number - Max slope for a collider to be detected as walkable (default: 0.5)

cameraBehavior Options

• collision: object | boolean - Enable camera collision (default: true)
• offset: number - Collision offset distance (default: 0.2)

• characterBaseOffset: Vector3 | [number, number, number] - Camera position relative to character (default: [0, 1.3, 0])

• rotation: object | boolean - Enable camera rotation (default: true)
• minPitch: number - Minimum pitch angle (default: -Math.PI/2)
• maxPitch: number - Maximum pitch angle (default: Math.PI/2)
• minYaw: number - Minimum yaw angle (default: -Infinity)
• maxYaw: number - Maximum yaw angle (default: +Infinity)
• speed: number - Rotation speed multiplier (default: 1000)

• zoom: object | boolean - Enable camera zoom (default: true)
• speed: number - Zoom speed multiplier (default: 1000)
• minDistance: number - Minimum camera distance (default: 1)
• maxDistance: number - Maximum camera distance (default: 7)

animation Options

• yawRotationBasedOn: 'camera' | 'movement' - Character rotation basis (default: 'movement')
• maxYawRotationSpeed: number - Maximum rotation speed (default: 10)
• crossFadeDuration: number - Animation blend time in seconds (default: 0.1)

The SimpleCharacter uses the following animations walk, run, idle, jumpForward, jumpUp, jumpLoop, jumpDown each with the following options:

• url: string - Animation file URL
• type: 'gltf' | 'vrma' | 'fbx' | 'bvh' - Animation file type (optional)
• boneMap - Allows to map the bone names of the animation amature to the standard VRM bone names
• removeXZMovement: boolean - Remove horizontal movement from animation
• trimTime: { start?: number; end?: number } - Trim animation timing
• scaleTime: number - Scale animation playback speed

PrototypeMaterial

The <prototypeMaterial> component provides a textured material for prototyping using kenney.nl's prototype texture.

• color: ColorRepresentation - Material color tint
• repeat: Vector2 - Texture repeat pattern (accessible as materialRef.current.repeat)
• All standard Three.js MeshPhongMaterial properties

// As JSX element
<mesh>
  <boxGeometry />
  <prototypeMaterial color="blue" />
</mesh>

Gameplay Quality

Use VIVERSE as the runtime and build games around a visible embodied player. Prefer the standard character for ordinary locomotion; choose a custom controller only when the requested mechanics need custom clips, layered animations, held items, camera-relative aim, or lower-level physics behavior.

Mechanics

• Drive play through normal input bindings and actions. Use state reads for observation and assertions, not to bypass interaction.
• If built-in input bindings do not fit the game or validation environment, a custom input component is fine when it is the real user-facing control path and writes the same VIVERSE actions or character inputs the player uses. Do not add hidden test-only controls.
• Keep one gameplay truth for collision and interaction. Objects that block, trigger, damage, score, or get collected should participate in the same physics, sensor, raycast, or state path used by the player and validation.
• If you replace an unreliable sensor or collider with an explicit range/raycast rule, remove or restyle the old sensor so the scene does not present one interaction truth while validation uses another.
• If an object is presented as a wall, cover, platform, obstacle, door, hazard, or physical blocker, do not make it only visual or only a state/raycast rule. It should affect the player through the expected movement, physics, or sensor path unless the design clearly marks it as non-solid.
• If a dynamic physical door or gate is not robust, redesign it as a clearly non-solid activation effect, portal state, or light barrier; do not present it as a player-blocking door/gate unless collision is part of the same validated gameplay path.
• If validating contact with a moving obstacle or blocker, attach the evidence to that moving object, its sensor, or the collision path it uses. Do not prove moving-obstacle contact with a separate broad static zone.
• For BvhPhysicsSensor, use real trigger geometry that remains part of the BVH, and keep decorative meshes outside the sensor body. Avoid fully removed or hidden trigger meshes; prefer simple visible or nearly transparent boxes and prove intersection behavior with a preflight.
• Make mechanics visible in the scene. Counters alone do not prove pickups, shots, checkpoints, damage, building, stealth, racing, or victory.
• When a held tool, weapon, beam, or light causes a mechanic, compute that mechanic from the tool's world transform, aim ray, or light volume. Do not label a plain player-radius proximity check as torch reveal, weapon range, or tool use.
• Held weapons, tools, torches, and props should be loaded models when an asset exists or the tutorial provides one; primitive boxes/cylinders are only acceptable as explicit placeholders, not as finished character equipment.
• For pickups, stations, switches, doors, portals, badges, and completion effects, validate a mechanic-specific visual change such as object visibility/removal, material/emissive change, pose/scale/transform change, portal opening, or a focused screenshot/pixel region. HUD text, state flags, and generic nonblank-canvas checks are not enough by themselves.
• For sensors placed on a traversal path, prefer flat floor-aligned pads or planes over tall trigger volumes; route-test that the sensor does not catch or block the character while still triggering the mechanic.
• Keep the player/controller subtree stable while gameplay state changes. Avoid per-frame React state updates for character position or validation telemetry that remount/rebind input; use refs, external stores, or throttled snapshots instead.
• Keep static level collision mounted independently from suspense-heavy character models or asset loaders, so character physics does not simulate before the ground and blockers exist.
• If movement animation matters, drive actual animation actions, layers, masks, bones, or model transforms from movement/action values instead of setting labels only. If a movement probe samples idle while movement input is held, fix the animation/pose path before final validation.

Common Game Shapes

• Obstacle course, platformer, parkour: use the standard character unless custom movement is the point; validate traversal, jumps, falls or hazards, checkpoints, and finish.
• Racing, time trial, checkpoint chase: validate route progression, checkpoint order, timing or score, collisions, and finish state.
• Collection or exploration: validate navigation to multiple pickups/areas, pickup visibility/removal, scoring, and completion.
• Physics puzzles: validate object movement through the same collision/sensor path the player uses, switch or pressure-plate activation, gate/key state, and exit completion.
• Social hubs or guided spaces: validate embodied navigation plus real interactions with NPCs, stations, emote/action pads, portals, badges, or tour milestones.
• Dark exploration or flashlight/torch games: attach the light/tool to the player or hand, keep the environment genuinely low-light, and validate that the held light itself visibly reveals paths, objects, hazards, or objectives during movement. In vitexec/play.ts, assert active visual evidence such as objective/path mesh visibility, scale, opacity, or material emissive values while the torch is revealing it.
• Combat or shooter: aim should come from camera/crosshair/player view; weapon feedback should be visible through held item, projectile/tracer, muzzle flash, impact, recoil, or target reaction.
• Third-person shooters, battle royale, and action-combat games should use the custom character controller architecture, including VIVERSE character physics, model/provider, bone attachments, and animation layers, when the game needs camera aim, directional combat movement, held weapons/tools, or attack/reload actions.
• Custom humanoid controllers should load a character model through VIVERSE model helpers. A homemade humanoid assembled from primitive meshes is not a substitute for the character model.
• When combat has aim, attack, reload, cast, or tool-use actions, connect those actions to visible weapon/tool pose, recoil, muzzle, upper-body, or character-layer feedback and validate the active frames, not only ammo or hit counters.
• For held weapons/tools under CharacterModelBone, use the hand attachment convention from the tutorials: local +X is side offset, local +Y is vertical/grip-up, and local -Z is forward/muzzle/aim direction. The tutorial pistol.glb has its barrel authored along model +Y, so rotation-x={-Math.PI / 2} maps it to attachment -Z; if another glTF is authored on a different barrel axis, rotate the wrapper so its muzzle also points -Z.
• For custom humanoid combat controllers, keep aim poses and tool overlays as separate animation layers: blend aim-up/forward/down with normal CharacterAnimationAction, then layer weapon/tool idle, shoot, reload, cast, or use clips with AdditiveCharacterAnimationAction against the forward aim reference pose.
• For recorded gameplay, pace the final route so a viewer can see each mechanic happen; do not compress traversal, aiming, attacks, reloads, blockers, and victory into a rapid state-machine run.
• Keep full gameplay routes representative, not repetitive. Prove each major mechanic with the smallest meaningful count, such as one miss, one block, one pickup, a few successful interactions, and completion, rather than adding extra targets or waypoints that only increase route brittleness.
• Do not satisfy minimum recording duration by idling after completion or victory. Pace traversal and mechanic dwell time before completion so the video shows representative play throughout.
• Shooter validation should prove aim fidelity: an off-crosshair shot should miss or be blocked, and a correctly aimed shot should hit. Avoid flattened aim rays or nearest-target hit selection unless the game explicitly has lock-on targeting.
• For camera-aim validation, use the same aim input a player would use. Prefer pointer/mouse-look controls for shooters; if keyboard aim is also a real control path, set its sensitivity and target margins before the first full-route rehearsal so precise crosshair hits do not require post-rehearsal control retuning.
• If the saved route needs adaptive crosshair aiming, expose observable screen-space aim evidence such as target projection, aim error, or camera ray distance during implementation/preflight. Do not add new aiming telemetry after the first full-route rehearsal.
• Match saved-route aim tolerances to the actual gameplay hit volume. Do not make the validation helper stricter than the shot rule; if the camera ray is already inside the visible hit volume, take the shot and assert the gameplay result instead of chasing exact screen-center alignment.
• Keep combat routes and targets inside generous camera-control margins. Do not place required hits so close, high, low, or occluded that the validation route must aim at pitch/yaw limits or thread a narrow collision gap.
• Keep the complete validation route playable rather than precision-scripted. Preflight the finish/completion path before the first full-route rehearsal, and place the final objective so it remains reachable with ordinary movement after the last major mechanic.
• Directional movement validation should prove visible side/back/diagonal feedback when the game uses combat strafing, not just that the player position changes sideways.
• Building or dynamic cover: build previews may be visual, but placed objects that block the player or shots must become collidable or otherwise share the gameplay collision path.
• AR, VR, avatar, publishing, or vanilla Three.js tasks: route to the specific tutorial/reference for that topic.

Validation

• Create a vitexec script for repeatable gameplay validation when the task asks for a playable game or demo.
• Use sustained, human-like input over real frames/time; include early, middle, and final milestones.
• Assert the core mechanics directly. Examples: checkpoint order, pickup count, hazard collision, lap completion, target hit, blocked shot, player-body collision, animation/pose evidence, score/victory.
• For collection and interaction games, pair state assertions with visible before/after evidence for representative pickups, activated stations, opened gates, or completion effects.
• Match validation to the game shape. A good route for a parkour course proves traversal and hazards; a racing route proves checkpoint order and timing; a puzzle route proves object/switch/gate causality; a social route proves actual station/NPC/pad interactions.
• Include browser visual evidence when visual behavior matters: canvas pixels, screenshot, recording, WebGL state, or an active-frame visual probe.
• For visual mechanics, capture evidence during the active mechanic window: before/after pixels, focused screenshot regions, or actual rendered object material/visibility state read from refs or the Three scene. A final screenshot plus computed snapshot flags is not enough.
• Before freezing a final route, use disposable probes to learn readiness, focus target, input signs, timing, and visual evidence windows.
• Save long route rehearsals as vitexec/rehearsal.ts with bounded per-step loops and milestone logs; avoid huge inline vitexec scripts for full gameplay routes. Once the rehearsal passes, copy the same route and assertions into vitexec/play.ts.
• For traversal-heavy games, probe representative segments before a full-route rehearsal: spawn settling, each jump or narrow obstacle type, hazard/reset behavior, checkpoint sensors, and finish trigger. If a segment is physically brittle, make the level more playable rather than baking a fragile route.
• Once a complete rehearsal works, save that same route into vitexec/play.ts and run the saved file by path. Reuse the same assertion timing, thresholds, helpers, and evidence checks; do not add stricter active-frame assertions while freezing the file unless those exact checks already passed in rehearsal.
• After copying a passing rehearsal into vitexec/play.ts, run vitexec/play.ts next; do not insert extra smoke, screenshot, or one-off probes between the proven route and the saved-route run.
• After a full-route rehearsal starts, avoid new one-off probes. If repeated rehearsal reruns keep failing on route precision, report the validation shortcoming or simplify the route before restarting from preflight.
• Treat navigation, dev-server reconnects, page errors, or frozen video during recording as failed visual evidence; produce a clean replay or report the limitation.
• Keep final visual assertions stable under recording replay. Prefer screenshots, recordings, WebGL state, or app-visible state over fragile canvas readback unless the canvas is configured and proven reliable.
• Run the production app build before the first full route. If the vitexec script uses browser-root imports such as /src/..., keep it outside the app TypeScript build before final validation starts.
• Finalize package scripts, dependencies, and TypeScript config before the first full route. After that route runs, do not edit the app or validation setup except to report a limitation.
• After the first full-route rehearsal starts, keep gameplay geometry, target sizes, target positions, win conditions, route intent, aim criteria, evidence criteria, and assertion meanings stable. Rehearsal input timing/helper fixes are acceptable if they preserve the same route and assertions; app/game retuning is not. Do not widen targets, move goals, loosen hit tolerances, or retune thresholds just to pass.
• Start recording after the app is ready and the camera has settled enough to show the avatar, world, and active mechanics.

<a id="doc-getting-started-index"></a>

Introduction

npm install three @react-three/fiber @react-three/viverse

What does it look like?

A prototype map with the <SimpleCharacter/> component and its default model

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10'
    }

Files:

File: /App.tsx

import { Sky } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse'

export default function App() {
  return (
    <Canvas shadows style={{ position: "absolute", inset: "0", touchAction: "none" }}>
      <Viverse>
        <Sky />
        <directionalLight intensity={1.2} position={[-10, 10, -10]} castShadow />
        <ambientLight intensity={1} />
        <SimpleCharacter />
        <BvhPhysicsBody>
          <PrototypeBox scale={[10, 1, 15]} position={[0, -0.5, 0]} />
        </BvhPhysicsBody>
      </Viverse>
    </Canvas>
  )
}

How to get started

Some familiarity with

react, threejs, and @react-three/fiber, is recommended.

Get started with [building a simple game](#doc-tutorials-simple-game), take a look at our [examples](#doc-getting-started-examples), or follow one of our tutorials:

• First person controls
• Augmented and virtual reality
• Accessing avatar and profile
• Equipping the character with items
• Using custom animations and models
• Actions
• Custom Character Controller
• How to remove the viverse integrations
• Publish to VIVERSE
• Vibe coding with @react-three/viverse (using AI)

Not into react?

No Problem

Check out how to build games using @pmndrs/viverse and only vanilla three.js.

Acknowledgments

This project would not be possible without the default model and default animations made by Quaternius, the prototype texture from kenney.nl, and the three-vrm project from the pixiv team!

<a id="doc-getting-started-examples"></a>

Examples

<Grid cols={2}> <li>  Simple Game Example w. a Player Tag </li> <li>  Simple Game Example using Vanilla Threejs </li> <li>  Augemented Reality Example using WebXR </li> <li>  Virtual Reality Example using WebXR </li> <li>  Fortnite Character Controller Example </li> </Grid>

<a id="doc-tutorials-publish-to-viverse"></a>

Publish to VIVERSE

Prerequisites

• Node.js version 22 or higher installed
• A VIVERSE account (create one at viverse.htcvive.com)

Step 1: Install the VIVERSE CLI

Install the official VIVERSE command-line interface:

npm install -g @viverse/cli

Step 2: Authenticate with VIVERSE

Before you can create apps or deploy, you need to authenticate with your VIVERSE account:

viverse-cli auth login -e your-email -p your-password

Step 3: Create a VIVERSE App

Create a new app entry in the VIVERSE platform:

viverse-cli app create

After creation, note the App ID - you'll need this for deployment.

Step 4: Configure Your App ID

Next, we need to provide the App ID to our VIVERSE component.

[!TIP]

Do not include the app ID in your local development environment. Keep it production-only to avoid conflicts during development.

Create a production environment file (.env.production) in your project root.

# .env.production
VITE_VIVERSE_APP_ID=your-app-id-here

This allows you to provide the app ID to your VIVERSE component using the environment variable VITE_VIVERSE_APP_ID

<Viverse clientId={import.meta.env.VITE_VIVERSE_APP_ID}>
  <YourGame />
</Viverse>

This only works when using vite. If you don't use vite you need to manually make sure the appId is provided to the VIVERSE clientId in the production build.

Step 5: Build Your Application

Build your application for production. The exact command depends on your build tool. For vite you need to run vite build.

Step 6: Deploy to VIVERSE

Deploy your built application to the VIVERSE platform:

viverse-cli app publish your-build-output-directly-here --app-id your-app-id-here

The CLI now shows you the URL with which you can preview your game in VIVERSE and how to submit it for review.

<a id="doc-tutorials-access-avatar-and-profile"></a>

Accessing Avatar and Profile

This tutorial shows you how to display a player tag above the character by accessing the user profile information from VIVERSE.

_Here's a preview of what we'll build in this tutorial:_

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10',
      "@react-three/uikit": "^1.0.41"
    }

Files:

File: /Playertag.tsx

import { useViverseProfile } from '@react-three/viverse'
import { Container, Image, Text } from '@react-three/uikit'
import { useRef } from 'react'
import { useFrame } from '@react-three/fiber'
import { Group } from 'three'

export function PlayerTag() {
  const profile = useViverseProfile() ?? {
    name: 'Anonymous',
    activeAvatar: { headIconUrl: 'https://picsum.photos/200' },
  }
  
  const ref = useRef<Group>(null)
  
  // Make the tag always face the camera
  useFrame((state) => {
    if (ref.current == null) {
      return
    }
    ref.current.quaternion.copy(state.camera.quaternion)
  })

return (

<group ref={ref} position-y={2.15}>
  <Container
    depthTest={false}
    renderOrder={1}
    borderRadius={10}
    paddingX={2}
    height={20}
    backgroundColor="rgba(255, 255, 255, 0.5)"
    flexDirection="row"
    alignItems="center"
    gap={4}
  >
    <Image
      width={16}
      height={16}
      borderRadius={14}
      depthTest={false}
      renderOrder={1}
      src={profile.activeAvatar?.headIconUrl}
    />
    <Text depthTest={false} renderOrder={1} fontWeight="bold" fontSize={12} marginRight={3}>
      {profile.name}
    </Text>
  </Container>
</group>
) }

File: /App.tsx

 import {Sky} from '@react-three/drei' import {Canvas} from '@react-three/fiber' import
{(Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox)} from '@react-three/viverse' import {PlayerTag} from
"./Playertag"

export default function App() {
  return (
    <Canvas shadows style={{ position: "absolute", inset: "0", touchAction: "none" }}>
      <Viverse>
        <Sky />
        <directionalLight intensity={1.2} position={[-10, 10, -10]} castShadow />
        <ambientLight intensity={1} />
        <SimpleCharacter>
          <PlayerTag />
        </SimpleCharacter>
        <BvhPhysicsBody>
          <PrototypeBox scale={[10, 1, 15]} position={[0, -0.5, 0]} />
        </BvhPhysicsBody>
      </Viverse>
    </Canvas>
  )
}

First, we use the useViverseProfile() hook to fetch the current user's profile from VIVERSE, including their name and avatar information. We provide a fallback for when the user isn't logged in:

const profile = useViverseProfile() ?? {
  name: 'Anonymous',
  activeAvatar: { headIconUrl: 'https://picsum.photos/200' },
}

Next, we need a 3D ui library, install it via

npm install @react-three/uikit

UIKit provides HTML-like components (Container, Image, Text) that work in 3D space. We create a card-like layout with flexbox:

<Container
  depthTest={false}
  renderOrder={1}
  borderRadius={10}
  paddingX={2}
  height={20}
  backgroundColor="rgba(255, 255, 255, 0.5)"
  flexDirection="row"
  alignItems="center"
  gap={4}
>
  <Image
    depthTest={false}
    renderOrder={1}
    width={16}
    height={16}
    borderRadius={14}
    src={profile.activeAvatar?.headIconUrl}
  />
  <Text depthTest={false} renderOrder={1} fontWeight="bold" fontSize={12} marginRight={3}>
    {profile.name}
  </Text>
</Container>

Next, we use useFrame to constantly update the tag's rotation to match the camera:

import { Group } from 'three'

const ref = useRef<Group>(null)

useFrame((state) => {
  if (ref.current == null) {
    return
  }
  ref.current.quaternion.copy(state.camera.quaternion)
})

The full PlayerTag component looks like this:

import { useViverseProfile } from '@react-three/viverse'
import { Container, Image, Text } from '@react-three/uikit'
import { useRef } from 'react'
import { useFrame } from '@react-three/fiber'
import { Group } from 'three'

export function PlayerTag() {
  const profile = useViverseProfile() ?? {
    name: 'Anonymous',
    activeAvatar: { headIconUrl: 'https://picsum.photos/200' },
  }

  const ref = useRef<Group>(null)

  // Make the tag always face the camera
  useFrame((state) => {
    if (ref.current == null) {
      return
    }
    ref.current.quaternion.copy(state.camera.quaternion)
  })

  return (
    <group ref={ref} position-y={2.15}>
      <Container
        depthTest={false}
        renderOrder={1}
        borderRadius={10}
        paddingX={2}
        height={20}
        backgroundColor="rgba(255, 255, 255, 0.5)"
        flexDirection="row"
        alignItems="center"
        gap={4}
      >
        <Image
          depthTest={false}
          renderOrder={1}
          width={16}
          height={16}
          borderRadius={14}
          src={profile.activeAvatar?.headIconUrl}
        />
        <Text depthTest={false} renderOrder={1} fontWeight="bold" fontSize={12} marginRight={3}>
          {profile.name}
        </Text>
      </Container>
    </group>
  )
}

Now finally lets add the PlayerTag as a child of SimpleCharacter to display it

<SimpleCharacter>
  <PlayerTag />
</SimpleCharacter>

<a id="doc-tutorials-actions"></a>

Actions and Action Bindings

Actions allow to decouple specific user inputs from game/business logic. Inputs (keyboard, mouse/touch, controllers, on‑screen UI) are converted by action bindings into actions that game systems consume on every frame or whenever an event happens to act upon the action.

• Input → Action Binding → Action → Effect
• Input: hardware or UI event (key, mouse move, touch, thumbstick, button)
• Action Binding: translates that input into a domain signal
• Action: a shared signal (event or state) consumed by systems
• Effect: the game changes on frame (e.g., camera rotates, character moves, jumps)

State vs. Event actions

• StateAction&lt;T&gt;
• Represents a continuous state that persists until changed (e.g., movement axes, “is running”).
• Merges multiple writers (e.g., keyboard + joystick) into one value each frame.
• Read anywhere via .get().

• EventAction&lt;T&gt;
• Represents instantaneous events (e.g., “jump pressed”, “rotate delta”, “zoom delta”).
• Produces values per frame via a reader; values are combined (sum, etc.) before consumption.

For example, the CharacterCameraBehavior consumes rotation and zoom event actions, which are StateActions, every frame. The StateAction returns the final value as an accumulation of all the inputs since the last frame.

Built-in actions:

• Movement: MoveForwardAction, MoveBackwardAction, MoveLeftAction, MoveRightAction, RunAction (State)
• Jumping: JumpAction (Event)
• Camera: RotateYawAction, RotatePitchAction, ZoomAction (Event)

Built-in action bindings

• Keyboard locomotion: useKeyboardLocomotionActionBindings(...)
• Mouse/touch camera (pointer capture): usePointerCaptureRotateZoomActionBindings(...)
• Mouse camera (pointer lock): usePointerLockRotateZoomActionBindings(...)
• Single key/button bindings: useKeyboardActionBinding(...), usePointerButtonActionBinding(...)
• Mobile UI: useScreenJoystickLocomotionActionBindings(...), useScreenButton(...)

These hooks connect hardware inputs to actions. Multiple bindings can feed the same action; values are safely merged.

Example: Rotate the camera with the mouse (Pointer Lock)

This shows the full pipeline: mouse movement → pointer-lock binding → rotation actions → camera rotates on frame.

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10'
    }

Files:

File: /App.tsx

import { Canvas } from '@react-three/fiber'
import { Sky } from '@react-three/drei'
import {
  Viverse,
  SimpleCharacter,
  BvhPhysicsBody,
  PrototypeBox,
  usePointerLockRotateZoomActionBindings,
} from '@react-three/viverse'

function Bindings() {
  // Binds mouse movement (while pointer is locked) to RotateYawAction/RotatePitchAction,
  // and mouse wheel to ZoomAction. The camera behavior consumes these every frame.
  usePointerLockRotateZoomActionBindings({ lockOnClick: true })
  return null
}

export default function App() {
  return (
    <Canvas
      style={{ position: 'absolute', inset: '0', touchAction: 'none' }}
      camera={{ fov: 90, position: [0, 2, 2] }}
      shadows
    >
      <Viverse>
        <Sky />
        <directionalLight intensity={1.2} position={[5, 10, 10]} castShadow />
        <ambientLight intensity={1} />

        {/* SimpleCharacter includes a default camera behavior that reads rotation + zoom actions */}
        <SimpleCharacter actionBindings={[]} />
        <Bindings />

        <BvhPhysicsBody>
          <PrototypeBox color="#ffffff" scale={[10, 0.5, 10]} position={[0, -2, 0]} />
          <PrototypeBox color="#cccccc" scale={[2, 1, 3]} position={[4, 0, 0]} />
          <PrototypeBox color="#ffccff" scale={[3, 1, 3]} position={[3, 1.5, -1]} />
        </BvhPhysicsBody>
      </Viverse>
    </Canvas>
  )
}

Click the canvas once to lock the pointer. Move the mouse to rotate the camera; use the mouse wheel to zoom. That’s the actions pipeline in action.

Example: Map Q/E keys to camera yaw rotation

You can also route keyboard inputs into camera rotation by mapping KeyboardEvent to yaw deltas and binding them to the same RotateYawAction.

import { useKeyboardActionBinding } from '@react-three/viverse'
import { RotateYawAction } from '@react-three/viverse'

function KeyboardYawBindings() {
  // Map KeyboardEvent → number (negative = left, positive = right)
  const rotateFromKeyboard = RotateYawAction.mapFrom((e: KeyboardEvent) => {
    if (e.code === 'KeyQ') return -0.02
    if (e.code === 'KeyE') return  0.02
    return 0
  })
  // Bind Q/E key presses to the mapped action
  useKeyboardActionBinding(rotateFromKeyboard, { keys: ['KeyQ', 'KeyE'] })
  return null
}

Place <KeyboardYawBindings /> alongside your other bindings. Now both mouse and keys contribute to RotateYawAction; their values are combined before the camera consumes them each frame.

Example: Keyboard locomotion and jump

To bind classic WASD + Shift + Space to movement and jump, use the built-in locomotion bindings:

import { useKeyboardLocomotionActionBindings } from '@react-three/viverse'

function LocomotionBindings() {
  useKeyboardLocomotionActionBindings({
    requiresPointerLock: false, // set to true if you want movement only when pointer is locked
  })
  return null
}

These bindings write to:

• MoveForwardAction, MoveBackwardAction, MoveLeftAction, MoveRightAction (State: 0..1)
• RunAction (State: boolean)
• JumpAction (Event)

Your character controller (e.g., SimpleCharacter) reads these to move and animate on frame, regardless of the input device that produced them.

Takeaways

• Decouple input from gameplay: actions provide a stable interface your systems consume.
• Compose inputs: multiple bindings can feed the same action; values merge predictably.
• Think “signals,” not devices: code against actions like “move forward” or “yaw delta,” not specific keys or hardware.

<a id="doc-tutorials-augmented-and-virtual-reality"></a>

Augmented and Virtual Reality

This tutorial shows how to add Augmented Reality (AR) and Virtual Reality (VR) support to your @react-three/viverse games. We'll start with AR and then show the additional changes needed for VR.

Prerequisites

Make sure you have the @react-three/xr package installed:

npm install @react-three/xr

Augmented Reality (AR)

Let's start by adding AR support to a basic @react-three/viverse game.

Here's the AR app we'll build now:

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10',
      '@react-three/xr': 'latest'
    }

Files:

File: /App.tsx

import { Canvas } from '@react-three/fiber'
import { Viverse } from '@react-three/viverse'
import { XR, XROrigin, createXRStore } from '@react-three/xr'
import { Scene } from './Scene'

const store = createXRStore({ offerSession: 'immersive-ar' })

export default function App() {
  return (
    <Viverse>
      <Canvas
        style={{ position: "absolute", inset: "0", touchAction: "none" }}
        camera={{ fov: 90, position: [0, 2, 2] }}
        shadows
        gl={{ antialias: true, localClippingEnabled: true }}
      >
        <XR store={store}>
            <XROrigin scale={10} position-y={-8} position-z={10} />
            <Scene />
        </XR>
      </Canvas>
    </Viverse>
  )
}

File: /Scene.tsx

import { useFrame } from '@react-three/fiber'
import {
  SimpleCharacter,
  BvhPhysicsBody,
  PrototypeBox,
  useXRControllerLocomotionActionBindings,
} from '@react-three/viverse'
import { useRef } from 'react'
import { Group } from 'three'

export function Scene() {
  const characterRef = useRef<Group>(null)
  useFrame(() => {
    if (characterRef.current == null) {
      return
    }
    if (characterRef.current.position.y < -10) {
      characterRef.current.position.set(0, 0, 0)
    }
  })
  useXRControllerLocomotionActionBindings()
  return (
    <>
      <directionalLight
        intensity={1.2}
        position={[5, 10, 10]}
        castShadow
        shadow-mapSize-width={1024}
        shadow-mapSize-height={1024}
        shadow-camera-left={-10}
        shadow-camera-right={10}
        shadow-camera-top={10}
        shadow-camera-bottom={-10}
      />
      <ambientLight intensity={1} />
      <SimpleCharacter cameraBehavior={false} ref={characterRef} />
      <BvhPhysicsBody>
        <PrototypeBox color="&#35;cccccc" scale={[2, 1, 3]} position={[3.91, 0, 0]} />
        <PrototypeBox color="&#35;ffccff" scale={[3, 1, 3]} position={[2.92, 1.5, -1.22]} />
        <PrototypeBox color="&#35;ccffff" scale={[2, 0.5, 3]} position={[1.92, 2.5, -3.22]} />
        <PrototypeBox color="&#35;ffccff" scale={[2, 1, 3]} position={[-2.92, 0, -2.22]} />
        <PrototypeBox color="&#35;ccffff" scale={[1, 1, 4]} position={[0.08, -1, 0]} />
        <PrototypeBox color="&#35;ffffcc" scale={[4, 1, 1]} position={[0.08, 3.5, 0]} />
        <PrototypeBox color="&#35;ffffff" scale={[10, 0.5, 10]} position={[0.08, -2, 0]} />
      </BvhPhysicsBody>
    </>
  )
}

Step 1: Set Up the XR Store

Create an XR store configured for AR. Add these imports and create the store:

import { XR, XROrigin, createXRStore } from '@react-three/xr'

const store = createXRStore({ offerSession: 'immersive-ar' })

The offerSession: 'immersive-ar' option tells the XR system that we want to create an AR experience.

Step 2: Wrap Your Scene with XR Components

Wrap your scene content with the XR component and add an XROrigin:

export function App() {
  return (
    <Viverse>
      <Canvas
        style={{ width: '100%', flexGrow: 1 }}
        camera={{ fov: 90, position: [0, 2, 2] }}
        shadows
        gl={{ antialias: true, localClippingEnabled: true }}
      >
        <Suspense fallback={<Text>Loading...</Text>}>
          <XR store={store}>
            <XROrigin scale={10} position-y={-8} position-z={10} />
            <Scene />
          </XR>
        </Suspense>
      </Canvas>
    </Viverse>
  )
}

Key points:

• XROrigin defines the coordinate system origin for AR tracking
• scale={10} makes the scene 10x larger relative to the real world
• position-y={-8} position-z={10} adjusts the initial positioning

Step 3: Remove Sky Component

For AR, you don't want a sky background since the camera feed should show through. Remove any Sky components from your scene:

export function Scene() {
  return (
    <>
      {/* Remove <Sky /> for AR */}
      <directionalLight intensity={1.2} position={[5, 10, 10]} castShadow />
      <ambientLight intensity={1} />
      {/* ... rest of your scene */}
    </>
  )
}

Step 4: Use XR Controller Action Bindings

Add XR controller action bindings using the useXRControllerLocomotionActionBindings hook:

import { useXRControllerLocomotionActionBindings } from '@react-three/viverse'

export function Scene() {
  useXRControllerLocomotionActionBindings()
  
  return (
    <>
      <SimpleCharacter 
        cameraBehavior={false} 
        ref={characterRef}
      >
        <PlayerTag />
      </SimpleCharacter>
      {/* ... rest of scene */}
    </>
  )
}

Key changes:

• cameraBehavior={false} - Disables automatic camera control (AR handles this)
• useXRControllerLocomotionActionBindings() - Hook that provides controller action bindings for movement

The useXRControllerLocomotionActionBindings hook binds XR controller inputs to character locomotion actions:

• Left thumbstick controls movement (forward/backward/left/right)
• Right controller A button triggers jumping
• Left trigger enables running

Virtual Reality (VR)

Now let's look at how to add VR support to a game building on the knowledge from adding AR support.

Here's the VR app we'll build now:

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10',
      '@react-three/xr': 'latest'
    }

Files:

File: /App.tsx

import { Canvas } from '@react-three/fiber'
import { Viverse } from '@react-three/viverse'
import { XR, createXRStore } from '@react-three/xr'
import { Scene } from './Scene'

const store = createXRStore({ offerSession: 'immersive-vr' })

export default function App() {
  return (
    <Viverse>
      <Canvas
        style={{ position: "absolute", inset: "0", touchAction: "none" }}
        camera={{ fov: 90, position: [0, 2, 2] }}
        shadows
        gl={{ antialias: true, localClippingEnabled: true }}
      >
        <XR store={store}>
            <Scene />
        </XR>
      </Canvas>
    </Viverse>
  )
}

File: /Scene.tsx

import { useFrame } from '@react-three/fiber'
import {
  SimpleCharacter,
  BvhPhysicsBody,
  PrototypeBox,
  useXRControllerLocomotionActionBindings,
} from '@react-three/viverse'
import { Sky } from '@react-three/drei'
import { XROrigin, useXRInputSourceState } from '@react-three/xr'
import { useRef } from 'react'
import { Group } from 'three'

export function Scene() {
  const characterRef = useRef<Group>(null)
  useFrame(() => {
    if (characterRef.current == null) {
      return
    }
    if (characterRef.current.position.y < -10) {
      characterRef.current.position.set(0, 0, 0)
    }
  })
  useXRControllerLocomotionActionBindings()
  return (
    <>
      <Sky />
      <directionalLight
        intensity={1.2}
        position={[5, 10, 10]}
        castShadow
        shadow-mapSize-width={1024}
        shadow-mapSize-height={1024}
        shadow-camera-left={-10}
        shadow-camera-right={10}
        shadow-camera-top={10}
        shadow-camera-bottom={-10}
      />
      <ambientLight intensity={1} />
      <SimpleCharacter cameraBehavior={false} model={false} ref={characterRef}>
        <SnapRotateXROrigin />
      </SimpleCharacter>
             <BvhPhysicsBody>
         <PrototypeBox color="&#35;cccccc" scale={[2, 1, 3]} position={[3.91, 0, 0]} />
         <PrototypeBox color="&#35;ffccff" scale={[3, 1, 3]} position={[2.92, 1.5, -1.22]} />
         <PrototypeBox color="&#35;ccffff" scale={[2, 0.5, 3]} position={[1.92, 2.5, -3.22]} />
         <PrototypeBox color="&#35;ffccff" scale={[2, 1, 3]} position={[-2.92, 0, -2.22]} />
         <PrototypeBox color="&#35;ccffff" scale={[1, 1, 4]} position={[0.08, -1, 0]} />
         <PrototypeBox color="&#35;ffffcc" scale={[4, 1, 1]} position={[0.08, 3.5, 0]} />
         <PrototypeBox color="&#35;ffffff" scale={[10, 0.5, 10]} position={[0.08, -2, 0]} />
       </BvhPhysicsBody>
    </>
  )
}

function SnapRotateXROrigin() {
  const ref = useRef<Group>(null)
  const rightController = useXRInputSourceState('controller', 'right')
  const prev = useRef(0)
  
  useFrame(() => {
    if (ref.current == null) return
    
    const current = Math.round(rightController?.gamepad?.['xr-standard-thumbstick']?.xAxis ?? 0)
    if (current < 0 && prev.current >= 0) {
      // Rotate left
      ref.current.rotation.y += Math.PI / 2
    }
    if (current > 0 && prev.current <= 0) {
      // Rotate right
      ref.current.rotation.y -= Math.PI / 2
    }
    prev.current = current
  })
  
  return <XROrigin ref={ref} />
}

Step 1: Change Session Type to VR

We can update the offer session to show the user a native "VR" enter button.

const store = createXRStore({ 
  offerSession: 'immersive-vr',
})

Step 2: Re-add the Sky for VR

Unlike AR, VR needs a sky background since there's no camera feed:

import { Sky } from '@react-three/drei'

export function Scene() {
  return (
    <>
      <Sky />
      <directionalLight intensity={1.2} position={[5, 10, 10]} castShadow />
      <ambientLight intensity={1} />
      {/* ... rest of scene */}
    </>
  )
}

Step 3: Hide the Character Model

In VR, you typically don't want to see your own character model:

<SimpleCharacter 
  cameraBehavior={false} 
  model={false}
  ref={characterRef}
>

Key change:

• model={false} - Hides the character model in VR

Step 4: Place the XROrigin into the Simple Character and Optionally Add Snap Rotation

As the XROrigin defines the player's position, we need to remove it from outside the Scene and add it into the SimpleCharacter.

import { useXRInputSourceState } from '@react-three/xr'

<SimpleCharacter ... >
    <XROrigin />
</SimpleCharacter>

For comfort in VR, you can add snap rotation using the right thumbstick by building a SnapRotateXROrigin which replaces the XROrigin component.

import { useXRInputSourceState } from '@react-three/xr'

<SimpleCharacter ... >
    <SnapRotateXROrigin />
</SimpleCharacter>

function SnapRotateXROrigin() {
  const ref = useRef<Group>(null)
  const rightController = useXRInputSourceState('controller', 'right')
  const prev = useRef(0)
  
  useFrame(() => {
    if (ref.current == null) return
    
    const current = Math.round(rightController?.gamepad?.['xr-standard-thumbstick']?.xAxis ?? 0)
    if (current < 0 && prev.current >= 0) {
      // Rotate left
      ref.current.rotation.y += Math.PI / 2
    }
    if (current > 0 && prev.current <= 0) {
      // Rotate right
      ref.current.rotation.y -= Math.PI / 2
    }
    prev.current = current
  })
  
  return <XROrigin ref={ref} />
}

Summary

For AR: 1. Use offerSession: 'immersive-ar' 2. Remove Sky component 3. Use useXRControllerLocomotionActionBindings() for to bind the locomotion actions 4. Set cameraBehavior={false} on SimpleCharacter 5. Add XROrigin with appropriate scaling/positioning

Additional changes for VR: 1. Change to offerSession: 'immersive-vr' 2. Re-add Sky component 3. Set model={false} to hide character 4. Place the XROrigin into the SimpleCharacter and optionally add snap rotation for comfort

<a id="doc-tutorials-custom-character-controller"></a>

Custom Character Controller

In this tutorial, you’ll build a custom extensible humanoid character controller in a fortnite style with support for aim, shoot, reload, run, and jump, using @react-three/viverse and @react-three/timeline.

What you’ll build

• Third-person character with physics and camera behavior
• WASD movement, run, jump, mouse look (pointer lock)
• Aiming up/forward/down with upper-body blending
• Pistol idle/shoot/reload with sound and muzzle flash
• Camera FOV and zoom effects while running/aiming
• Map collisions using a BVH physics body
• Simple HUD with name, health bar, ammo, and crosshair

Prereqs and install

Use pnpm and ensure these packages are installed. VRM support is optional but recommended for humanoids.

pnpm add three @react-three/fiber @react-three/drei @react-three/timeline @react-three/viverse zustand

Step 1 — App shell (scene, provider, UI)

First, we start by loading the tutorial assets. You can reference the exact URLs below directly, or download those binary files into your app public/ folder and use local /filename.glb paths. Do not invent sibling asset URLs: the default idle and jump clips come from @react-three/viverse exports, while the custom directional, aim, pistol, map, avatar, sound, and flash assets are the files listed here.

export const FortniteAsset = {
  map: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/map.glb',
  avatar: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/avatar.vrm',
  pistol: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol.glb',
  jogForward: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-forward.glb',
  jogForwardRight:
    'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-forward-right.glb',
  jogRight: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-right.glb',
  jogBackwardRight:
    'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-backward-right.glb',
  jogBackward: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-backward.glb',
  jogBackwardLeft:
    'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-backward-left.glb',
  jogLeft: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-left.glb',
  jogForwardLeft: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-forward-left.glb',
  aimUp: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-up.glb',
  aimForward: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-forward.glb',
  aimDown: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-down.glb',
  pistolIdle: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-idle.glb',
  pistolShoot: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-shoot.glb',
  pistolReload: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-reload.glb',
  pistolReloadSound:
    'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-reload-sound.mp3',
  pistolShootSound:
    'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-shoot-sound.mp3',
  muzzleFlash: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/muzzleflash.png',
} as const

Use IdleAnimationUrl, JumpUpAnimationUrl, JumpLoopAnimationUrl, and JumpDownAnimationUrl from @react-three/viverse for idle and jump defaults. For animations, we are using the Universal Animation Library - Pro from Quaternius. If you want to use these animations for your project, please make sure to buy the Pro version at their website.

Next, we start adding a <Canvas> and <Viverse> component, a environment that includes a sky, clouds, fog, lights, a HUD overlay, the character, and the map. In the next steps, we will create the Map, Character, and HUD components. We use <Viverse> up front so profile features, shared actions, and integrations are available everywhere; you could delay it, but then you’d pass more props around later.

Key ideas:

• Provider: <Viverse> enables account/profile APIs and shared actions/state.
• Suspense Fallback: Show a simple “Loading...” UI.
• Environment: Sky, Clouds, directionalLight with shadows.
• Composition: <Character /> and <Map /> live inside the canvas.

Add the following snippet into your src/app.tsx:

return (
  <Viverse clientId={import.meta.env.VITE_VIVERSE_APP_ID}>
    <HUD />
    <Canvas style={{ width: '100%', flexGrow: 1 }} shadows gl={{ antialias: true, localClippingEnabled: true }}>
      <fog attach="fog" args={[0xd3e1ec]} near={12} far={60} />
      <Sky rayleigh={0.2} turbidity={0.6} sunPosition={[9.2, 9, 5]} />
      <Suspense fallback={null}>
        <Clouds material={MeshBasicMaterial}>
          <Cloud position-y={40} segments={40} bounds={[50, 1, 50]} volume={20} color="gray" />
          <Cloud position-y={60} segments={40} bounds={[20, 5, 20]} volume={20} color="gray" />
        </Clouds>
        <directionalLight /* with shadows */ />
        <ambientLight intensity={1} />
        <Character />
        <Map />
      </Suspense>
    </Canvas>
  </Viverse>
)

Step 2 — Map and collisions

Create src/map.tsx and add the full component below. It loads the map, enables collisions via BvhPhysicsBody, and tweaks the ground material to receive shadows properly.

export function Map() {
  const [map, setMap] = useState<Group | null>(null)
  useEffect(
    () =>
      map?.traverse(
        (object) =>
          object.name === 'Plane' &&
          object instanceof Mesh &&
          ((object.receiveShadow = true),
          (object.material = new MeshStandardMaterial({
            roughness: 1,
            metalness: 0,
            map: (object.material as MeshStandardMaterial).map,
          }))),
      ),
    [map],
  )
  return (
    <BvhPhysicsBody>
      <Gltf
        ref={setMap}
        scale={0.3}
        src="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/map.glb"
      />
    </BvhPhysicsBody>
  )
}

Notes:

• Wrapping the map in BvhPhysicsBody builds a BVH acceleration structure, necessary for performing collision detections.
• We set receiveShadow = true on the ground mesh and re-create the material to ensure proper PBR/shadowing with the embedded base color map.
• Load the map from FortniteAsset.map, or from /map.glb if you installed the listed assets into public/.
• Loading the exact listed asset URL keeps paths simple; copying the same binary into public/ is useful when you want the generated project to be self-contained.

Step 3 — The Character component

Next, we create the Character component, which uses the VRM model at https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/avatar.vrm, displays the pistol from https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol.glb, adds animations (jogging, aiming, pistol actions), and sets up audio and visual effects. We prefer VRM for humanoids because it standardizes bone names (handy for retargeting); plain glTF works too if your bone names match the boneMap.

Before creating the component, we need to create several custom actions, specifically reloading, shooting, and aiming actions outside of the character component.

export const ReloadAction = new EventAction()
export const ShootAction = new EventAction()
export const AimAction = new StateAction<boolean>(BooleanOr, false)

Next, we create the character component and start by loading the character model and setting its height to the spawn height.

const model = useCharacterModelLoader({
  castShadow: true,
  url: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/avatar.vrm',
})
useEffect(() => void (model.scene.position.y = 70), [model])

We set the spawn height once so the character drops onto the level instead of intersecting it. Next, we set up physics on the character and apply the movement actions to the physics. We use the built-in helper rather than writing raw velocity math to keep input → motion deterministic and consistent with other examples.

const physics = useBvhCharacterPhysics(model.scene)
useFrame((state) => updateSimpleCharacterVelocity(state.camera, physics))

Then, we bind all actions to the inputs (keyboard and mouse). Notice, that we have not yet added a camera behavior or used the custom actions, so even though they are bound, they have no effect yet.

// action bindings
usePointerLockRotateZoomActionBindings()
useKeyboardLocomotionActionBindings({ requiresPointerLock: true })
useKeyboardActionBinding(ReloadAction, { keys: ['KeyR'], requiresPointerLock: true })
usePointerButtonActionBinding(ShootAction, { buttons: [0], requiresPointerLock: true })
usePointerButtonActionBinding(AimAction, { buttons: [2], requiresPointerLock: true })

Render the character and attach the pistol so you can already walk around. We will create the animation components, specifically LowerBodyAnimation, SpineAnimation, UpperBodyAimAnimation, UpperBodyAdditiveAnimation later.

<CharacterModelProvider model={model}>
  <LowerBodyAnimation physics={physics} />
  <SpineAnimation />
  <UpperBodyAimAnimation />
  <UpperBodyAdditiveAnimation />
  {/* pistol and model */}
  <CharacterModelBone bone="rightHand">
    <Gltf
      scale={0.13}
      position={[0.1, -0.03, 0]}
      rotation-x={-Math.PI / 2}
      src="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol.glb"
    />
  </CharacterModelBone>
  <primitive object={model.scene} />
</CharacterModelProvider>

Keep the pistol as a loaded model (FortniteAsset.pistol, or /pistol.glb after copying assets). Primitive box or cylinder meshes are useful while sketching, but they do not prove hand-bone orientation, muzzle direction, or held-item animation.

Step 3.1 — Create ammo store

We’ll track ammo with a tiny Zustand store so reload/shoot actions can update it and the HUD can display it.

Add this for example in src/app.tsx or in a small src/state.ts file:

import { create } from 'zustand'

export const useAmmo = create(() => ({ ammo: 12 }))

Step 4 — Camera behavior and rotation sync

We use the character camera behavior and keep the character’s yaw aligned with the camera. Using the provided behavior avoids re‑implementing orbit/zoom/offset logic; if you need full control later, you can swap it for your own.

const cameraBehaviorRef = useCharacterCameraBehavior(model.scene, {
  zoom: { speed: 0 },
  characterBaseOffset: [0.5, 1.3, 0],
})

// character rotation matches camera Y
useFrame((state) => (model.scene.rotation.y = state.camera.rotation.y))

Step 5 — Camera effects: FOV while running, zoom while aiming

Two small hooks manage the cinematic feel. We apply gentle, framerate‑independent easing to avoid jarring changes:

FOV while running: When RunAction is active, we increase the camera’s field of view (from 60 → 75) to convey speed. We use exponential smoothing (t = 1 - exp(-k * delta)) so the transition feels responsive yet stable at any framerate. After changing fov, we call updateProjectionMatrix() to apply it.

function useCameraFovControl() {
  useFrame((state, delta) => {
    if ('fov' in state.camera) {
      const targetFov = RunAction.get() ? 75 : 60
      const t = 1 - Math.exp(-10 * delta)
      state.camera.fov += (targetFov - state.camera.fov) * t
      state.camera.updateProjectionMatrix?.()
    }
  })
}

Zoom while aiming: We adjust the CharacterCameraBehavior’s zoomDistance (2.0 → 0.7) while the AimAction is active. This narrows composition around the crosshair and subtly reduces parallax. We use a slightly faster smoothing constant for snappier aim‑down‑sights behavior.

function useAimZoomControl(behaviorRef: RefObject<CharacterCameraBehavior | undefined>) {
  useFrame((_state, delta) => {
    const behavior = behaviorRef.current
    if (behavior == null) return
    const targetDistance = AimAction.get() ? 0.7 : 2.0
    const t = 1 - Math.exp(-20 * delta)
    behavior.zoomDistance += (targetDistance - behavior.zoomDistance) * t
  })
}

Step 6 — Bone map and masks

We provide a bone map for retargeting and define a mask for “upper body without spine”.

Why exclude the spine?

• We manually control the spine rotation in Step 8 to keep the torso aiming consistently forward inline with the cross-hair. If the aim layer also animated the spine, it would fight our manual rotation.
• Therefore, the “upper body without spine” mask includes shoulders, arms, hands, chest, etc., but excludes the spine so we can drive it explicitly.
• And because rigs vary, the boneMap lets us translate from your model’s bone names to VRM’s, so timeline clips target the right joints.

export const upperBodyWithoutSpine = (name: VRMHumanBoneName) => upperBody(name) && name !== 'spine'
export const boneMap: Record<string, VRMHumanBoneName> = {
  'DEF-hips': 'hips',
  'DEF-spine001': 'spine',
  'DEF-spine002': 'chest',
  'DEF-spine003': 'upperChest',
  'DEF-neck': 'neck',
  'DEF-head': 'head',
  'DEF-shoulderL': 'leftShoulder',
  'DEF-upper_armL': 'leftUpperArm',
  'DEF-forearmL': 'leftLowerArm',
  'DEF-handL': 'leftHand',
  'DEF-thumb.01L': 'leftThumbMetacarpal',
  'DEF-thumb.02L': 'leftThumbProximal',
  'DEF-thumb.03L': 'leftThumbDistal',
  'DEF-f_index.01L': 'leftIndexProximal',
  'DEF-f_index.02L': 'leftIndexIntermediate',
  'DEF-f_index.03L': 'leftIndexDistal',
  'DEF-f_middle.01L': 'leftMiddleProximal',
  'DEF-f_middle.02L': 'leftMiddleIntermediate',
  'DEF-f_middle.03L': 'leftMiddleDistal',
  'DEF-f_ring.01L': 'leftRingProximal',
  'DEF-f_ring.02L': 'leftRingIntermediate',
  'DEF-f_ring.03L': 'leftRingDistal',
  'DEF-f_pinky.01L': 'leftLittleProximal',
  'DEF-f_pinky.02L': 'leftLittleIntermediate',
  'DEF-f_pinky.03L': 'leftLittleDistal',
  'DEF-shoulderR': 'rightShoulder',
  'DEF-upper_armR': 'rightUpperArm',
  'DEF-forearmR': 'rightLowerArm',
  'DEF-handR': 'rightHand',
  'DEF-thumb01R': 'rightThumbMetacarpal',
  'DEF-thumb02R': 'rightThumbProximal',
  'DEF-thumb03R': 'rightThumbDistal',
  'DEF-f_index01R': 'rightIndexProximal',
  'DEF-f_index02R': 'rightIndexIntermediate',
  'DEF-f_index03R': 'rightIndexDistal',
  'DEF-f_middle01R': 'rightMiddleProximal',
  'DEF-f_middle02R': 'rightMiddleIntermediate',
  'DEF-f_middle03R': 'rightMiddleDistal',
  'DEF-f_ring01R': 'rightRingProximal',
  'DEF-f_ring02R': 'rightRingIntermediate',
  'DEF-f_ring03R': 'rightRingDistal',
  'DEF-f_pinky01R': 'rightLittleProximal',
  'DEF-f_pinky02R': 'rightLittleIntermediate',
  'DEF-f_pinky03R': 'rightLittleDistal',
  'DEF-thighL': 'leftUpperLeg',
  'DEF-shinL': 'leftLowerLeg',
  'DEF-footL': 'leftFoot',
  'DEF-toeL': 'leftToes',
  'DEF-thighR': 'rightUpperLeg',
  'DEF-shinR': 'rightLowerLeg',
  'DEF-footR': 'rightFoot',
  'DEF-toeR': 'rightToes',
}

Step 7 — Lower-body locomotion and jumping

For the lower body, which represents the character’s movement and jumping, we blend eight move directions plus idle, then add a jump state machine. This uses @react-three/timeline graphs and viverse helpers.

Concepts:

• Compute normalized input direction from actions.
• Scale animation speed when running.
• Jump state machine: start → loop → land → move.

Create src/lower-body-animation.tsx. We’ll build it in small parts so each piece is clear and easy to place. This timeline approach keeps animation state readable and composable, compared to ad‑hoc useFrame toggles.

1. Setup: compute input direction and time scaling

We convert the action values into a normalized 2D direction (for selecting locomotion clips) and speed up locomotion when running.

export function LowerBodyAnimation({ physics }) {
  const normalizedDirection = useMemo(() => new Vector2(), [])
  useFrame(() =>
    normalizedDirection
      .set(MoveRightAction.get() - MoveLeftAction.get(), MoveForwardAction.get() - MoveBackwardAction.get())
      .normalize(),
  )

  const forwardRef = useRef(null)
  const backwardRef = useRef(null)
  const leftRef = useRef(null)
  const rightRef = useRef(null)
  const forwardRightRef = useRef(null)
  const forwardLeftRef = useRef(null)
  const backwardRightRef = useRef(null)
  const backwardLeftRef = useRef(null)

  useFrame(() => {
    const timeScale = RunAction.get() ? 2 : 1
    for (const ref of [
      forwardRef,
      backwardRef,
      leftRef,
      rightRef,
      forwardRightRef,
      forwardLeftRef,
      backwardRightRef,
      backwardLeftRef,
    ]) {
      ref.current && (ref.current.timeScale = timeScale)
    }
  })
  // ...
}

2. Timeline scaffold and movement state (add this inside the return)

We use a small timeline graph:

• RunTimeline evaluates the timeline every frame.
• CharacterAnimationLayer groups animation actions for a specific body region.
• Graph declares states and transitions.
• GrapthState is a named state node with transition rules.

return (
  <RunTimeline>
    <CharacterAnimationLayer name="lower-body">
      <Graph enterState="move">
        <GrapthState
          name="move"
          transitionTo={{
            jumpStart: { whenUpdate: () => shouldJump(physics, lastJumpTimeRef.current) },
            jumpLoop: { whenUpdate: () => !physics.isGrounded },
          }}
        >
          {/* Add the directional Switch in substep 2a below */}
        </GrapthState>
        {/* jump states below */}
      </Graph>
    </CharacterAnimationLayer>
  </RunTimeline>
)

2a) Directional clip selection (Switch)

Inside the move state, we select one of eight directional clips plus idle based on the normalized input. The scaleTime values are tuned so diagonals feel consistent with straight movement.

<Switch>
  <SwitchCase index={0} condition={() => Math.abs(normalizedDirection.x) < 0.5 && normalizedDirection.y > 0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={1.5}
      boneMap={boneMap}
      ref={forwardRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-forward.glb"
    />
  </SwitchCase>
  <SwitchCase index={1} condition={() => normalizedDirection.x > 0.5 && normalizedDirection.y > 0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={1.5}
      boneMap={boneMap}
      ref={forwardRightRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-forward-right.glb"
    />
  </SwitchCase>
  <SwitchCase index={2} condition={() => normalizedDirection.x > 0.5 && Math.abs(normalizedDirection.y) < 0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={0.9}
      boneMap={boneMap}
      ref={rightRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-right.glb"
    />
  </SwitchCase>
  <SwitchCase index={3} condition={() => normalizedDirection.x > 0.5 && normalizedDirection.y < -0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={1.3}
      boneMap={boneMap}
      ref={backwardRightRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-backward-right.glb"
    />
  </SwitchCase>
  <SwitchCase index={4} condition={() => Math.abs(normalizedDirection.x) < 0.5 && normalizedDirection.y < -0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={1.4}
      boneMap={boneMap}
      ref={backwardRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-backward.glb"
    />
  </SwitchCase>
  <SwitchCase index={5} condition={() => normalizedDirection.x < -0.5 && normalizedDirection.y < -0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={1.3}
      boneMap={boneMap}
      ref={backwardLeftRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-backward-left.glb"
    />
  </SwitchCase>
  <SwitchCase index={6} condition={() => normalizedDirection.x < -0.5 && Math.abs(normalizedDirection.y) < 0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={0.9}
      boneMap={boneMap}
      ref={leftRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-left.glb"
    />
  </SwitchCase>
  <SwitchCase index={7} condition={() => normalizedDirection.x < -0.5 && normalizedDirection.y > 0.5}>
    <CharacterAnimationAction
      mask={lowerBody}
      sync
      scaleTime={1.5}
      boneMap={boneMap}
      ref={forwardLeftRef}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/jog-forward-left.glb"
    />
  </SwitchCase>
  <SwitchCase index={8}>
    <CharacterAnimationAction mask={lowerBody} url={IdleAnimationUrl} />
  </SwitchCase>
</Switch>

3. Jump states

Jumping splits into short phases so we can apply upward velocity once, loop while airborne, and land smoothly when grounded again.

const lastJumpTimeRef = useRef(0)

/* place inside <Graph> after the "move" state */
<GrapthState name="jumpStart" transitionTo={{ jumpDown: { whenUpdate: () => !physics.isGrounded }, finally: 'jumpUp' }}>
  <CharacterAnimationAction
    until={() => timePassed(0.2, 'seconds')}
    update={() => void physics.inputVelocity.multiplyScalar(0.3)}
    mask={lowerBody}
    paused
    url={JumpUpAnimationUrl}
  />
</GrapthState>
<GrapthState name="jumpLoop" transitionTo={{ jumpDown: { whenUpdate: () => physics.isGrounded } }}>
  <CharacterAnimationAction mask={lowerBody} url={JumpLoopAnimationUrl} />
</GrapthState>
<GrapthState
  name="jumpUp"
  transitionTo={{
    jumpDown: { whenUpdate: (_, _clock, actionTime) => actionTime > 0.3 && physics.isGrounded },
    finally: 'jumpLoop',
  }}
>
  <CharacterAnimationAction
    loop={LoopOnce}
    mask={lowerBody}
    init={() => {
      lastJumpTimeRef.current = performance.now() / 1000
      physics.applyVelocity(new Vector3(0, 8, 0))
    }}
    url={JumpUpAnimationUrl}
  />
</GrapthState>
<GrapthState name="jumpDown" transitionTo={{ finally: 'move' }}>
  <CharacterAnimationAction
    mask={lowerBody}
    until={() => timePassed(150, 'milliseconds')}
    loop={LoopOnce}
    url={JumpDownAnimationUrl}
  />
</GrapthState>

Step 8 — Spine: keep upright and match camera yaw

Create src/spine-animation.tsx. Add these parts:

1. Resolve the spine bone once

We cache the lowest upper‑body bone so updates are fast and don’t require repeated lookups.

export function SpineAnimation() {
  const model = useCharacterModel()
  const spineBone = useMemo(() => {
    // VRM or plain glTF
    return model instanceof VRM ? model.humanoid.getNormalizedBoneNode('spine') : model.scene.getObjectByName('spine')
  }, [model])
  // ...
}

2. Align the spine each frame (keep upright, match camera yaw)

The spine should stay upright and rotate only around Y to match camera yaw; this keeps the torso aligned with aim and avoids double transforms.

const eulerYXZ = new Euler(0, 0, 0, 'YXZ')
const qWorld = new Quaternion()
const qParentWorldInv = new Quaternion()
const qLocal = new Quaternion()
const cameraRotationOffsetY = -0.5

useFrame((state) => {
  if (spineBone == null) return
  state.camera.getWorldQuaternion(qWorld)
  eulerYXZ.setFromQuaternion(qWorld, 'YXZ')
  const cameraYaw = eulerYXZ.y + (model instanceof VRM ? 0 : Math.PI) + cameraRotationOffsetY
  eulerYXZ.set(0, cameraYaw, 0, 'YXZ')
  qWorld.setFromEuler(eulerYXZ)
  const parent = spineBone.parent
  if (parent != null) {
    parent.getWorldQuaternion(qParentWorldInv).invert()
    qLocal.copy(qParentWorldInv).multiply(qWorld)
    spineBone.quaternion.copy(qLocal)
  } else {
    spineBone.quaternion.copy(qWorld)
  }
  spineBone.updateMatrixWorld()
})

Step 9 — Aim up/forward/down blending

Create src/upper-body-aim-animation.tsx. Add these parts:

1. Weight aim clips by camera pitch

We blend “up/forward/down” by the camera’s pitch so the upper body points toward where you look. Using three focused clips keeps pose fidelity better than stretching a single generic clip. We also introduce Parallel, which plays its children at the same time—we’ll reuse it for simultaneous clip layers and effects.

export function UpperBodyAimAnimation() {
  const aimUpRef = useRef(null)
  const aimForwardRef = useRef(null)
  const aimDownRef = useRef(null)

  useFrame((state) => {
    if (!aimUpRef.current || !aimForwardRef.current || !aimDownRef.current) return
    const pitch = -state.camera.rotation.x
    if (pitch <= 0) {
      aimUpRef.current.weight = Math.min(1, Math.max(0, -pitch / (Math.PI / 2)))
      aimForwardRef.current.weight = 1 - aimUpRef.current.weight
      aimDownRef.current.weight = 0
    } else {
      aimDownRef.current.weight = Math.min(1, Math.max(0, pitch / (Math.PI / 2)))
      aimForwardRef.current.weight = 1 - aimDownRef.current.weight
      aimUpRef.current.weight = 0
    }
  })
  // ...
}

2. Layer the three aim clips (mask excludes the spine)

We play all three clips together and only change their weights; the mask excludes the spine to avoid conflicts with our manual spine rotation.

return (
  <RunTimeline>
    <Parallel type="all">
      <CharacterAnimationLayer name="aim">
        <CharacterAnimationAction
          boneMap={boneMap}
          mask={upperBodyWithoutSpine}
          url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-up.glb"
          crossFade={false}
          ref={aimUpRef}
        />
        <CharacterAnimationAction
          boneMap={boneMap}
          mask={upperBodyWithoutSpine}
          url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-forward.glb"
          ref={aimForwardRef}
        />
        <CharacterAnimationAction
          boneMap={boneMap}
          mask={upperBodyWithoutSpine}
          url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-down.glb"
          crossFade={false}
          ref={aimDownRef}
        />
      </CharacterAnimationLayer>
    </Parallel>
  </RunTimeline>
)

Step 10 — Additive upper-body: idle, shoot, reload, audio, muzzle flash

Create src/upper-body-additive-animation.tsx. Add these parts:

1. Attach muzzle flash and audio under the right hand

Audio and flash sit where the muzzle is, so sounds and visuals feel spatially correct. Notice, that the sound only plays when we execute .play() on the attached ref.

<CharacterModelBone bone="rightHand">
  <group position={[0.3, 0, -0.1]}>
    <PositionalAudio
      ref={reloadAudioRef}
      loop={false}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-reload-sound.mp3"
    />
    <PositionalAudio
      ref={muzzleFlashAudioRef}
      loop={false}
      url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-shoot-sound.mp3"
    />
    <Billboard scale={0.4}>
      <mesh visible={false} ref={muzzleFlashVisualRef}>
        <planeGeometry />
        <meshBasicMaterial color="#ffcc88" transparent opacity={0.7} map={muzzleflashTexture} />
      </mesh>
    </Billboard>
  </group>
</CharacterModelBone>

2. Timeline for idle → reload/shoot transitions (additive layer)

An additive layer lets us overlay weapon actions on top of locomotion/aim; timeline transitions keep behavior deterministic and easy to expand.

<RunTimeline>
  <CharacterAnimationLayer name="upper-body">
    <Graph enterState="idle">
      <GrapthState
        name="idle"
        transitionTo={{
          reload: { whenPromise: () => ReloadAction.waitFor() },
          shoot: {
            whenPromise: async () => {
              await ShootAction.waitFor()
              if (useAmmo.getState().ammo === 0) await new Promise(() => {})
            },
          },
        }}
      >
        <AdditiveCharacterAnimationAction
          referenceClip={{
            url: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-forward.glb',
          }}
          url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-idle.glb"
          boneMap={boneMap}
          mask={upperBodyWithoutSpine}
        />
      </GrapthState>
      <GrapthState name="reload" transitionTo={{ finally: 'idle' }}>
        <AdditiveCharacterAnimationAction
          referenceClip={{
            url: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-forward.glb',
          }}
          boneMap={boneMap}
          loop={LoopOnce}
          init={() => {
            reloadAudioRef.current?.play(0.3)
            useAmmo.setState({ ammo: 12 })
          }}
          mask={upperBodyWithoutSpine}
          scaleTime={0.5}
          url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-reload.glb"
        />
      </GrapthState>
      <GrapthState name="shoot" transitionTo={{ finally: 'idle' }}>
        <Parallel type="all">
          <Action
            update={(state) => {
              const jitter = 0.01
              state.camera.rotation.set(
                state.camera.rotation.x + (Math.random() - 0.5) * jitter,
                state.camera.rotation.y + (Math.random() - 0.5) * jitter,
                0,
              )
            }}
            until={() => timePassed(0.11, 'seconds')}
          />
          <Action
            init={() => {
              useAmmo.setState({ ammo: useAmmo.getState().ammo - 1 })
              muzzleFlashAudioRef.current?.stop()
              muzzleFlashAudioRef.current?.play()
              if (muzzleFlashVisualRef.current) {
                muzzleFlashVisualRef.current.visible = true
                return () => (muzzleFlashVisualRef.current!.visible = false)
              }
            }}
            until={() => timePassed(0.07, 'seconds')}
          />
          <AdditiveCharacterAnimationAction
            referenceClip={{
              url: 'https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/aim-forward.glb',
            }}
            boneMap={boneMap}
            loop={LoopOnce}
            mask={upperBodyWithoutSpine}
            fadeDuration={0}
            scaleTime={0.5}
            url="https://raw.githubusercontent.com/pmndrs/viverse/main/examples/fortnite/public/pistol-shoot.glb"
          />
        </Parallel>
      </GrapthState>
    </Graph>
  </CharacterAnimationLayer>
</RunTimeline>

The brief “jitter” recoil in the shoot state nudges the camera by a tiny random amount only while the state is active. Because the Action has an until={() => timePassed(0.11,'seconds')}, the shake starts exactly when the state begins and stops automatically, letting the regular camera behavior bring the view back smoothly (we keep roll at 0 to avoid unwanted tilt).

Step 11 — HUD and crosshair

The HUD is plain React DOM absolutely positioned over the canvas. It reads the player profile and ammo, shows a health bar, and renders a minimal crosshair.

Highlights from src/hud.tsx:

const { name } = useViverseProfile() ?? { name: 'Anonymous', activeAvatar: null }
const ammo = useAmmo((s) => s.ammo)
// ... name at top-left, health bar bottom-left, ammo bottom-right ...
// ... a simple crosshair (centered) composed of small divs ...

Create src/hud.tsx and add the full component. It overlays the canvas (no pointer events on the crosshair) and uses the same system font stack as the example.

export function HUD() {
  const [health, setHealth] = useState(50)
  const ammo = useAmmo((s) => s.ammo)

  const { name } = useViverseProfile() ?? { name: 'Anonymous', activeAvatar: null }

  const percent = Math.max(0, Math.min(100, health))

  return (
    <>
      <div
        style={{
          position: 'absolute',
          top: 16,
          left: 16,
          color: '#fff',
          zIndex: 100000,
          fontFamily: 'system-ui, -apple-system, Segoe UI, Roboto, sans-serif',
          textShadow: '0 1px 2px rgba(0,0,0,0.4)',
        }}
      >
        <div style={{ fontWeight: 800, fontSize: 14, letterSpacing: 2 }}>{name}</div>
      </div>

      <div
        style={{
          position: 'absolute',
          bottom: 28,
          left: 28,
          zIndex: 100000,
          display: 'flex',
          alignItems: 'center',
          gap: 12,
          background: 'rgba(0,0,0,0.2)',
          padding: '8px 12px',
          color: '#fff',
          fontFamily: 'system-ui, -apple-system, Segoe UI, Roboto, sans-serif',
        }}
      >
        <div style={{ fontWeight: 800, fontSize: 22, lineHeight: 1, transform: 'translate(0, -2px)' }}>+</div>
        <div
          style={{
            width: 260,
            height: 20,
            background: 'rgba(255,255,255,0.3)',
            overflow: 'hidden',
          }}
        >
          <div
            style={{
              width: `${percent}%`,
              height: '100%',
              background: 'linear-gradient(90deg,rgb(31, 224, 102), #2dbb5f)',
            }}
          />
        </div>
        <div style={{ fontWeight: 800, fontSize: 18, minWidth: 36, textAlign: 'right' }}>{Math.round(health)}</div>
      </div>

      <div
        style={{
          zIndex: 100000,
          position: 'absolute',
          bottom: 28,
          right: 28,
          color: '#fff',
          textAlign: 'right',
          fontFamily: 'system-ui, -apple-system, Segoe UI, Roboto, sans-serif',
          textShadow: '0 1px 2px rgba(0,0,0,0.4)',
        }}
      >
        <div style={{ fontSize: 13, opacity: 0.85, marginBottom: 4, fontWeight: 700 }}>AMMO</div>
        <div style={{ fontWeight: 800, fontSize: 38 }}>
          {ammo}
          <span style={{ fontSize: 11, opacity: 0.7, fontWeight: 'normal' }}>/ 12</span>
        </div>
      </div>

      {/* Crosshair */}
      <div
        style={{
          zIndex: 100000,
          position: 'absolute',
          top: '50%',
          left: '50%',
          transform: 'translate(-50%, -50%)',
          pointerEvents: 'none',
        }}
      >
        {/* center dot */}
        <div
          style={{
            position: 'absolute',
            width: 2,
            height: 2,
            borderRadius: 1,
            background: 'rgba(255,255,255,0.55)',
            boxShadow: '0 0 0 1px rgba(0,0,0,0.25)',
            transform: 'translate(-1px, -1px)',
          }}
        />
        {/* top line */}
        <div
          style={{
            position: 'absolute',
            left: -1,
            top: -22,
            width: 2,
            height: 8,
            borderRadius: 1,
            background: 'rgba(255,255,255,0.55)',
            boxShadow: '0 0 0 1px rgba(0,0,0,0.15)',
          }}
        />
        {/* bottom line */}
        <div
          style={{
            position: 'absolute',
            left: -1,
            top: 14,
            width: 2,
            height: 8,
            borderRadius: 1,
            background: 'rgba(255,255,255,0.55)',
            boxShadow: '0 0 0 1px rgba(0,0,0,0.15)',
          }}
        />
        {/* left line */}
        <div
          style={{
            position: 'absolute',
            top: -1,
            left: -22,
            width: 8,
            height: 2,
            borderRadius: 1,
            background: 'rgba(255,255,255,0.55)',
            boxShadow: '0 0 0 1px rgba(0,0,0,0.15)',
          }}
        />
        {/* right line */}
        <div
          style={{
            position: 'absolute',
            top: -1,
            left: 14,
            width: 8,
            height: 2,
            borderRadius: 1,
            background: 'rgba(255,255,255,0.55)',
            boxShadow: '0 0 0 1px rgba(0,0,0,0.15)',
          }}
        />
      </div>
    </>
  )
}

At this point your project should behave exactly like examples/fortnite. If anything does not work, compare your project with the files in examples/fortnite.

<a id="doc-tutorials-custom-models-and-animations"></a>

Custom Models and Animations

Using Custom Character Models

By default, the SimpleCharacter component uses a built-in robot avatar. You can easily replace this with your own 3D model by providing a URL to the model file in any of the following formats:

• VRM - The standardized VRM format for avatars requires no additional configuration
• GLTF - (or also glb) Standard 3D Model format - make sure to use the standard vrm bone names as shown below or provide a boneMap.

<details> <summary>VRM 1.0 humanoid bone names (click to expand)</summary>

The following are the standard VRM 1.0 humanoid bone names your GLTF rig should use (aligned with the VRM specification). If your model uses these names, animations and retargeting will work reliably:

</details>

import { SimpleCharacter } from '@react-three/viverse'

export function MyCharacter() {
  return <SimpleCharacter model={{ url: '/path/to/your-model.vrm' }} />
}

If your GLTF model does not use the standard VRM bone names, you can provide a boneMap to map your model's bone names to the VRM standard:

import { SimpleCharacter } from '@react-three/viverse'

const myBoneMap = {
  'mixamorig:Hips': 'hips',
  'mixamorig:Spine': 'spine',
  // ... other bones
}

export function MyCharacter() {
  return (
    <SimpleCharacter
      model={{
        url: '/path/to/your-model.glb',
        boneMap: myBoneMap
      }}
    />
  )
}

Adding Custom Animations

Your can replace the default animations of the SimpleCharacter component with files in any of these three supported animation formats:

• VRMA (VRM Animation) - The native VRM animation format
• FBX - Popular file format for character animations
• GLTF - Standard 3D format with animations
• Mixamo - deprecated - use remove type: 'mixamo' and add boneMap: mixamoBoneMap instead

Make sure to either use a bone map, e.g. when your bone names follow the mixamo naming conventions use the mixamoBoneMap or use the standard VRM bones for the animation amature as shown above under "VRM 1.0 humanoid bone names".

Each animation type can be configured individually:

<SimpleCharacter
  animation={{
    walk: {
      url: '/animations/walking.fbx',
      boneMap: mixamoBoneMap,
      removeXZMovement: true,
      scaleTime: 0.8,
    },
    run: {
      url: '/animations/running.vrma',
    },
    idle: {
      url: '/animations/idle.gltf',
      trimTime: { start: 0.5, end: 3.0 },
    },
  }}
/>

You can customize any of these animation slots:

• walk - Walking animation
• run - Running animation
• idle - Standing idle animation
• jumpStart - Beginning of jump
• jumpUp - Ascending during jump
• jumpLoop - Mid-air loop animation
• jumpDown - Landing animation

<a id="doc-tutorials-equipping-items"></a>

Equipping the Character With Items

This tutorial shows you how to equip your character with items by attaching 3D objects to specific bones. We'll create a simple sword using just two meshes and attach it to the character's right hand.

_Here's a preview of what we'll build in this tutorial:_

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10'
    }

Files:

File: /App.tsx

import { Sky } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox, CharacterModelBone } from '@react-three/viverse'

export default function App() {
  return (
    <Canvas shadows style={{ position: 'absolute', inset: '0', touchAction: 'none' }}>
      <Viverse>
        <Sky />
        <directionalLight intensity={1.2} position={[-10, 10, -10]} castShadow />
        <ambientLight intensity={1} />
        <SimpleCharacter>
          <CharacterModelBone bone="rightHand">
            <group scale={0.5} position-y={-0.02} position-x={0.07}>
              {/* Blade */}
              <mesh position={[0, 0.8, 0]} castShadow>
                <boxGeometry args={[0.08, 1.9, 0.04]} />
                <meshStandardMaterial color="#c0c0c0" metalness={0.9} roughness={0.1} />
              </mesh>

              {/* Handle */}
              <mesh position={[0, 0.2, 0]} castShadow>
                <boxGeometry args={[0.3, 0.04, 0.04]} />
                <meshStandardMaterial color="#654321" metalness={0.1} roughness={0.8} />
              </mesh>
            </group>
          </CharacterModelBone>
        </SimpleCharacter>
        <BvhPhysicsBody>
          <PrototypeBox scale={[10, 1, 15]} position={[0, -0.5, 0]} />
        </BvhPhysicsBody>
      </Viverse>
    </Canvas>
  )
}

Understanding Bone Attachment

The CharacterModelBone component allows you to attach any 3D object to specific bones in the character's skeleton. This is perfect for equipping weapons, accessories, or any items that should move with the character.

For hand attachments, use the same local-axis convention as the custom controller tutorial: +X offsets across the hand, +Y is vertical or grip-up, and -Z is forward for a muzzle, beam, torch cone, or other aimed tool. If a glTF weapon is authored with its barrel on another axis, wrap or rotate it so the muzzle points local -Z after attachment; the tutorial pistol.glb uses rotation-x={-Math.PI / 2} because its barrel is authored on model +Y.

Step 1: Import the CharacterModelBone Component

First, import the CharacterModelBone component from @react-three/viverse:

import { CharacterModelBone } from '@react-three/viverse'

Step 2: Add a Simple Sword to the "rightHand"

Next, we place the CharacterModelBone inside the SimpleCharacter component and attach it to the "rightHand". We then build a simple sword using two meshes. For better looks, you probably want to import your own 3D model.

<SimpleCharacter>
  <CharacterModelBone bone="rightHand">
    <group scale={0.5} position-y={-0.02} position-x={0.07}>
      {/* Blade */}
      <mesh position={[0, 0.8, 0]} castShadow>
        <boxGeometry args={[0.08, 1.9, 0.04]} />
        <meshStandardMaterial color="#c0c0c0" metalness={0.9} roughness={0.1} />
      </mesh>

      {/* Handle */}
      <mesh position={[0, 0.2, 0]} castShadow>
        <boxGeometry args={[0.3, 0.04, 0.04]} />
        <meshStandardMaterial color="#654321" metalness={0.1} roughness={0.8} />
      </mesh>
    </group>
  </CharacterModelBone>
</SimpleCharacter>

<a id="doc-tutorials-first-person"></a>

First Person Controls

In this tutorial we will configure the SimpleCharacter to use first person controls with the following result:

_Here's a preview of this tutorial's result:_

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10'
    }

Files:

File: /App.tsx

import { Sky } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import {
  Viverse,
  SimpleCharacter,
  BvhPhysicsBody,
  PrototypeBox,
  FirstPersonCharacterCameraBehavior,
  PointerLockRotateZoomActionBindings,
  KeyboardLocomotionActionBindings,
} from '@react-three/viverse'

export default function App() {
  return (
   <Canvas
      style={{ position: 'absolute', inset: '0', touchAction: 'none' }}
    >
      <Viverse>
        <Sky />
        <directionalLight intensity={1.2} position={[-10, 10, -10]} />
        <ambientLight intensity={1} />
        <SimpleCharacter
          model={false}
          actionBindings={[KeyboardLocomotionActionBindings, PointerLockRotateZoomActionBindings]}
          cameraBehavior={FirstPersonCharacterCameraBehavior}
        />
        <BvhPhysicsBody>
          <PrototypeBox scale={[10, 1, 15]} position={[0, -0.5, 0]} />
        </BvhPhysicsBody>
      </Viverse>
    </Canvas>
  )
}

First, we switch from third-person to first-person camera behavior and hide the character model to prevent the model from occluding the players view.

<SimpleCharacter
  model={false}
  cameraBehavior={FirstPersonCharacterCameraBehavior}
  // ... other props
/>

Changes:

• model={false} - Hides the character model since in first-person view, you don't want to see your own character
• cameraBehavior={FirstPersonCharacterCameraBehavior} - Switches from the default third-person camera to first-person camera behavior

Next, you need to set up the appropriate action bindings for first-person movement and looking around:

<SimpleCharacter
  actionBindings={[KeyboardLocomotionActionBindings, PointerLockRotateZoomActionBindings]}
  // ... other props
/>

• KeyboardLocomotionActionBindings - Handles WASD movement action bindings for walking around
• PointerLockRotateZoomActionBindings - Enables mouse look action bindings for rotating the camera/view direction

Tutorials

Read the smallest relevant tutorial file rather than loading every tutorial.

• references/tutorials/simple-game.md: Building a Simple Game
• references/tutorials/first-person.md: First Person Controls
• references/tutorials/augmented-and-virtual-reality.md: Augmented and Virtual Reality
• references/tutorials/access-avatar-and-profile.md: Accessing Avatar and Profile
• references/tutorials/equipping-items.md: Equipping the Character With Items
• references/tutorials/custom-models-and-animations.md: Custom Models and Animations
• references/tutorials/actions.md: Actions and Action Bindings
• references/tutorials/custom-character-controller.md: Custom Character Controller
• references/tutorials/remove-viverse-integrations.md: Remove VIVERSE Integrations

<a id="doc-tutorials-remove-viverse-integrations"></a>

Remove VIVERSE Integrations

To remove the VIVERSE integrations replace the <Viverse> component with <BvhPhysicsWorld> and remove all VIVERSE-specific hooks from your components as these hooks will no longer work without the VIVERSE context.

Dependencies:

{
      'three': 'latest',
      '@react-three/fiber': '<9',
      '@react-three/viverse': 'latest',
      '@react-three/drei': '<10'
    }

Files:

File: /App.tsx

import { Sky } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import { BvhPhysicsWorld, SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse'

export default function App() {
  return (
    <Canvas shadows style={{ position: "absolute", inset: "0", touchAction: "none" }}>
      <BvhPhysicsWorld>
        <Sky />
        <directionalLight intensity={1.2} position={[-10, 10, -10]} castShadow />
        <ambientLight intensity={1} />
        <SimpleCharacter />
        <BvhPhysicsBody>
          <PrototypeBox scale={[10, 1, 15]} position={[0, -0.5, 0]} />
        </BvhPhysicsBody>
      </BvhPhysicsWorld>
    </Canvas>
  )
}