Agents Ui

LiveKit Agents UI

Build React frontends for LiveKit voice AI agents with shadcn-based components.

LiveKit MCP server tools

This skill works alongside the LiveKit MCP server, which provides direct access to the latest LiveKit documentation, code examples, and changelogs. Use these tools when you need up-to-date information that may have changed since this skill was created.

Available MCP tools:

• docs_search - Search the LiveKit docs site
• get_pages - Fetch specific documentation pages by path
• get_changelog - Get recent releases and updates for LiveKit packages
• code_search - Search LiveKit repositories for code examples
• get_python_agent_example - Browse 100+ Python agent examples

When to use MCP tools:

• You need the latest API documentation or feature updates
• You're looking for recent examples or code patterns
• You want to check if a feature has been added in recent releases
• The local references don't cover a specific topic

When to use local references:

• You need quick access to core concepts covered in this skill
• You're working offline or want faster access to common patterns
• The information in the references is sufficient for your needs

Use MCP tools and local references together for the best experience.

References

Consult these resources as needed:

• ./references/livekit-overview.md -- LiveKit ecosystem overview and how these skills work together
• ./references/components.md -- All component APIs, props, and usage examples

Prerequisites

• Node.js >= 18
• React 19
• Tailwind CSS 4
• shadcn/ui initialized in your project

Installation

1. Add the LiveKit registry to your shadcn config

In your components.json:

{
  "registries": {
    "@agents-ui": "https://livekit.io/ui/r/{name}.json"
  }
}

2. Install components

# Install individual components
npx shadcn@latest add @agents-ui/agent-session-provider
npx shadcn@latest add @agents-ui/agent-control-bar
npx shadcn@latest add @agents-ui/agent-audio-visualizer-bar

# Or install multiple at once
npx shadcn@latest add @agents-ui/agent-session-provider @agents-ui/agent-control-bar

Components are copied to your components/agents-ui/ directory for full customization.

Quick start

Installation

Agents UI components require both livekit-client and @livekit/components-react:

npm install livekit-client @livekit/components-react

Setting up a TokenSource

Before using Agents UI components, you need a TokenSource from livekit-client to handle authentication.

For development (using LiveKit Cloud Sandbox):

import { TokenSource } from 'livekit-client';

const tokenSource = TokenSource.sandboxTokenServer({
  sandboxId: 'your-sandbox-id',
});

For production (using your own token endpoint):

import { TokenSource } from 'livekit-client';

const tokenSource = TokenSource.endpoint('/api/token');

Basic voice agent interface

Create a session using useSession from @livekit/components-react, then pass it to AgentSessionProvider:

'use client';

import { useRef, useEffect } from 'react';
import { useSession } from '@livekit/components-react';
import { TokenSource, TokenSourceConfigurable } from 'livekit-client';
import { AgentSessionProvider } from '@/components/agents-ui/agent-session-provider';
import { AgentControlBar } from '@/components/agents-ui/agent-control-bar';
import { AgentAudioVisualizerBar } from '@/components/agents-ui/agent-audio-visualizer-bar';

export function VoiceAgent() {
  // Use useRef to prevent recreating TokenSource on each render
  const tokenSource: TokenSourceConfigurable = useRef(
    TokenSource.sandboxTokenServer({ sandboxId: 'your-sandbox-id' })
  ).current;

  // Create session using useSession hook (required for AgentSessionProvider)
  const session = useSession(tokenSource, {
    agentName: 'your-agent-name',
  });

  // Auto-start session with cleanup
  useEffect(() => {
    session.start();
    return () => session.end();
  }, []);

  return (
    <AgentSessionProvider session={session}>
      <div className="flex flex-col items-center gap-8 p-8">
        <AgentAudioVisualizerBar />
        <AgentControlBar />
      </div>
    </AgentSessionProvider>
  );
}

Production example with token endpoint

'use client';

import { useRef, useEffect } from 'react';
import { useSession } from '@livekit/components-react';
import { TokenSource, TokenSourceConfigurable } from 'livekit-client';
import { AgentSessionProvider } from '@/components/agents-ui/agent-session-provider';
import { AgentControlBar } from '@/components/agents-ui/agent-control-bar';
import { AgentAudioVisualizerBar } from '@/components/agents-ui/agent-audio-visualizer-bar';

export function VoiceAgent() {
  const tokenSource: TokenSourceConfigurable = useRef(
    TokenSource.endpoint('/api/token')
  ).current;

  const session = useSession(tokenSource, {
    roomName: 'my-room',
    participantIdentity: 'user-123',
    participantName: 'John',
    agentName: 'my-agent',
  });

  useEffect(() => {
    session.start();
    return () => session.end();
  }, []);

  return (
    <AgentSessionProvider session={session}>
      <div className="flex flex-col items-center gap-8 p-8">
        <AgentAudioVisualizerBar />
        <AgentControlBar />
      </div>
    </AgentSessionProvider>
  );
}

Core components

AgentSessionProvider

Required wrapper that provides session state to all child components. It wraps SessionProvider from @livekit/components-react and includes RoomAudioRenderer for audio playback.

You must create a session using useSession from @livekit/components-react and pass it to AgentSessionProvider:

import { useRef, useEffect } from 'react';
import { useSession } from '@livekit/components-react';
import { TokenSource, TokenSourceConfigurable } from 'livekit-client';
import { AgentSessionProvider } from '@/components/agents-ui/agent-session-provider';

function MyApp() {
  // Create tokenSource with useRef to prevent recreation
  const tokenSource: TokenSourceConfigurable = useRef(
    TokenSource.endpoint('/api/token')
  ).current;

  // Create session using useSession hook (required)
  const session = useSession(tokenSource, { agentName: 'your-agent' });

  // Start session when component mounts
  useEffect(() => {
    session.start();
    return () => session.end();
  }, []);

  return (
    <AgentSessionProvider session={session}>
      {/* All Agents UI components must be inside this provider */}
    </AgentSessionProvider>
  );
}

AgentControlBar

Combined media controls with visualizer:

<AgentControlBar />

Audio visualizers

Five visualization styles:

// Bar visualizer (horizontal bars)
<AgentAudioVisualizerBar />

// Grid visualizer (dot matrix)
<AgentAudioVisualizerGrid />

// Radial visualizer (circular)
<AgentAudioVisualizerRadial />

// Wave visualizer (waveform)
<AgentAudioVisualizerWave />

// Aura visualizer (ambient glow effect)
<AgentAudioVisualizerAura />

Media controls

Individual track controls:

// Toggle microphone
<AgentTrackToggle source="microphone" />

// Toggle camera
<AgentTrackToggle source="camera" />

// Toggle screen share
<AgentTrackToggle source="screen_share" />

// Full track control with label
<AgentTrackControl source="microphone" />

Chat components

Display conversation transcripts:

// Full chat transcript
<AgentChatTranscript />

// Typing/thinking indicator
<AgentChatIndicator />

Session controls

// Disconnect button
<AgentDisconnectButton />

// Start audio (for browsers requiring user interaction)
<StartAudioButton />

Agent states

The AgentSessionProvider tracks these states:

• initializing - Agent is starting up
• listening - Agent is listening for user input
• thinking - Agent is processing/generating response
• speaking - Agent is speaking

Use these states to customize your UI with the local useAgentState hook (installed with the Agents UI components):

// This hook is local to your project (copied via shadcn CLI)
import { useAgentState } from '@/components/agents-ui/hooks/use-agent-state';

function StatusIndicator() {
  const state = useAgentState();
  
  return (
    <div className="flex items-center gap-2">
      <div className={cn(
        "w-2 h-2 rounded-full",
        state === "speaking" && "bg-green-500",
        state === "listening" && "bg-blue-500",
        state === "thinking" && "bg-yellow-500 animate-pulse",
      )} />
      <span className="capitalize">{state}</span>
    </div>
  );
}

Styling

Components use Tailwind CSS and support className overrides:

<AgentAudioVisualizerBar 
  className="h-32 w-64 bg-slate-900 rounded-lg"
/>

<AgentControlBar 
  className="gap-4 p-4 bg-white/10 backdrop-blur rounded-full"
/>

Customization

Since components are copied to your project, you can modify them directly:

// components/agents-ui/agent-control-bar.tsx
export function AgentControlBar({ className }: { className?: string }) {
  return (
    <div className={cn("flex items-center gap-2", className)}>
      {/* Customize the layout, add/remove controls */}
      <AgentTrackToggle source="microphone" />
      <AgentAudioVisualizerBar className="flex-1" />
      <AgentDisconnectButton />
    </div>
  );
}

Required packages

Install both livekit-client and @livekit/components-react:

npm install livekit-client @livekit/components-react

These packages provide:

• TokenSource from livekit-client - Factory for creating token sources (sandbox, endpoint, custom)
• useSession from @livekit/components-react - Required hook for creating sessions for AgentSessionProvider

You do NOT need UI components from @livekit/components-react (like LiveKitRoom, BarVisualizer, or VoiceAssistantControlBar) when using Agents UI components. Use Agents UI components instead for the UI.

Using hooks from @livekit/components-react

Agents UI requires @livekit/components-react for the useSession hook. You can also use additional hooks from this package for custom behavior. These hooks work inside AgentSessionProvider:

Required hook:

• useSession - Creates the session object required by AgentSessionProvider

Additional hooks for custom behavior:

• useAgent - Get full agent state with lifecycle helpers (requires session from useSession)
• useVoiceAssistant - Get agent state, tracks, and transcriptions
• useTrackToggle - Build custom track toggle buttons
• useChat - Send and receive chat messages
• useParticipants - Access all participants in the room
• useConnectionState - Monitor connection status

import { useVoiceAssistant } from '@livekit/components-react';

// This works inside AgentSessionProvider
function CustomAgentDisplay() {
  const { state, audioTrack, agentTranscriptions } = useVoiceAssistant();
  
  return (
    <div>
      <p>Agent is {state}</p>
      {agentTranscriptions.map((t) => (
        <p key={t.id}>{t.text}</p>
      ))}
    </div>
  );
}

See the livekit-react-hooks skill for full hook documentation.

Best practices

1. Always wrap with AgentSessionProvider - All Agents UI components require this context. 2. Use useSession to create sessions - Create a session with useSession from @livekit/components-react and pass it to AgentSessionProvider. 3. Use useRef for TokenSource - Always wrap TokenSource creation in useRef to prevent recreation on each render. 4. Start and end sessions properly - Call session.start() in a useEffect and session.end() in the cleanup function. 5. Handle audio permissions - Use StartAudioButton for browsers requiring user interaction. 6. Customize via Tailwind - Use className props for styling adjustments. 7. Modify source directly - Components are copied to your project for full control.

Components reference

All Agents UI components, their props, and usage examples.

AgentSessionProvider

Required wrapper that provides session state to all child components. It wraps SessionProvider from @livekit/components-react and includes RoomAudioRenderer for audio playback.

You must create a session using useSession from @livekit/components-react and pass it to AgentSessionProvider:

import { useRef, useEffect } from 'react';
import { useSession } from '@livekit/components-react';
import { TokenSource, TokenSourceConfigurable } from 'livekit-client';
import { AgentSessionProvider } from '@/components/agents-ui/agent-session-provider';

function MyApp() {
  // Use useRef to prevent recreating TokenSource on each render
  const tokenSource: TokenSourceConfigurable = useRef(
    TokenSource.endpoint('/api/token')
  ).current;

  // Create session using useSession hook (required)
  const session = useSession(tokenSource, { agentName: 'your-agent' });

  // Start session when component mounts
  useEffect(() => {
    session.start();
    return () => session.end();
  }, []);

  return (
    <AgentSessionProvider session={session}>
      {children}
    </AgentSessionProvider>
  );
}

AgentControlBar

Combined control bar with track toggles and visualizer.

import { AgentControlBar } from '@/components/agents-ui/agent-control-bar';

<AgentControlBar className="gap-4" />

AgentAudioVisualizerBar

Horizontal bar audio visualizer responding to agent speech.

import { AgentAudioVisualizerBar } from '@/components/agents-ui/agent-audio-visualizer-bar';

<AgentAudioVisualizerBar 
  className="h-16 w-48"
  barCount={5}
  barWidth={4}
  barGap={2}
/>

AgentAudioVisualizerGrid

Grid/dot matrix audio visualizer.

import { AgentAudioVisualizerGrid } from '@/components/agents-ui/agent-audio-visualizer-grid';

<AgentAudioVisualizerGrid 
  className="w-32 h-32"
  rows={4}
  cols={4}
/>

AgentAudioVisualizerRadial

Circular/radial audio visualizer.

import { AgentAudioVisualizerRadial } from '@/components/agents-ui/agent-audio-visualizer-radial';

<AgentAudioVisualizerRadial 
  className="w-48 h-48"
  barCount={32}
/>

AgentAudioVisualizerWave

Waveform-style audio visualizer.

import { AgentAudioVisualizerWave } from '@/components/agents-ui/agent-audio-visualizer-wave';

<AgentAudioVisualizerWave 
  className="w-64 h-16"
/>

AgentAudioVisualizerAura

Ambient aura-style audio visualizer designed in partnership with Unicorn Studio. Creates a glowing visual effect that responds to agent audio.

import { AgentAudioVisualizerAura } from '@/components/agents-ui/agent-audio-visualizer-aura';

<AgentAudioVisualizerAura 
  className="w-64 h-64"
/>

AgentTrackToggle

Toggle button for a specific track source.

import { AgentTrackToggle } from '@/components/agents-ui/agent-track-toggle';

<AgentTrackToggle source="microphone" />
<AgentTrackToggle source="camera" />
<AgentTrackToggle source="screen_share" />

AgentTrackControl

Track control with label and toggle.

import { AgentTrackControl } from '@/components/agents-ui/agent-track-control';

<AgentTrackControl source="microphone" />

AgentChatTranscript

Displays the conversation transcript between user and agent.

import { AgentChatTranscript } from '@/components/agents-ui/agent-chat-transcript';

<AgentChatTranscript 
  className="h-96 overflow-y-auto"
  showTimestamps={true}
/>

AgentChatIndicator

Shows when the agent is thinking or typing.

import { AgentChatIndicator } from '@/components/agents-ui/agent-chat-indicator';

<AgentChatIndicator />

AgentDisconnectButton

Button to disconnect from the session.

import { AgentDisconnectButton } from '@/components/agents-ui/agent-disconnect-button';

<AgentDisconnectButton className="bg-red-500 hover:bg-red-600" />

StartAudioButton

Button to start audio playback (required by some browsers).

import { StartAudioButton } from '@/components/agents-ui/start-audio-button';

<StartAudioButton label="Click to enable audio" />

Local hooks (from Agents UI)

These hooks are installed with the Agents UI components via the shadcn CLI. They are copied to your project at @/components/agents-ui/hooks/ and can be customized directly.

These are NOT from the `@livekit/components-react` npm package. For hooks from that package (like useSession, useAgent, useVoiceAssistant, useTrackToggle, useChat), see the livekit-react-hooks skill. Note that useSession from @livekit/components-react is required to use AgentSessionProvider.

useAgentState

Get the current agent state.

// Local hook from Agents UI (copied to your project)
import { useAgentState } from '@/components/agents-ui/hooks/use-agent-state';

function MyComponent() {
  const state = useAgentState();
  // state: "initializing" | "listening" | "thinking" | "speaking"
  
  return <div>Agent is {state}</div>;
}

useAgentControlBar

Hook for building custom control bars.

// Local hook from Agents UI (copied to your project)
import { useAgentControlBar } from '@/components/agents-ui/hooks/use-agent-control-bar';

function CustomControlBar() {
  const { isMuted, toggleMute, isConnected, disconnect } = useAgentControlBar();
  
  return (
    <div>
      <button onClick={toggleMute}>
        {isMuted ? 'Unmute' : 'Mute'}
      </button>
      <button onClick={disconnect}>
        Disconnect
      </button>
    </div>
  );
}

useAgentAudioVisualizerBar

Hook for building custom audio visualizers.

// Local hook from Agents UI (copied to your project)
import { useAgentAudioVisualizerBar } from '@/components/agents-ui/hooks/use-agent-audio-visualizer-bar';

function CustomVisualizer() {
  const { volumes, state } = useAgentAudioVisualizerBar({ barCount: 5 });
  
  return (
    <div className="flex gap-1">
      {volumes.map((volume, i) => (
        <div 
          key={i}
          className="w-2 bg-blue-500 transition-all"
          style={{ height: `${volume * 100}%` }}
        />
      ))}
    </div>
  );
}

Complete example

'use client';

import { useRef, useEffect } from 'react';
import { useSession } from '@livekit/components-react';
import { TokenSource, TokenSourceConfigurable } from 'livekit-client';
import { AgentSessionProvider } from '@/components/agents-ui/agent-session-provider';
import { AgentControlBar } from '@/components/agents-ui/agent-control-bar';
import { AgentAudioVisualizerRadial } from '@/components/agents-ui/agent-audio-visualizer-radial';
import { AgentChatTranscript } from '@/components/agents-ui/agent-chat-transcript';
import { AgentDisconnectButton } from '@/components/agents-ui/agent-disconnect-button';
import { StartAudioButton } from '@/components/agents-ui/start-audio-button';

export function VoiceAssistant() {
  // Use useRef to prevent recreating TokenSource on each render
  const tokenSource: TokenSourceConfigurable = useRef(
    TokenSource.endpoint('/api/token')
  ).current;

  // Create session using useSession hook (required)
  const session = useSession(tokenSource, {
    roomName: 'my-room',
    participantIdentity: 'user-123',
    agentName: 'my-agent',
  });

  // Start session when component mounts
  useEffect(() => {
    session.start();
    return () => session.end();
  }, []);

  return (
    <AgentSessionProvider session={session}>
      <div className="flex flex-col h-screen bg-slate-950 text-white">
        {/* Header */}
        <header className="flex justify-between items-center p-4 border-b border-slate-800">
          <h1 className="text-xl font-semibold">Voice Assistant</h1>
          <AgentDisconnectButton className="text-red-400 hover:text-red-300" />
        </header>
        
        {/* Main content */}
        <main className="flex-1 flex flex-col items-center justify-center gap-8 p-8">
          <AgentAudioVisualizerRadial className="w-48 h-48" />
          <AgentControlBar className="gap-4" />
          <StartAudioButton className="text-sm text-slate-400" />
        </main>
        
        {/* Chat transcript */}
        <aside className="h-64 border-t border-slate-800 p-4 overflow-y-auto">
          <AgentChatTranscript showTimestamps />
        </aside>
      </div>
    </AgentSessionProvider>
  );
}

Installation reference

# All components
npx shadcn@latest add @agents-ui/agent-session-provider
npx shadcn@latest add @agents-ui/agent-control-bar
npx shadcn@latest add @agents-ui/agent-track-toggle
npx shadcn@latest add @agents-ui/agent-track-control
npx shadcn@latest add @agents-ui/agent-audio-visualizer-bar
npx shadcn@latest add @agents-ui/agent-audio-visualizer-grid
npx shadcn@latest add @agents-ui/agent-audio-visualizer-radial
npx shadcn@latest add @agents-ui/agent-audio-visualizer-wave
npx shadcn@latest add @agents-ui/agent-audio-visualizer-aura
npx shadcn@latest add @agents-ui/agent-chat-transcript
npx shadcn@latest add @agents-ui/agent-chat-indicator
npx shadcn@latest add @agents-ui/agent-disconnect-button
npx shadcn@latest add @agents-ui/start-audio-button

LiveKit overview

LiveKit is a realtime communication platform for building AI-native applications with audio, video, and data streaming. This overview helps you understand the LiveKit ecosystem and how to use these skills effectively.

Platform components

LiveKit Cloud

LiveKit Cloud is a fully managed platform for building, deploying, and operating AI agent applications. It includes:

• Realtime media infrastructure - Global mesh of servers for low-latency audio, video, and data streaming
• Managed agent hosting - Deploy agents without managing servers or orchestration
• LiveKit Inference - Run AI models directly within LiveKit Cloud without API keys
• Native telephony - Provision phone numbers and connect PSTN calls directly to rooms
• Observability - Built-in analytics, logs, and quality metrics

Agents framework

The Agents framework lets you build Python or Node.js programs that join LiveKit rooms as realtime participants. Key capabilities:

• Voice pipelines - Stream audio through STT-LLM-TTS pipelines
• Realtime models - Use models like OpenAI Realtime API that handle speech directly
• Tool calling - Define functions the LLM can invoke during conversations
• Multi-agent workflows - Hand off between specialized agents
• Turn detection - State-of-the-art model for natural conversation flow

Architecture

┌─────────────┐     WebRTC      ┌─────────────┐     HTTP/WS      ┌─────────────┐
│   Frontend  │ ◄─────────────► │   LiveKit   │ ◄──────────────► │    Agent    │
│  (Web/App)  │                 │    Room     │                  │   Server    │
└─────────────┘                 └─────────────┘                  └─────────────┘
                                       │                                │
                                       │                                │
                                       ▼                                ▼
                                ┌─────────────┐                  ┌─────────────┐
                                │  Telephony  │                  │  AI Models  │
                                │    (SIP)    │                  │ (STT/LLM/TTS)│
                                └─────────────┘                  └─────────────┘

How these skills work together

The LiveKit skills cover the full stack for building voice AI applications:

Typical workflow:

1. Choose your backend - Use agents-py or agents-ts based on your team's preference 2. Build the frontend - Use agents-ui for React-based web interfaces 3. Connect via LiveKit - Both connect to the same LiveKit room for realtime communication

Using the skills effectively

When to use each skill

• Building a new voice agent? Start with agents-py or agents-ts for the backend logic
• Need a web interface? Add agents-ui for pre-built React components
• Full-stack project? Use both a backend skill and agents-ui together

Combining skills

The skills are designed to work together. A typical project structure:

my-voice-app/
├── agent/           # Use agents-py or agents-ts skill
│   └── agent.py     # or agent.ts
├── frontend/        # Use agents-ui skill
│   └── src/
│       └── app/
└── .env.local       # Shared LiveKit credentials

Environment setup

All skills require LiveKit credentials:

LIVEKIT_API_KEY=your_api_key
LIVEKIT_API_SECRET=your_api_secret
LIVEKIT_URL=wss://your-project.livekit.cloud

Get these from your LiveKit Cloud dashboard or self-hosted deployment.

Resources

• LiveKit documentation
• Agents framework guide
• LiveKit Cloud
• GitHub repositories