Agents

ElevenLabs Agents Platform

Build voice AI agents with natural conversations, multiple LLM providers, custom tools, and easy web embedding.

Setup: See Installation Guide for CLI and SDK setup.

Quick Start with CLI

The ElevenLabs CLI is the recommended way to create and manage agents:

# Install CLI and authenticate
npm install -g @elevenlabs/cli
elevenlabs auth login

# Initialize project and create an agent
elevenlabs agents init
elevenlabs agents add "My Assistant" --template complete

# Push to ElevenLabs platform
elevenlabs agents push

Available templates: complete, minimal, voice-only, text-only, customer-service, assistant

Python

from elevenlabs import ElevenLabs

client = ElevenLabs()

agent = client.conversational_ai.agents.create(
    name="My Assistant",
    enable_versioning=True,
    conversation_config={
        "agent": {
            "first_message": "Hello! How can I help?",
            "language": "en",
            "prompt": {
                "prompt": "You are a helpful assistant. Be concise and friendly.",
                "llm": "gemini-2.0-flash",
                "temperature": 0.7
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
    }
)

JavaScript

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
const client = new ElevenLabsClient();

const agent = await client.conversationalAi.agents.create({
  name: "My Assistant",
  enableVersioning: true,
  conversationConfig: {
    agent: {
      firstMessage: "Hello! How can I help?",
      language: "en",
      prompt: {
        prompt: "You are a helpful assistant.",
        llm: "gemini-2.0-flash",
        temperature: 0.7
      }
    },
    tts: { voiceId: "JBFqnCBsd6RMkjVDRZzb" }
  }
});

cURL

curl -X POST "https://api.elevenlabs.io/v1/convai/agents/create?enable_versioning=true" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "My Assistant", "conversation_config": {"agent": {"first_message": "Hello!", "language": "en", "prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'

Starting Conversations

Server-side (Python): Get signed URL for client connection:

signed_url = client.conversational_ai.conversations.get_signed_url(
    agent_id="your-agent-id",
    environment="staging",
)

Client-side (JavaScript):

import { Conversation } from "@elevenlabs/client";

const conversation = await Conversation.startSession({
  agentId: "your-agent-id",
  environment: "staging",
  onMessage: (msg) => console.log("Agent:", msg.message),
  onUserTranscript: (t) => console.log("User:", t.message),
  onError: (e) => console.error(e)
});

React Hook:

import { useConversation } from "@elevenlabs/react";

const conversation = useConversation({ onMessage: (msg) => console.log(msg) });
// Get a signed URL for the target environment from your backend, then:
await conversation.startSession({ signedUrl: token });

Configuration

Use GET /v1/convai/llm/list to inspect the current model catalog, including deprecation state, token/context limits, and capability flags such as image-input support.

Popular voices: JBFqnCBsd6RMkjVDRZzb (George), EXAVITQu4vr4xnSDxMaL (Sarah), onwK4e9ZLuTAKqWW03F9 (Daniel), XB0fDUnXU5powFXDhCwa (Charlotte)

Turn eagerness: patient (waits longer for user to finish), normal, or eager (responds quickly)

See Agent Configuration for all options.

Tools

Extend agents with webhook, client, or built-in system tools. Tools are defined inside conversation_config.agent.prompt:

Workspace environment variables can resolve per-environment server tool URLs, headers, and auth connections, and runtime system variables such as {{system__conversation_history}} can pass full conversation context into tool calls when needed.

"prompt": {
    "prompt": "You are a helpful assistant that can check the weather.",
    "llm": "gemini-2.0-flash",
    "tools": [
        # Webhook: server-side API call
        {"type": "webhook", "name": "get_weather", "description": "Get weather",
         "api_schema": {"url": "https://api.example.com/weather", "method": "POST",
             "request_body_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}},
        # Client: runs in the browser
        {"type": "client", "name": "show_product", "description": "Display a product",
         "parameters": {"type": "object", "properties": {"productId": {"type": "string"}}, "required": ["productId"]}}
    ],
    "built_in_tools": {
        "end_call": {},
        "transfer_to_number": {"transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]}
    }
}

Client tools run in browser:

clientTools: {
  show_product: async ({ productId }) => {
    document.getElementById("product").src = `/products/${productId}`;
    return { success: true };
  }
}

See Client Tools Reference for complete documentation.

Widget Embedding

<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>

Customize with attributes: avatar-image-url, action-text, start-call-text, end-call-text.

See Widget Embedding Reference for all options.

Outbound Calls

Make outbound phone calls using your agent via Twilio integration:

Python

response = client.conversational_ai.twilio.outbound_call(
    agent_id="your-agent-id",
    agent_phone_number_id="your-phone-number-id",
    to_number="+1234567890",
    call_recording_enabled=True
)
print(f"Call initiated: {response.conversation_id}")

JavaScript

const response = await client.conversationalAi.twilio.outboundCall({
  agentId: "your-agent-id",
  agentPhoneNumberId: "your-phone-number-id",
  toNumber: "+1234567890",
  callRecordingEnabled: true,
});

cURL

curl -X POST "https://api.elevenlabs.io/v1/convai/twilio/outbound-call" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
  -d '{"agent_id": "your-agent-id", "agent_phone_number_id": "your-phone-number-id", "to_number": "+1234567890", "call_recording_enabled": true}'

See Outbound Calls Reference for configuration overrides and dynamic variables.

Managing Agents

Using CLI (Recommended)

# List agents and check status
elevenlabs agents list
elevenlabs agents status

# Import agents from platform to local config
elevenlabs agents pull                      # Import all agents
elevenlabs agents pull --agent <agent-id>   # Import specific agent

# Push local changes to platform
elevenlabs agents push              # Upload configurations
elevenlabs agents push --dry-run    # Preview changes first

# Add tools
elevenlabs tools add-webhook "Weather API"
elevenlabs tools add-client "UI Tool"

Project Structure

The CLI creates a project structure for managing agents:

your_project/
├── agents.json       # Agent definitions
├── tools.json        # Tool configurations
├── tests.json        # Test configurations
├── agent_configs/    # Individual agent configs
├── tool_configs/     # Individual tool configs
└── test_configs/     # Individual test configs

SDK Examples

# List
agents = client.conversational_ai.agents.list()

# Get
agent = client.conversational_ai.agents.get(agent_id="your-agent-id")

# Update (partial - only include fields to change)
client.conversational_ai.agents.update(agent_id="your-agent-id", name="New Name")
client.conversational_ai.agents.update(agent_id="your-agent-id",
    conversation_config={
        "agent": {"prompt": {"prompt": "New instructions", "llm": "claude-sonnet-4"}}
    })

# Delete
client.conversational_ai.agents.delete(agent_id="your-agent-id")

See Agent Configuration for all configuration options and SDK examples.

Error Handling

try:
    agent = client.conversational_ai.agents.create(...)
except Exception as e:
    print(f"API error: {e}")

Common errors: 401 (invalid key), 404 (not found), 422 (invalid config), 429 (rate limit)

References

• Installation Guide - SDK setup and migration
• Agent Configuration - All config options and CRUD examples
• Client Tools - Webhook, client, and system tools
• Widget Embedding - Website integration
• Outbound Calls - Twilio phone call integration

Agent Configuration

Complete reference for configuring conversational AI agents.

Configuration Structure

agent = client.conversational_ai.agents.create(
    name="My Agent",
    conversation_config={
        "agent": {
            "first_message": "Hello!",
            "language": "en",
            "prompt": {           # LLM, system prompt, tools, and knowledge base
                "prompt": "You are helpful.",
                "llm": "gemini-2.0-flash",
                "tools": [...],
                "built_in_tools": {...}
            }
        },
        "tts": {...},             # Voice and TTS model settings
        "asr": {...},             # Speech recognition settings
        "turn": {...},            # Turn-taking behavior
        "conversation": {...},    # Duration, events, monitoring
        "vad": {...},             # Voice activity detection config
        "language_presets": {...}  # Language-specific overrides
    },
    platform_settings={...}       # Auth, call limits
)

conversation_config

Controls the real-time conversation behavior.

agent

conversation_config={
    "agent": {
        "first_message": "Hello! How can I help you today?",
        "language": "en",
        "disable_first_message_interruptions": False,
        "prompt": {
            "prompt": "You are a helpful assistant.",
            "llm": "gemini-2.0-flash",
            "temperature": 0.7
        }
    }
}

tts (Text-to-Speech)

conversation_config={
    "tts": {
        "voice_id": "JBFqnCBsd6RMkjVDRZzb",
        "model_id": "eleven_flash_v2_5",
        "stability": 0.5,
        "similarity_boost": 0.8,
        "speed": 1.0,
        "optimize_streaming_latency": 3,
        "expressive_mode": True
    }
}

Available TTS models for agents:

asr (Automatic Speech Recognition)

conversation_config={
    "asr": {
        "quality": "high",
        "keywords": ["ElevenLabs", "TechCorp"],
        "user_input_audio_format": "pcm_16000"
    }
}

turn (Turn-Taking)

conversation_config={
    "turn": {
        "turn_timeout": 7,
        "turn_eagerness": "normal",
        "silence_end_call_timeout": -1
    }
}

soft_timeout_config:

prompt (nested in conversation_config.agent)

Configures the LLM behavior. This object lives at conversation_config.agent.prompt:

conversation_config={
    "agent": {
        "prompt": {
            "prompt": "You are a helpful customer service agent...",
            "llm": "gemini-2.0-flash",
            "temperature": 0.7,
            "max_tokens": 500,
            "tools": [...],
            "built_in_tools": {...},
            "knowledge_base": [...]
        }
    }
}

Workspace environment variables let one agent configuration span multiple deployments. Use {{system_env__label}} in server tool and MCP server URLs, { "env_var_label": "orders_api_key" } for secret-backed tool headers, and { "env_var_label": "orders_oauth" } in auth_connection to resolve per-environment auth connections at runtime.

LLM Providers

Use GET /v1/convai/llm/list to inspect the current model catalog, including deprecation state, token/context limits, and capability flags such as image-input support.

Custom LLM

The custom_llm field is nested inside conversation_config.agent.prompt:

conversation_config={
    "agent": {
        "prompt": {
            "prompt": "You are helpful.",
            "llm": "custom-llm",
            "custom_llm": {
                "url": "https://your-llm-endpoint.com/v1/chat/completions",
                "model_id": "your-model-id",
                "api_key": {"secret_id": "your-secret-id"},
                "api_type": "chat_completions"  # or "responses"
            }
        }
    }
}

platform_settings

Platform-level configuration for security, limits, summaries, and widget behavior.

platform_settings={
    "summary_language": "en",
    "widget": {
        "show_agent_status": True,
        "show_conversation_id": True
    },
    "auth": {
        "enable_auth": True,
        "allowlist": [{"hostname": "example.com"}]
    },
    "call_limits": {
        "agent_concurrency_limit": 10,
        "daily_limit": 100
    }
}

Top-Level Fields

auth

call_limits

guardrails

Use platform_settings.guardrails to configure built-in safety controls for user input and agent behavior. The fields below cover the current schema additions that are most relevant in agent configs.

focus / prompt_injection:

content:

content.config:

content.config.\<category\>:

Blocking content guardrails and custom guardrails support a trigger_action that either ends the session immediately or retries the response. Retry removes the blocked reply, injects your feedback as a system message, and re-generates up to 3 times before the platform falls back to ending the session. Feedback templates can use {{trigger_reason}} and {{agent_message}}.

privacy

Use platform_settings.privacy to control recording, retention, and redaction behavior. The redaction-specific field is:

conversation_history_redaction:

widget

Use platform_settings.widget to configure the hosted widget and shareable page defaults. For client-side embed attributes, see the widget embedding reference.

conversation (inside conversation_config)

Additional Top-Level Fields

Knowledge Base / RAG

Knowledge base is configured inside conversation_config.agent.prompt:

agent = client.conversational_ai.agents.create(
    name="Support Agent",
    conversation_config={
        "agent": {
            "prompt": {
                "prompt": "You are a support agent. Use the knowledge base to answer questions.",
                "llm": "gemini-2.0-flash",
                "knowledge_base": [
                    {"type": "file", "id": "doc-id", "name": "Product Guide", "usage_mode": "auto"}
                ],
                "rag": {
                    "enabled": True,
                    "embedding_model": "qwen3_embedding_4b",
                    "max_documents_length": 50000,
                    "max_retrieved_rag_chunks_count": 20
                }
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
    }
)

rag.embedding_model supports e5_mistral_7b_instruct, multilingual_e5_large_instruct, and qwen3_embedding_4b.

CRUD Operations

Using CLI (Recommended)

# Initialize project
elevenlabs agents init

# Create agent from template
elevenlabs agents add "My Agent" --template complete
elevenlabs agents add "Support Bot" --template customer-service

# List agents
elevenlabs agents list

# Check status
elevenlabs agents status

# Push local changes to platform
elevenlabs agents push
elevenlabs agents push --dry-run    # Preview changes first

# Import agents from platform
elevenlabs agents pull                      # Import all
elevenlabs agents pull --agent <agent-id>   # Import specific agent
elevenlabs agents pull --update             # Override local configs

# View available templates
elevenlabs agents templates list
elevenlabs agents templates show <template-name>

# Add tools
elevenlabs tools add-webhook "API Tool"
elevenlabs tools add-client "UI Tool"

# Generate widget code
elevenlabs agents widget <agent-id>

SDK: List Agents

agents = client.conversational_ai.agents.list()
for agent in agents.agents:
    print(f"{agent.name}: {agent.agent_id}")

const agents = await client.conversationalAi.agents.list();

curl -X GET "https://api.elevenlabs.io/v1/convai/agents" -H "xi-api-key: $ELEVENLABS_API_KEY"

SDK: Get Agent

agent = client.conversational_ai.agents.get(agent_id="your-agent-id")

const agent = await client.conversationalAi.agents.get("your-agent-id");

curl -X GET "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" -H "xi-api-key: $ELEVENLABS_API_KEY"

SDK: Update Agent

Only include fields you want to change. All other settings remain unchanged.

Python:

# Update name
client.conversational_ai.agents.update(agent_id="id", name="New Name")

# Update TTS voice
client.conversational_ai.agents.update(agent_id="id", conversation_config={
    "tts": {"voice_id": "EXAVITQu4vr4xnSDxMaL", "model_id": "eleven_flash_v2_5"}
})

# Update prompt/LLM (nested in agent)
client.conversational_ai.agents.update(agent_id="id", conversation_config={
    "agent": {"prompt": {"prompt": "New instructions.", "llm": "claude-sonnet-4", "temperature": 0.8}}
})

# Update first message
client.conversational_ai.agents.update(agent_id="id", conversation_config={
    "agent": {"first_message": "Welcome back!"}
})

# Update platform settings
client.conversational_ai.agents.update(agent_id="id", platform_settings={
    "auth": {"enable_auth": True, "allowlist": [{"hostname": "myapp.com"}]}
})

JavaScript:

await client.conversationalAi.agents.update("id", { name: "New Name" });
await client.conversationalAi.agents.update("id", {
  conversationConfig: { tts: { voiceId: "EXAVITQu4vr4xnSDxMaL" } }
});
await client.conversationalAi.agents.update("id", {
  conversationConfig: { agent: { prompt: { prompt: "New instructions.", llm: "claude-sonnet-4" } } }
});

cURL:

curl -X PATCH "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "New Name"}'

Updatable Fields

SDK: Delete Agent

client.conversational_ai.agents.delete(agent_id="your-agent-id")

await client.conversationalAi.agents.delete("your-agent-id");

curl -X DELETE "https://api.elevenlabs.io/v1/convai/agents/your-agent-id" -H "xi-api-key: $ELEVENLABS_API_KEY"

CI/CD Integration

Use the CLI in your deployment pipeline:

# Set API key as environment variable
export ELEVENLABS_API_KEY="your-api-key"

# Push changes (non-interactive)
elevenlabs agents push

Example Configurations

Customer Support Agent

agent = client.conversational_ai.agents.create(
    name="Support Agent",
    conversation_config={
        "agent": {
            "first_message": "Hi! Thanks for calling TechCorp support.",
            "language": "en",
            "prompt": {
                "prompt": "You are a customer support agent. Be helpful, professional, concise.",
                "llm": "gemini-2.0-flash",
                "temperature": 0.5,
                "built_in_tools": {
                    "end_call": {},
                    "transfer_to_number": {
                        "transfers": [{"transfer_destination": {"type": "phone", "phone_number": "+1234567890"}, "condition": "User asks for human support"}]
                    }
                }
            }
        },
        "tts": {"voice_id": "XB0fDUnXU5powFXDhCwa", "model_id": "eleven_flash_v2_5"},
        "turn": {"turn_eagerness": "normal", "turn_timeout": 7},
        "conversation": {"max_duration_seconds": 900}
    }
)

Low-Latency Assistant

agent = client.conversational_ai.agents.create(
    name="Quick Assistant",
    conversation_config={
        "agent": {
            "first_message": "Hey! What do you need?",
            "prompt": {
                "prompt": "Fast, efficient assistant. Brief answers.",
                "llm": "gemini-2.0-flash",
                "temperature": 0.3,
                "max_tokens": 100
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5", "optimize_streaming_latency": 4},
        "turn": {"turn_eagerness": "eager", "turn_timeout": 3}
    }
)

Client Tools

Extend your agent with custom capabilities. Tools let the agent take actions beyond just talking.

Tool Types

Where Tools Live

Tools are defined inside conversation_config.agent.prompt. Webhook and client tools go in the tools array. System tools go in built_in_tools:

conversation_config={
    "agent": {
        "prompt": {
            "prompt": "You are helpful.",
            "llm": "gemini-2.0-flash",
            "tools": [...],            # Webhook and client tools
            "built_in_tools": {...}     # System tools (end_call, transfer, etc.)
        }
    }
}

Webhook Tools

Execute server-side logic when the agent needs external data or actions.

Basic Webhook

agent = client.conversational_ai.agents.create(
    name="Weather Assistant",
    conversation_config={
        "agent": {
            "prompt": {
                "prompt": "You are a helpful assistant that can check the weather.",
                "llm": "gemini-2.0-flash",
                "tools": [{
                    "type": "webhook",
                    "name": "get_weather",
                    "description": "Get current weather for a city. Use when user asks about weather.",
                    "api_schema": {
                        "url": "https://api.example.com/weather",
                        "method": "POST",
                        "request_headers": {
                            "Authorization": "Bearer {{API_KEY}}"
                        },
                        "request_body_schema": {
                            "type": "object",
                            "properties": {
                                "city": {
                                    "type": "string",
                                    "description": "City name, e.g., 'San Francisco'"
                                },
                                "units": {
                                    "type": "string",
                                    "enum": ["celsius", "fahrenheit"],
                                    "description": "Temperature units"
                                }
                            },
                            "required": ["city"]
                        }
                    }
                }]
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
    }
)

Webhook Request Format

When the agent calls a webhook tool, ElevenLabs sends:

{
  "tool_call_id": "call_abc123",
  "tool_name": "get_weather",
  "parameters": {
    "city": "San Francisco",
    "units": "fahrenheit"
  },
  "conversation_id": "conv_xyz789"
}

Webhook Response Format

Your server should respond with:

{
  "result": "The weather in San Francisco is 68°F and sunny."
}

Or for structured data:

{
  "result": {
    "temperature": 68,
    "condition": "sunny",
    "humidity": 45
  }
}

Webhook with Authentication

# Inside conversation_config.agent.prompt.tools:
{
    "type": "webhook",
    "name": "lookup_order",
    "description": "Look up order status by order ID",
    "response_timeout_secs": 10,
    "api_schema": {
        "url": "https://api.mystore.com/orders/lookup",
        "method": "POST",
        "request_headers": {
            "Authorization": "Bearer {{ORDER_API_KEY}}",
            "X-Store-ID": "store_123"
        },
        "request_body_schema": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "Order ID (e.g., ORD-12345)"
                }
            },
            "required": ["order_id"]
        }
    }
}

Use workspace environment variables to keep a single server tool configuration working across staging and production. {{system_env__label}} works in server tool URLs, secret environment variables can populate request_headers, and auth-connection environment variables can populate api_schema.auth_connection. The same environment-variable resolution model also applies to MCP server connections.

{
  "api_schema": {
    "url": "https://{{system_env__api_host}}.example.com/orders",
    "method": "GET",
    "request_headers": {
      "X-Api-Key": { "env_var_label": "orders_api_key" }
    },
    "auth_connection": { "env_var_label": "orders_oauth" }
  }
}

Workspace auth connections support OAuth2 client credentials, OAuth2 JWT, private key JWT, basic auth, bearer auth, and custom header auth.

System dynamic variables are also available in tool parameters and headers. Use {{system__conversation_history}} when a webhook or sub-agent needs the full conversation context as a lazily evaluated JSON history object with user, agent, and tool entries.

Webhook Tool Options

Note: The default api_schema.method is GET. Always set "method": "POST" explicitly for webhook tools that send request bodies.

Server Implementation (Node.js)

app.post("/webhook/get_weather", async (req, res) => {
  const { parameters, conversation_id } = req.body;
  const { city, units = "fahrenheit" } = parameters;

  // Fetch weather from your data source
  const weather = await weatherService.get(city, units);

  res.json({
    result: `It's ${weather.temp}°${units === "celsius" ? "C" : "F"} and ${weather.condition} in ${city}.`,
  });
});

Server Implementation (Python)

@app.post("/webhook/get_weather")
async def get_weather(request: Request):
    data = await request.json()
    city = data["parameters"]["city"]
    units = data["parameters"].get("units", "fahrenheit")

    # Fetch weather from your data source
    weather = weather_service.get(city, units)

    return {
        "result": f"It's {weather['temp']}°{'C' if units == 'celsius' else 'F'} and {weather['condition']} in {city}."
    }

Client Tools

Execute JavaScript in the user's browser. Useful for UI updates, navigation, or accessing browser APIs.

Defining Client Tools

Client tools are registered when starting a conversation:

import { Conversation } from "@elevenlabs/client";

const conversation = await Conversation.startSession({
  agentId: "your-agent-id",
  clientTools: {
    show_product: async ({ productId }) => {
      // Update UI to show product
      const modal = document.getElementById("product-modal");
      modal.innerHTML = await fetchProductCard(productId);
      modal.showModal();
      return { success: true, message: "Showing product" };
    },

    navigate_to: async ({ page }) => {
      // Navigate to a page
      window.location.href = `/${page}`;
      return { success: true };
    },

    save_preference: async ({ key, value }) => {
      // Store in localStorage
      localStorage.setItem(key, value);
      return { saved: true };
    },
  },
});

Registering Client Tools with Agent

Tell the agent about available client tools in conversation_config.agent.prompt.tools:

agent = client.conversational_ai.agents.create(
    name="Shopping Assistant",
    conversation_config={
        "agent": {
            "prompt": {
                "prompt": """You are a shopping assistant.
When users want to see a product, use show_product.
When users want to go somewhere, use navigate_to.""",
                "llm": "gemini-2.0-flash",
                "tools": [
                    {
                        "type": "client",
                        "name": "show_product",
                        "description": "Display a product card to the user",
                        "parameters": {
                            "type": "object",
                            "properties": {
                                "productId": {
                                    "type": "string",
                                    "description": "Product ID to display"
                                }
                            },
                            "required": ["productId"]
                        }
                    },
                    {
                        "type": "client",
                        "name": "navigate_to",
                        "description": "Navigate user to a different page",
                        "parameters": {
                            "type": "object",
                            "properties": {
                                "page": {
                                    "type": "string",
                                    "enum": ["cart", "checkout", "account", "home"],
                                    "description": "Page to navigate to"
                                }
                            },
                            "required": ["page"]
                        }
                    }
                ]
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}
    }
)

Client Tool Options

Client Tool Return Values

Return data that the agent can use in conversation:

clientTools: {
  check_cart: async () => {
    const cart = JSON.parse(localStorage.getItem("cart") || "[]");
    return {
      itemCount: cart.length,
      total: cart.reduce((sum, item) => sum + item.price, 0),
      items: cart.map((item) => item.name),
    };
  };
}

The agent receives this data and can say: "You have 3 items in your cart totaling $45.99."

System Tools (built_in_tools)

Built-in tools provided by ElevenLabs. These are configured in conversation_config.agent.prompt.built_in_tools (not in the tools array):

"built_in_tools": {
    "end_call": {},
    "transfer_to_number": {...},
    "transfer_to_agent": {...},
    "language_detection": {},
    "skip_turn": {},
    "voicemail_detection": {...},
    "play_keypad_touch_tone": {}
}

Current API schemas also expose agent_prompt_change, memory_entry_create, memory_entry_delete, memory_entry_search, and memory_entry_update in built_in_tools.

end_call

Ends the current conversation:

"built_in_tools": {
    "end_call": {}
}

The agent can say "Goodbye!" and then end the call programmatically.

transfer_to_number

Transfer to a phone number (requires telephony integration):

"built_in_tools": {
    "transfer_to_number": {
        "transfers": [{
            "transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
            "condition": "User asks to speak with a human agent"
        }]
    }
}

transfer_to_agent

Transfer to another ElevenLabs agent:

"built_in_tools": {
    "transfer_to_agent": {
        "transfers": [{
            "agent_id": "other-agent-id",
            "condition": "User asks about sales"
        }]
    }
}

Best Practices

Tool Descriptions

Write clear descriptions so the LLM knows when to use tools:

# Good - specific and actionable
"description": "Look up order status. Use when customer asks about their order, delivery, or shipping."

# Bad - vague
"description": "Order tool"

Parameter Descriptions

Help the LLM extract correct values:

"parameters": {
    "type": "object",
    "properties": {
        "order_id": {
            "type": "string",
            "description": "Order ID in format ORD-XXXXX (e.g., ORD-12345)"
        },
        "email": {
            "type": "string",
            "description": "Customer email address for verification"
        }
    }
}

Error Handling

Configure how tool errors are shared with the agent using tool_error_handling_mode:

Return helpful error messages:

// Server webhook
app.post("/webhook/lookup_order", async (req, res) => {
  const { order_id } = req.body.parameters;

  const order = await db.orders.find(order_id);

  if (!order) {
    return res.json({
      result: {
        error: true,
        message: `Order ${order_id} not found. Please verify the order ID.`,
      },
    });
  }

  res.json({ result: order });
});

Timeouts

Set reasonable timeouts for webhooks using response_timeout_secs (5-120 seconds, default 20):

{
    "type": "webhook",
    "name": "slow_operation",
    "description": "Run a slow operation",
    "response_timeout_secs": 30,
    "api_schema": {
        "url": "https://api.example.com/slow-operation",
        "method": "POST"
    }
}

Complete Example

agent = client.conversational_ai.agents.create(
    name="E-commerce Assistant",
    conversation_config={
        "agent": {
            "first_message": "Hi! How can I help you today?",
            "language": "en",
            "prompt": {
                "prompt": """You are an e-commerce support assistant.

Available actions:
- lookup_order: Check order status
- show_product: Display products to customer
- end_call: End conversation politely
- transfer_to_number: Transfer to human support

Always verify order ID before lookup. Offer transfer for complex issues.""",
                "llm": "gemini-2.0-flash",
                "tools": [
                    # Webhook: Server-side order lookup
                    {
                        "type": "webhook",
                        "name": "lookup_order",
                        "description": "Look up order status by order ID or email",
                        "api_schema": {
                            "url": "https://api.mystore.com/orders/lookup",
                            "method": "POST",
                            "request_headers": {"Authorization": "Bearer {{API_KEY}}"},
                            "request_body_schema": {
                                "type": "object",
                                "properties": {
                                    "order_id": {"type": "string"},
                                    "email": {"type": "string"}
                                }
                            }
                        }
                    },
                    # Client: Browser-side product display
                    {
                        "type": "client",
                        "name": "show_product",
                        "description": "Display product details to the customer",
                        "parameters": {
                            "type": "object",
                            "properties": {
                                "product_id": {"type": "string"}
                            },
                            "required": ["product_id"]
                        }
                    }
                ],
                "built_in_tools": {
                    "end_call": {},
                    "transfer_to_number": {
                        "transfers": [{
                            "transfer_destination": {"type": "phone", "phone_number": "+1234567890"},
                            "condition": "User asks for human support"
                        }]
                    }
                }
            }
        },
        "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_flash_v2_5"}
    }
)

Installation

CLI (Recommended)

The ElevenLabs CLI is the recommended way to create and manage agents:

npm install -g @elevenlabs/cli
# or
pnpm add -g @elevenlabs/cli
# or
yarn global add @elevenlabs/cli

Requires Node.js 16.0.0 or higher.

Authentication

elevenlabs auth login          # Authenticate with API key
elevenlabs auth whoami         # Verify current login status
elevenlabs auth logout         # Remove stored credentials

API keys are securely stored in ~/.agents/api_keys.json.

Quick Start

# Initialize a new project
elevenlabs agents init

# Create an agent from template
elevenlabs agents add "My Assistant" --template complete

# Push to ElevenLabs platform
elevenlabs agents push

JavaScript / TypeScript SDK

For programmatic access and client-side integration:

npm install @elevenlabs/elevenlabs-js

Important: Always use @elevenlabs/elevenlabs-js. The old elevenlabs npm package (v1.x) is deprecated and should not be used.

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

// Option 1: Environment variable (recommended)
// Set ELEVENLABS_API_KEY in your environment
const client = new ElevenLabsClient();

// Option 2: Pass directly
const client = new ElevenLabsClient({ apiKey: "your-api-key" });

Migrating from deprecated packages

If you have old packages installed, remove them:

# Remove deprecated packages
npm uninstall elevenlabs

# Install the current packages
npm install @elevenlabs/elevenlabs-js

# For client-side/browser usage, also install:
npm install @elevenlabs/client  # Browser client
npm install @elevenlabs/react   # React hooks

Import changes:

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
import { Conversation } from "@elevenlabs/client";
import { useConversation } from "@elevenlabs/react";

Python

pip install elevenlabs

from elevenlabs import ElevenLabs

# Option 1: Environment variable (recommended)
# Set ELEVENLABS_API_KEY in your environment
client = ElevenLabs()

# Option 2: Pass directly
client = ElevenLabs(api_key="your-api-key")

cURL / REST API

Set your API key as an environment variable:

export ELEVENLABS_API_KEY="your-api-key"

Include in requests via the xi-api-key header:

curl -X POST "https://api.elevenlabs.io/v1/convai/agents/create" \
  -H "xi-api-key: $ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "My Agent", "conversation_config": {"agent": {"prompt": {"prompt": "You are helpful.", "llm": "gemini-2.0-flash"}}, "tts": {"voice_id": "JBFqnCBsd6RMkjVDRZzb"}}}'

Getting an API Key

1. Sign up at elevenlabs.io 2. Go to API Keys 3. Click Create API Key 4. Copy and store securely

Or use the setup-api-key skill for guided setup.

Environment Variables

Outbound Calls

Make outbound phone calls using your ElevenLabs agent via Twilio integration.

Prerequisites

1. A configured ElevenLabs agent 2. A Twilio phone number linked to your agent (obtain agent_phone_number_id from ElevenLabs dashboard) 3. Your ElevenLabs API key

Basic Usage

See the main agents skill for basic Python, JavaScript, and cURL examples.

Request Parameters

Response

{
  "success": true,
  "message": "Call initiated successfully",
  "conversation_id": "conv_abc123",
  "callSid": "CA1234567890abcdef"
}

Customizing the Call

Override agent settings for a specific call using conversation_initiation_client_data:

Python

response = client.conversational_ai.twilio.outbound_call(
    agent_id="your-agent-id",
    agent_phone_number_id="your-phone-number-id",
    to_number="+1234567890",
    call_recording_enabled=True,
    conversation_initiation_client_data={
        "conversation_config_override": {
            "agent": {
                "first_message": "Hello! This is a reminder about your appointment tomorrow.",
                "language": "en"
            },
            "tts": {
                "voice_id": "JBFqnCBsd6RMkjVDRZzb"
            }
        },
        "dynamic_variables": {
            "customer_name": "John",
            "appointment_time": "2:00 PM"
        }
    }
)

JavaScript

const response = await client.conversationalAi.twilio.outboundCall({
  agentId: "your-agent-id",
  agentPhoneNumberId: "your-phone-number-id",
  toNumber: "+1234567890",
  callRecordingEnabled: true,
  conversationInitiationClientData: {
    conversationConfigOverride: {
      agent: {
        firstMessage: "Hello! This is a reminder about your appointment tomorrow.",
        language: "en",
      },
      tts: {
        voiceId: "JBFqnCBsd6RMkjVDRZzb",
      },
    },
    dynamicVariables: {
      customer_name: "John",
      appointment_time: "2:00 PM",
    },
  },
});

Configuration Overrides

Agent Settings

TTS Settings

Telephony Call Configuration

Dynamic Variables

Pass custom data to your agent's prompt using dynamic_variables. Reference them in your agent's prompt with {{variable_name}} syntax.

When assigning dynamic variables, you can use the sanitize option to remove sensitive values from tool responses before they are sent to the LLM and transcript, while still allowing variable assignment:

Complete Example

from elevenlabs import ElevenLabs

client = ElevenLabs()

# Make personalized outbound calls
customers = [
    {"name": "Alice", "phone": "+1234567890", "balance": "$150.00"},
    {"name": "Bob", "phone": "+0987654321", "balance": "$75.50"},
]

for customer in customers:
    try:
        response = client.conversational_ai.twilio.outbound_call(
            agent_id="payment-reminder-agent",
            agent_phone_number_id="your-phone-number-id",
            to_number=customer["phone"],
            call_recording_enabled=True,
            conversation_initiation_client_data={
                "conversation_config_override": {
                    "agent": {
                        "first_message": f"Hello {customer['name']}, this is a friendly reminder about your account."
                    }
                },
                "dynamic_variables": {
                    "customer_name": customer["name"],
                    "balance": customer["balance"]
                }
            }
        )
        print(f"Called {customer['name']}: {response.conversation_id}")
    except Exception as e:
        print(f"Failed to call {customer['name']}: {e}")

Widget Embedding

Add a voice AI agent to any website with the ElevenLabs conversation widget.

Basic Embed

<elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
<script src="https://unpkg.com/@elevenlabs/convai-widget-embed" async type="text/javascript"></script>

This creates a floating button that users can click to start a voice conversation.

Note: Widgets currently require public agents with authentication disabled. For authenticated flows, use the SDKs.

Widget Attributes

Required

Appearance

Text Labels

Behavior

Examples

Custom Avatar

<elevenlabs-convai
  agent-id="your-agent-id"
  avatar-image-url="https://example.com/your-avatar.png"
></elevenlabs-convai>

Custom Colors

<elevenlabs-convai
  agent-id="your-agent-id"
  avatar-orb-color-1="#ff6b6b"
  avatar-orb-color-2="#ffd93d"
></elevenlabs-convai>

Custom Text

<elevenlabs-convai
  agent-id="your-agent-id"
  action-text="Chat with our AI assistant"
  start-call-text="Begin conversation"
  end-call-text="Hang up"
></elevenlabs-convai>

Expanded Variant

<elevenlabs-convai
  agent-id="your-agent-id"
  variant="expanded"
></elevenlabs-convai>

Full Customization

<elevenlabs-convai
  agent-id="your-agent-id"
  avatar-image-url="https://example.com/support-agent.png"
  avatar-orb-color-1="#4f46e5"
  avatar-orb-color-2="#818cf8"
  action-text="Talk to Support"
  start-call-text="Start voice chat"
  end-call-text="End conversation"
  expand-text="Open assistant"
  collapse-text="Minimize"
></elevenlabs-convai>

CSS Customization

The widget uses Shadow DOM but exposes CSS custom properties:

elevenlabs-convai {
  --elevenlabs-convai-widget-width: 400px;
  --elevenlabs-convai-widget-height: 600px;
}

Positioning

By default, the widget appears in the bottom-right corner. Override with CSS:

elevenlabs-convai {
  position: fixed;
  bottom: 20px;
  right: 20px;
  /* Or position differently */
  left: 20px;
  right: auto;
}

Z-Index

elevenlabs-convai {
  z-index: 9999;
}

JavaScript Control

Access the widget element to control it programmatically:

<elevenlabs-convai id="my-widget" agent-id="your-agent-id"></elevenlabs-convai>

<script>
  const widget = document.getElementById("my-widget");

  // Start a conversation
  widget.startConversation();

  // End the conversation
  widget.endConversation();

  // Listen for events
  widget.addEventListener("conversationStarted", () => {
    console.log("Conversation started");
  });

  widget.addEventListener("conversationEnded", () => {
    console.log("Conversation ended");
  });
</script>

Custom Trigger Button

Hide the default widget and use your own button:

<style>
  elevenlabs-convai {
    display: none;
  }
</style>

<button onclick="document.getElementById('widget').startConversation()">
  Talk to AI
</button>

<elevenlabs-convai id="widget" agent-id="your-agent-id"></elevenlabs-convai>

Authentication

For agents with authentication enabled, pass a signed URL:

<elevenlabs-convai id="widget" agent-id="your-agent-id"></elevenlabs-convai>

<script>
  async function startAuthenticatedConversation() {
    // Get signed URL from your backend
    const response = await fetch("/api/get-signed-url");
    const { signedUrl } = await response.json();

    const widget = document.getElementById("widget");
    widget.setAttribute("signed-url", signedUrl);
    widget.startConversation();
  }
</script>

Your backend:

@app.get("/api/get-signed-url")
def get_signed_url():
    signed_url = client.conversational_ai.conversations.get_signed_url(
        agent_id="your-agent-id"
    )
    return {"signedUrl": signed_url.signed_url}

Mobile Considerations

Responsive Positioning

/* Desktop: bottom-right */
elevenlabs-convai {
  position: fixed;
  bottom: 20px;
  right: 20px;
}

/* Mobile: full-width bottom */
@media (max-width: 768px) {
  elevenlabs-convai {
    bottom: 0;
    right: 0;
    left: 0;
    --elevenlabs-convai-widget-width: 100%;
  }
}

Touch-Friendly

The widget is touch-optimized by default. For better mobile UX:

@media (max-width: 768px) {
  elevenlabs-convai {
    /* Larger touch target */
    transform: scale(1.1);
    transform-origin: bottom right;
  }
}

Multiple Widgets

You can have multiple widgets for different agents:

<elevenlabs-convai
  agent-id="support-agent-id"
  action-text="Support"
  style="right: 20px"
></elevenlabs-convai>

<elevenlabs-convai
  agent-id="sales-agent-id"
  action-text="Sales"
  style="right: 100px"
></elevenlabs-convai>

Framework Integration

React

function App() {
  useEffect(() => {
    // Load widget script
    const script = document.createElement("script");
    script.src = "https://unpkg.com/@elevenlabs/convai-widget-embed";
    script.async = true;
    document.body.appendChild(script);

    return () => document.body.removeChild(script);
  }, []);

  return (
    <div>
      <elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
    </div>
  );
}

Vue

<template>
  <div>
    <elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
  </div>
</template>

<script setup>
import { onMounted } from "vue";

onMounted(() => {
  const script = document.createElement("script");
  script.src = "https://unpkg.com/@elevenlabs/convai-widget-embed";
  script.async = true;
  document.body.appendChild(script);
});
</script>

Next.js

import Script from "next/script";

export default function Page() {
  return (
    <>
      <Script
        src="https://unpkg.com/@elevenlabs/convai-widget-embed"
        strategy="lazyOnload"
      />
      <elevenlabs-convai agent-id="your-agent-id"></elevenlabs-convai>
    </>
  );
}

Troubleshooting

Widget Not Appearing

1. Check that the agent ID is correct 2. Verify the script is loaded (check Network tab) 3. Check for JavaScript errors in console 4. Ensure no CSS is hiding the widget

Audio Issues

1. Ensure HTTPS (microphone requires secure context) 2. Check browser permissions for microphone 3. Test in a supported browser (Chrome, Firefox, Safari, Edge)

CORS Errors

If using authentication, ensure your domain is in the agent's allowlist:

platform_settings={
    "auth": {
        "enable_auth": True,
        "allowlist": ["https://yourdomain.com"]
    }
}