Elevenlabs

ElevenLabs Plugin

Comprehensive ElevenLabs AI audio integration for voice-enabled applications with TTS, STT, voice cloning, conversational AI agents, and Vercel AI SDK support.

Overview

The ElevenLabs plugin provides complete integration with ElevenLabs voice AI platform, enabling developers to build production-ready voice applications with text-to-speech, speech-to-text, voice cloning, and conversational agents with MCP tool calling.

Features

Core Capabilities

• Text-to-Speech (TTS): 4 voice models (v3 Alpha, Flash v2.5, Turbo v2.5, Multilingual v2)
• Speech-to-Text (STT): Scribe v1 with 99 languages, speaker diarization, timestamps
• Voice Cloning: Instant (1 min) and professional (30+ min) voice cloning
• Voice Library: Access to 70+ pre-made voices
• Voice Design: Generate voices from text descriptions

Advanced Features

• Agents Platform: Conversational AI agents with full MCP integration
• MCP Support: Zapier MCP server and custom MCP servers for tool calling
• Streaming: Real-time WebSocket audio streaming (TTS and STT)
• Sound Effects: AI-generated cinematic sound effects
• Dubbing: Multi-language audio dubbing (70+ languages)
• Voice Changer: Transform voice characteristics
• Voice Isolator: Remove background noise

Vercel AI SDK Integration

• @ai-sdk/elevenlabs Provider: Official Vercel AI SDK provider
• experimental_transcribe: STT with AI workflows
• Multi-modal Chat: Voice input → LLM → Voice output pipelines

Production Features

• Rate Limiting: Concurrent request management
• Monitoring: Usage tracking, latency metrics, cost estimation
• Error Handling: Retry logic, circuit breakers, graceful degradation
• Security: Secure API key management, input validation
• Cost Optimization: Model selection, caching strategies

Commands

Setup & Initialization

• /elevenlabs:init [project-name] - Initialize ElevenLabs project with SDK and auth

Feature Commands

• /elevenlabs:add-text-to-speech - Add TTS with all 4 voice models
• /elevenlabs:add-speech-to-text - Add STT with Scribe v1 and Vercel AI SDK
• /elevenlabs:add-vercel-ai-sdk - Integrate Vercel AI SDK provider
• /elevenlabs:add-voice-management - Add voice cloning and library features
• /elevenlabs:add-agents-platform - Add conversational agents with MCP
• /elevenlabs:add-streaming - Add real-time WebSocket streaming
• /elevenlabs:add-advanced-features - Add sound effects, dubbing, voice changer
• /elevenlabs:add-production - Add rate limiting, monitoring, security

Orchestrator

• /elevenlabs:build-full-stack [app-name] - Build complete voice application (chains all commands)

Agents

• elevenlabs-setup: Initialize project with SDK and framework-specific setup
• elevenlabs-tts-integrator: Implement TTS with all voice models
• elevenlabs-stt-integrator: Implement STT with Vercel AI SDK integration
• elevenlabs-voice-manager: Implement voice cloning and library features
• elevenlabs-agents-builder: Build conversational agents with MCP
• elevenlabs-production-agent: Implement production features

Quick Start

1. Initialize Project

/elevenlabs:init my-voice-app

2. Add Features

# Add text-to-speech
/elevenlabs:add-text-to-speech

# Add speech-to-text with Vercel AI SDK
/elevenlabs:add-speech-to-text

# Add Vercel AI SDK for multi-modal chat
/elevenlabs:add-vercel-ai-sdk

3. Build Full Stack (All-in-One)

/elevenlabs:build-full-stack my-voice-app

Framework Support

• Next.js 15: App Router, React Server Components, Vercel AI SDK integration
• React: Client-side voice applications
• Python/FastAPI: Backend APIs with async processing
• Node.js: Server-side voice processing

Voice Models

Model	Latency	Languages	Best For
Eleven v3 Alpha	~250-300ms	70+	Highest quality, emotional range
Flash v2.5	~75ms	32	Ultra-low latency, real-time
Turbo v2.5	~250-300ms	32	Balanced speed/quality
Multilingual v2	~300ms	29	Stable, long-form content

MCP Integration

ElevenLabs has full native MCP support via the Agents Platform:

• Zapier MCP Server: Access hundreds of tools (https://zapier.com/mcp)
• Custom MCP Servers: Build your own tool integrations
• Security Controls: Fine-grained approval modes (always ask, auto-approve, disabled)
• Real-time: SSE and HTTP transport support

MCP Dashboard

Configure MCP servers at: https://elevenlabs.io/app/agents/integrations

Pricing Tiers

Text-to-Speech

• Free: 10,000 chars/month
• Starter: $5/month - 30,000 chars
• Creator: $22/month - 100,000 chars
• Pro: $99/month - 500,000 chars

Speech-to-Text

• Starter: $5/month - 12.5 hours
• Creator: $22/month - ~63 hours
• Pro: $99/month - 300 hours

Environment Variables

Create .env file with:

ELEVENLABS_API_KEY=your_api_key_here

Get your API key: https://elevenlabs.io/app/settings/api-keys

Documentation

Official ElevenLabs Docs

• Main Docs: https://elevenlabs.io/docs/overview
• API Reference: https://elevenlabs.io/docs/api-reference/introduction
• Developer Quickstart: https://elevenlabs.io/docs/quickstart
• Models: https://elevenlabs.io/docs/models
• Agents Platform: https://elevenlabs.io/docs/agents-platform/overview
• MCP Integration: https://elevenlabs.io/docs/agents-platform/customization/tools/mcp

Vercel AI SDK Integration

• ElevenLabs Provider: https://ai-sdk.dev/providers/ai-sdk-providers/elevenlabs
• Transcription Guide: https://ai-sdk.dev/docs/ai-sdk-core/transcription
• STT Cookbook: https://elevenlabs.io/docs/cookbooks/speech-to-text/vercel-ai-sdk

Local Documentation

• docs/elevenlabs-documentation-complete-links.md - Complete URL reference
• docs/mcp-integration-examples.md - MCP integration patterns

Examples

Basic TTS (TypeScript)

import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";

const client = new ElevenLabsClient({
  apiKey: process.env.ELEVENLABS_API_KEY
});

const audio = await client.textToSpeech.convert({
  text: "Hello world!"
  voice_id: "JBFqnCBsd6RMkjVDRZzb"
  model_id: "eleven_flash_v2_5"
});

STT with Vercel AI SDK (TypeScript)

import { elevenlabs } from '@ai-sdk/elevenlabs';
import { experimental_transcribe as transcribe } from 'ai';

const result = await transcribe({
  model: elevenlabs.transcription('scribe_v1')
  audio: audioFile
  providerOptions: {
    elevenlabs: {
      languageCode: 'en'
      diarize: true
      timestampsGranularity: 'word'
    }
  }
});

Python TTS

from elevenlabs.client import ElevenLabs
import os

client = ElevenLabs(api_key=os.getenv("ELEVENLABS_API_KEY"))

audio = client.text_to_speech.convert(
    text="Hello world!"
    voice_id="JBFqnCBsd6RMkjVDRZzb"
    model_id="eleven_flash_v2_5"
)

Contributing

This plugin is part of the ai-dev-marketplace. For issues, feature requests, or contributions, please visit the repository.

License

MIT

Support

• ElevenLabs Docs: https://elevenlabs.io/docs
• Discord Community: https://discord.gg/elevenlabs
• Support: https://elevenlabs.io/support