
Tavily Web
- 375 installs
- 2 repo stars
- Updated April 21, 2026
- benedictking/tavily-web
tavily-web is a skill that calls the Tavily API to run web search, content extraction, site crawling, site mapping, and structured research from inside an agent.
About
This skill lets an agent run web research through the Tavily API, choosing among five endpoints: search for latest info, extract for content from URLs, crawl for traversing a site, map for discovering site structure, and research for structured schema output. It uses a two-phase architecture where the main context assembles the JSON payload and a forked sub-skill executes the HTTP call to conserve tokens. A developer uses it when they need current web information, page content, or a site map inside an agent workflow.
- Wraps the Tavily API for search, extract, crawl, map, and structured research
- Two-phase design: main skill builds the JSON payload, a sub-skill runs the HTTP call
- Requires a TAVILY_API_KEY via env var or a .env file
Tavily Web by the numbers
- 375 all-time installs (skills.sh)
- Ranked #463 of 2,715 Automation & Workflows skills by installs in the Skillselion catalog
- Data as of Jul 28, 2026 (Skillselion catalog sync)
tavily-web capabilities & compatibility
Requires a Tavily API key set via TAVILY_API_KEY env var or .env file
- Capabilities
- web search · web scraping · research · site map
- Use cases
- research · web search · web scraping
- Pricing
- Bring your own API key
What tavily-web says it does
Web research skill using Tavily API for search, extract, crawl, map, and structured research tasks. Use when you need latest information, extract content from URLs, or discover site structure.
Two ways to configure API Key (priority: environment variable > `.env`):
npx skills add https://github.com/benedictking/tavily-web --skill tavily-webAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 375 |
|---|---|
| repo stars | ★ 2 |
| Last updated | April 21, 2026 |
| Repository | benedictking/tavily-web ↗ |
What it does
Run web search, content extraction, crawling, or structured research through the Tavily API from inside an agent.
Who is it for?
Fetching the latest web information, extracting content from URLs, or discovering a site's structure during agent research tasks.
Skip if: Working offline or without a Tavily API key.
When should I use this skill?
You need latest information, need to extract content from URLs, or need to discover a site's structure.
What you get
The agent gets search results, extracted page content, crawled pages, a site map, or schema-structured research without wasting conversation tokens on the raw call.
- Web search results, extracted page content, crawled pages, a site map, or schema-structured research output
By the numbers
- Five Tavily endpoints: search, extract, crawl, map, research
Files
Tavily Web Skill
Trigger Conditions & Endpoint Selection
Choose Tavily endpoint based on user intent:
- search: Need to "search web / latest info / find sources / find links"
- extract: Given URL(s), need to extract/summarize content
- crawl: Need to traverse site following instructions and scrape page content
- map: Need to discover site page list/structure (without full content or metadata only)
- research: Need structured research output following given
output_schema
Recommended Architecture (Main Skill + Sub-skill)
This skill uses a two-phase architecture:
1. Main skill (current context): Understand user question → Choose endpoint → Assemble JSON payload 2. Sub-skill (fork context): Only responsible for HTTP call execution, avoiding conversation history token waste
Execution Method
Use Task tool to invoke tavily-fetcher sub-skill, passing command and JSON (stdin):
Task parameters:
- subagent_type: Bash
- description: "Call Tavily API"
- prompt: cat <<'JSON' | node .claude/skills/tavily-web/tavily-api.cjs <search|extract|crawl|map|research>
{ ...payload... }
JSONPayload Examples (Based on Provided curl)
1) Search the web
cat <<'JSON' | node .claude/skills/tavily-web/tavily-api.cjs search
{
"query": "who is Leo Messi?",
"auto_parameters": false,
"topic": "general",
"search_depth": "basic",
"chunks_per_source": 3,
"max_results": 1,
"time_range": null,
"start_date": "2025-02-09",
"end_date": "2025-12-29",
"include_answer": false,
"include_raw_content": false,
"include_images": false,
"include_image_descriptions": false,
"include_favicon": false,
"include_domains": [],
"exclude_domains": [],
"country": null,
"include_usage": false
}
JSON2) Extract webpages
cat <<'JSON' | node .claude/skills/tavily-web/tavily-api.cjs extract
{
"urls": "https://en.wikipedia.org/wiki/Artificial_intelligence",
"query": "<string>",
"chunks_per_source": 3,
"extract_depth": "basic",
"include_images": false,
"include_favicon": false,
"format": "markdown",
"timeout": "None",
"include_usage": false
}
JSON3) Crawl webpages
cat <<'JSON' | node .claude/skills/tavily-web/tavily-api.cjs crawl
{
"url": "docs.tavily.com",
"instructions": "Find all pages about the Python SDK",
"chunks_per_source": 3,
"max_depth": 1,
"max_breadth": 20,
"limit": 50,
"select_paths": null,
"select_domains": null,
"exclude_paths": null,
"exclude_domains": null,
"allow_external": true,
"include_images": false,
"extract_depth": "basic",
"format": "markdown",
"include_favicon": false,
"timeout": 150,
"include_usage": false
}
JSON4) Map webpages
cat <<'JSON' | node .claude/skills/tavily-web/tavily-api.cjs map
{
"url": "docs.tavily.com",
"instructions": "Find all pages about the Python SDK",
"max_depth": 1,
"max_breadth": 20,
"limit": 50,
"select_paths": null,
"select_domains": null,
"exclude_paths": null,
"exclude_domains": null,
"allow_external": true,
"timeout": 150,
"include_usage": false
}
JSON5) Create Research Task
cat <<'JSON' | node .claude/skills/tavily-web/tavily-api.cjs research
{
"input": "What are the latest developments in AI?",
"model": "auto",
"stream": false,
"output_schema": {
"properties": {
"company": {
"type": "string",
"description": "The name of the company"
},
"key_metrics": {
"type": "array",
"description": "List of key performance metrics",
"items": {
"type": "string"
}
},
"financial_details": {
"type": "object",
"description": "Detailed financial breakdown",
"properties": {
"operating_income": {
"type": "number",
"description": "Operating income for the period"
}
}
}
},
"required": [
"company"
]
},
"citation_format": "numbered"
}
JSONEnvironment Variables & API Key
Two ways to configure API Key (priority: environment variable > .env):
1. Environment variable: TAVILY_API_KEY 2. .env file: Place in .claude/skills/tavily-web/.env, can copy from .env.example
# Tavily API Key Configuration
# Get your API key from: https://app.tavily.com/home
TAVILY_API_KEY=your_api_key_here
.env
node_modules/
*.log
#!/usr/bin/env node
/**
* Tavily API Helper Script
* Provides a small CLI wrapper around Tavily endpoints for skill integration.
*
* Usage:
* node tavily-api.js <search|extract|crawl|map|research> [<json-string>]
* cat payload.json | node tavily-api.js search
* node tavily-api.js search --file ./payload.json
*/
const https = require('https');
const fs = require('fs');
const path = require('path');
const API_BASE = 'https://api.tavily.com';
function loadApiKey() {
if (process.env.TAVILY_API_KEY) {
return process.env.TAVILY_API_KEY;
}
const envPath = path.join(__dirname, '.env');
if (!fs.existsSync(envPath)) {
return null;
}
const envContent = fs.readFileSync(envPath, 'utf8');
const match = envContent.match(/TAVILY_API_KEY\s*=\s*(.+)/);
if (!match) {
return null;
}
return match[1].trim().replace(/^[\"']|[\"']$/g, '');
}
function usage() {
const cmd = path.basename(process.argv[1] || 'tavily-api.js');
console.error(
[
'Usage:',
` node ${cmd} <search|extract|crawl|map|research> [<json-string>]`,
` cat payload.json | node ${cmd} search`,
` node ${cmd} search --file ./payload.json`,
'',
'Env:',
' TAVILY_API_KEY (env var) or .env file next to this script',
].join('\n'),
);
}
function readStdin() {
return new Promise((resolve, reject) => {
let data = '';
process.stdin.setEncoding('utf8');
process.stdin.on('data', (chunk) => {
data += chunk;
});
process.stdin.on('end', () => resolve(data));
process.stdin.on('error', reject);
});
}
async function readPayload(args) {
const fileFlagIndex = args.findIndex((arg) => arg === '--file');
if (fileFlagIndex !== -1) {
const filePath = args[fileFlagIndex + 1];
if (!filePath) {
throw new Error('Missing value for --file');
}
const content = fs.readFileSync(filePath, 'utf8');
return JSON.parse(content);
}
const dataFlagIndex = args.findIndex((arg) => arg === '--data');
if (dataFlagIndex !== -1) {
const json = args[dataFlagIndex + 1];
if (!json) {
throw new Error('Missing value for --data');
}
return JSON.parse(json);
}
if (args[0] && !args[0].startsWith('-')) {
return JSON.parse(args[0]);
}
if (process.stdin.isTTY) {
throw new Error('No payload provided (pass JSON arg, --data, --file, or pipe via stdin)');
}
const stdin = await readStdin();
if (!stdin.trim()) {
throw new Error('Empty stdin payload');
}
return JSON.parse(stdin);
}
function validatePayload(command, payload) {
const required = {
search: 'query',
extract: 'urls',
crawl: 'url',
map: 'url',
research: 'input'
};
const field = required[command];
if (field && !payload[field]) {
throw new Error(`Missing required field: '${field}'`);
}
}
function postJson(endpointPath, apiKey, payload) {
return new Promise((resolve, reject) => {
const body = JSON.stringify(payload);
const url = new URL(endpointPath, API_BASE);
const req = https.request(
url,
{
method: 'POST',
headers: {
Authorization: `Bearer ${apiKey}`,
'Content-Type': 'application/json',
'Content-Length': Buffer.byteLength(body),
'User-Agent': 'Tavily-Skill/1.0',
},
timeout: 60_000,
},
(res) => {
let data = '';
res.setEncoding('utf8');
res.on('data', (chunk) => {
data += chunk;
});
res.on('end', () => {
const ok = res.statusCode && res.statusCode >= 200 && res.statusCode < 300;
if (!ok) {
reject(new Error(`API Error ${res.statusCode}: ${data}`));
return;
}
try {
resolve(JSON.parse(data));
} catch {
resolve(data);
}
});
},
);
req.on('error', reject);
req.on('timeout', () => {
req.destroy(new Error('Request timed out'));
});
req.write(body);
req.end();
});
}
const ENDPOINT_BY_COMMAND = {
search: '/search',
extract: '/extract',
crawl: '/crawl',
map: '/map',
research: '/research',
};
(async () => {
const command = process.argv[2];
if (!command || command === '--help' || command === '-h') {
usage();
process.exit(command ? 0 : 1);
}
const endpoint = ENDPOINT_BY_COMMAND[command];
if (!endpoint) {
usage();
process.exit(1);
}
const apiKey = loadApiKey();
if (!apiKey) {
console.error('Missing Tavily API key: set TAVILY_API_KEY or create .env next to tavily-api.js');
process.exit(1);
}
try {
const args = process.argv.slice(3);
const outputIndex = args.findIndex(arg => arg === '--output');
const outputFile = outputIndex !== -1 ? args[outputIndex + 1] : null;
const payloadArgs = outputFile ? args.filter((_, i) => i !== outputIndex && i !== outputIndex + 1) : args;
const payload = await readPayload(payloadArgs);
validatePayload(command, payload);
const result = await postJson(endpoint, apiKey, payload);
const output = JSON.stringify(result, null, 2);
if (outputFile) {
fs.writeFileSync(outputFile, output, 'utf8');
console.error(`Results saved to: ${outputFile}`);
} else {
console.log(output);
}
} catch (error) {
console.error(`Error: ${error.message}`);
process.exit(1);
}
})();
Tavily Fetcher Sub-skill
Note: This is an internal sub-skill, invoked by the tavily-web main skill through the Task tool.Purpose
Execute Tavily API calls in an independent context with context: fork, avoiding carrying main conversation context, reducing token consumption.
Received Parameters
Receives complete command through Task's prompt, using stdin for JSON:
cat <<'JSON' | node .claude/skills/tavily-web/tavily-api.cjs <search|extract|crawl|map|research>
{ ...payload... }
JSONOutput
Returns Tavily API's JSON response as-is (pretty printed).
Related skills
How it compares
Pick tavily-web for Tavily-powered agent web research; use firecrawl-scraper or exa-search skills when scraping, screenshots, or semantic search are the primary need.
FAQ
Which Tavily endpoint should I use?
search for finding sources or latest info, extract for summarizing given URLs, crawl for traversing and scraping a site, map for discovering a site's page list, and research for structured output following an output_schema.
How is the API key configured?
Via the TAVILY_API_KEY environment variable or a .env file placed in .claude/skills/tavily-web/.env, with the environment variable taking priority.