Tavily Search

Name: Tavily Search
Author: framix-team

framix-team/openclaw-tavily

Crawl a seed URL through Tavily’s API with depth, breadth, and format controls so your agent can harvest site content for research.

Overview

tavily-search is an agent skill for the Idea phase that crawls a seed URL via the Tavily API with depth, breadth, limit, and markdown or text output for research.

Install

npx skills add https://github.com/framix-team/openclaw-tavily --skill tavily-search

What is this skill?

Node CLI: crawl.mjs with url plus --depth, --breadth, --limit, --instructions, --format
Output formats: markdown or text
Requires TAVILY_API_KEY environment variable
Posts to https://api.tavily.com/crawl with configurable max depth and page limits
Fits agent pipelines that need offline-readable crawl bundles instead of one-off search snippets
CLI flags: --depth, --breadth, --limit, --instructions, --format (markdown|text)

Compatible agents: Claude Code, Cursor, Codex, any compatible agent

Adoption & trust: 737 installs on skills.sh; 75 GitHub stars; 2/3 security scanners passed (skills.sh audits).

What problem does it solve?

You need more than one search hit—a controlled multi-page extract from a site tree for competitive or technical research.

Who is it for?

Solo builders researching docs sites, competitor footprints, or niche communities with a capped Tavily crawl.

Skip if: High-frequency production scraping, sites that block crawlers, or research that must stay entirely on-device without third-party APIs.

When should I use this skill?

User needs structured multi-page extraction from a URL using Tavily Crawl with API key configured.

What do I get? / Deliverables

Running crawl.mjs returns Tavily-crawled pages in your chosen format so the agent can summarize, compare, or cite sources in downstream specs.

Crawled page content in markdown or text
Parameterized crawl invocation via crawl.mjs

Recommended Skills

Microsoft Foundrymicrosoft/azure-skills

Microsoft Foundry skill guides agents through the full Azure AI Foundry lifecycle—containerizing agents, pushing to ACR,…377k installs·1.2k stars

Azure Aimicrosoft/azure-skills

azure-ai is a Prism-oriented quick reference for Microsoft Azure AI work, with the published body centered on the Azure …375k installs·1.2k stars

Azure Hosted Copilot Sdkmicrosoft/azure-skills

Azure Hosted Copilot SDK is Microsoft's entry skill for repos using @github/copilot-sdk—it detects CopilotClient usage, …346k installs·1.2k stars

Lark Eventlarksuite/cli

Lark real-time subscription skill via lark-cli event consume for building bots and streaming webhook-style agent workers…208k installs·13.7k stars

Running Claude Code Via Litellm Copilotxixu-me/skills

Running Claude Code via LiteLLM Copilot walks through pointing Claude Code at a local LiteLLM proxy that forwards Anthro…200k installs·61 stars

Setup Matt Pocock Skillsmattpocock/skills

One-time per-repo setup so Matt Pocock engineering skills share correct issue tracker, triage strings, and domain docume…180k installs·121k stars

Journey fit

Primary fit

IdeaOpportunity & market research

Structured site crawling supports competitor and docs research before you commit to scope—canonical shelf is Idea research. The bundled crawl.mjs client targets Tavily Crawl for multi-page extraction, which maps directly to research subphase workflows.

Also useful

ValidateScope & plan

How it compares

Tavily multi-page crawl CLI—not a generic web_search one-liner or self-hosted Playwright spider.

Common Questions / FAQ

Who is tavily-search for?

Indie developers and agent users who already use Tavily and want scripted, parameterized site crawls during discovery work.

When should I use tavily-search?

In Idea research when mapping a docs portal or competitor site—set depth and limit, then feed markdown output into validation or scope notes.

Is tavily-search safe to install?

Review Prism Security Audits on this page; treat TAVILY_API_KEY as a secret and audit crawl targets and data handling before production use.

SKILL.md

READMESKILL.md - Tavily Search

#!/usr/bin/env node

function usage() {
  console.error('Usage: crawl.mjs "url" [--depth N] [--breadth N] [--limit N] [--instructions "..."] [--format markdown|text]');
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const url = args[0];
let depth = null;
let breadth = null;
let limit = null;
let instructions = null;
let format = null;

for (let i = 1; i < args.length; i++) {
  const a = args[i];
  if (a === "--depth") {
    depth = Number.parseInt(args[i + 1] ?? "2", 10);
    i++;
  } else if (a === "--breadth") {
    breadth = Number.parseInt(args[i + 1] ?? "10", 10);
    i++;
  } else if (a === "--limit") {
    limit = Number.parseInt(args[i + 1] ?? "10", 10);
    i++;
  } else if (a === "--instructions") {
    instructions = args[i + 1] ?? null;
    i++;
  } else if (a === "--format") {
    format = args[i + 1] ?? "markdown";
    i++;
  } else {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
}

const apiKey = (process.env.TAVILY_API_KEY ?? "").trim();
if (!apiKey) {
  console.error("Missing TAVILY_API_KEY");
  process.exit(1);
}

const body = { url };
if (depth !== null) body.max_depth = depth;
if (breadth !== null) body.max_breadth = breadth;
if (limit !== null) body.limit = limit;
if (instructions) body.instructions = instructions;
if (format) body.format = format;

const resp = await fetch("https://api.tavily.com/crawl", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify(body),
});

if (!resp.ok) {
  const text = await resp.text().catch(() => "");
  console.error(`Tavily Crawl failed (${resp.status}): ${text}`);
  process.exit(1);
}

const data = await resp.json();

console.log(`## Crawl: ${data.base_url ?? url}\n`);

const results = data.results ?? [];
console.log(`Found ${results.length} page(s)\n`);

for (const r of results) {
  const pageUrl = String(r?.url ?? "").trim();
  const content = String(r?.raw_content ?? "").trim();
  console.log(`### ${pageUrl}\n`);
  if (content) {
    console.log(content.slice(0, 2000) + (content.length > 2000 ? "\n\n... (truncated)" : ""));
  } else {
    console.log("(no content extracted)");
  }
  console.log("\n---\n");
}


#!/usr/bin/env node

function usage() {
  console.error('Usage: extract.mjs "url1" ["url2" ...] [--format markdown|text] [--query "..."]');
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const urls = [];
let format = "markdown";
let query = null;

for (let i = 0; i < args.length; i++) {
  const a = args[i];
  if (a === "--format") {
    format = args[i + 1] ?? "markdown";
    i++;
  } else if (a === "--query") {
    query = args[i + 1] ?? null;
    i++;
  } else if (!a.startsWith("-")) {
    urls.push(a);
  } else {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
}

if (urls.length === 0) {
  console.error("No URLs provided");
  usage();
}

const apiKey = (process.env.TAVILY_API_KEY ?? "").trim();
if (!apiKey) {
  console.error("Missing TAVILY_API_KEY");
  process.exit(1);
}

const extractBody = { urls, format };
if (query) extractBody.query = query;

const resp = await fetch("https://api.tavily.com/extract", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify(extractBody),
});

if (!resp.ok) {
  const text = await resp.text().catch(() => "");
  console.error(`Tavily Extract failed (${resp.status}): ${text}`);
  process.exit(1);
}

const data = await resp.json();

for (const r of data.results ?? []) {
  const url = String(r?.url ?? "").trim();
  const content = String(r?.raw_content ?? "").trim();
  console.log(`# ${url}\n`);
  console.log(content || "(no content extracted)");
  console.log("\n---\n");
}

const failed = data.failed_results ?? [];
if (failed.length > 0) {
  console.log("## Failed URL

What is this skill?

Node CLI: crawl.mjs with url plus --depth, --breadth, --limit, --instructions, --format

Output formats: markdown or text

Requires TAVILY_API_KEY environment variable

Posts to https://api.tavily.com/crawl with configurable max depth and page limits

Fits agent pipelines that need offline-readable crawl bundles instead of one-off search snippets

CLI flags: --depth, --breadth, --limit, --instructions, --format (markdown|text)

Compatible agents: Claude Code, Cursor, Codex, any compatible agent

Adoption & trust: 737 installs on skills.sh; 75 GitHub stars; 2/3 security scanners passed (skills.sh audits).

Journey fit

Primary fit

IdeaOpportunity & market research

Also useful

ValidateScope & plan

SKILL.md

READMESKILL.md - Tavily Search

#!/usr/bin/env node

function usage() {
  console.error('Usage: crawl.mjs "url" [--depth N] [--breadth N] [--limit N] [--instructions "..."] [--format markdown|text]');
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const url = args[0];
let depth = null;
let breadth = null;
let limit = null;
let instructions = null;
let format = null;

for (let i = 1; i < args.length; i++) {
  const a = args[i];
  if (a === "--depth") {
    depth = Number.parseInt(args[i + 1] ?? "2", 10);
    i++;
  } else if (a === "--breadth") {
    breadth = Number.parseInt(args[i + 1] ?? "10", 10);
    i++;
  } else if (a === "--limit") {
    limit = Number.parseInt(args[i + 1] ?? "10", 10);
    i++;
  } else if (a === "--instructions") {
    instructions = args[i + 1] ?? null;
    i++;
  } else if (a === "--format") {
    format = args[i + 1] ?? "markdown";
    i++;
  } else {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
}

const apiKey = (process.env.TAVILY_API_KEY ?? "").trim();
if (!apiKey) {
  console.error("Missing TAVILY_API_KEY");
  process.exit(1);
}

const body = { url };
if (depth !== null) body.max_depth = depth;
if (breadth !== null) body.max_breadth = breadth;
if (limit !== null) body.limit = limit;
if (instructions) body.instructions = instructions;
if (format) body.format = format;

const resp = await fetch("https://api.tavily.com/crawl", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify(body),
});

if (!resp.ok) {
  const text = await resp.text().catch(() => "");
  console.error(`Tavily Crawl failed (${resp.status}): ${text}`);
  process.exit(1);
}

const data = await resp.json();

console.log(`## Crawl: ${data.base_url ?? url}\n`);

const results = data.results ?? [];
console.log(`Found ${results.length} page(s)\n`);

for (const r of results) {
  const pageUrl = String(r?.url ?? "").trim();
  const content = String(r?.raw_content ?? "").trim();
  console.log(`### ${pageUrl}\n`);
  if (content) {
    console.log(content.slice(0, 2000) + (content.length > 2000 ? "\n\n... (truncated)" : ""));
  } else {
    console.log("(no content extracted)");
  }
  console.log("\n---\n");
}


#!/usr/bin/env node

function usage() {
  console.error('Usage: extract.mjs "url1" ["url2" ...] [--format markdown|text] [--query "..."]');
  process.exit(2);
}

const args = process.argv.slice(2);
if (args.length === 0 || args[0] === "-h" || args[0] === "--help") usage();

const urls = [];
let format = "markdown";
let query = null;

for (let i = 0; i < args.length; i++) {
  const a = args[i];
  if (a === "--format") {
    format = args[i + 1] ?? "markdown";
    i++;
  } else if (a === "--query") {
    query = args[i + 1] ?? null;
    i++;
  } else if (!a.startsWith("-")) {
    urls.push(a);
  } else {
    console.error(`Unknown arg: ${a}`);
    usage();
  }
}

if (urls.length === 0) {
  console.error("No URLs provided");
  usage();
}

const apiKey = (process.env.TAVILY_API_KEY ?? "").trim();
if (!apiKey) {
  console.error("Missing TAVILY_API_KEY");
  process.exit(1);
}

const extractBody = { urls, format };
if (query) extractBody.query = query;

const resp = await fetch("https://api.tavily.com/extract", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: `Bearer ${apiKey}`,
  },
  body: JSON.stringify(extractBody),
});

if (!resp.ok) {
  const text = await resp.text().catch(() => "");
  console.error(`Tavily Extract failed (${resp.status}): ${text}`);
  process.exit(1);
}

const data = await resp.json();

for (const r of data.results ?? []) {
  const url = String(r?.url ?? "").trim();
  const content = String(r?.raw_content ?? "").trim();
  console.log(`# ${url}\n`);
  console.log(content || "(no content extracted)");
  console.log("\n---\n");
}

const failed = data.failed_results ?? [];
if (failed.length > 0) {
  console.log("## Failed URL

Overview

Install

What is this skill?

What problem does it solve?

Who is it for?

When should I use this skill?

What do I get? / Deliverables

Recommended Skills

Journey fit

Who is tavily-search for?

When should I use tavily-search?

Is tavily-search safe to install?

SKILL.md

This week for builders

Overview

Install

What is this skill?

What problem does it solve?

Who is it for?

When should I use this skill?

What do I get? / Deliverables

Recommended Skills

Journey fit

Who is tavily-search for?

When should I use tavily-search?

Is tavily-search safe to install?

SKILL.md